Configuring storage quota service

281 views 1

How to configure your environment to use quota service for storage usage calculation.

Overview

Quota service is used for calculating web and mail storage usage for customer. Additionally it has a possibility of notifying customer when predefined storage limit has been reached by sending an email. Configuration of separate limit for email account only is also possible.

Calculations are done per package. Quota service is per default set under CsBase package, however it is also possible to create package that includes Quota service but not the CsBase package. In this use case additional provisioning handling for this package is necessary.

Requirements

Following set of requirements must be met for the Quota service to work correctly:

  • XFS Quota – it is required that you have xfs_quota storage calculation on your environment
  • Quota Agent – one new virtual machine with Quota Agent package installed on it
  • Quota service – quota service definition must be set for every customer package that will use this quota service

Quota Agent

Important!

This section is about an obsolete Atomia Quota agent (latest version 20.3.2) which will no longer get any updates. We strongly recommend that you switch to the latest atomia agents package.

Quota agent virtual machine must have Quota Agent package installed on it and properly configured. One MySql database has to be installed on the same machine with quota_statistics table added with following structure:

Column name Column type and attributes
id int NOT NULL PRIMARY KEY AUTO_INCREMENT
usage_id varchar(50) NOT NULL
scope varchar(255) NOT NULL
type varchar(255) NOT NULL
current bigint NOT NULL
quota bigint NOT NULL
timestamp timestamp NOT NULL

Configuration file of quota agent is located at /etc/atomia-quota-agent.config.

Following is an example of default configuration file:

{
    "auth":{
        "username":"someusername",
        "password":"somepassword"
    },
    "host":"http://127.0.0.1:5000",
    "db":{
        "user":"root",
        "password":"mysqlpassword",
        "name":"mysql",
        "host":"localhost"
    },
    "batch_insert_size":500,
    "usage_folder_path":"/***/usage_folder",
    "usage_parsing_folder_path":"/***/usage_parsing_folder",
    "usage_processed_folder_path":"/***/usage_processed_folder",
    "error_folder_path":"/***/error_logs"
}
  • auth API credentials for authorizing API requests towards Quota agent
  • host must be set to IP address of Quota Agent machine with port 5000
  • db section is for MySql database credentials Quota Agent will use to connect to database
  • batch_insert_size number of inputs that will be inserted in database at once
  • usage_folder_path path to the folder where all usage reports reside that need to be processed
  • usage_parsing_folder_path path to the folder where report will be moved just before it is processed (note: this folder needs to be created if it does not exist)
  • usage_processed_folder_path path to the folder where report will be moved after it is processed with timestamp added to the name of the file (note: this folder needs to be created if it does not exist)
  • error_folder_path path to the error folder where all the error logs will be stored (note: this folder needs to be created if it does not exist)

Resource.xml file located at Atomia/AutomationServer/Common must contain module Atomia.Provisioning.Modules.Quota.Quota with following information:

<bindings>
    <moduleList>
        <module name="Atomia.Provisioning.Modules.Quota.Quota" resourceAsignmentPolicy="RoundRobin" />
    </moduleList>
    <resourceList>
        <resource name="QuotaResource">
            <property name="Username">someusername</property>
            <property name="Password">somepassword</property>
            <property name="ApiEndpoint">http://127.0.0.1:5000</property>
        </resource>
    </resourceList>
</bindings>

Where Username and Password properties must match credentials from the configuration file of Quota Agent explained above. ApiEndpoint property must be set to the IP address of Quota Agent machine with 5000 port.

Periodic update of storage usage for Quota service

Atomia Quota Service has default settings for updating the storage usage. These defaults are:

These default values can be updated to suit your business and technical requirements of your Atomia installation. To change the behaviour and how usage is being collected you will need to update following files.

  • in cron job script located on your storage machine in /opt/atomia change trigger period
  • in atomia-quota-agent.cron cron job script located on Quota Agent machine in /etc/cron.d change trigger period
  • change expiresAfter value for all properties for Quota service and MailAccount service in Provisioning Description (value is in seconds)
  • change trigger period for QuotaNotificationScheduledEventHandler if it is enabled and for QuotaEnforceScheduledEventHandler also
  • change QuotaCacheTimeoutInMinutes pluginSetting to wanted value in Atomia/HostingControlPanel/bin/Atomia.Web.Plugin.Meter.dll.config (value is in minutes)

Enabling Quota service in User control panel

  • To enable Quota service to used for displaying storage usage in Meters bar in User control panel you need to set UseQuotaStorageStatistics pluginSetting in Atomia/HostingControlPanel/bin/Atomia.Web.Plugin.HCP.Provisioning.dll.config to true.
  • To enable possibility of setting the limit for email accounts you need to change UseQuotaMailbox globalSetting to True in Atomia/HostingControlPanel/App_Data/appConfig.config file.
    Also for this to work you need to set expiresAfter attribute to QuotaUsage property for MailAccount simple service like this:

    <property name="QuotaUsage" description="Quota Usage" defaultValue="0" expiresAfter="120" />
    

    keep in mind that expiresAfter value is in seconds

  • If there are packages that include Quota but not CsBase, you need to set QuotaEmptyPathPackages globalSetting to those packages in Atomia/HostingControlPanel/App_Data/appConfig.config file.

Scheduled tasks

  • Quota Notification Scheduled Event Handler – It is used to notify customer if some of his packages exceeding some of the defined storage usage limits by sending him an email with necessary data. More information about this scheduled task can be found here.
  • Quota Enforce Scheduled Event Handler – It is used for enforcing quota limit after storage usage for some package exceed 100% + tolerance, where tolerance is set in plugin configuration for this scheduled task. More information about this scheduled task can be found here.

Mail templates

Email templates used for Quota Notification Scheduled Event Handler are:

  • StorageQuotaUsageBody and StorageQuotaUsageSubject are used for emails regarding the package exceeding certain storage limit
  • MailboxQuotaUsageBody and MailboxQuotaUsageSubject are used for emails regarding the mailbox (Email Account) exceeding certain storage limit

More information can be found here.

Was this helpful?