Atomia Agents

Upgrading to the new Atomia Agents

7 views 0

This manual covers the upgrade and data migration from the old atomia agents to the new ones.

Overview

The new Atomia agents 21.5.1 package should be used as the next version for the following obsoleted packages:

  • Atomia Cron Agent 18.10.2
  • Atomia File System Agent 20.3.3
  • Atomia Apache Agent 18.10.2
  • Atomia AWStats Agent 1.1.34
  • Atomia Quota Agent 20.3.2
  • Atomia SSL Redirect Agent 18.10.1

This article will explain the upgrade steps for each of them. Data migration is only needed for the Cron agent.

Before you begin

The preconditions are that the machine where the new agent is installed is a Linux server with either Ubuntu 16.04 or Ubuntu 18.04 (recommended) installed with at least Python 3.5 version. Some agents require a MySQL DB to be installed prior to agent installation.

Upgrading from legacy Apache agent

If you have a legacy Apache agent installed on the machine you should locate your settings file (settings.cfg) in your installation folder (usually /usr/local/apache-agent).

After installing the new Apache agent you should copy those settings to /etc/atomia/apache-agent.json

apache-agent.json settings.cfg default value comment
server_web_root SERVER_WEB_ROOT /storage/content
vhost_map HOSTS_MAP_FILE /storage/configuration/maps/vhost.map
parking_map USERS_MAP_FILE /storage/configuration/maps/users.map
parking_map PARKING_MAP_FILE /storage/configuration/maps/parks.map
redirection_map REDIRECTION_MAP_FILE /storage/configuration/maps/redrs.map
frame_redirection_map FRAME_REDIRECTION_MAP_FILE /storage/configuration/maps/frmrs.map
suspended_map SUSPENDED_MAP_FILE /storage/configuration/maps/sspnd.map
package_map PACKAGE_MAP_FILE /storage/configuration/maps/packg.map
php_version_map PHPVERSION_MAP_FILE /storage/configuration/maps/phpvr.map
proxy_map PROXY_MAP_FILE /storage/configuration/maps/proxy.map
port SERVICE_PORT 9999
ssl_config_dir SSL_CONFIG_DIR /etc/httpd/ssl.d
ssl_cert_dir SSL_CERT_DIR /storage/content/%%s/%%s/ssl
apachectl_path APACHECTL_PATH /usr/sbin/apachectl
SERVICE_AUTH False always True in the new agent
auth.username SERVICE_AUTH_USER
auth.password SERVICE_AUTH_PW
SERVE_HTTPS False missing entry
CERTIFICATE_FILE /usr/local/apache-agent/server.crt
PRIVATE_KEY_FILE /usr/local/apache-agent/server.key

Cleanup

Stop and remove legacy apache service and package:

service apache-agent stop

service apache-agent disable

apt-get remove atomia-pa-apache

Cleanup remaining data (installation folder, backups, etc).

Update Atomia Automation Server configuration

Update provisioning description file by changing providing module from Atomia.Provisioning.Modules.Apache.Apache to Atomia.Provisioning.Modules.ApacheRest.Apache.

Update resource description file by changing binding module name from Atomia.Provisioning.Modules.Apache.Apache to Atomia.Provisioning.Modules.ApacheRest.Apache.

Upgrading from legacy Awstats agent

If you have a legacy Awstats agent installed on the machine you should locate your settings file (settings.cfg) in your installation folder (usually /usr/local/awstats-agent).

After installing the new Awstats agent you should copy those settings to /etc/atomia/awstats-agent.json

awstats-agent.json settings.cfg default value comment
awstats_config.dir AWSTATS_CONFIG_DIR /etc/awstats/
awstats_config.https_force_enabled HTTPS_FORCE_ENABLED False
port SERVICE_PORT 9999
SERVICE_AUTH True always True
auth.username SERVICE_AUTH_USER
auth.password SERVICE_AUTH_PW

Cleanup

Stop and remove legacy Awstats service and package:

service awstats-agent stop

service awstats-agent disable

apt-get remove atomia-pa-awstats

Cleanup remaining data (installation folder, backups, etc).

Update Atomia Automation Server configuration

Update provisioning description file by changing providing module from Atomia.Provisioning.Modules.AWStats.AWStats to Atomia.Provisioning.Modules.AWStatsRest.AWStats.

Update resource description file by changing binding module name from Atomia.Provisioning.Modules.AWStats.AWStats to Atomia.Provisioning.Modules.AWStatsRest.AWStats. In addition, a property name needs to be changed in the AWStats resource from URL to AwstatsBaseUrl.

Upgrading from legacy Cron agent

If you have a legacy Cron agent installed on the machine you should have its settings in the environment variables. They can be found in /etc/default/cronagent.

After installing the new Cron agent you should copy those settings to /etc/atomia/cron-agent.json. If there is no environment variable specified you should put default value.

cron-agent.json environment variable default value comment
port ‘CRON_LISTEN_PORT’ 10101
‘CRON_GLOBAL_AUTH_TOKEN’ null Replaced by auth.user and auth.password
min_part ‘MIN_PART’ 0
max_part ‘MAX_PART’ 1000
interval ‘INTERVAL’ 5 seconds
‘USE_LOCALTIME’ true timezone is now used
mail.host MAIL_HOST “localhost”
mail.port MAIL_PORT 25
mail.user MAIL_USER
mail.password MAIL_PASS
mail.ssl MAIL_SSL false
mail.from MAIL_FROM [email protected]

Migrate data

Data should be migrated from MongoDB to the new MySQL DB.

First, we need to export data to the CSV file from the legacy agent MongoDB. The export command is:

mongoexport -u cronagent -p cronagent  --collection=tasks --db=cronagent --csv -f Nextdate,Enabled,Url,Email,OnlyOnError,Owner,Name,Lastdate,Lastresult,className,Interval,Months,Days,Weekdays,Weekday,Which > tasks.csv

Make sure to update the command parameters according to the data in /etc/default/cronagent.

Second, we import data from generated tasks.csv file into the new agent with the db_port.py migration script:

cd /usr/lib/atomia/cron
python3 db_port.py <some_path>/tasks.csv

The report should look something like this:

Rows imported: 4, Errors: 0

If there are some errors they will be added to import_errors.csv as original rows with the addition of import error column. Check the errors and try to fix the CSV data. It can happen that some cron run time is in the past and that’s why the import failed – just change the date to some time in the future. You can copy/rename import_errors.csv to fixed_import_errors.csv, fix errors and run the script with it:

`python3 db_port.py <some_path>/fixed_import_errors.csv`.

Cleanup

Stop and remove legacy Cron service and package:

service atomia-cronagent stop

service atomia-cronagent disable

apt-get remove atomia-cronagent

Cleanup remaining data (installation folder, backups, etc).

Update Atomia Automation Server configuration

Update provisioning description file by changing providing module from tomia.Provisioning.Modules.CronAgent.CronAgent to Atomia.Provisioning.Modules.CronAgentRest.CronAgent.

Update resource description file by changing binding module name from tomia.Provisioning.Modules.CronAgent.CronAgent to Atomia.Provisioning.Modules.CronAgentRest.CronAgent. In addition, a property AuthToken should be removed and 2 new properties (Username, Password) added in the Cron resource:

<resource name="CronAgent01">
	<property name="CronBaseUrl">http://192.168.33.13:10101</property>
	<property name="Username">SomeUsername</property>
	<property name="Password">SomePassword</property>
</resource>

Upgrading from legacy File System agent

If you have a legacy File System agent installed on the machine you should have its settings in the environment variables. They can be found in /etc/default/fsagent.

After installing the new File System agent you should copy those settings to /etc/atomia/filesystem-agent.json. If there is no environment variable specified you should put default value.

filesystem-agent.json environment variable default value comment
content_path ‘FS_STORAGE_PATH’ ‘/storage/content’
mail_content_path ‘FS_MAIL_STORAGE_PATH’ ‘/storage/mailcontent’
backup_path ‘FS_STORAGE_BACKUP_PATH’ ‘/storage/content_backup’
parent_root_folders_pattern ‘FS_ROOT_DIRS_PATTERNS’ [‘^/[0-9][0-9]$’] split into list by ‘||’
user_root_folders_pattern ‘FS_USER_ROOT_DIRS_PATTERNS’ [‘^/[0-9][0-9]/(\d{6,}).*$’] split into list by ‘||’
arbitrary_root_folders_pattern ‘FS_ARBITRARY_DIRS_PATTERNS’ [‘^/awstats-sites$’] split into list by ‘||’
auth.username ‘FS_USERNAME’ ‘atomia’
auth.password ‘FS_PASSWORD’ ‘atomia13’
parent_accounts ‘FS_PARENT_ACCOUNT’ ‘100000’
additional_group ‘FS_ADD_GROUP’ 33 Number
port ‘FS_AGENT_PORT’ 10201 Number
list_symbolic_links ‘FS_LIST_SYMBOLIC_LINKS’ 0 Number

Note: If your legacy configuration file holds multiple regex conditions separated by logical OR (||) for FS_USER_ROOT_DIRS_PATTERNS and/or FS_ARBITRARY_DIRS_PATTERNS like:

FS_USER_ROOT_DIRS_PATTERNS='^/[0-9][0-9]/([0-9]{6,}).*$||^/ssl/([0-9]{6,}).*$'
FS_ARBITRARY_DIRS_PATTERNS='^/awstats-sites$||^/ssl$'

It needs to be replaced with a list in the new agent configuration, like:

"user_root_folders_pattern":["^\/?[0-9][0-9]\/([0-9]{6,})", "^/?ssl/([0-9]{6,}).*$"],
"arbitrary_root_folders_pattern":["^/awstats-sites$", "^/ssl$"],

Cleanup

Stop and remove legacy File System service and package:

service atomia-fsagent stop

service atomia-fsagent disable

apt-get remove atomia-fsagent

Cleanup remaining data (installation folder, backups, etc).

Upgrading from legacy Quota agent

If you have legacy Quota Agent installed on the machine you should have its settings in the /etc/atomia-quota-agent.config.

After installing the new Quota agent you should copy those settings to /etc/atomia/quota-agent.json.

The settings properties are the same. The only difference is with the property usage_folder_path which is no longer at the top-level item in the JSON list. It is now part of servers list item:

{   "...": "...",
    "servers":[
        {
            "usage_folder_path": "/storage/quota_reports/reports",
        },
        {
            "...": "..."
        }
    ]
}

If the legacy agent was parsing reports from multiple folders, then there is a CRON job that puts all files into one folder. That cron can now be removed and each folder should have its own record (with the usage_folder_path property) in the servers list item.

The legacy agent also has the cron job (/etc/cron.d/atomia-quota-agent-cron) that triggers parsing which can also be removed. It is now substituted with a daemon (atomia-quota-daemon). Parsing interval for a daemon is set in the settings file, interval property, which represents the interval value in seconds.

Cleanup

Stop and remove legacy Quota service and package:

service atomia-quota-agent stop

service atomia-quota-agent disable

apt-get remove atomia-quota-agent

Cleanup remaining data (installation folder, backups, etc).

Upgrading from legacy SslRedirects agent

If you have a legacy SslRedirects agent installed on the machine you should have its settings in the environment variables. They can be found in /etc/default/ssl-redirects-agent.

After installing the new SSL Redirect agent you should copy those settings to /etc/atomia/sslredirect-agent.json. If there is no environment variable specified you should put default value.

sslredirect-agent.json environment variable default value comment
ssl_domain_file_path ‘SR_FILE_PATH’ ‘/storage/configuration/ssl-redirects.lst’
auth.username ‘SR_USERNAME’
auth.password ‘SR_PASSWORD’
port ‘SR_AGENT_PORT’ 10203 Number

Cleanup

Stop and remove legacy Quota service and package:

service atomia-sslredirects-agent stop

service atomia-sslredirects-agent disable

apt-get remove atomia-sslredirects-agent

Cleanup remaining data (installation folder, backups, etc).

Was this helpful?