VPS configuration

How to enable customers to buy and manage virtual private servers that are hosted on Azure and/or Amazon AWS. It can work alongside your own OpenStack cloud in your local data center, or on it’s own.

Overview

This feature enables provisioning of virtual private servers on public clouds. Currently, the feature works with Amazon AWS and Azure. All regions of these providers can be used, and a region can be disclosed or not, if required. In addition, it’s supported to set up multiple accounts used for provisioning for one provider, and round robin selects provider account when provisioning. The feature uses the same VPS manager graphical interface that is being used in the OpenStack cloud setup.

When creating a new machine, you first select which provider you wish to use (if more than one are configured), and fill out the rest of necessary data for that provider. A new component called Atomia Public Cloud Agent is created as part of the feature. It is used as communication buffer between Atomia and public cloud providers. This component is required for operation. More information about it can be found below.

Before you begin

Before you begin, you need to set up accounts for the providers you wish to use. This should be repeated for any account you wish to use as a separate resource.

Setting up Azure account

1. Register your account on Azure portal.
2. Upload the certificate used for API access authorization.
3. On Linux, generate the certificate file using the following commands:

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout azure_cert.pem -out azure_cert.pem
openssl x509 -inform pem -in azure_cert.pem -outform der -out azure_cert.cer

4. When the certificate is generated, go to Azure Management Portal: https://manage.windowsazure.com
5. Select Settings > Management certificates > Upload.
6. Upload the generated .cer file.

For authorization on Azure API, Azure subscription id and generated .pem file are used. Subscription id can be seen on Settings page in Azure portal. Generated .pem should be uploaded somewhere on the machine where Public Cloud Agent is installed. Read more about this below.

Setting up Amazon AWS account

1. Register your account on Amazon AWS portal.
2. When your account is fully operational (this can take up to 24 hours), select Services > IAM from the top menu.
3. Go to Users and create a new user.
4. Set permissions for this user. For example attach AmazonEC2FullAccess and AdministratorAccess policies on Permissions tab.
5. Go to the Security credentials tab and create an access key.
6. Save this key for later as this is used for authorization on Amazon AWS API.

Installing Atomia Public Cloud Agent

1. Install a resource with name Public Cloud Agent through the Atomia Installer.
2. When installation is completed, open file /etc/atomialibcloudagent.conf and check if hostname and port are OK under [ServerSection].
3. If you wish to use HTTPS for communication with the Agent, you can configure locations for .cert and .key files with ssl_cert_file and ssl_key_file config options, respectively. With config option processes you can set a number of processes that will handle requests sent to the Agent. There is also possible option with name threaded which marks if the process should handle each request in a separate thread.
4. If you are using Azure as a provider, you can now upload .pem file(s) somewhere on the Agent machine.

Configuring Atomia Automation Server

Configuring resources

Start with configuring the Resources.xml file. There is already a binding for module Atomia.Provisioning.Modules.PublicCloud.PublicCloud, with commented out example resource configs. These examples are also listed below. The resources for all providers should be put inside the same element. The names for each resource should be unique.

Azure resource example:

<resource name="Azure01">
    <property name="Provider">azure</property>
    <property name="ResourceProviderName">azure</property>        
    <property name="InitParams"><![CDATA[{'subscription_id': '05h67fed-4c92-4537-841a-52566ca2d485', 'key_file': '/home/atomia/azure_cert.pem'}]]></property>
    <property name="Region">West Europe</property>
    <property name="UrlBase">http://127.0.0.1:4000</property>
    <property name="MaxItems">20</property>
    <property name="HasProjects">true</property>
    <property name="HasFirewalls">false</property>
    <property name="HasKeyPairs">false</property>
    <property name="DnsUrlBase">.cloudapp.net</property>
    <property name="InstanceExistsMsgPart">already exists</property>
    <property name="InstanceDeletedMsgPart">Not Found</property>
    <property name="ProjectExistsMsgPart">DNS name is already taken</property>
    <property name="ProjectDeletedMsgPart">The hosted service does not exist</property>
    <property name="AddDefaultPorts">false</property>
</resource>

Amazon AWS resource example:

<resource name="Amazon01">
    <property name="Provider">amazon</property>
    <property name="ResourceProviderName">ec2</property>        
    <property name="InitParams"><![CDATA[{"key": "IHSKJHBII6FVBBVXGRPA", "secret": "VdRsx7/CzF+UgrfGkjsTEERrQtUhDadX89bMjtPy"}]]></property>
    <property name="Region">US East</property>    
    <property name="UrlBase">http://127.0.0.1:4000</property>
    <property name="MaxItems">20</property>
    <property name="HasProjects">false</property>
    <property name="HasFirewalls">true</property>
    <property name="HasKeyPairs">true</property>
    <property name="DnsUrlBase"></property>
    <property name="InstanceExistsMsgPart"></property>
    <property name="InstanceDeletedMsgPart"></property>
    <property name="AddDefaultPorts">true</property>
</resource>

Properties explanation:

• Provider – Value for this property for Azure/Amazon should either be “azure”/”amazon”, or start with “azure-“/”amazon-“. For example: “azure-westeurope”/”amazon-useast”. Part after the “-” is not relevant and can be anything. Resources with the same value for “Provider” are considered as one group and shown to the user as one provider. If you have more that one account set up on a provider, you can set up multiple resources like this that are shown to the user as one provider. Round robin is used for selection of the resource. It is also possible to set Azure to display all regions as one. This is not possible for Amazon AWS, since Amazon has different image ids for different regions.
• ResourceProviderName – This is the name the Public Cloud Agent uses. For Azure, it should always be “azure”. For Amazon AWS, one of the following should be used, based on region: “ec2_eu_west”, “ec2_ap_southeast”, “ec2_us_east”, “ec2_us_west_oregon”, “ec2_us_west”, “ec2_ap_northeast”, “ec2”, “ec2_sa_east”, “ec2_eu”, “ec2_ap_southeast2”.
• InitParams – This property contains json with authentication info for provider API. Format should stay the same as respective examples, but some values should be changed. For Azure, subscription_id and key_file values should be changed so replace 1e687fed-4c92-4537-841a-52566ca81f56 with your subscription id and /home/atomia/azure_cert.pem with location to the .pem file on your Agent machine. For Amazon AWS, set key and secret, so replace AKIAJHBII6FVBBVXGRPA with your API key id and LpyYx7/CzF+UgrfGkjsTEERrQtUhDadP53bMjtPy with your secret.
• Region – Region to be used for provider. For Azure, it must be name of the region. For Amazon AWS is not used because ResourceProviderName specifies the region already.
• UrlBase – It’s the hostname where Public Cloud Agent is running. It’s the same for both Azure and Amazon AWS.
• MaxItems – Number of instances limit on resource.

The rest of the properties are behavioural configuration and should be the same as in respective examples.

Configuring provisioning description

There is a complex service named CsPublicCloud, which is the base complex service for all public cloud services. This service has a property EnablePublicCloud and its default value has to be changed from false to true.

This service also needs to be configured to be part of all the packages where public cloud should be available.

Configuring public cloud images and sizes

The public cloud images and sizes that are available for buying are configured in Atomia.Provisioning.Modules.PublicCloud.dll.config file within Automation Server’s Modules folder. There is a section that holds the json configuration. In the configuration, the top level are provider identifiers that hold images and instance types (sizes) configuration.

The next level is the same as the OpenStack module configuration. Elements image_id, instance_type and ssh_username must be provided and represent provider/region ids. For Azure, ssh_username is used for both Linux and Windows images. These values can be set to anything, as they will be set as username when creating machine, and for Amazon AWS it must be predefined image username. Here are some of the Amazons usernames: https://alestic.com/2014/01/ec2-ssh-username/ . For Amazon AWS, ssh_port should also be configured.

Example:

    <cloudInfo>
        <![CDATA[ { 'azure': { 'AvailableImages': [ { 'name': { 'default': 'Azure Linux Ubuntu Server 12.04' }, 'description': { 'default': 'Ubuntu is a fast, secure and easy-to-use operating system used by millions of people around the world.', 'sv': 'Ubuntu är ett fritt operativsystem baserat på linuxkärnan. Ubuntus målsättning är bland annat att vara ett aktuellt, stabilt operativsystem för den genomsnittlige användaren, med ett starkt fokus på användarvänlighet och enkel installation.' }, 'image_id': 'b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu-12_04_3-LTS-amd64-server-20140127-en-us-30GB', 'product_id': 'VPS-LINUX', 'admin_password_set' : false, 'ssh_username' : 'ubuntu' }, { 'name': { 'default': 'Microsoft Windows Server 2008 R2 SP1' }, 'description': { 'default': 'Ubuntu is a fast, secure and easy-to-use operating system used by millions of people around the world.', 'sv': 'Ubuntu är ett fritt operativsystem baserat på linuxkärnan. Ubuntus målsättning är bland annat att vara ett aktuellt, stabilt operativsystem för den genomsnittlige användaren, med ett starkt fokus på användarvänlighet och enkel installation.' }, 'image_id': 'a699494373c04fc0bc8f2bb1389d6106__Win2K8R2SP1-Datacenter-20151214-en.us-127GB.vhd', 'product_id': 'VPS-WINDOWS', 'admin_password_set' : false, 'ssh_username': 'azureuser' } ], 'AvailableInstanceTypes': [ { 'name': { 'default': 'Azure extra small instance', 'sv': 'Miniatyr-VPS' }, 'description': { 'default': 'Our smallest instance type. 512 MB of RAM.', 'sv': 'Var allra minsta VPS. 512 MB RAM-minne.' }, 'instance_type': 'ExtraSmall', 'available_images': [ 'b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu-12_04_3-LTS-amd64-server-20140127-en-us-30GB' ] }, { 'name': { 'default': 'Azure standard D1 instance', 'sv': 'Miniatyr-VPS' }, 'description': { 'default': 'Our smallest instance type. 512 MB of RAM.', 'sv': 'Var allra minsta VPS. 512 MB RAM-minne.' }, 'instance_type': 'Standard_D1', 'available_images': [ 'b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu-12_04_3-LTS-amd64-server-20140127-en-us-30GB', 'a699494373c04fc0bc8f2bb1389d6106__Win2K8R2SP1-Datacenter-20151214-en.us-127GB.vhd' ] } ] }, 'amazon': { 'AvailableImages': [ { 'name': { 'default': 'Amazon Linux Ubuntu Server 12.04' }, 'description': { 'default': 'Ubuntu is a fast, secure and easy-to-use operating system used by millions of people around the world.', 'sv': 'Ubuntu är ett fritt operativsystem baserat på linuxkärnan. Ubuntus målsättning är bland annat att vara ett aktuellt, stabilt operativsystem för den genomsnittlige användaren, med ett starkt fokus på användarvänlighet och enkel installation.' }, 'image_id': 'ami-fce3c696', 'product_id': 'VPS-LINUX', 'admin_password_set' : false, 'ssh_username' : 'ubuntu', 'ssh_port' : '22' } ], 'AvailableInstanceTypes': [ { 'name': { 'default': 'Amazon T2 instance', 'sv': 'Miniatyr-VPS' }, 'description': { 'default': 'Our smallest instance type. 512 MB of RAM.', 'sv': 'Var allra minsta VPS. 512 MB RAM-minne.' }, 'instance_type': 't2.micro', 'available_images': [ 'ami-fce3c696' ] } ] } } ]]>
    </cloudInfo>

Setting up Billing

For each available image that you have configured, there must be a usage based product with article number like the one you set for product_id image property. There also needs to be a counter for each instance type that the image can be installed on.

Setting up VPS Manager

In the Hosting Control Panel’s bin folder, Atomia.Web.Plugin.VPS.dll.config has to be edited. The new pluginSetting element needs to be added with name “Providers”, and the values of all providers used has to be separated by comma. If you have your own OpenStack, it should be added as at the beginning, for example:

<pluginSetting name="Providers" value="Openstack, Azure, Amazon" />

Note that these names when lowercased should be the same as the ones configured in Provider properties of resources. The VPS Manager’s resource files also has to have entries for all the names configured here, including OpenStack. This way you can set your own name instead of OpenStack.

The following settings should also be added like provided here. These disable unsupported options:

<pluginSetting name="UnsupportedOpsAmazon" value="reboot"/>
<pluginSetting name="UnsupportedOpsAzure" value="reboot, stop, start"/>

There are some additional settings that have default values. These do not require to be edited, but it is possible to change them.

• PublicInstanceAsyncStatusCheck – If VPS Manager index page takes long to load instances, it can be due to overhead of many providers. In that case you can turn on asynchronous instance status check, to not block the page. The default value is: false.
• CacheValidMinutes – e.g. CacheValidMinutesAzure, sets for how long the data is obtained asynchronously. Default value: 1440.
• AppendAccountWhenNoProject – If the provider doesn’t support projects for each customer, all services are in one place. Appending the account id before service name avoids duplicate name error. The default value is: true.

Current limitations

• In the current version, Windows images on Amazon AWS are not supported. This means that it is possible to provision them, but that the administrator password is randomly generated. There is currently no way to obtain the password. If password were to be manually retrieved, everything else would work. This will be fixed in future releases.
• Since Amazon AWS has different images ids across different regions, it is not currently possible to abstract the regions and show amazon as one provider, and use the regions as multiple resources. All regions must be shown as separate providers.
• Currently this version only supports old Default theme.