Automating VM creation

Parent page: OpenStack

The below virtual machine (VM) setup files are intended to be used to quickly create VMs and clusters of VMs. The files come in two forms, Heat templates, and CloudInit files; both use the YAML file format. CloudInit files are used to initialize a particular VM and run within that VM. They can be used to update the operating system, install and configuring applications, create files, run commands, and create users and groups. Heat templates are even more powerful. They can create multiple VMs at once, configure security groups to access VMs from the outside world, create and configure networks, create and attach volumes to VMs, and pass information about other resources to CloudInit (e.g. floating IPs). They can also be used in conjunction with CloudInit files.

Using CloudInit

The creation of CloudInit YAML files is not covered here, instead see the official CloudInit documentation . When you are creating a new VM a CloudInit file can be used to describe how the VM is to be configured after the selected image has been loaded. CloudInit files can be used with the Horizon dashboard (OpenStack's web GUI), the CLI, or the Python API. Here we describe how to use a CloudIinit file with Horizon.

Specifying a CloudInit File

Start as normal when launching an instance, by clicking under Project->Compute->Instances and specifying your VM's configuration as described in Cloud Quick Start#Launching a VM.
Before clicking Launch, select the Post-Creation tab.
Specify your Customization Script Source, in this case a CloudInit YAML file, by either coping and pasting into a text box (Direct Input method) or uploading from a file from your desktop computer (File method). Older versions of OpenStack, in particular IceHouse, only provide a text box to copy and past your CloudInit file into.
Once the usual selections for your VM, as described in Launching a VM, have been made and the CloudInit YAML file is included, click Launch to create the VM.
It may take some time for CloudInit to complete depending on what has been specified in the CloudInit YAML file.

Checking CloudInit Progress

To see the progress of CloudInit on a VM, check the console log of the VM by:

Selecting Project->Compute->Instances in the left hand menu.
Click on the Instance Name of the VM. This will provide more information about the particular VM.
Select the Log tab and look for lines containing 'clout-init' for information about the various phases of CloudInit.
When CloudInit has finished running the following line will appear near or at the end of the log:

 Cloud-init v. 0.7.5 finished at Wed, 22 Jun 2016 17:52:29 +0000. Datasource DataSourceOpenStack [net,ver=2].  Up 44.33 seconds

Using Heat Templates

As with CloudInit the creation of Heat Orchestration Template (HOT) files is not covered here, instead see the official documentation. Heat allows automation of operations performed in Horizon as well as the ability to pass information into the embeded CloudInit files, such as an IP of another server. It also allows one to add wait conditions which signal to OpenStack when the creation of the stack is complete. These wait conditions can be triggered from within the running VMs once various setup steps have been completed. Finally Heat allows output to be created from the resulting stack, such as indicating the URL for a particular service created by the stack.

To use a HOT file:

Select Project->Ochestration->Stacks
Click the Launch Stack button to start creating a new stack
Provide a HOT file. This can be done using a URL, File, or Direct Input. To use a HOT file from the table below, select URL for Template Source and copy and paste the URL in the table into the Template URL box.
Click Next to begin setting your stack parameters. Stack parameters will very depending on the template, however all Stacks have the following parameters by default:
- The Stack Name parameter which identifies a given stack, choose a name which is meaningful.
- The Creation Timeout parameter indicates how long after stack creation before OpenStack will give up trying to create the stack if it hasn't finished.
- The Password for user parameter sets the password required for later stack changes.
Click Launch to begin creating your stack.

To graphically see the progress of you stack creation click on the Stack Name and select the Topology tab. Gray nodes indicate creation in progress, green nodes have finished being created, and red nodes indicate failures. Once the stack has completed successfully click the Overview tab to see any information that the stack may provide (e.g. url to access a service).

Available Setups

Setup	Description	Type	Compatible Images	Minimum Required OpenStack Version	Status	URL
Hadoop+Spark	Creates a Hadoop cluster with Spark configured to run ontop of HDFS and submit jobs using YARN.	Heat Template	ubuntu-server-14.04-amd64	Kilo	Has been used successfully with the c1-7.5gb-30 flavor at west-cloud. Needs further testing and streamlining.	https://raw.githubusercontent.com/cgeroux/heat-hadoop-spark/master/hadoop%2Bspark.yaml

See CC-Cloud Resources for the OpenStack versions running on the CC clouds.