Automating VM creation

The below virtual machine (VM) setup files are intended to be used to quickly create VMs and clusters of VMs. The files come in two forms, Heat templates, and CloudInit files; both use the YAML file format. CloudInit files are used to initialize a particular VM and run within that VM. They can be used to update the operating system, install and configuring applications, create files, run commands, and create users and groups. Heat templates are even more powerful. They can create multiple VMs at once, configure security groups to access VMs from the outside world, create and configure networks, create and attach volumes to VMs, and pass information about other resources to CloudInit (e.g. floating IPs). They can also be used in conjunction with CloudInit files.

Using CloudInit

The creation of CloudInit YAML files is not covered here, instead see the official CloudInit documentation. When you are creating a new VM a CloudInit file can be used to describe how the VM is to be configured after the selected image has been loaded. CloudInit files can be used with the Horizon dashboard (OpenStack's web GUI), the CLI, or the Python API. Here we describe how to use a CloudIinit file with Horizon.

Specifying a CloudInit File

• Start as normal when launching an instance, by clicking under Project->Compute->Instances and specifying your VM's configuration as described in Cloud_Quick_Start#Launching_a_VM.
• Before clicking Launch, select the Post-Creation tab.
• Specify your Customization Script Source, in this case a CloudInit YAML file, by either coping and pasting into a text box (Direct Input method) or uploading from a file from your desktop computer (File method). Older versions of OpenStack, in particular IceHouse, only provide a text box to copy and past your CloudInit file into.
• Once the usual selections for your VM, as described in Cloud_Quick_Start#Launching_a_VM, have been made and the CloudInit YAML file is included, click Launch to create the VM.
• It may take some time for CloudInit to complete depending on what has been specified in the CloudInit YAML file.

Checking CloudInit Progress

To see the progress of CloudInit on a VM, check the console log of the VM by:

• Selecting Project->Compute->Instances in the left hand menu.
• Click on the Instance Name of the VM. This will provide more information about the particular VM.
• Select the Log tab and look for lines containing 'clout-init' for information about the various phases of CloudInit.
• When CloudInit has finished running the following line will appear near or at the end of the log:

 Cloud-init v. 0.7.5 finished at Wed, 22 Jun 2016 17:52:29 +0000. Datasource DataSourceOpenStack [net,ver=2].  Up 44.33 seconds

Using Heat Templates

TODO: finish writing up these sections and add more setup files and test/streamline setup files.

Available Setups

Setup	Description	Type	Status	URL
Hadoop+Spark	Creates a Hadoop cluster with Spark configured to run ontop of HDFS and submit jobs using YARN.	Heat Template	Has been used successfully with the ubuntu-server-14.04-amd64 image and the c1-7.5gb-30 flavor at west-cloud. Needs further testing and streamlining.	https://raw.githubusercontent.com/cgeroux/heat-hadoop-spark/master/hadoop%2Bspark.yaml