Terraform: Difference between revisions

[https://www.terraform.io/ Terraform] is a tool for defining and provisioning data centre infrastructure, including virtual machines.  Terraform is seeing growing use within the Compute Canada Federation.  Its infrastructure-as-code model allows one to maintain OpenStack resources as a collection of definitions which can be easily updated using favourite text editors, shared among members of a group, and stored in version control.

There are two ways to provide your OpenStack credentials in a command-line environment: via environment variables or in a configuration file.  We'll need to use one of these methods with Terraform, described in the [[#Defining_OpenStack_provider|next section]].  Regardless of your preferred method, the OpenStack web interface offers a simple way to download credentials: once logged in, click on ''API Access'' in the navigation bar, and on that page is a drop-down menu entitled “Download OpenStack RC File”. From here you may download a <code>clouds.yaml</code> file or an RC file which can be sourced from your shell session.

<source lang="shell">$ source openrc.sh</source>

The other method is to create a configuration in <code>$HOME/.config/openstack/clouds.yaml</code>. If you don’t have such a file already, you can download `clouds.yaml` as described above and move it into place.  We recommend changing the name given to the cloud in the downloaded file to something meaningful, especially if you use more than one OpenStack cloud. Then, to use the CLI tools described below, simply create an environment variable <code>$OS_CLOUD</code> with the name of the cloud you want to use.

<source lang="shell">$ export OS_CLOUD=arbutus</source>

=== OpenStack session ===

The following is an example of a provider specification with connection and credential information:

<source lang="terraform">provider "openstack" {

}</source>

For some OpenStack instances the above would specify the complete set of information necessary to connect to the instance and manage resources in the given project (“tenant”). However, Terraform supports ''partial credentials'' in which you could leave some values out of the Terraform configuration and supply them a different way. This would allow us, for example, to leave the password out of the configuration file, in which case it would need to be specified in the environment with <code>$OS_PASSWORD</code>.

Alternatively, if you prefer to use <code>clouds.yaml</code>, specify <code>cloud</code> in the provider stanza:

<source lang="terraform">provider "openstack" {

}</source>

It's acceptable to leave the provider definition completely empty:

<source lang="terraform">provider "openstack" {

}</source>

In this case, either <code>$OS_CLOUD</code> or the variables set by the appropriate RC file would need to be in the executing environment for Terraform to proceed.

To ensure we have the provider set up correctly, initialize Terraform and check the configuration so far. With the provider definition in a file called, for example, <code>nodes.tf</code>, run <code>terraform init</code>:

<source lang="shell">$ terraform init

rerun this command to reinitialize your working directory. If you forget, other

commands will detect it and remind you to do so if necessary.</source>

This shows success in initializing Terraform and downloading the OpenStack provider plugin so the OpenStack stanzas will be handled correctly. This does not test out the credentials because this operation doesn’t actually try to connect to the defined provider.

</blockquote>

A minimal OpenStack VM may be defined as follows in Terraform:

<source lang="terraform">resource "openstack_compute_instance_v2" "myvm" {

   security_groups = ["default"]

}</source>

This will create a VM with the given name, image and flavor, and associate with it a key pair and the default security group.

The command <code>terraform plan</code> compiles the Terraform definition and attempts to determine how to reconcile the resulting desired state with the actual state on the cloud, and produces a plan of what it would if the changes were applied.

<source lang="shell">$ terraform plan

"terraform apply" is subsequently run.</source>

Read through this output.  This is a lot of information but it’s ''definitely'' required to check this before applying changes to ensure there are no surprises.

If you are in a hurry and don’t mind risking destroying or rebuilding resources by mistake, at ''least'' make sure you double-check the last line of the plan:

<source lang="shell">Plan: 1 to add, 0 to change, 0 to destroy.</source>

In this case we know we’re adding a resource so this looks right. If the other values were non-zero then we’d better have another look at our configuration, state and what’s actually defined in OpenStack and make whatever corrections are necessary.

Now, use <code>terraform apply</code> to actually effect the changes described in the plan.

<source lang="shell">$ terraform apply

   on nodes.tf line 4, in resource "openstack_compute_instance_v2" "myvm":

   4: resource "openstack_compute_instance_v2" "myvm" {</source>

This fails in this example.  OpenStack projects in Compute Canada have at least two networks defined: one private and one public.  Terraform needs to know which one to use.

The name of the private network differs from project to project and the naming convention can differ from cloud to cloud within Compute Canada, but typically they are on a 192.168.X.Y network, and can be found in the CLI using `network list` or on Horizon under ''Network''->''Networks''.  If your project's private network is <code>my-tenant-net</code>, you will add a <code>network</code> resource sub-block to your VM definition similar to the following:

<source lang="terraform">resource "openstack_compute_instance_v2" "myvm" {

}</source>

Try again:

<source lang="shell">$ terraform apply

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.</source>

You now have a VM created by Terraform. You should see your new VM on Horizon or in the output of <code>server list</code> in your OpenStack terminal window:

<pre class="plaintext">(openstack) server list -c ID -c Name -c Status

| 9b42cbf3-3782-4472-bdd0-9028bbb73460 | lbr    | ACTIVE |

+--------------------------------------+--------+--------+</pre>

In this example output, there are three other VMs created previously which survive untouched by Terraform.

Assuming you do not already have a floating IP allocated for this use, declare a desired floating IP resource like the following example. The only thing you need is to know the pool from which to allocate the floating IP; in Compute Canada clouds this is the external network (<code>ext_net</code> in this example).

<source lang="terraform">resource "openstack_networking_floatingip_v2" "myvm_fip" {

   pool = "ext_net"

}</source>

You may either apply this change immediately or just use <code>terraform plan</code> to show what would happen.

<source lang="shell">$ terraform apply

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.</source>

This floating IP is now ''allocated'' but not yet associated with your VM.  Add the following definition:

<source lang="terraform">resource "openstack_compute_floatingip_associate_v2" "myvm_fip" {

}</source>

This new resource defines as its attributes references to other resources and their attributes.

</blockquote>

References like this are typically <code>&lt;resource type&gt;.&lt;resource name&gt;.&lt;attribute&gt;</code>. Others you may soon see include <code>var.&lt;variable name&gt;</code>. At any rate, this resource forms an association between the created earlier, and the floating IP allocated in the next step.

<source lang="shell">$ terraform apply

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.</source>

Note that it has an associated floating IP, you could probably SSH into the new VM right now.

<source lang="shell">$ ssh centos@X.Y.Z.W hostname

myvm.novalocal</source>

If not, it may be necessary to add your workstation's IP address to the project's default security group.

Since this is a root volume, create it as part of the compute instance, as another subblock along with the network subblock:

<source lang="terraform">  block_device {

     delete_on_termination = true

   }</source>

Set the <code>uuid</code> attribute to the UUID of the image you want to use and remove <code>image_id</code> from the outer block definition. The other attributes are self-explanatory, except for <code>destination_type</code>, which is here set to <code>volume</code> to indicate this is to be stored with an OpenStack-provided volume rather than using disk on the hypervisor. <code>delete_on_termination</code> is important&mdash;for testing, you will probably want this to be <code>true</code> so you don’t have to remember to constantly clean up leftover volumes, but for real use you should consider setting it to <code>false</code> as a last defence against accidental deletion of resources.

</blockquote>

Here’s how the plan looks:

<source lang="shell">An execution plan has been generated and is shown below.

         }

     }</source>

So note there are several warnings of what’s going to be replaced and what’s going to change, not to mention this line:

Your VM will be created with a new SSH key, so if you connected previously you'll need to remove the SSH key from your <code>known_hosts</code> file (or the equivalent).  After this, the first thing to do is log on and ''apply all available updates''.

<source lang="shell">[centos@myvm ~]$ sudo yum update -y

...

[ goes for ages ]</source>

So you now have a working, Terraformed VM and a way to get to it and a place on it to store data once we get there, with the latest OS patches applied.

== The full example ==

<source lang="terraform">provider "openstack" {

   instance_id = openstack_compute_instance_v2.myvm.id

}</source>

== Appendix ==

This material is adapted from a posting which appeared on [https://acme.c3.ca the ACME group blog] (Compute Canada staff only; login required).  The original is [https://acme.c3.ca/blog/getting-started-with-terraform/ here] and is the first article in a series on Terraform.