Managing your cloud resources with OpenStack

Parent page: Cloud

OpenStack is the software suite used on the Compute Canada Cloud to control hardware resources such as computers, storage and networking. It allows the creation and management of virtual machines ("VMs", or "instances"), which act like separate individual machines, by emulation in software. This allows users complete control over their computing environment, from choosing an operating system to software installation and configuration. Diverse use cases are supported, from hosting websites to creating virtual clusters. More documentation on OpenStack can be found at the OpenStack web site.

This page describes how to perform common tasks encountered while working with OpenStack. It is assumed that you have already read Cloud Quick Start and understand the basic operations of launching and connecting to a VM. Most tasks can be performed using either the dashboard (as described below), CLI, or a tool called terraform; however, some require using command line tools, for example sharing an image with another project.

Projects

OpenStack projects group VMs together and provide a quota out of which VMs and related resources can be created. All accounts which are members of a project have the same level of permissions, meaning anyone can create or delete a VM within a project they are a member of. You can view the projects you are a member of by logging into an OpenStack dashboard for the cloud(s) you have access to (see cloud resources for a list of cloud URLs). The active project name will be displayed in the top left of the dashboard to the right cloud logo. If you are a member of more than one project you can switch between active projects by clicking on the drop down menu and clicking a project name.

Security Groups

A security group is a set of rules to control network traffic into and out of your virtual machines. To manage security groups go to Project->Compute->Access & Security and select the Security Groups tab. You will see a list of currently defined security groups. If you have not previously defined any security groups, there will be single default security group.

To add or remove rules from a security group, click Manage Rules beside that group. When the group description is displayed, you can add or remove rules by clicking the +Add Rule and Delete Rule buttons.

Default security group

Default Security Group Rules (Click for larger image)

The default security group contains rules which allow a VM access out to the internet, for example to download operating system upgrades or package installations, but does not allow another machine to access it, except for other VMs belonging to the same default security group. We recommend you do not remove rules from the default security group as this may cause problems when creating new VMs. The image on the right shows the default security group rules that should be present:

• 2 Egress rules, allow your instance to access outside network without any limitation, there is one rule for IPV4 and one for IPV6.
• 2 Ingress rules, allow communication for all the vms that belong to that security group, for both IPV4 and IPV6.

It is safe to add rules to the default security group and you may recall that we did this in Cloud Quick Start by either adding security rule for SSH or RDP to your default security group so that you were able to connect to your VM.

Managing security groups

You can define multiple security groups and a VM can belong to more than one security group. When deciding on how to manage your security groups and rules, think carefully about what needs to be accessed and who needs to access it. Strive to minimize the IP addresses and ports in your Ingress rules. For example, if you will always be connecting to your VM via SSH from the same computer with a static IP it makes sense to allow SSH access only from that IP. To specify the allowed IP or IP range use the CIDR box (use this web based tool for converting IP ranges to CIDR rules). Further, if you only need to connect to one VM via SSH from the outside and then can use that as a gateway to any other Cloud VMs, it makes sense to put the SSH rule in a separate security group and add that group only to the gateway VM. However, you will also need to ensure your SSH keys are configured correctly to allow you to use SSH between VMs (see SSH Keys). In addition to CIDR, security rules can be limited within a project using security groups. For example, you can configure a security rule for a VM in your project running a MySQL Database to be accessible form other VMs in the default security group.

The security groups a VM belongs to can be chosen when it is created on the Launch Instance screen under the Access & Security tab, or after the VM has been launched by selecting Edit Security Groups form the drop down menu of actions for the VM on the Project->Compute->Instances page.

Using CIDR rules

CIDR stands for Classless Inter-Domain Routing and is a standardized way of defining IP ranges (see also this wikipedia page on CIDR).

An example of a CIDR rule is 192.168.1.1/24. This looks just like a normal IP address with a /24 appended to it. IP addresses are made up of 4, 1 byte (8 bit) numbers ranging from 0 to 255. What this /24 means is that this CIDR rule will match the first left most 24 bits (3 bytes) of an IP address. In this case any IP address starting with 192.168.1 will match this CIDR rule. If /32 is appended the full 32 bits are specified the IP address must match exactly, likely wise if a /0 is appended no bits must match and therefor any IP address will match it.

Working with Volumes

A volume provides storage which is not destroyed when a VM is terminated. On the Compute Canada cloud resources, volumes use Ceph storage with a 3-fold replication factor to provide safety against hardware failure. More documentation about OpenStack volumes can be found here.

Creating a Volume

OpenStack Dashboard, Volumes page (Click for larger image)

Create Volume dialog (Click for larger image)

To create a volume click and fill in the following fields:

• Volume Name: data, for example
  
• Description:
  Optional text
• Volume Source: No source, empty volume
  
• Type: No volume type
  
• Size (GB): 40, or some suitable size
  
• Availability Zone: Any Availability Zone
  

Finally click the blue "Create Volume" button.

Accessing a Volume from a VM

To access the volume you must first attach it to a running VM. This is analogous to inserting a USB key or plugging an external drive into your personal computer. You can attach a volume from the Compute->Volumes page.

• At the right-hand end of the line describing the volume will be a drop-down menu; select "Manage Attachments".
• In the "Attach To Instance" drop-down box select a VM.
• Click the blue "Attach Volume" button.

The process should complete in a few seconds. Then the Compute->Volumes page will show the newly created volume is attached to your selected VM on /dev/vdb or some similar location.

Formatting and Mounting an Empty Volume

Before a newly created and attached volume can be used it must be formatted and mounted. See instructions for doing this on a Linux or Windows VM.

Booting from a Volume

If you want to run a persistent machine, it is safest to boot from a volume. When you boot a VM from an image rather than a volume, the VM is stored on the local disk of the actual machine running the VM. If something goes wrong with that machine or its disk the VM may be lost. Volume storage has redundancy which protects the VM from hardware failure. Typically when booting from a volume VM flavors starting with a 'p' are used (see Virtual machine flavors).

There are several ways to boot a VM from a volume. You can

• boot from an image, creating a new volume, or
• boot from a pre-existing volume, or
• boot from a volume snapshot, creating a new volume.

If you have not done this before, then the first one is your only option. The other two are only possible if you have already created a bootable volume or a volume snapshot.

If creating a volume as part of the process of launching the VM, select Boot from image (creates a new volume), select the image to use, and the size of the volume. If this volume is something you would like to remain longer than the VM ensure that the "Delete on Terminate" box is unchecked. If you are unsure about this option, it is better to leave this box unchecked. You can manually delete the volume later.

Creating an Image from a Volume

Upload to Image form (Click for larger image)

Creating an image from a volume allows you to download the image. Do this if you want to save it as a backup, or to spin up a VM somewhere other than the CC Cloud, e.g. with VirtualBox. To create an image of a volume it must first be detached from a VM. If it is a root volume, it can only be detached from a VM if the VM is terminated/deleted.

Using the Dashboard

1. Click on the Volumes left hand menu.
2. Under the volume you wish to create an image of click on the drop down Actions menu and select Upload to Image.
3. Choose a name for your new image.
4. Choose a Disk Format. QCOW2 is recommended for using within the OpenStack cloud as it is relatively compact compared to Raw image format and works well with OpenStack. If you wish to use the image with Virtualbox the vmdk or vdi image formats might be better suited.
5. Finally click Upload.

Using the Command Line Clients

The command line client can do this:

[name@server ~]$ openstack image create --disk-format <format> --volume <volume_name> <image_name>

where

• <format> is the disk format (two possible values are qcow2 and vmdk),
• <volume_name> can be found from the OpenStack dashboard by clicking on the volume name, and
• <image_name> is a name you choose for the image.

You can then download the image as described below. It is best to detach the volume from the VM before you create an image from the volume. If the volume is a boot volume you will likely need to delete your VM to detach it, however, make sure you have not checked "Delete Volume on Instance Delete" when creating the VM.

Working with images

Creating an Image from a VM

Using the OpenStack Command Line Clients issue the command

[name@server ~]$  openstack server image create <server-name>

where <server-name> should be replaced with the name of your server. This action will only include the VM's root drive (e.g. /dev/vda) in the image. Ephemeral drives and non-boot attached volumes will not be included in the image so additional measures should be taken to preserve this data. In addition, if the VM is writing to disk while the image is being created the file system may be captured in an inconsistent state. We recommend the VM be shut off (not deleted) before an image is created from it.

Sharing an image with another project

Sharing an image with another project is a two step process.

1. A member of the project owning the image must share it with a second project.
2. A member of the second project must accept the newly shared image.

To share an image a member in the project owning the image uses the OpenStack command below.

[name@server]$  glance member-create <IMAGE_ID> <MEMBER_ID>
+------------+-------------+---------+
| Image ID   | Member ID   | Status  |
+------------+-------------+---------+
| <IMAGE_ID> | <MEMBER_ID> | pending |
+------------+-------------+---------+

where <IMAGE_ID> is the ID of image to be shared, and <MEMBER_ID> is the ID of project to share with.

To accept the shared image a member in the second project uses the OpenStack command below.

[name@server]$  glance member-update <IMAGE_ID> <MEMBER_ID> <MEMBER_STATUS>
+------------+-------------+----------+
| Image ID   | Member ID   | Status   |
+------------+-------------+----------+
| <IMAGE_ID> | <MEMBER_ID> | accepted |
+------------+-------------+----------+

where <IMAGE_ID> is ID of the image to update, <MEMBER_ID> is the ID of the second project, and <MEMBER_STATUS> is the new status of the image. Valid Values for status are accepted, rejected, and pending. The image will then be available for use and appear in the OpenStack dashboard's list of images in the second project.

To check the status of image membership use the below command.

[name@server]$ glance member-list --image-id <IMAGE_ID>
+------------+-------------+----------+
| Image ID   | Member ID   | Status   |
+------------+-------------+----------+
| <IMAGE_ID> | <MEMBER_ID> | accepted |
+------------+-------------+----------+

where <IMAGE_ID> is the ID of the image to check the membership status of.

Downloading an Image

The first step is to install the OpenStack client and download the OpenStack RC file and source it (see OpenStack Command Line Clients). The OpenStack client can list the available images on your OpenStack project with

[name@server ~]$ openstack image list

producing something like:

+--------------------------------------+---------------------------------------+-------------+------------------+-------------+--------+
| ID                                   | Name                                  | Disk Format | Container Format | Size        | Status |
+--------------------------------------+---------------------------------------+-------------+------------------+-------------+--------+
| 982761b2-c77b-4852-8ae3-bf98b32b8894 | Hadoop-2.2.4                          | qcow2       | bare             | 10253107200 | active |
| b7bd3033-9836-406d-a8f2-2e91978026b4 | hadoopmaster                          | qcow2       | bare             | 3493527552  | active |
| 2c751755-854d-49c3-af82-d501e51e7159 | hadoopmaster-active                   | qcow2       | bare             | 13134004224 | active |
| c41012f4-ed82-4478-a81f-5efb96a31b1a | hadoopmaster-old                      | qcow2       | bare             | 3493527552  | active |
| 78e61a3f-b546-441a-b476-a7077b04ca36 | hadoopslave                           | qcow2       | bare             | 3490971648  | active |
| 516845c3-b256-4c6d-a2cb-e31e822c7e34 | hadoopslave1-active                   | qcow2       | bare             | 8345026560  | active |
| 1546bd86-5314-4fce-9576-e2f6930dad30 | hadoopslave1-old                      | qcow2       | bare             | 3490971648  | active |
| baf78e8d-8288-4854-a66b-812cdf3ccbca | TestVM                                | qcow2       | bare             | 13167616    | active |
| 2faf97d7-5b0b-44ce-8024-3bef5a634570 | test_ubuntu_initial                   | qcow2       | bare             | 1799487488  | active |
| 308b6614-396a-4360-9c33-4e86f41ea0ec | trusty                                | qcow2       | bare             | 256180736   | active |
| 9b3c3fda-2aca-43b5-a3e7-662a94f5e7fb | Ubuntu_14.04_Trusty-amd64-20150708    | qcow2       | bare             | 257884672   | active |
| f93e66cf-fec1-4460-8fc7-506e716fbf30 | ucernvm-prod.1.18-10                  | raw         | bare             | 20971520    | active |
+--------------------------------------+---------------------------------------+-------------+------------------+-------------+--------+

You can then download a particular image with

[name@server ~]$ openstack image save --file ./<file-name-for-image>.<format> <ID>

where <format> matches the value in the Disk format column and <ID> matches the value in the ID column.

Uploading an Image

The first step is to install the OpenStack client and download the OpenStack RC file and source it (see OpenStack Command Line Clients). Then run the command

[name@server ~]$ openstack image create --file <path-to-local-file-image> --disk-format <format> <new-image-name>

where

• <path-to-local-file-image> is the path to the file containing the image you wish to upload from your local machine,
• <format> is the disk format; if not specified, the raw format is assumed, which is incorrect since it can cause issues when using the image in OpenStack,
• <new-image-name> is the name of the image as it appears on the OpenStack dashboard.

Creating a VirtualBox VM from a Cloud Image

VirtualBox is a software package which allows you to create and run virtual machines on your desktop or laptop. It can be run on many different operating systems (Windows, Linux, Mac) and the virtual machines it creates may run one of many different operating systems.

To use a QCOW2 image downloaded from an OpenStack cloud, as shown above, with VirtualBox you will need to convert the image in the qcow2 format to the vmdk format. This can be done with the qemu-img tool. This can be installed with something like

[name@server ~]$ sudo apt-get install qemu-utils

(previously the package was called qemu-img) then do the conversion with

[name@server ~]$ qemu-img convert -f qcow2 vdisk.qcow2 -O vmdk vdisk.vmdk

Then you can create a new virtual machine and attach the vmdk image to it (see how to run a vmdk file in virtualbox for detailed instructions on this).

Working with VMs

Locking VMs

When working with multiple people in a project or to protect a VM from accidental deletion or shutdown it can be useful to lock it.

To lock a VM, click on the "Lock Instance" from the action menu in the dashboard.
Once a vm is locked most of the action menu item will not be able to be executed until the instance is unlocked. There is an icon indicating the lock state for every instance.

To unlcok a VM, select the "Unlock Instance" from the action menu in the dashboard.

Resizing VMs

It is possible to resize a VM by changing its flavor. However, there are some things to be aware of when choosing to resize a VM which depends on whether you have a "p" flavor or a "c" flavor VM (see Virtual machine flavors). Resizing a VM may involve some risk as it is similar to deleting and recreating your VM with a new flavor, if in doubt contact cloud technical support.

c flavors

"c" flavors often have extra ephemeral drives, which will be resized when you choose a new "c" flavor. These ephemeral drives can not become smaller, and as such "c" flavor VMs can only be resized to flavors with equal or larger ephemeral drives. After the resize however, you will not immediately see a larger ephemeral drive within your VM (e.g. the df -h command will not show the size increase). To see this extra space you will need to resize your file system (see the resize2fs command). However, filesystem resizes should be treated with caution and can take considerable time if the partitions are large. Before resizing a filesystem it is recommend to create backups of its contents (see backing up your VM).

p flavors

Unlike "c" flavors, "p" flavors do not typically have extra ephemeral drives associated with them, so they can be resized to larger and smaller flavors.

Linux VM User Management

There are a number of ways to allow more than one person to log into a VM. We recommend creating new user accounts and adding public SSH Keys to these accounts.

Creating an account and keys

A new user account can be created on Ubuntu with the command

[name@server ~]$ sudo adduser --disabled-password USERNAME

To be able to connect, the new user will need to have a key pair, see generating SSH keys in windows or creating a key pair in Linux or Mac depending on the operating system they will be connecting from. Then their public key must be added to /home/USERNAME/.ssh/authorized_keys on the VM, ensuring permissions and ownership are correct as described in steps 2 and 3 of connecting using a key pair.

Granting admin privileges

In Ubuntu administrative, or root user privliges, can be given to a new user with the command

[name@server ~]$ sudo visudo -f /etc/sudoers.d/90-cloud-init-users

which opens an editor where a line like

USERNAME ALL=(ALL) NOPASSWD:ALL

can be added. For more detailed information about the visudo command and how to edit this file see this digitalocean tutorial on the topic.

Add users with CloudInit during VM creation

Cloud init to add multiple users (Click for larger image)

Alternatively, you can do this during the creation of a VM using CloudInit. The following CloudInit script adds two users gretzky and lemieux with and without sudo permissions respectively.

 #cloud-config
users:
  - name: gretzky
    shell: /bin/bash
    sudo: ALL=(ALL) NOPASSWD:ALL
    ssh_authorized_keys:
      - <Gretzky's public key goes here>
  - name: lemieux
    shell: /bin/bash
    ssh_authorized_keys:
      - <Lemieux's public key goes here>

For more about the YAML format used by CloudInit, see YAML Preview. Note that YAML is very picky about white space formatting, so that there must be a space after the "-" before your public key string. Also, this configuration overwrites the default user that is added when no CloudInit script is specified, so the users listed in this configuration script will be the only users on the newly created VM. It is therefore vital to have at least one user with sudo permission. More users can be added by simply including more - name: username sections.

If you wish to preserve the default user created by the distribution (users debian, centos, ubuntu, etc.), use the following form:

 #cloud-config
users:
  - default
  - name: gretzky
    shell: /bin/bash
    sudo: ALL=(ALL) NOPASSWD:ALL
    ssh_authorized_keys:
      - <Gretzky's public key goes here>
  - name: lemieux
    shell: /bin/bash
    ssh_authorized_keys:
      - <Lemieux's public key goes here>

After the VM has finished spawning, look at the log to ensure that the public keys have been added correctly for those users. The log can be found by clicking on the name of the instance on the "Compute->Instances" panel and then selecting the "log" tab. The log should show something like this:

ci-info: ++++++++Authorized keys from /home/gretzky/.ssh/authorized_keys for user gretzky++++++++
ci-info: +---------+-------------------------------------------------+---------+------------------+
ci-info: | Keytype |                Fingerprint (md5)                | Options |     Comment      |
ci-info: +---------+-------------------------------------------------+---------+------------------+
ci-info: | ssh-rsa | ad:a6:35:fc:2a:17:c9:02:cd:59:38:c9:18:dd:15:19 |    -    | rsa-key-20160229 |
ci-info: +---------+-------------------------------------------------+---------+------------------+
ci-info: ++++++++++++Authorized keys from /home/lemieux/.ssh/authorized_keys for user lemieux++++++++++++
ci-info: +---------+-------------------------------------------------+---------+------------------+
ci-info: | Keytype |                Fingerprint (md5)                | Options |     Comment      |
ci-info: +---------+-------------------------------------------------+---------+------------------+
ci-info: | ssh-rsa | ad:a6:35:fc:2a:17:c9:02:cd:59:38:c9:18:dd:15:19 |    -    | rsa-key-20160229 |
ci-info: +---------+-------------------------------------------------+---------+------------------+

Once this is done, users can log into the VM with their private keys as usual (see SSH Keys).