Managing your cloud resources with OpenStack: Difference between revisions

<languages />

<translate>

''Parent page: [[CC-Cloud]]''

OpenStack is the software suite used on the Compute Canada Cloud to control hardware resources such as computers, storage and networking. It allows the creation and management of virtual machines ("VMs", or "instances"), which act like separate individual machines, by emulation in software. This allows users complete control over their computing environment, from choosing an operating system to software installation and configuration. Diverse use cases are supported, from hosting websites to creating virtual clusters. More documentation on OpenStack can be found at the [http://docs.openstack.org/ OpenStack web site].

This page describes how to perform common tasks encountered while working with OpenStack. It is assumed that you have already read [[Cloud Quick Start]] and understand the basic operations of launching and connecting to a VM. Most tasks can be performed using the dashboard, as described there and below. But some require use of the command line tools, for example [[#Creating an Image From a Volume|creating an image]]. See [[OpenStack Command Line Clients]] for more information.

Security groups allow you to control network traffic into and out of your virtual machines. To manage security groups go to ''Project->Compute->Access & Security'' and select the ''Security Groups'' tab. You will see a list of currently defined security groups. If you have not previously defined any security groups, there will be single ''default'' security group.  

To add or remove security rules from a security group click ''Manage Rules'' for that particular security group. To add a new rule click ''Add Rule'' button in the top right, to remove a rule click ''Delete Rule'' beside the rule you wish to delete.

The ''default'' security group contains a number of rules by default. These rules allow network traffic for any port, from any ip, into (Ingress) a VM originating from another VM in the ''default'' security group for internet protocols version 4 and 6. They also allow network traffic out (Egress) of a VM from any port to any IP for both internet protocol versions. In other words these rules allow a VM which belongs to the ''Default'' security group access out to the internet, to download content (e.g. operating system upgrades, package installations) but does not allow another machine outside the ''default'' security group access to the VM. These default rules allow you to correctly launch a VM, removing them may cause problems when creating new VMs and is not recommended. These rules do not allow access to your VM from outside the default security group which is why to connect to your VM via SSH a security rule was added for port 22 to allow incoming (Ingress) traffic so that you were able to connect to your VM (see [[Cloud Quick Start#Firewall, add rules to allow SSH| Firewall, add rules to allow SSH]]).

You can define multiple security groups and a VM can belong to multiple security groups with each security group having a number of security rules. When deciding on how to manage your security groups and rules, think carefully about what needs to be accessed and who needs to access it. For example, if you will always be connecting to your VM from the same computer with a static IP via SSH it makes sense to allow SSH access only from that IP. To specify the allowed IP or IP range use the [https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing CIDR] box. Further, if you are only connecting to one VM via ssh from the outside, and then connecting to other VMs within the default security group from that VM it makes sense to add the SSH security rule to a separate group and add that group to the VM you are SSHing to, however, you will also need to ensure your ssh keys are configured correctly to ssh between VMs (see [[Ssh keys | SSH Keys]]).

The security group a VM belongs to can be configured when they are created on the ''Launch Instance'' screen under the ''Access & Security'' tab, or after the VM has been launched by selecting ''Edit Security Groups'' form the drop down menu of actions for the VM on the ''Project->Compute->Instances'' page.

A '''volume''' provides storage which is not destroyed when a VM is terminated. On the Compute Canada cloud resources, volumes use [https://en.wikipedia.org/wiki/Ceph_(software) Ceph] storage with a 3-fold replication factor to provide safety against hardware failure. More documentation about OpenStack volumes can be found [http://docs.openstack.org/user-guide/common/cli_manage_volumes.html here].

[[File:OpenStack-Dashboard-Compute-Volumes.png|800px|thumb| OpenStack Dashboard, Volumes page (Click for larger image)]]

[[File:Create-Volume-Form.png|400px|thumb| Create Volume dialog (Click for larger image)]]

To create a volume click [[File:Create-Volume-Button.png]] and fill in the following fields:

*Volume Name: <code>data</code>, for example<br/>

*Description:<br/>Optional text

*Availability Zone: <code>Any Availability Zone</code><br/>

Finally click the blue "Create Volume" button.

To access the volume you must first '''attach''' it to a running VM. This is analogous to inserting a USB key or plugging an external drive into your personal computer. You can attach a volume from the Compute->Volumes page.

* At the right-hand end of the line describing the volume will be a drop-down menu. Select "Manage Attachments" or "Edit Attachments".

The process should complete in a few seconds. Then the Compute->Volumes page will show the newly created volume is attached to your selected VM on <code>/dev/vdb</code> or some similar location.

On most Linux distributions the following steps can be used to partition, format, and mount the newly created volume. NOTE: If this is not a newly created volume the partition and format steps should be skipped as they will result in loss of data on that volume, and only the steps to mount the volume should be followed.

<ol>

<li>Create a partition on the volume with

<code>fdisk</code> will prompt you to enter a command. Use this sequence of single-character commands to create a new partition on your volume.</li>

  p => primary, only one partition on disk

  1 => partition number 1

  w => write partition table to disk and exit

<li>Format the newly created partition with  

{{Command|sudo mkfs -t ext4 /dev/vdb1}}

</ol>

If the VM is rebooted for some reason the volume will need to be remounted. To cause the VM to mount the volume automatically at boot time, edit <code>/etc/fstab</code> and add a line like

If you are not rebooting, you can mount the device just added to <code>/etc/fstab</code> with  

{{Command|sudo mount -a}}

If you want to run a persistent machine, it is safest to boot from a volume. When you boot a VM from an image rather than a volume, the VM is stored on the local disk of the actual machine running the VM. If something goes wrong with that machine or its disk the VM may be lost. Volume storage has redundancy which protects the VM from hardware failure.

There are several ways to boot a VM from a volume. You can  

* boot from an image, creating a new volume, or  

* boot from a volume snapshot, creating a new volume.

If you have not done this before, then the first one is your only option. The other two are only possible if you have already created a bootable volume or a volume snapshot.

If creating a volume as part of the process of launching the VM, select <code>Boot from image (creates a new volume)</code>, select the image to use, and the size of the volume. If this volume is something you would like to remain longer than the VM ensure that the "Delete on Terminate" box is unchecked. If you are unsure about this option, it is better to leave this box unchecked. You can manually delete the volume later.

Creating an image from a volume allows you to download the image. Do this if you want to save it as a backup, or to spin up a VM somewhere other than the CC Cloud, e.g. with [https://www.virtualbox.org/ VirtualBox].

Use the [[OpenStack Command Line Clients|command line client]] to do this:

{{Command|openstack image create --disk-format <format> --volume <volume_name> <image_name>}}

You can then download the image as described in [[OpenStack#Downloading an image into a local VirtualBox | Downloading an image into a local VirtualBox]].

[[File:VM multi user cloud init.png|400px|thumb| Cloud init to add multiple users (Click for larger image)]]

There are a number of ways to approach this, but basically what is needed is to add another public ssh key to the VM (see [[ssh keys]] about working with and creating keys). This could be done by adding to the <code>authorized_keys</code> of the primary user account provided for the VM, usually named after the operating system, or it could be added to a newly created user account. In Ubuntu, when logged in with the primary user's account this can be accomplished with the command {{Command|sudo adduser USERNAME}}

Then add the supplied public key to that user's <code>.ssh/authorized_keys</code> file as described in [[ssh keys]] and linked pages.

Another alternative would be to do this during the creation of your VM using [http://cloudinit.readthedocs.org/en/latest/index.html# CloudInit]. The following CloudInit script (which uses the YAML format, see [http://www.yaml.org/spec/1.2/spec.html#Preview YAML Preview]):

  users:

   - name: sysadmin

       - <Insert public key>

adds two users <code>sysadmin</code> and <code>user</code> with and without sudo permissions respectively. The <code><Insert public key></code> must be replaced with the public key to use for that user. Note that YAML is very picky about white space formatting, so that there must be a space after the "-" before your public key string. Also, this configuration overwrites the default user that is added when no CloudInit script is specified, so that the users listed in this configuration script will be the only users on the newly created VM so it is important to have at least one user with sudo permission. More users can be added by simply including another <code>- name: username</code> section.

If you wish to preserve the default user created by the distribution (debian, centos, ubuntu, etc.), use the following form:

  users:

   - default

       - <Insert public key>

After the VM has finished spawning, take a look at the log to ensure that the public keys have been added correctly for those users. The log can be found by click the name of the instance under the "Compute->Instances" panel and then selecting the "log" tab.  The log should show something like:

  ci-info: +---------+-------------------------------------------------+---------+------------------+

  ci-info: | Keytype |                Fingerprint (md5)                | Options |    Comment      |

  ci-info: +---------+-------------------------------------------------+---------+------------------+

Once this is done, users can log into the VM with their private keys as usual (see [[ssh keys]]).

The first step is to install the OpenStack client and download the OpenStack RC file and source it (see [[OpenStack Command Line Clients]]).

The OpenStack client can list the available images on your OpenStack project with

producing something like:

  | ID                                  | Name                                  | Disk Format | Container Format | Size        | Status |

  +--------------------------------------+---------------------------------------+-------------+------------------+-------------+--------+

  +--------------------------------------+---------------------------------------+-------------+------------------+-------------+--------+

You can then download a particular image with

{{Command|openstack image save --file ./<file-name-for-image>.<format> 2c751755-854d-49c3-af82-d501e51e7159}}

where <format> matches that given in the table above for your selected image andthe last argument is the <code>ID</code> of the image you wish to download.

Then to use this image in VirtualBox you will need to convert the image in the qcow2 format to the vmdk format. This can be done with the <code>qemu-img</code> tool. This can be installed with something like

{{Command|sudo apt-get install qemu-img}}