Managing your cloud resources with OpenStack/en: Difference between revisions

Jump to navigation Jump to search

Managing your cloud resources with OpenStack/en (view source)

Revision as of 21:23, 8 November 2022

20,197 bytes removed ,  2 years ago
Updating to match new version of source page
m (FuzzyBot moved page OpenStack/en to Managing your cloud resources with OpenStack/en without leaving a redirect: Part of translatable page "OpenStack")
(Updating to match new version of source page)
Line 12: Line 12:


Projects can be thought of as owned by primary investigators (PIs) and new projects and quota adjustments can only be requested by PIs. In addition, request for access to an existing project must be confirmed by the PI owning the project.
Projects can be thought of as owned by primary investigators (PIs) and new projects and quota adjustments can only be requested by PIs. In addition, request for access to an existing project must be confirmed by the PI owning the project.
=Working with volumes=
Please see [[Working_with_volumes|this page]] for more information about creating and managing storage volumes.
=Working with images=
Please see [[Working_with_images|this page]] for more information about creating and managing disk image files.
= Working with VMs=
Please see [[Working_with_VMs|this page]] for more information about managing certain characteristics of your VMs in the Dashboard.


=Availability Zones=
=Availability Zones=
Line 38: Line 47:
An example of a CIDR rule is <code>192.168.1.1/24</code>. This looks just like a normal IP address with a <code>/24</code> appended to it. IP addresses are made up of 4, 1-byte (8 bits) numbers ranging from 0 to 255. What this <code>/24</code> means is that this CIDR rule will match the first left most 24 bits (3 bytes) of an IP address. In this case, any IP address starting with <code>192.168.1</code> will match this CIDR rule. If <code>/32</code> is appended, the full 32 bits of the IP address must match exactly; if <code>/0</code> is appended, no bits must match and therefore any IP address will match it.
An example of a CIDR rule is <code>192.168.1.1/24</code>. This looks just like a normal IP address with a <code>/24</code> appended to it. IP addresses are made up of 4, 1-byte (8 bits) numbers ranging from 0 to 255. What this <code>/24</code> means is that this CIDR rule will match the first left most 24 bits (3 bytes) of an IP address. In this case, any IP address starting with <code>192.168.1</code> will match this CIDR rule. If <code>/32</code> is appended, the full 32 bits of the IP address must match exactly; if <code>/0</code> is appended, no bits must match and therefore any IP address will match it.


=Working with Volumes=
=Working with CloudInit=
 
A volume provides storage which is not destroyed when a VM is terminated. On our clouds, volumes use [https://en.wikipedia.org/wiki/Ceph_(software) Ceph] storage with either a 3-fold replication factor or [https://en.wikipedia.org/wiki/Erasure_code erasure codes] to provide safety against hardware failure. On arbutus the ''Default'' volume type uses erasure codes to provide data safety while reducing the extra storage costs of 3-fold replication while the ''OS or Database'' volume type still uses the 3-fold replication factor. The More documentation about OpenStack volumes can be found [https://docs.openstack.org/cinder/latest/cli/cli-manage-volumes.html here]
 
==Creating a Volume==
 
[[File:Creating_a_volume_EN.png|300px|thumb| Create Volume dialog (Click for larger image)]]
 
To create a volume click [[File:Create-Volume-Button.png]] and fill in the following fields:
 
*Volume Name: <code>data</code>, for example<br/>
*Description: Optional text
*Volume Source: <code>No source, empty volume</code><br/>
*Type: <code>No volume type</code><br/>
*Size (GiB): <code>40</code>, or some suitable size for your data or operating system<br/>
*Availability Zone: On the East and Arbutus clouds, the only option is <code>nova</code><br/>
 
Finally, click the blue "Create Volume" button.
 
==Mounting a Volume on a VM==
===Attaching a Volume===
[[File:Manage_attachments_EN.png|400px|thumb| Managing attachments command in Actions menu (Click for larger image)]]
* '''Attaching''' is the process of associating a volume with a VM. This is analogous to inserting a USB key or plugging an external drive into your personal computer.
* You can attach a volume from the Volumes page in the Dashboard.
* At the right-hand end of the line describing the volume is the Actions column; from the drop-down menu, select "Manage Attachments."
* In the "Attach To Instance" drop-down menu, select a VM.
* Click the blue "Attach Volume" button.
Attaching should complete in a few seconds. Then the Volumes page will show the newly created volume attached to your selected VM on <code>/dev/vdb</code> or some similar location.
===Formatting a newly created Volume===
* '''Formatting''' is the process of preparing a volume to store directories and files.
* Before a newly created and attached volume can be used, it must be formatted.
* Formatting erases all existing information on a volume and therefore should be done with care.
* See instructions for doing this on a [[Using a new empty volume on a Linux VM|Linux]] or [[Using a new empty volume on a Windows VM|Windows]] VM.
===Mounting a Volume===
* '''Mounting''' is the process of mapping the volume's filesystem logically within the VM's directory and file structure.
* To mount the volume, use a command similar to <code>[name@server ~]$ sudo mount /dev/vdb1 /mnt</code> depending on the device name, disk layout, and the desired mount point in your filesystem.
This command makes the volume's directory and file structure available under the VM's /mnt directory.
 
==Booting from a Volume==
If you want to run a persistent machine, it is safest to boot from a volume. When you boot a VM from an image rather than a volume, the VM is stored on the local disk of the actual machine running the VM. If something goes wrong with that machine or its disk the VM may be lost. Volume storage has redundancy which protects the VM from hardware failure. Typically when booting from a volume VM flavors starting with a 'p' are used (see [[Virtual machine flavors]]).
 
There are several ways to boot a VM from a volume. You can
* boot from an image, creating a new volume, or
* boot from a pre-existing volume, or
* boot from a volume snapshot, creating a new volume.
 
If you have not done this before, then the first one is your only option. The other two are only possible if you have already created a bootable volume or a volume snapshot.
 
If creating a volume as part of the process of launching the VM, select <code>Boot from image (creates a new volume)</code>, select the image to use, and the size of the volume. If this volume is something you would like to remain longer than the VM ensure that the "Delete on Terminate" box is unchecked. If you are unsure about this option, it is better to leave this box unchecked. You can manually delete the volume later.
 
==Creating an Image from a Volume==
[[File:Upload_volume_from_image_EN.png|400px|thumb| Upload to Image form (Click for larger image)]]<!--Note to translator: there is a FR version of this screen shot at [[File:Os-upload-volume-to-image-fr.png]]-->
Creating an image from a volume allows you to download the image. Do this if you want to save it as a backup, or to spin up a VM somewhere other than our cloud, e.g. with [https://www.virtualbox.org/ VirtualBox]. If you want to copy a volume to a new volume within the same cloud see [[#Cloning a Volume|cloning a volume]] instead. To create an image of a volume, it must first be detached from a VM. If it is a boot volume, it can only be detached from a VM if the VM is terminated/deleted.
 
===Using the Dashboard===
# Click on the ''Volumes'' left-hand menu.
# Under the volume you wish to create an image of click on the drop down ''Actions'' menu and select ''Upload to Image''.
# Choose a name for your new image.
# Choose a ''Disk Format''. QCOW2 is recommended for using within the OpenStack cloud as it is relatively compact compared to Raw image format and works well with OpenStack. If you wish to use the image with Virtualbox the vmdk or vdi image formats might be better suited.
# Finally click ''Upload''.
 
===Using the Command Line Clients===
The [[OpenStack Command Line Clients|command line client]] can do this:
{{Command|openstack image create --disk-format <format> --volume <volume_name> <image_name>}}
where
* <format> is the disk format (two possible values are [https://en.wikipedia.org/wiki/Qcow qcow2] and [https://en.wikipedia.org/wiki/VMDK vmdk]),
* <volume_name> can be found from the OpenStack dashboard by clicking on the volume name, and
* <image_name> is a name you choose for the image.
You can then [[OpenStack#Downloading an image |download the image]] as described below. It is best to detach the volume from the VM before you create an image from the volume. If the volume is a boot volume you will likely need to delete your VM to detach it, however, make sure you have not checked "Delete Volume on Instance Delete" when creating the VM.
 
==Cloning a Volume==
Cloning is the recommended method for copying volumes. While it is possible to make an image of an existing volume and use it to create a new volume, cloning is much faster and requires less movement of data behind the scenes. This method is handy if you have a persistent VM and you want to test out something before doing it on your production site. It is highly recommended to shut down your VM before creating a clone of the volume as the newly created volume may be left in an inconsistent state if there was writing to the source volume during the time the clone was created. To create a clone you must use the [[OpenStack Command Line Clients|command line client]] with this command
{{Command|openstack volume create --source <source-volume-id> --size <size-of-new-volume> <name-of-new-volume>}}
 
==Detaching a Volume==
Before detaching a volume, it is important to make sure that the operating system and other programs running on your VM are not accessing files on this volume. If so, the detached volume can be left in a corrupted state or the programs could show unexpected behaviours. To avoid this, you can either shut down the VM before you detach the volume or [[Using_a_new_empty_volume_on_a_Linux_VM#Unmounting_a_volume_or_device|unmount the volume]].
 
To detach a volume, log in to the OpenStack dashboard (see the [[Cloud#Using_the_Cloud|list of links]]) and select the project containing the volume you wish to detach. Selecting ''Volumes -> Volumes'' displays the project’s volumes. For each volume, the ''Attached to'' column indicates where the volume is attached.
 
*If attached to <code>/dev/vda</code>, it is a boot volume; you must delete the attached VM before the volume can be detached otherwise you will get the error message ''Unable to detach volume''.
 
*With volumes attached to <code>/dev/vdb</code>, <code>/dev/vdc</code>, etc. you do not need to delete the VM it is attached to before proceeding. In the ''Actions'' column drop-down list, select ''Manage Attachments'', click on the ''Detach Volume'' button and again on the next ''Detach Volume'' button to confirm.
 
=Working with images=
 
Images are files which contain the contents of a virtual disk. Often Images contain a base operating system used to create a volume or an ephemeral disk from which a virtual machine boots. An ephemeral disk is a virtual disk file which resides on the host (or hypervisor) where the virtual machine runs. Ephemeral disk files are destroyed when a VM is destroyed, in contrast to [[Working_with_volumes|volumes]]. Images are portable in that they can be downloaded from the cloud, used to create a virtual machine using virtual box or similar on your laptop, and uploaded to another cloud and used to create a new virtual machine. This is not the case with volumes or ephemeral disks. Images come in a variety of formats. Some commonly encountered formats are, raw, qcow2, vmdk, and vdi.
 
If sharing your virtual machine images, be sure to remove sensitive information such as public/private keys, configuration files containing passwords, etc. If uploading an image created from a virtual box virtual machine to our clouds, it must have cloud-init installed and configured correctly (see openstack docs on [https://docs.openstack.org/image-guide/create-images-manually.html creating images manually] for more details).
 
For a list of images provided by our staff, see [[Cloud resources#Images| images]].
 
==Creating an Image from a VM==
The procedure for creating an image of a VM depends on whether it is booting from a volume (typically "p" flavors), or from an ephemeral disk (typically "c" flavors).
 
===If booting from an ephemeral disk===
the [[OpenStack Command Line Clients]] can be used with the command:
{{Command| openstack server image create <server-name>}}
where <code><server-name></code> should be replaced with the name of your server. This action will only include the VM's root drive (e.g. /dev/vda) in the image. Ephemeral drives and non-boot attached volumes will not be included in the image, so additional measures should be taken to preserve this data. In addition, if the VM is writing to disk while the image is being created the filesystem may be captured in an inconsistent state. We recommend the VM be shut off (not deleted) before an image is created from it.
 
===If booting from a volume===
see [[Working_with_images#Creating_an_Image_from_a_VM|Creating an Image from a Volume]]
 
==Sharing an image with another project==
Sharing an image with another project is a two-step process.
 
# A member of the project owning the image must share it with a second project.
# A member of the second project must accept the newly shared image.
 
To share an image a member in the project owning the image uses the [[OpenStack Command Line Clients|OpenStack]] command below.
<source lang="console">
[name@server]$  glance member-create <IMAGE_ID> <MEMBER_ID>
+------------+-------------+---------+
| Image ID  | Member ID  | Status  |
+------------+-------------+---------+
| <IMAGE_ID> | <MEMBER_ID> | pending |
+------------+-------------+---------+
</source>
where
<code><IMAGE_ID></code> is the ID of the image to be shared, and <code><MEMBER_ID></code> is the ID of the project to share it with.
 
To accept the shared image a member in the second project uses the [[OpenStack_Command_Line_Clients#Separate_Command-line_interfaces|OpenStack]] command below.
<source lang="console">
[name@server]$  glance member-update <IMAGE_ID> <MEMBER_ID> <MEMBER_STATUS>
+------------+-------------+----------+
| Image ID  | Member ID  | Status  |
+------------+-------------+----------+
| <IMAGE_ID> | <MEMBER_ID> | accepted |
+------------+-------------+----------+
</source>
where <code><IMAGE_ID></code> is ID of the image to update, <code><MEMBER_ID></code> is the ID of the second project, and <code><MEMBER_STATUS></code> is the new status of the image. Valid Values for status are <code>accepted</code>, <code>rejected</code>, and <code>pending</code>. The image will then be available for use and appear in the OpenStack dashboard's list of images in the second project.
 
To check the status of image membership use the following command.
<source lang="console">
[name@server]$ glance member-list --image-id <IMAGE_ID>
+------------+-------------+----------+
| Image ID  | Member ID  | Status  |
+------------+-------------+----------+
| <IMAGE_ID> | <MEMBER_ID> | accepted |
+------------+-------------+----------+
</source>
 
==Downloading an Image==
The first step is to install the OpenStack client and download the OpenStack RC file and source it (see [[OpenStack Command Line Clients]]).
The OpenStack client can list the available images on your OpenStack project with
{{Command|openstack image list}}
producing something like:
 
  +--------------------------------------+---------------------------------------+-------------+------------------+-------------+--------+
| ID                                  | Name                                  | Disk Format | Container Format | Size        | Status |
+--------------------------------------+---------------------------------------+-------------+------------------+-------------+--------+
| 982761b2-c77b-4852-8ae3-bf98b32b8894 | Hadoop-2.2.4                          | qcow2      | bare            | 10253107200 | active |
| b7bd3033-9836-406d-a8f2-2e91978026b4 | hadoopmaster                          | qcow2      | bare            | 3493527552  | active |
| 2c751755-854d-49c3-af82-d501e51e7159 | hadoopmaster-active                  | qcow2      | bare            | 13134004224 | active |
| c41012f4-ed82-4478-a81f-5efb96a31b1a | hadoopmaster-old                      | qcow2      | bare            | 3493527552  | active |
| 78e61a3f-b546-441a-b476-a7077b04ca36 | hadoopslave                          | qcow2      | bare            | 3490971648  | active |
| 516845c3-b256-4c6d-a2cb-e31e822c7e34 | hadoopslave1-active                  | qcow2      | bare            | 8345026560  | active |
| 1546bd86-5314-4fce-9576-e2f6930dad30 | hadoopslave1-old                      | qcow2      | bare            | 3490971648  | active |
| baf78e8d-8288-4854-a66b-812cdf3ccbca | TestVM                                | qcow2      | bare            | 13167616    | active |
| 2faf97d7-5b0b-44ce-8024-3bef5a634570 | test_ubuntu_initial                  | qcow2      | bare            | 1799487488  | active |
| 308b6614-396a-4360-9c33-4e86f41ea0ec | trusty                                | qcow2      | bare            | 256180736  | active |
| 9b3c3fda-2aca-43b5-a3e7-662a94f5e7fb | Ubuntu_14.04_Trusty-amd64-20150708    | qcow2      | bare            | 257884672  | active |
| f93e66cf-fec1-4460-8fc7-506e716fbf30 | ucernvm-prod.1.18-10                  | raw        | bare            | 20971520    | active |
+--------------------------------------+---------------------------------------+-------------+------------------+-------------+--------+
 
You may need an additional option(s), such as <code>--long</code>, to see all the columns above depending on the version of the client you are using.
 
You can then download a particular image with
{{Command|openstack image save --file ./<file-name-for-image>.<format> <ID>}}
where <format> matches the value in the ''Disk format'' column and <ID> matches the value in the ''ID'' column.
 
==Uploading an Image==
The first step is to install the OpenStack client and download the OpenStack RC file and source it (see [[OpenStack Command Line Clients]]).
Then run the command
{{Command|openstack image create --file <path-to-local-file-image> --disk-format <format> <new-image-name>}}
 
where
* <path-to-local-file-image> is the path to the file containing the image you wish to upload from your local machine,
* <format> is the disk format; if not specified, the raw format is assumed, which is incorrect since it can cause issues when using the image in OpenStack,
* <new-image-name> is the name of the image as it appears on the OpenStack dashboard.
 
==Creating a VirtualBox VM from a Cloud Image==
[https://www.virtualbox.org/ VirtualBox] is a software package which allows you to create and run virtual machines on your desktop or laptop. It can be run on many different operating systems (Windows, Linux, Mac) and the virtual machines it creates may run one of many different operating systems.
 
To use a QCOW2 image downloaded from an OpenStack cloud, as shown above, with VirtualBox you will need to convert the image in the qcow2 format to the vmdk format. This can be done with the <code>qemu-img</code> tool. This can be installed with something like
{{Command|sudo apt-get install qemu-utils}}
(previously the package was called <code>qemu-img</code>) then do the conversion with
{{Command|qemu-img convert -f qcow2 vdisk.qcow2 -O vmdk vdisk.vmdk}}
You can then create a new virtual machine and attach the vmdk image to it (see [http://techathlon.com/how-to-run-a-vmdk-file-in-oracle-virtualbox/ how to run a vmdk file in virtualbox] for detailed instructions on this).
 
= Working with VMs=
== Locking VMs ==
When working on a project with multiple people or to protect a VM from accidental deletion or shutdown, it can be useful to lock it.
 
To '''lock''' a VM, click on the "Lock Instance" option from the Actions drop-down menu on the dashboard.<br/>
Once a vm is locked most of the Actions menu items will not be able to be executed until the instance is unlocked. There is an icon indicating the lock state for every instance.
 
To '''unlock''' a VM, select the "Unlock Instance" from the Actions drop-down menu on the dashboard.<br/>
 
==Resizing VMs==
It is possible to resize a VM by changing its flavor. However, there are some things to be aware of when choosing to resize a VM which depends on whether you have a "p" flavor or a "c" flavor VM (see [[Virtual machine flavors]]). Resizing a VM may involve some risk as it is similar to deleting and recreating your VM with a new flavor, if in doubt contact cloud [[technical support]].
 
===c flavors===
"c" flavors often have extra ephemeral drives, which will be resized when you choose a new "c" flavor. These ephemeral drives cannot become smaller, and as such "c" flavor VMs can only be resized to flavors with equal or larger ephemeral drives. After resizing however, you will not immediately see a larger ephemeral drive within your VM (e.g. the [https://en.wikipedia.org/wiki/Df_(Unix) <code>df -h</code>] command will not show the size increase). To see this extra space you will need to resize your filesystem (see the [https://linux.die.net/man/8/resize2fs <code>resize2fs</code>] command). However, filesystem resizes should be treated with caution and can take considerable time if the partitions are large. Before resizing a filesystem it is recommended to create backups of its contents (see [[backing up your VM]]).
 
===p flavors===
Unlike "c" flavors, "p" flavors do not typically have extra ephemeral drives associated with them, so they can be resized to larger and smaller flavors.
 
=Linux VM User Management=
There are a number of ways to allow more than one person to log into a VM. We recommend creating new user accounts and adding public [[SSH Keys]] to these accounts.
 
==Creating an account and keys==
A new user account can be created on Ubuntu with the command {{Command|sudo adduser --disabled-password USERNAME}} To be able to connect, the new user will need to have a key pair, see [[Generating_SSH_keys_in_Windows|generating SSH keys in windows]] or [[Using_SSH_keys_in_Linux#Creating a Key Pair|creating a key pair in Linux or Mac]] depending on the operating system they will be connecting from. Then their public key must be added to <code>/home/USERNAME/.ssh/authorized_keys</code> on the VM, ensuring permissions and ownership are correct as described in steps 2 and 3 of [[Using_SSH_keys_in_Linux#Connecting using a Key Pair|connecting using a key pair]].


==Granting admin privileges==
You can customize your instance after it has launched using CloudInit. This can be done via the OpenStack Command Line Interface or the Dashboard when launching an instance under the "Configuration" tab of the "Launch Instance" pop-up screen. You can paste your CloudInit script into the "Customization Script" field.  
In Ubuntu administrative, or root user privleges, can be given to a new user with the command
{{Command|sudo visudo -f /etc/sudoers.d/90-cloud-init-users}}
which opens an editor where a line like
USERNAME ALL=(ALL) NOPASSWD:ALL
can be added. For more detailed information about the visudo command and how to edit this file see this [https://www.digitalocean.com/community/tutorials/how-to-edit-the-sudoers-file-on-ubuntu-and-centos#what-is-visudo digitalocean] tutorial on the topic.


==Add users with CloudInit during VM creation==
==Add users with CloudInit during VM creation==
FuzzyBot
Bots
38,939

edits

Retrieved from "https://docs.alliancecan.ca/wiki/Special:MobileDiff/122530"

Navigation menu