Backing up your VM: Difference between revisions
(First cut at describing backup of a Compute VM - more work needed.) |
No edit summary |
||
| Line 2: | Line 2: | ||
''Parent page: [[Cloud]]'' | ''Parent page: [[Cloud]]'' | ||
There are a few different strategies to backup your virtual machine and to recover from disasters, which you choose will depend on your requirements and your particular use case. A few common methods to backup or preserve the state of your virtual machine are mentioned followed by a suggested strategy of how a few of these methods might be used together to create a complete backup strategy. | |||
==Persistent VMs== | ==File backup== | ||
Many of the backup strategies for physical machines still apply to virtual machines, for example tools such as [https://rsync.samba.org/ rsync] or [https://nongnu.org/duplicity/ duplicity] can be used to perform backups of the data within your VM to remote locations. | |||
==Setup automation== | |||
Provisioning tools such as [https://www.ansible.com/ ansible], [https://puppet.com/ puppet], [https://www.chef.io/ chef], and [https://saltstack.com/ saltstack] can be used to automate setup and configuration of software and operating systems. Recreating your virtual machine then becomes a trivial matter given specification files. These specification files can be managed with version control tools like [https://git-scm.com/ git]. Provisioning tools can be combined with orchestration tools such as heat and terraform (see [Automating VM creation]) to automate the entire process of virtual machine creation and software configuration. Any data which is generated or created outside this automation would then need to be backed up likely using a [#File backup] method. | |||
==OpenStack specific== | |||
OpenStack provides tools to create disk images and snapshots of your virtual machines. The two main VM flavors persistent (p) and compute (c) have different behaviors and creating disk images or snapshots should be considered differently for each. | |||
===Persistent VMs=== | |||
Persistent VMs are designed to boot from volumes (see [[OpenStack#Booting_from_a_Volume| booting from a volume]]) thus creating a copy of the volume(s) the VM has attached to it will produce a backup. However this would not preserver things like VM flavor, public IP, and security rules. The best way to create a copy of a volume for backup purposes is to create an image from that volume. An image can be [[OpenStack#Downloading_an_Image|downloaded]] and reused to create multiple new VMs, can be [[OpenStack#Creating_a_VirtualBox_VM_from_a_Cloud_Image|accessed by VirtualBox]] on your Desktop or laptop, and [[OpenStack#Uploading_an_Image|uploaded]] to other clouds. | Persistent VMs are designed to boot from volumes (see [[OpenStack#Booting_from_a_Volume| booting from a volume]]) thus creating a copy of the volume(s) the VM has attached to it will produce a backup. However this would not preserver things like VM flavor, public IP, and security rules. The best way to create a copy of a volume for backup purposes is to create an image from that volume. An image can be [[OpenStack#Downloading_an_Image|downloaded]] and reused to create multiple new VMs, can be [[OpenStack#Creating_a_VirtualBox_VM_from_a_Cloud_Image|accessed by VirtualBox]] on your Desktop or laptop, and [[OpenStack#Uploading_an_Image|uploaded]] to other clouds. | ||
| Line 13: | Line 22: | ||
Once you have created images of all the volumes you wish to backup you can then re-create your VM booting from the original VM's root volume and attaching any additional volumes you may have had attached to the original VM. | Once you have created images of all the volumes you wish to backup you can then re-create your VM booting from the original VM's root volume and attaching any additional volumes you may have had attached to the original VM. | ||
===What about volume snapshots?=== | ====What about volume snapshots?==== | ||
Another alternative might be to create a snapshot of a volume, which will save the state of a volume at the time the snapshot was created, however volume snapshots depend on the original volume remaining intact and as such are not ideal for backups. It is also not possible to download volume snapshots as they depend on the original volume. However, they do allow you to create a new volume from a snapshot of a volume at a given point in time. For example, if there were file changes since the last volume snapshot you would like to revert. Or if there were file changes specific to that VM which should not be included in other VMs. | Another alternative might be to create a snapshot of a volume, which will save the state of a volume at the time the snapshot was created, however volume snapshots depend on the original volume remaining intact and as such are not ideal for backups. It is also not possible to download volume snapshots as they depend on the original volume. However, they do allow you to create a new volume from a snapshot of a volume at a given point in time. For example, if there were file changes since the last volume snapshot you would like to revert. Or if there were file changes specific to that VM which should not be included in other VMs. | ||
===What about VM snapshots?=== | ====What about VM snapshots?==== | ||
Unfortunately, OpenStack uses the word "snapshot" to mean two different things. There are volume snapshosts, as discussed above, and snapshots of VMs. Snapshots of VMs behave in different ways depending on the flavor of your VM. If you have a persistent VM and create a snapshot of the VM, OpenStack creates a nearly empty image, which contains pointers to volume snapshots. These pointers point to volume snapshot(s) of the persistent VM's boot volume and any attached volumes which were created as part of creating a snapshot of the VM. You can then create a new VM (boot from image (creates a new volume)) which will create new volumes from the snapshots of the volumes taken previously, boot a new VM from the root volume and attach any other duplicated volumes. | Unfortunately, OpenStack uses the word "snapshot" to mean two different things. There are volume snapshosts, as discussed above, and snapshots of VMs. Snapshots of VMs behave in different ways depending on the flavor of your VM. If you have a persistent VM and create a snapshot of the VM, OpenStack creates a nearly empty image, which contains pointers to volume snapshots. These pointers point to volume snapshot(s) of the persistent VM's boot volume and any attached volumes which were created as part of creating a snapshot of the VM. You can then create a new VM (boot from image (creates a new volume)) which will create new volumes from the snapshots of the volumes taken previously, boot a new VM from the root volume and attach any other duplicated volumes. | ||
==Compute VMs== | ===Compute VMs=== | ||
As with creating backups of persistent VMs, the main goal is to create an image of at least the root drive, and perhaps also other attached volumes if needed. However, differences with compute flavor VMs change the process of creating that image. Compute VMs are not designed to boot from volumes accessed over the network as persistent VMs are, instead they are meant to be booted from disk images which reside locally on the computer which is actually running your VM. This means there is no volume which you can click on in the OpenStack dashboard to create an image of your root disk. Instead you can do this by clicking "Create Snapshot" on your VM's drop down menu on the "Instances" tab. As with creating a snapshot with a persistent VM this creates an image. However, in this case the image is not nearly empty containing only pointers to volume snapshots. Instead the image contains a copy of the VMs root drive. | As with creating backups of persistent VMs, the main goal is to create an image of at least the root drive, and perhaps also other attached volumes if needed. However, differences with compute flavor VMs change the process of creating that image. Compute VMs are not designed to boot from volumes accessed over the network as persistent VMs are, instead they are meant to be booted from disk images which reside locally on the computer which is actually running your VM. This means there is no volume which you can click on in the OpenStack dashboard to create an image of your root disk. Instead you can do this by clicking "Create Snapshot" on your VM's drop down menu on the "Instances" tab. As with creating a snapshot with a persistent VM this creates an image. However, in this case the image is not nearly empty containing only pointers to volume snapshots. Instead the image contains a copy of the VMs root drive. | ||
Compute VMs also come with an extra data drive mounted at <code>/mnt</code>, the data on this drive is not captured in the image created of a compute VM. Other arrangements must be made to save this data, such as coping it off the disk before the VM is terminated. | Compute VMs also come with an extra data drive mounted at <code>/mnt</code>, the data on this drive is not captured in the image created of a compute VM. Other arrangements must be made to save this data, such as coping it off the disk before the VM is terminated. | ||
==A basic backup strategy== | |||
Very large disk images (larger than 10-20 GB) can become difficult to manage with relatively long upload times and long VM creation times. | |||
A good basic strategy might be to separate large data sets from your operating system and software stack. The operating system and software stack can be backup either using a disk image or recreated using some provisioning software (see [#Setup automation]). The data sets can then be backup with using normal [#File backup] methods to a remote location. | |||
==See also== | |||
* Volumes versus ephemeral drives? | * Volumes versus ephemeral drives? | ||
* [[OpenStack Command Line Clients]] | * [[OpenStack Command Line Clients]] | ||
Revision as of 19:16, 4 July 2018
This is not a complete article: This is a draft, a work in progress that is intended to be published into an article, which may or may not be ready for inclusion in the main wiki. It should not necessarily be considered factual or authoritative.
Parent page: Cloud
There are a few different strategies to backup your virtual machine and to recover from disasters, which you choose will depend on your requirements and your particular use case. A few common methods to backup or preserve the state of your virtual machine are mentioned followed by a suggested strategy of how a few of these methods might be used together to create a complete backup strategy.
File backup
Many of the backup strategies for physical machines still apply to virtual machines, for example tools such as rsync or duplicity can be used to perform backups of the data within your VM to remote locations.
Setup automation
Provisioning tools such as ansible, puppet, chef, and saltstack can be used to automate setup and configuration of software and operating systems. Recreating your virtual machine then becomes a trivial matter given specification files. These specification files can be managed with version control tools like git. Provisioning tools can be combined with orchestration tools such as heat and terraform (see [Automating VM creation]) to automate the entire process of virtual machine creation and software configuration. Any data which is generated or created outside this automation would then need to be backed up likely using a [#File backup] method.
OpenStack specific
OpenStack provides tools to create disk images and snapshots of your virtual machines. The two main VM flavors persistent (p) and compute (c) have different behaviors and creating disk images or snapshots should be considered differently for each.
Persistent VMs
Persistent VMs are designed to boot from volumes (see booting from a volume) thus creating a copy of the volume(s) the VM has attached to it will produce a backup. However this would not preserver things like VM flavor, public IP, and security rules. The best way to create a copy of a volume for backup purposes is to create an image from that volume. An image can be downloaded and reused to create multiple new VMs, can be accessed by VirtualBox on your Desktop or laptop, and uploaded to other clouds.
To be able to create an image from a volume, that volume must be detached from the VM. In addition, if the volume is the root volume of the VM it can not be detached unless the VM is deleted. If you are sure that when you created the VM you did not check the box "Delete Volume on Instance Delete" then you can delete your VM knowing you will not loose any data. However, if you are unsure if you checked this box OpenStack, unfortunately, doesn't tell you if this box was checked when you created a VM. One trick which maybe useful for getting around this, is to create a snapshot of the volume provided you have a storage quota which allows it (Snapshots count towards your storage quota). Since volumes can not be deleted if there is a volume snapshot created from them, when you delete the VM the volume will not be deleted even if you checked the box.
At this point all the volumes you wish to create images of should be in the "Available" state. To create an image from a volume, select 'Upload to Image' from the drop down menu for the volume. Select the 'QCOW2' disk format and give your image a name. There are several formats for disk images but QCOW2 works well with OpenStack and typically does not take up as much space as "Raw" images. Other formats "vmdk" and "vdi" can be useful when working with other virtualization tools such as VirtualBox.
Once you have created images of all the volumes you wish to backup you can then re-create your VM booting from the original VM's root volume and attaching any additional volumes you may have had attached to the original VM.
What about volume snapshots?
Another alternative might be to create a snapshot of a volume, which will save the state of a volume at the time the snapshot was created, however volume snapshots depend on the original volume remaining intact and as such are not ideal for backups. It is also not possible to download volume snapshots as they depend on the original volume. However, they do allow you to create a new volume from a snapshot of a volume at a given point in time. For example, if there were file changes since the last volume snapshot you would like to revert. Or if there were file changes specific to that VM which should not be included in other VMs.
What about VM snapshots?
Unfortunately, OpenStack uses the word "snapshot" to mean two different things. There are volume snapshosts, as discussed above, and snapshots of VMs. Snapshots of VMs behave in different ways depending on the flavor of your VM. If you have a persistent VM and create a snapshot of the VM, OpenStack creates a nearly empty image, which contains pointers to volume snapshots. These pointers point to volume snapshot(s) of the persistent VM's boot volume and any attached volumes which were created as part of creating a snapshot of the VM. You can then create a new VM (boot from image (creates a new volume)) which will create new volumes from the snapshots of the volumes taken previously, boot a new VM from the root volume and attach any other duplicated volumes.
Compute VMs
As with creating backups of persistent VMs, the main goal is to create an image of at least the root drive, and perhaps also other attached volumes if needed. However, differences with compute flavor VMs change the process of creating that image. Compute VMs are not designed to boot from volumes accessed over the network as persistent VMs are, instead they are meant to be booted from disk images which reside locally on the computer which is actually running your VM. This means there is no volume which you can click on in the OpenStack dashboard to create an image of your root disk. Instead you can do this by clicking "Create Snapshot" on your VM's drop down menu on the "Instances" tab. As with creating a snapshot with a persistent VM this creates an image. However, in this case the image is not nearly empty containing only pointers to volume snapshots. Instead the image contains a copy of the VMs root drive.
Compute VMs also come with an extra data drive mounted at /mnt, the data on this drive is not captured in the image created of a compute VM. Other arrangements must be made to save this data, such as coping it off the disk before the VM is terminated.
A basic backup strategy
Very large disk images (larger than 10-20 GB) can become difficult to manage with relatively long upload times and long VM creation times.
A good basic strategy might be to separate large data sets from your operating system and software stack. The operating system and software stack can be backup either using a disk image or recreated using some provisioning software (see [#Setup automation]). The data sets can then be backup with using normal [#File backup] methods to a remote location.
See also
- Volumes versus ephemeral drives?
- OpenStack Command Line Clients
- OpenStack#Creating_an_Image_from_a_VM
- OpenStack#Downloading_an_Image
- OpenStack#Uploading_an_Image