Arbutus Migration Guide: Difference between revisions

<translate>

This document aims to describe how to migrate virtual machine (VM) instances from the legacy West Cloud to the new Arbutus Cloud.  

You know your workload best, so we recommend that you migrate your instances according to your own application requirements and schedule.

Note the following URLs for accessing the Horizon Web UI for the two Clouds:

'''West Cloud (legacy):''' [https://west.cloud.computecanada.ca https://west.cloud.computecanada.ca]

'''Arbutus Cloud (new):''' [https://arbutus.cloud.computecanada.ca https://arbutus.cloud.computecanada.ca]

Firefox and Chrome browsers are supported. Safari and Edge may work but have not been validated.

Your Project (Tenant), Network, and Router will be pre-created for you in Arbutus Cloud. User access will also be pre-populated.

Prior to migrating instances, we recommend that you complete the following preliminaries to prepare the necessary environment for migration.

# '''IMPORTANT''': Back up any critical data! While the Cloud has redundant storage systems, no backups of any instances are taken.

# Get RC files (used to set environment variables used by the OpenStack command-line tools) after logging in to the URLs above with your Compute Canada credentials:

# Plan an outage window. Generally, shutting down services and then shutting down the instance is the best way to avoid corrupt or inconsistent data after the migration. Smaller volumes can be copied over fairly quickly, e.g. a 10GB volume will copy over in less than 5 minutes, but larger volumes, e.g. 100GB can take 30 to 40 minutes. Plan for this. Additionally, floating IP addresses will change, so ensure the TTL of your DNS records is set to a small value so that the changes propagate as quickly as possible.  

There are three general migration scenarios to consider.  

* [[#Manual or orchestrated migration|Manual or orchestrated migration]]

Depending on your current setup, you may use any or all of these scenarios to migrate from the West Cloud to the Arbutus Cloud.

In this scenario, instances and volumes are created in Arbutus with the same specifications as that on West Cloud. The general approach is:

# Copy any Glance images from West Cloud to Arbutus Cloud if you are using any customized images. You may also simply start with a fresh base image in Arbutus Cloud.

# Install and configure services on the instance (or instances).

# Decommission the old instances and delete old volumes.

The above steps can be done manually or orchestrated via various configuration management tools such as

[https://docs.ansible.com/ansible/2.5/modules/list_of_cloud_modules.html Ansible],  

The use of such tools is beyond the scope of this document, but if you were already using orchestration tools on West Cloud, they should work with Arbutus Cloud as well.

Volume-backed instances, as their name implies, have a persistent volume attached to them containing the operating system and any required data. Best practice is to use separate volumes for the operating system and for data.

This method is recommended for volumes less than 150GB in size. For volumes larger than that, the approach described in [[#Manual or orchestrated migration|Manual or orchestrated migration]] above is preferred.

# Open two SSH sessions to the migration host ''cloudmigration.computecanada.ca'' with your Compute Canada credentials.

# In one session, source the OpenStack RC file for West Cloud. In the other session, source the OpenStack RC file for Arbutus Cloud. As mentioned earlier, use of the <code>screen</code> command is recommended in case of SSH disconnections.

# Once you have migrated and validated your instances and volumes, and once all associated DNS records updated, please delete your old instances and volumes on the legacy West Cloud.

# Launch an instance on West Cloud with the smallest flavor possible “p1-1.5gb”. We will call this the "temporary migration host". The instructions below assume you choose CentOS 7 for this instance, but any Linux distribution with Python and Pip available should work.

# Log in to the instance via SSH and install the OpenStack CLI in a root shell:

# Once you have migrated and validated your volumes and instances, and once any associated DNS records updated, please delete your old instances and volumes on the legacy West Cloud.

An ephemeral instance is an instance without a backing volume.

This method is recommended for instances with ephemeral storage less than 150GB in size. For instances larger than that, the approach described in [[#Manual or orchestrated migration|Manual or orchestrated migration]] above is preferred.

In either case you will still need to copy data from any non-boot ephemeral storage (i.e. mounted under <code>/mnt</code>) separately. Consult [[#Methods to copy data|methods to copy data]] below for this.

# Open two SS sessions to the migration host ''cloudmigration.computecanada.ca'' with your Compute Canada credentials.

# In one session, source the OpenStack RC file for West Cloud. In the other session, source the OpenStack RC file for Arbutus Cloud. As mentioned earlier, use of the <code>screen</code> command is recommended in case of SSH disconnections.

# Once you have migrated and validated your volumes and instances, and after any associated DNS records are updated, please delete your old instances on the legacy West Cloud.

# Login to the instance running on West Cloud via SSH. When migrating an ephemeral instance, it is important to shut down as many services as possible on the instance prior to migration e.g. httpd, databases, etc. Ideally, leave only SSH running.

# As root, install the OpenStack CLI if not already installed:

# Once you have migrated and validated your volumes and instances, and after any associated DNS records are updated, please delete your old instances on the legacy West Cloud.

Here are two recommended approaches for copying data between instances running in the two clouds. The most appropriate method depends upon the size of the data volumes in your tenant. For very large volumes (e.g. greater than 5TB) Globus is recommended. Please see here for configuration details: [https://computecanada.github.io/DHSI-cloud-course/globus/ https://computecanada.github.io/DHSI-cloud-course/globus/]

If you have very large volumes, we recommend you submit a support ticket as well.

For smaller volumes, rsync+ssh provides good transfer speeds and can (like Globus) work in an incremental way. A typical use case would be:

# SSH to the West Cloud instance which has the large volume attached. Note the absolute path you want to copy to the instance on Arbutus Cloud.

# Execute rsync over SSH. The example below assumes that password-less login via [[SSH keys]] has already been setup between the instances. Replace the placeholders below with real values:

# Verify that the data has been successfully copied on the instance in Arbutus Cloud. Then delete the data from the legacy West Cloud.

You may also use any other method you are familiar with for transferring data.

Support requests can be sent to the usual Cloud support address at [mailto:cloud@computecanada.ca cloud@computecanada.ca]

</translate>