Using cloud vGPUs

From Alliance Doc
Revision as of 14:23, 21 May 2024 by Bott (talk | contribs)
Jump to navigation Jump to search
Other languages:

This page describes how to

  • allocate virtual GPU (vGPU) resources to a virtual machine (VM),
  • install the necessary drivers and
  • check whether the vGPU can be used.

Access to repositories as well as to the vGPUs is currently only available within Arbutus Cloud. Please note that the documentation below only covers the vGPU driver installation. The CUDA toolkit is not pre-installed but you can install it directly from NVIDIA or load it from the CVMFS software stack. If you choose to install the toolkit directly from NVIDIA, please ensure that the vGPU driver is not overwritten with the one from the CUDA package.

Supported flavors

To use a vGPU within a VM, the instance needs to be deployed on one of the flavors listed below. The vGPU will be available to the operating system via the PCI bus.

  • g1-8gb-c4-22gb
  • g1-16gb-c8-40gb


Preparation of a VM running Almalinux8

Once the VM is available, make sure to update the OS to the latest available software, including the kernel. Then, reboot the VM to have the latest kernel running. To have access to the DKMS package, the EPEL repository is required. 

[root@vgpu almalinux]# dnf -y update && dnf -y install epel-release && reboot

After the reboot of the VM, the Arbutus Cloud repository needs to be installed. 

[root@almalinux8]# dnf install http://repo.arbutus.cloud.computecanada.ca/pulp/repos/alma8/Packages/a/arbutus-cloud-vgpu-repo-1.0-1.el8.noarch.rpm

The next step is to install the vGPU packages, which will install the required driver and user-space tools. 

[root@vgpu almalinux]# dnf -y install nvidia-vgpu-gridd.x86_64 nvidia-vgpu-tools.x86_64 nvidia-vgpu-kmod.x86_64

After a successful installation, nvidia-smi can be used to verify the proper functionality. 

[root@almalinux8]# nvidia-smi 
Tue Apr 23 16:37:31 2024       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.54.15              Driver Version: 550.54.15      CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  GRID V100D-8C                  On  |   00000000:00:06.0 Off |                    0 |
| N/A   N/A    P0             N/A /  N/A  |       0MiB /   8192MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
                                                                                         
+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

Preparation of a VM running Almalinux8

Once the VM is available, make sure to update the OS to the latest available software, including the kernel. Then, reboot the VM to have the latest kernel running. To have access to the DKMS package, the EPEL repository is required. 

[root@vgpu almalinux]# dnf -y update && dnf -y install epel-release && reboot

After the reboot of the VM, the Arbutus Cloud repository needs to be installed. 

[root@almalinux8]# dnf install http://repo.arbutus.cloud.computecanada.ca/pulp/repos/centos/8/x86_64/Packages/a/arbutus-cloud-vgpu-repo-1.2-1.el8.noarch.rpm

The next step is to install the vGPU packages, which will install the required driver and user-space tools. 

[root@vgpu almalinux]# dnf -y install nvidia-vgpu-gridd.x86_64 nvidia-vgpu-tools.x86_64 nvidia-vgpu-kmod.x86_64

After a successful installation, nvidia-smi can be used to verify the proper functionality. 

[root@almalinux8]# nvidia-smi 
Wed Apr  5 15:07:23 2023       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.182.03   Driver Version: 470.182.03   CUDA Version: 11.4     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GRID V100D-8C       On   | 00000000:00:06.0 Off |                    0 |
| N/A   N/A    P0    N/A /  N/A |    560MiB /  8192MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

Preparation of a VM running Debian10

Ensure that the latest packages are installed and the system has been booted with the latest stable kernel, as DKMS will request the latest one available from the Debian repositories. 

root@debian10:~# apt-get update && apt-get -y dist-upgrade && reboot

After a successful reboot, the system should have the latest available kernel running and the repository can be installed, by installing the arbutus-cloud-repo package. This package also contains the gpg key all packages are signed with. 

root@debian10:~# apt-get -y install gnupg
root@debian10:~# wget http://repo.arbutus.cloud.computecanada.ca/pulp/deb/debian/pool/main/arbutus-cloud-repo_0.3_all.deb
root@debian10:~# dpkg -i arbutus-cloud-repo_0.3_all.deb

The installation of the package will display a warning, since the key is directly imported (for convenience) via the package's post-installation procedure. 

Setting up arbutus-cloud-repo (0.1) ...
Warning: apt-key should not be used in scripts (called from postinst maintainerscript of the package arbutus-cloud-repo)
OK

Update the local apt cache and install the vGPU packages: 

root@debian10:~# apt-get update && apt-get -y install nvidia-vgpu-kmod nvidia-vgpu-tools nvidia-vgpu-gridd

If your installation was successful, the vGPU will be accessible and licensed. Test this by running nvidia-smi as shown above for CentOS7.


Preparation of a VM running Debian11

Ensure that the latest packages are installed and the system has been booted with the latest stable kernel, as DKMS will request the latest one available from the Debian repositories. 

root@debian11:~# apt-get update && apt-get -y dist-upgrade && reboot

After a successful reboot, the system should have the latest available kernel running and the repository can be installed, by installing the arbutus-cloud-repo package. This package also contains the gpg key all packages are signed with. 

root@debian11:~# wget http://repo.arbutus.cloud.computecanada.ca/pulp/deb/debian11/pool/main/arbutus-cloud-repo_0.2_all.deb
root@debian11:~# apt-get install -y ./arbutus-cloud-repo_0.2_all.deb

Update the local apt cache and install the vGPU packages: 

root@debian11:~# apt-get update && apt-get -y install nvidia-vgpu-kmod nvidia-vgpu-tools nvidia-vgpu-gridd


Preparation of a VM running Ubuntu22

Ensure that the OS is up to date, that all the latest patches are installed, and that the latest stable kernel is running. 

root@ubuntu22:~# apt-get update && apt-get -y dist-upgrade && reboot


After a successful reboot, the system should have the latest available kernel running. Now the repository can be installed by installing the arbutus-cloud-repo package. This package also contains the gpg key all packages are signed with. 

root@ubuntu22:~# wget http://repo.arbutus.cloud.computecanada.ca/pulp/deb/ubuntu22/pool/main/arbutus-cloud-repo_0.2_all.deb
root@ubuntu22:~# dpkg -i arbutus-cloud-repo_0.2_all.deb

A warning will be displayed since the signature key is added in the post-install stage; this warning can be ignored. Update the local apt cache and install the vGPU packages: 

root@ubuntu22:~# apt-get update && apt-get -y install nvidia-vgpu-kmod nvidia-vgpu-tools nvidia-vgpu-gridd

If your installation was successful, the vGPU will be accessible and licensed. Test this by running nvidia-smi as shown above for CentOS7.


Preparation of a VM running Ubuntu20

Ensure that the OS is up to date, that all the latest patches are installed, and that the latest stable kernel is running. 

root@ubuntu20:~# apt-get update && apt-get -y dist-upgrade && reboot

After a successful reboot, the system should have the latest available kernel running. Now the repository can be installed by installing the arbutus-cloud-repo package. This package also contains the gpg key all packages are signed with. 

root@ubuntu20:~# wget http://repo.arbutus.cloud.computecanada.ca/pulp/deb/ubuntu/pool/main/arbutus-cloud-repo_0.2ubuntu20_all.deb
root@ubuntu20:~# dpkg -i arbutus-cloud-repo_0.2ubuntu20_all.deb

A warning will be displayed since the signature key is added in the post-install stage; this warning can be ignored. Update the local apt cache and install the vGPU packages: 

root@ubuntu20:~# apt-get update && apt-get -y install nvidia-vgpu-kmod nvidia-vgpu-tools nvidia-vgpu-gridd

If your installation was successful, the vGPU will be accessible and licensed. Test this by running nvidia-smi as shown above for CentOS7.

Retrieved from "https://docs.alliancecan.ca/mediawiki/index.php?title=Using_cloud_vGPUs&oldid=154096"