Bureaucrats, cc_docs_admin, cc_staff
2,879
edits
Hyper-Q / MPS: Difference between revisions
Marked this version for translation
(mark for translation) |
(Marked this version for translation) |
||
| Line 3: | Line 3: | ||
<translate> | <translate> | ||
==Overview== | ==Overview== <!--T:1--> | ||
<!--T:2--> | |||
Hyper-Q (or MPS) is a feature of NVIDIA GPUs. | Hyper-Q (or MPS) is a feature of NVIDIA GPUs. | ||
It is available in GPUs with CUDA compute capability 3.5 and higher,<ref>For a table relating NVIDIA GPU model names, architecture names, and CUDA compute capabilties, see [https://en.wikipedia.org/wiki/Nvidia_Tesla https://en.wikipedia.org/wiki/Nvidia_Tesla]</ref> | It is available in GPUs with CUDA compute capability 3.5 and higher,<ref>For a table relating NVIDIA GPU model names, architecture names, and CUDA compute capabilties, see [https://en.wikipedia.org/wiki/Nvidia_Tesla https://en.wikipedia.org/wiki/Nvidia_Tesla]</ref> | ||
which is all GPUs currently deployed on Alliance general-purpose clusters (Béluga, Cedar, Graham, and Narval). | which is all GPUs currently deployed on Alliance general-purpose clusters (Béluga, Cedar, Graham, and Narval). | ||
<!--T:3--> | |||
[https://docs.nvidia.com/deploy/mps/index.html According to NVIDIA], | [https://docs.nvidia.com/deploy/mps/index.html According to NVIDIA], | ||
<blockquote> | <blockquote> | ||
| Line 14: | Line 16: | ||
</blockquote> | </blockquote> | ||
<!--T:4--> | |||
In our tests, MPS may increase the total GPU flop rate even when the GPU is being shared by unrelated CPU processes. That means that MPS is great for CUDA applications with relatively small problem sizes, which on their own cannot efficiently saturate modern GPUs with thousands of cores. | In our tests, MPS may increase the total GPU flop rate even when the GPU is being shared by unrelated CPU processes. That means that MPS is great for CUDA applications with relatively small problem sizes, which on their own cannot efficiently saturate modern GPUs with thousands of cores. | ||
<!--T:5--> | |||
MPS is not enabled by default, but it is straightforward to do. Execute the following commands before running your CUDA application: | MPS is not enabled by default, but it is straightforward to do. Execute the following commands before running your CUDA application: | ||
export CUDA_MPS_PIPE_DIRECTORY=/tmp/nvidia-mps | <!--T:6--> | ||
export CUDA_MPS_PIPE_DIRECTORY=/tmp/nvidia-mps | |||
export CUDA_MPS_LOG_DIRECTORY=/tmp/nvidia-log | export CUDA_MPS_LOG_DIRECTORY=/tmp/nvidia-log | ||
nvidia-cuda-mps-control -d | nvidia-cuda-mps-control -d | ||
<!--T:7--> | |||
Then you can use the MPS feature if you have more than one CPU thread accessing the GPU. This will happen if you run a hybrid MPI/CUDA application, a hybrid OpenMP/CUDA application, or multiple instances of a serial CUDA application ("GPU farming"). | Then you can use the MPS feature if you have more than one CPU thread accessing the GPU. This will happen if you run a hybrid MPI/CUDA application, a hybrid OpenMP/CUDA application, or multiple instances of a serial CUDA application ("GPU farming"). | ||
<!--T:8--> | |||
Additional details on MPS can be found here: [https://docs.nvidia.com/deploy/mps/index.html CUDA Multi Process Service (MPS) - NVIDIA Documentation]. | Additional details on MPS can be found here: [https://docs.nvidia.com/deploy/mps/index.html CUDA Multi Process Service (MPS) - NVIDIA Documentation]. | ||
==GPU farming== | ==GPU farming== <!--T:9--> | ||
<!--T:10--> | |||
One situation when the MPS feature can be very useful is when you need to run multiple instances of a CUDA application, but the application is too small to saturate a modern GPU. MPS allows you to run multiple instances of the application sharing a single GPU, as long as there is enough of GPU memory for all of the instances of the application. In many cases this should result in a significantly increased throughput from all of your GPU processes. | One situation when the MPS feature can be very useful is when you need to run multiple instances of a CUDA application, but the application is too small to saturate a modern GPU. MPS allows you to run multiple instances of the application sharing a single GPU, as long as there is enough of GPU memory for all of the instances of the application. In many cases this should result in a significantly increased throughput from all of your GPU processes. | ||
<!--T:11--> | |||
Here is an example of a job script to set up GPU farming: | Here is an example of a job script to set up GPU farming: | ||
#!/bin/bash | <!--T:12--> | ||
#!/bin/bash | |||
#SBATCH --gpus-per-node=v100:1 | #SBATCH --gpus-per-node=v100:1 | ||
#SBATCH -t 0-10:00 | #SBATCH -t 0-10:00 | ||
| Line 50: | Line 60: | ||
wait | wait | ||
<!--T:13--> | |||
In the above example, we share a single V100 GPU between 8 instances of "my_code" (which takes a single argument - the loop index $i). We request 8 CPU cores (#SBATCH -c 8) so there is one CPU core per application instance. The two important elements are "&" on the code execution line, which sends the code processes to the background, and the "wait" command at the end of the script, which ensures that the job runs until all background processes end. | In the above example, we share a single V100 GPU between 8 instances of "my_code" (which takes a single argument - the loop index $i). We request 8 CPU cores (#SBATCH -c 8) so there is one CPU core per application instance. The two important elements are "&" on the code execution line, which sends the code processes to the background, and the "wait" command at the end of the script, which ensures that the job runs until all background processes end. | ||
<!--T:14--> | |||
[[Category:Software]] | [[Category:Software]] | ||
</translate> | </translate> | ||