Hyper-Q / MPS: Difference between revisions

One situation when the MPS feature can be very useful is when you need to run multiple instances of your CUDA code, when your code is too small to saturate a modern GPU. What you can do is to run multiple instances of your code sharing a single GPU. (This will work as long as there is enough of GPU memory for all of your code instances.) In many cases this should result in a significantly increased collective throughput from all of your GPU processes.

In the above example, we are sharing a single V100 gpu between 8 instances of "my_code" (which takes a single argument - the loop index $i). We request 8 CPU cores (#SBATCH -c 8) for the farm, so there is one CPU core per code instance. The two important elements are "&" on the code execution line (this sends the code processes to the background), and the "wait" command at the end of the script (which ensures that the job runs until all background processes finished running.)