ARM software

Introduction

ARM DDT (formerly know as Allinea DDT) is a powerful commercial parallel debugger with a graphical user interface. It can be used to debug serial, MPI, multi-threaded, and CUDA programs, or any combination of the above, written in C, C++, and FORTRAN. MAP - an efficient parallel profiler - is another very useful tool from ARM (formerly Allinea).

This software is available on Graham as two separate modules:

• allinea-cpu, for CPU debugging and profiling;
• allinea-gpu, for GPU or mixed CPU/GPU debugging.

As this is a GUI application, you should log in using ssh -Y, and use an SSH client like MobaXTerm (Windows) or XQuartz (Mac) to ensure proper X11 tunnelling.

Both DDT and MAP are normally used interactively through their GUI, which is normally accomplished using the salloc command (see below for details). MAP can also be used non-interactively, in which case it can be submitted to the scheduler with the sbatch command.

The current license limits the use of DDT/MAP to a maximum of 512 CPU cores across all users at any given time. DDT-GPU is likewise limited to 8 GPUs.

Usage

CPU-only code, no GPUs

Allocate the node or nodes on which to do the debugging or profiling with salloc, e.g.:

salloc --x11 --time=0-1:00 --mem-per-cpu=4G --ntasks=4

This will open a shell session on the allocated node. Then load the appropriate module:

module load allinea-cpu

This may fail with a suggestion to load an older version of OpenMPI first. If that happens, reload the OpenMPI module with the suggested command, and then reload the allinea-cpu module:

module load openmpi/2.0.2
module load allinea-cpu

You can then run the ddt or map command as:

ddt path/to/code
map path/to/code

Make sure the MPI implementation is the default "OpenMPI" in the Allinea application window, before pressing the Run button. If this is not the case, press the Change button next to the "Implementation:" string, and pick the correct option from the drop down menu.

When done, exit the shell. This will terminate the allocation.

CUDA code

Allocate the node or nodes on which to do the debugging or profiling with salloc, e.g.:

salloc --x11 --time=0-1:00 --mem-per-cpu=4G --ntasks=1 --gres=gpu:1

This will open a shell session on the allocated node. Then load the appropriate module:

module load allinea-gpu

This may fail with a suggestion to load an older version of OpenMPI first. If that happens, reload the OpenMPI module with the suggested command, and then reload the allinea-gpu module:

module load openmpi/2.0.2
module load allinea-gpu

Ensure a cuda module is loaded:

module load cuda

You can then run the ddt command as:

ddt path/to/code

When done, exit the shell. This will terminate the allocation.

Known issues

MPI DDT

• For some reason the debugger doesn't show queued MPI messages (e.g. when paused in an MPI deadlock).

OpenMP DDT

• Memory debugging module (which is off by default) doesn't work.

CUDA DDT

• Memory debugging module (which is off by default) doesn't work.

MAP

• MAP currently doesn't work correctly on Graham. We are working on resolving this issue. For now the workaround is to request a SHARCNET account from your Compute Canada account (ccdb), and then run MAP on the SHARCNET's legacy cluster orca's development nodes using these instructions.

Message Passing Interface