Advanced MPI scheduling: Difference between revisions

In addition to the time limit needed for ''any'' Slurm job, an MPI job requires that you specify how many MPI processes Slurm should start. The simplest way to do this is with <code>--ntasks</code>. Since the default memory allocation of 256MB per core is often insufficient, you may also wish to specify how much memory is needed. Using <code>--ntasks</code> you cannot know in advance how many cores will reside on each node, so you should request memory with <code>--mem-per-cpu</code>. For example:

Not every application runs with maximum efficiency on a multiple of 32 (or 40, or 48) cores. Choosing the number of cores to request, and whether or not to request whole nodes, may be a trade-off between ''running'' time (or efficient use of the computer) and ''waiting'' time (or efficient use of your time). If you want help evaluating these factors, please contact [[Technical support]].

It is important to understand that the number of ''tasks'' requested of Slurm is the number of ''processes'' that will be started by <code>srun</code>. So for a hybrid job that will use both MPI processes and OpenMP threads or Posix threads, you should set the MPI process count with <code>--ntasks</code> or <code>-ntasks-per-node</code>, and set the thread count with <code>--cpus-per-task</code>.

This job is the same size as the last one: 16 tasks (that is, 16 MPI processes), each with 4 threads. The difference here is that we are sure of getting exactly 2 whole nodes. Recall that <code>--mem</code> requests memory ''per node'', so we use it instead of <code>--mem-per-cpu</code> for the reason described earlier.