Running jobs: Difference between revisions

Memory may be requested with <code>--mem-per-cpu</code> (memory per core) or <code>--mem</code> (memory per node).  On general-purpose (GP) clusters a default memory amount of 256 MB per core will be allocated unless you make some other request.  At [[Niagara]] only whole nodes are allocated along with all available memory, so a memory specification is not required there.

A common source of confusion comes from the fact that some memory on a node is not available to the job (reserved for the OS, etc.).  The effect of this is that each node-type has a maximum amount available to jobs - for instance, nominally "128G" nodes are typically configured to permit 125G of memory to user jobs.  If you request more memory than a node-type provides, your job will be constrained to run on higher-memory nodes, which may be fewer in number.

Adding to this confusion, Slurm interprets K, M, G, etc., as [https://en.wikipedia.org/wiki/Binary_prefix binary prefixes], so <code>--mem=125G</code> is equivalent to <code>--mem=128000M</code>.  See the "available memory" column in the "Node characteristics" table for each GP cluster for the Slurm specification of the maximum memory you can request on each node: [[Béluga/en#Node_characteristics|Béluga]], [[Cedar#Node_characteristics|Cedar]], [[Graham#Node_characteristics|Graham]], [[Narval/en#Node_characteristics|Narval]].

The general command for checking the status of Slurm jobs is <code>squeue</code>, but by default it supplies information about ''all'' jobs in the system, not just your own.  You can use the shorter <code>sq</code> to list only your own jobs:

'''Do not''' run <code>sq</code> or <code>squeue</code> from a script or program at high frequency, ''e.g.,'' every few seconds. Responding to <code>squeue</code> adds load to Slurm, and may interfere with its performance or correct operation.  See [[#Email_notification|Email notification]] below for a much better way to learn when your job starts or ends.

By default the output is placed in a file named "slurm-", suffixed with the job ID number and ".out", ''e.g.'' <code>slurm-123456.out</code>, in the directory from which the job was submitted.

Every job must have an associated ''account name'' corresponding to a [[Frequently_Asked_Questions_about_the_CCDB#What_is_a_RAP.3F|Resource Allocation Project]] (RAP). If you are a member of only one account, the scheduler will automatically associate your jobs with that account.

a given project is under the column '''Group Name'''. Note that a Resource  

Slurm will use the value of <code>SBATCH_ACCOUNT</code> in place of the <code>--account</code> directive in the job script. Note that even if you supply an account name inside the job script, ''the environment variable takes priority.'' In order to override the environment variable, you must supply an account name as a command-line argument to <code>sbatch</code>.