Job arrays: Difference between revisions

Job arrays (view source)

Revision as of 19:23, 14 May 2018

975 bytes added , 6 years ago

import text from ACENET wiki

Rdickson

Bureaucrats, cc_docs_admin, cc_staff

2,879

edits

@@ Line 1: / Line 1: @@
 {{Draft}}
-Also known as a ''task array'' or an ''array job'', a job array is a way to submit a whole set of jobs with one command. The individual jobs in the array are distinguished by an environment variable, <code>$SLURM_ARRAY_TASK_ID</code>, which is set to a different value for each instance of the job.
+A large number of tasks which differ only in some parameter can be conveniently submitted as a ''job array,'' also known as a ''task array'' or an ''array job''. The individual tasks in the array are distinguished by an environment variable, <code>$SLURM_ARRAY_TASK_ID</code>, which Slurm sets to different values according to the range you supply with the --array parameter.
   sbatch --array=0-7 ...      # $SLURM_ARRAY_TASK_ID will take values from 0 to 7 inclusive
   sbatch --array=1,3,5,7 ...  # $SLURM_ARRAY_TASK_ID will take the listed values
@@ Line 9: / Line 10: @@
 See [https://slurm.schedmd.com/job_array.html Job Array Support] at SchedMD.com for detailed documentation.
-== Examples ==
+== A simple example ==
+ $ sbatch --array=1-10 runme
+ Your job-array 54321.1-10:1 ("runme") has been submitted
+Job 54321 will be scheduled as 10 independent tasks which may start at different times on different hosts. The individual tasks are differentiated by the value of an environment variable $SLURM_ARRAY_TASK_ID. The script can reference $SLURM_ARRAY_TASK_ID to select an input file, for example, or to set a command-line argument for the application code:
+ my_app <input.$SLURM_ARRAY_TASK_ID
+ my_app $SLURM_ARRAY_TASK_ID some_arg another_arg
+Using a job array instead of a large number of separate serial jobs has advantages for other users and the system as a whole. A waiting job array only produces one line of output in squeue, making it easier for you to read its output. The scheduler does not have to analyze job requirements for each array task separately, so it can run more efficiently too.