Job arrays: Difference between revisions

Job arrays (view source)

Revision as of 13:21, 15 May 2018

312 bytes removed , 6 years ago

→‎A simple example

Rdickson

Bureaucrats, cc_docs_admin, cc_staff

2,879

edits

@@ Line 13: / Line 13: @@
   $ sbatch --array=1-10 runme
-  Your job-array 54321.1-10:1 ("runme") has been submitted
+  Submitted batch job 54321
 Job 54321 will be scheduled as 10 independent tasks which may start at different times on different hosts. The individual tasks are differentiated by the value of an environment variable $SLURM_ARRAY_TASK_ID. The script can reference $SLURM_ARRAY_TASK_ID to select an input file, for example, or to set a command-line argument for the application code:
@@ Line 20: / Line 20: @@
   my_app $SLURM_ARRAY_TASK_ID some_arg another_arg
-Here is an example bash script that you can insert into your slurm batch script that uses the $SLURM_ARRAY_TASK_ID to select a different input file from the datafiles directory:
+Using a job array instead of a large number of separate serial jobs has advantages for you and other users. A waiting job array only produces one line of output in squeue, making it easier for you to read its output. The scheduler does not have to analyze job requirements for each array task separately, so it can run more efficiently too.
- FILES=(datadir/*)
- input_file="${FILES[$SLURM_ARRAY_TASK_ID]}"
- my_app $input_file
-Using a job array instead of a large number of separate serial jobs has advantages for other users and the system as a whole. A waiting job array only produces one line of output in squeue, making it easier for you to read its output. The scheduler does not have to analyze job requirements for each array task separately, so it can run more efficiently too.
 == Running the same script in multiple directories ==