cc_staff
5
edits
Job arrays: Difference between revisions
Added bash code for the selection of input file based on array task
(import text from ACENET wiki) |
(Added bash code for the selection of input file based on array task) |
||
| Line 19: | Line 19: | ||
my_app <input.$SLURM_ARRAY_TASK_ID | my_app <input.$SLURM_ARRAY_TASK_ID | ||
my_app $SLURM_ARRAY_TASK_ID some_arg another_arg | my_app $SLURM_ARRAY_TASK_ID some_arg another_arg | ||
Here is an example bash script that you can insert into your slurm batch script that uses the $SLURM_ARRAY_TASK_ID to select a different input file from the datafiles directory | |||
FILES=($(ls -1 datafiles/*)) | |||
input_file=${FILES[$SLURM_ARRAY_TASK_ID]} | |||
my_app $input_file | |||
Using a job array instead of a large number of separate serial jobs has advantages for other users and the system as a whole. A waiting job array only produces one line of output in squeue, making it easier for you to read its output. The scheduler does not have to analyze job requirements for each array task separately, so it can run more efficiently too. | Using a job array instead of a large number of separate serial jobs has advantages for other users and the system as a whole. A waiting job array only produces one line of output in squeue, making it easier for you to read its output. The scheduler does not have to analyze job requirements for each array task separately, so it can run more efficiently too. | ||