cc_staff
318
edits
GNU Parallel: Difference between revisions
Jump to navigation
Jump to search
no edit summary
No edit summary |
No edit summary |
||
| Line 42: | Line 42: | ||
== File Content as Command List == <!--T:8--> | == File Content as Command List == <!--T:8--> | ||
Gnu parallel can also interpret the lines of a file as the actual sub-jobs to be run in parallel, by using redirection. | Gnu parallel can also interpret the lines of a file as the actual sub-jobs to be run in parallel, by using redirection. For example, if you have a list of sub-jobs in the file <code>my_commands.txt</code> (one per line), you may run them in parallel as follows: | ||
{{Command|parallel < | {{Command|parallel < my_commands.txt}} | ||
Note that there is no command-argument given to parallel. This usage mode can be particularly useful if the sub-jobs contain symbols that are special to gnu parallel, or the sub-command are to contain a few commands (e.g. <code>cd dir1 && ./executable</code>). | |||
Note that there is no command-argument given to parallel. This usage mode can be particularly useful if the sub-jobs contain symbols that are special to gnu parallel, or the sub-command are to contain a few commands (e.g. < | |||
Here is an example how to run a job with GNU parallel with 4 tasks running in parallel | |||
<tabs> | |||
<tab name="Script"> | |||
{{File | |||
|name=run_gnuparallel_test.sh | |||
|lang="bash" | |||
|contents= | |||
#!/bin/bash | |||
#SBATCH --account=def-someuser | |||
#SBATCH --cpus-per-task=4 | |||
#SBATCH --time=00-02:00 | |||
#SBATCH --mem=4000M # Total memory for all tasks | |||
parallel < ./my_commands.txt | |||
}} | |||
</tab> | |||
<tab name="List of Tasks"> | |||
{{File | |||
|name=my_commands.txt | |||
|lang="txt" | |||
|contents= | |||
command1 | |||
command2 | |||
command3 | |||
command4 | |||
command5 | |||
command6 | |||
command7 | |||
command8 | |||
command9 | |||
}} | |||
</tab> | |||
</tabs> | |||
==Running on Multiple Nodes== <!--T:10--> | ==Running on Multiple Nodes== <!--T:10--> | ||