GNU Parallel: Difference between revisions

GNU Parallel (view source)

Revision as of 14:04, 1 April 2020

553 bytes added , 4 years ago

no edit summary

Skhan

cc_staff

318

edits

@@ Line 42: / Line 42: @@
 == File Content as Command List == <!--T:8-->
-Gnu parallel can also interpret the lines of a file as the actual sub-jobs to be run in parallel, by using redirection.  For example, if you have a list of sub-jobs in the file <tt>mycommands.txt</tt> (one per line), you may run them in parallel as follows:
+Gnu parallel can also interpret the lines of a file as the actual sub-jobs to be run in parallel, by using redirection. For example, if you have a list of sub-jobs in the file <code>my_commands.txt</code> (one per line), you may run them in parallel as follows:
-{{Command|parallel < mycommands.txt}}
+{{Command|parallel < my_commands.txt}}
-<!--T:9-->
+Note that there is no command-argument given to parallel. This usage mode can be particularly useful if the sub-jobs contain symbols that are special to gnu parallel, or the sub-command are to contain a few commands (e.g. <code>cd dir1 && ./executable</code>).
-Note that there is no command-argument given to parallel. This usage mode can be particularly useful if the sub-jobs contain symbols that are special to gnu parallel, or the sub-command are to contain a few commands (e.g. <tt>cd dir1 && ./executable</tt>).
+Here is an example how to run a job with GNU parallel with 4 tasks running in parallel
+<tabs>
+<tab name="Script">
+{{File
+  |name=run_gnuparallel_test.sh
+  |lang="bash"
+  |contents=
+#!/bin/bash
+#SBATCH --account=def-someuser
+#SBATCH --cpus-per-task=4
+#SBATCH --time=00-02:00
+#SBATCH --mem=4000M     # Total memory for all tasks
+parallel < ./my_commands.txt
+}}
+</tab>
+<tab name="List of Tasks">
+{{File
+  |name=my_commands.txt
+  |lang="txt"
+  |contents=
+command1
+command2
+command3
+command4
+command5
+command6
+command7
+command8
+command9
+}}
+</tab>
+</tabs>
 ==Running on Multiple Nodes== <!--T:10-->