rsnt_translations
56,420
edits
GNU Parallel: Difference between revisions
Jump to navigation
Jump to search
no edit summary
(Marked this version for translation) |
No edit summary |
||
| Line 22: | Line 22: | ||
<!--T:5--> | <!--T:5--> | ||
Note that GNU Parallel refers to each of the commands executed as <i>jobs</i>. This can be confusing because on many | Note that GNU Parallel refers to each of the commands executed as <i>jobs</i>. This can be confusing because on many of our systems, a job is a batch script run by a scheduler, and GNU Parallel would be used inside that job. From that perspective, GNU Parallel's jobs are <i>subjobs</i>. | ||
== Multiple Arguments == <!--T:6--> | == Multiple Arguments == <!--T:6--> | ||
| Line 94: | Line 94: | ||
|title=Not recommended | |title=Not recommended | ||
|content= | |content= | ||
While GNU parallel can be used across multiple nodes, it can have problems doing so, and it is not recommended, in particular in the context of a lot of short jobs. That is because it needs to start SSH session on remote nodes, which is an operation which can hang, and which requires multiple seconds. If you choose to use it, make sure you add a delay between jobs of 30 seconds or more, using the option <tt>--sshdelay 30</tt> | While GNU parallel can be used across multiple nodes, it can have problems doing so, and it is not recommended, in particular in the context of a lot of short jobs. That is because it needs to start an SSH session on remote nodes, which is an operation which can hang, and which requires multiple seconds. If you choose to use it, make sure you add a delay between jobs of 30 seconds or more, using the option <tt>--sshdelay 30</tt> | ||
}} | }} | ||
<!--T:38--> | <!--T:38--> | ||
You can also use GNU Parallel to distribute a workload across multiple nodes in a cluster, such as in the context of a job on | You can also use GNU Parallel to distribute a workload across multiple nodes in a cluster, such as in the context of a job on our servers. An example of this use is the following: | ||
{{Command | {{Command | ||
|scontrol show hostname > ./node_list_${SLURM_JOB_ID} | |scontrol show hostname > ./node_list_${SLURM_JOB_ID} | ||