Bureaucrats, cc_docs_admin, cc_staff
2,879
edits
Using node-local storage: Difference between revisions
→Signal trapping
No edit summary |
|||
| Line 63: | Line 63: | ||
You can ask Slurm to send your script a signal a short time before the run-time expires with | You can ask Slurm to send your script a signal a short time before the run-time expires with | ||
[https://slurm.schedmd.com/sbatch.html <code>--signal</code>], and write a shell function which | [https://slurm.schedmd.com/sbatch.html <code>--signal</code>], and write a shell function which | ||
copies your output from <code>$SLURM_TMPDIR</code> back to network when that happens. | copies your output from <code>$SLURM_TMPDIR</code> back to network storage when that happens. | ||
This will not address unexpected node failures, but may be useful if your run-time estimate is uncertain, | This will not address unexpected node failures, but may be useful if your run-time estimate is uncertain, | ||
or if you are chaining together several Slurm jobs to complete a long calculation. | or if you are chaining together several Slurm jobs to complete a long calculation. | ||