Bureaucrats, cc_docs_admin, cc_staff
2,879
edits
Data management at Niagara: Difference between revisions
Marked this version for translation
mNo edit summary |
(Marked this version for translation) |
||
| Line 36: | Line 36: | ||
For consistency with the general purpose clusters Cedar and Graham, the environment variable $SLURM_TMPDIR will be set on Niagara compute jobs. Note that this variable will point to RAMdisk, not to local hard drives. The $SLURM_TMPDIR directory will be empty when your jobs starts and its content gets deleted after the job has finished. | For consistency with the general purpose clusters Cedar and Graham, the environment variable $SLURM_TMPDIR will be set on Niagara compute jobs. Note that this variable will point to RAMdisk, not to local hard drives. The $SLURM_TMPDIR directory will be empty when your jobs starts and its content gets deleted after the job has finished. | ||
=== Per-job temporary burst buffer space ($BB_JOB_DIR) === | === Per-job temporary burst buffer space ($BB_JOB_DIR) === <!--T:48--> | ||
For every job on Niagara, the scheduler creates a temporary directory on the burst buffer called <tt>$BB_JOB_DIR</tt>. The <tt>$BB_JOB_DIR</tt> directory will be empty when your jobs starts and its content gets deleted after the job has finished. This directory is accessible from all nodes of a job. | For every job on Niagara, the scheduler creates a temporary directory on the burst buffer called <tt>$BB_JOB_DIR</tt>. The <tt>$BB_JOB_DIR</tt> directory will be empty when your jobs starts and its content gets deleted after the job has finished. This directory is accessible from all nodes of a job. | ||
<!--T:49--> | |||
<tt>$BB_JOB_DIR</tt> is intended as a place for applications that generate many small temporary files or that create files that are accessed very frequently (i.e., high IOPS applications), but that do not fit in ramdisk. | <tt>$BB_JOB_DIR</tt> is intended as a place for applications that generate many small temporary files or that create files that are accessed very frequently (i.e., high IOPS applications), but that do not fit in ramdisk. | ||
<!--T:50--> | |||
It should be emphasized that if the temporary files do fit in ramdisk, then that is generally a better location for them as both the bandwidth and iops of ramdisk far exceeds that of the burst buffer. To use ramdisk, you can either directly access /dev/shm or use the environment variable <tt>$SLURM_TMPDIR</tt>. | It should be emphasized that if the temporary files do fit in ramdisk, then that is generally a better location for them as both the bandwidth and iops of ramdisk far exceeds that of the burst buffer. To use ramdisk, you can either directly access /dev/shm or use the environment variable <tt>$SLURM_TMPDIR</tt>. | ||
<!--T:51--> | |||
Note that Niagara compute nodes have no local disks, so <tt>$SLURM_TMPDIR</tt> lives in memory (ramdisk), in contrast to the general purpose Compute Canada systems Cedar and Graham, where this variable points to a directory on a node-local ssd disk. | Note that Niagara compute nodes have no local disks, so <tt>$SLURM_TMPDIR</tt> lives in memory (ramdisk), in contrast to the general purpose Compute Canada systems Cedar and Graham, where this variable points to a directory on a node-local ssd disk. | ||