Data management at Niagara: Difference between revisions

Line 26: Line 26:


=== /dev/shm (RAM) ===
=== /dev/shm (RAM) ===
On the Niagara nodes a [[User_Ramdisk | ramdisk]] is available. [[User_Ramdisk | Ramdisk]] is much faster than real disk, and faster than Burst Buffer. Up to 70 percent of the RAM on the node (i.e. 202GB) may be used as a temporary '''local''' file system. This is particularly useful in the early stages of migrating desktop-computing codes to a HPC platform such as Niagara, especially those that use a lot of file I/O (Input/Output). Using a lot of I/O is a bottleneck in large scale computing, especially on parallel file systems (such as the GPFS used on Niagara), since the files are synchronized across the whole network.
On the Niagara nodes a [https://docs.scinet.utoronto.ca/index.php/User_Ramdisk ramdisk] is available. [https://docs.scinet.utoronto.ca/index.php/User_Ramdisk Ramdisk] is much faster than real disk, and faster than Burst Buffer. Up to 70 percent of the RAM on the node (i.e. 202GB) may be used as a temporary '''local''' file system. This is particularly useful in the early stages of migrating desktop-computing codes to a HPC platform such as Niagara, especially those that use a lot of file I/O (Input/Output). Using a lot of I/O is a bottleneck in large scale computing, especially on parallel file systems (such as the GPFS used on Niagara), since the files are synchronized across the whole network.


= Quotas and purging =
= Quotas and purging =
cc_staff
290

edits