Using nearline storage: Difference between revisions

Using nearline storage (view source)

67 bytes added , 4 years ago

try to make the use-case clearer, and remove reference to scratch.

cc_staff

176

edits

@@ Line 2: / Line 2: @@
 <translate>
 <!--T:30-->
-The nearline storage space is tape-based and intended for the storage of data that needs to be conserved but does not need to be read except quite infrequently. The nearline filesystem is only accessible from the login and data transfer nodes of the cluster in question and it is intended for the storage of very large archival files, e.g. using the [[Tar]] and [[Dar]] format. These archival files that will be transferred to nearline should be created on a data transfer node in the scratch space.
+The nearline storage space is tape-based and designed to store '''large''' files which are '''infrequently accessed'''.
+The nearline filesystem is only accessible from the login and data-transfer nodes of the cluster (not compute nodes).  If you have many and/or small files, you should use a tool like [[Tar]] or [[Dar]] to aggregate them into large archive files.
+Because such operations can be resource-intensive, they may not be appropriate on login nodes so use data-transfer nodes,
+or create the archive files on another filesystem before moving to nearline.
 ==Nearline is a filesystem virtualized onto tape== <!--T:1-->