cc_staff
127
edits
Using nearline storage: Difference between revisions
Jump to navigation
Jump to search
no edit summary
No edit summary |
No edit summary |
||
| Line 4: | Line 4: | ||
==Nearline is a filesystem virtualized onto tape== <!--T:1--> | ==Nearline is a filesystem virtualized onto tape== <!--T:1--> | ||
Nearline storage is a disk-tape hybrid filesystem with a layout like [[Project layout|Project]], except that it can virtualize files by moving them to tape-based storage on criteria such as age and size, and then back again upon read or recall operations. This is a way to manage less-used files. On tape, the files do not consume your disk quota, but they can still be accessed, albeit slower than with the home, scratch and project filesystems. | Nearline storage is a disk-tape hybrid filesystem with a layout like [[Project layout|Project]], except that it can virtualize files by moving them to tape-based storage on criteria such as age and size, and then back again upon read or recall operations. This is a way to manage less-used files. On tape, the files do not consume your disk quota, but they can still be accessed, albeit slower than with the home, scratch and project filesystems. | ||
<!--T:2--> | <!--T:2--> | ||
This is useful because the capacity of our tape libraries is both large and expandable. When a file has been moved to tape (or ''virtualized''), it still appears in the directory listing. If the virtual file is read, the reading process will block for some time, probably a few minutes, while the file contents is copied from tape to disk. | This is useful because the capacity of our tape libraries is both large and expandable. When a file has been moved to tape (or ''virtualized''), it still appears in the directory listing. If the virtual file is read, the reading process will block for some time, probably a few minutes, while the file contents is copied from tape to disk. | ||
You can tell that a file is on tape or still on disk with the <code>lfs hsm_state</code> command: | |||
<source lang="bash"> | |||
# Here, <FILE> is still on the disk | |||
$ lfs hsm_state <FILE> | |||
<FILE>: [...]: exists archived, [...] | |||
# Here, <FILE> is archived on tape, there will be a lag when opening it. | |||
$ lfs hsm_state <FILE> | |||
<FILE>: [...]: released archived, [...] | |||
</source> | |||
== Using nearline == <!--T:3--> | == Using nearline == <!--T:3--> | ||
| Line 39: | Line 52: | ||
<!--T:11--> | <!--T:11--> | ||
To use nearline, just put files into your <tt>~/nearline/PROJECT</tt> directory. After a period of time (24 hours as of February 2019), they will be copied onto tape. If the file remains unchanged for another period (24 hours as of February 2019), the copy on disk will be removed, making the file virtualized on tape. | To use nearline, just put files into your <tt>~/nearline/PROJECT</tt> directory. After a period of time (24 hours as of February 2019), they will be copied onto tape. If the file remains unchanged for another period (24 hours as of February 2019), the copy on disk will be removed, making the file virtualized on tape. | ||
<!--T:8--> | <!--T:8--> | ||
| Line 68: | Line 82: | ||
<!--T:20--> | <!--T:20--> | ||
<tab name="Béluga"> | <tab name="Béluga"> | ||
Nearline service similar to that on Graham | Nearline service similar to that on Graham. | ||
</tab> | </tab> | ||
</tabs> | </tabs> | ||
</translate> | </translate> | ||