Using nearline storage: Difference between revisions

There are three methods to access nearline:

1. By submitting HPSS-specific commands <tt>htar</tt> or <tt>hsi</tt> to the Slurm scheduler as a job in one of the archive partitions; see [https://docs.scinet.utoronto.ca/index.php/HPSS the HPSS documentation] for detailed examples. Using job scripts offers the benefit of automating nearline transfers and is the best method if you use HPSS regularly. Your HPSS files can be found in the $ARCHIVE directory, which is like '$PROJECT' but with ''/project'' replaced by ''/archive''.  

2. To manage a small number of files in HPSS, you can use the VFS (''Virtual File System'') node, which is accessed with the command <tt>salloc --time=1:00:00 -pvfsshort</tt>. Your HPSS files can be found in the $ARCHIVE directory, which is like '$PROJECT' but with ''/project'' replaced by ''/archive''.  

3. By using [[Globus]] for transfers to and from HPSS using the endpoint <b>computecanada#hpss</b>. This is useful for occasional usage and for transfers to and from other sites.