Bureaucrats, cc_docs_admin, cc_staff
2,879
edits
Data management at Niagara: Difference between revisions
Jump to navigation
Jump to search
change section headers to sentence case
No edit summary |
(change section headers to sentence case) |
||
| Line 112: | Line 112: | ||
<li><p><code>$BBUFFER</code> stands for [https://docs.scinet.utoronto.ca/index.php/Burst_Buffer Burst Buffer], a faster parallel storage tier for temporary data.</p></li></ul> | <li><p><code>$BBUFFER</code> stands for [https://docs.scinet.utoronto.ca/index.php/Burst_Buffer Burst Buffer], a faster parallel storage tier for temporary data.</p></li></ul> | ||
==How much | ==How much disk space do I have left?== <!--T:14--> | ||
The <tt>'''/scinet/niagara/bin/diskUsage'''</tt> command, available on the login nodes and datamovers, provides information in a number of ways on the home, scratch, project and archive file systems. For instance, how much disk space is being used by yourself and your group (with the -a option), or how much your usage has changed over a certain period ("delta information") or you may generate plots of your usage over time. Please see the usage help below for more details. | The <tt>'''/scinet/niagara/bin/diskUsage'''</tt> command, available on the login nodes and datamovers, provides information in a number of ways on the home, scratch, project and archive file systems. For instance, how much disk space is being used by yourself and your group (with the -a option), or how much your usage has changed over a certain period ("delta information") or you may generate plots of your usage over time. Please see the usage help below for more details. | ||
<pre> | <pre> | ||
| Line 127: | Line 127: | ||
Note: information on usage and quota is only updated every 3 hours! | Note: information on usage and quota is only updated every 3 hours! | ||
==Scratch | ==Scratch disk purging policy== <!--T:17--> | ||
In order to ensure that there is always significant space available for running jobs '''we automatically delete files in /scratch that have not been accessed or modified for more than 2 months by the actual deletion day on the 15th of each month'''. Note that we recently changed the cut out reference to the ''MostRecentOf(atime,ctime)''. This policy is subject to revision depending on its effectiveness. More details about the purging process and how users can check if their files will be deleted follows. If you have files scheduled for deletion you should move them to more permanent locations such as your departmental server or your /project space or into HPSS (for PIs who have either been allocated storage space by the RAC on project or HPSS). | In order to ensure that there is always significant space available for running jobs '''we automatically delete files in /scratch that have not been accessed or modified for more than 2 months by the actual deletion day on the 15th of each month'''. Note that we recently changed the cut out reference to the ''MostRecentOf(atime,ctime)''. This policy is subject to revision depending on its effectiveness. More details about the purging process and how users can check if their files will be deleted follows. If you have files scheduled for deletion you should move them to more permanent locations such as your departmental server or your /project space or into HPSS (for PIs who have either been allocated storage space by the RAC on project or HPSS). | ||
| Line 183: | Line 183: | ||
* Storage space on HPSS is allocated through the annual [https://www.computecanada.ca/research-portal/accessing-resources/resource-allocation-competitions Compute Canada RAC allocation]. | * Storage space on HPSS is allocated through the annual [https://www.computecanada.ca/research-portal/accessing-resources/resource-allocation-competitions Compute Canada RAC allocation]. | ||
= File | = File ownership management and access control lists = <!--T:32--> | ||
* By default, at SciNet, users within the same group already have read permission to each other's files (not write) | * By default, at SciNet, users within the same group already have read permission to each other's files (not write) | ||
* You may use access control list ('''ACL''') to allow your supervisor (or another user within your group) to manage files for you (i.e., create, move, rename, delete), while still retaining your access and permission as the original owner of the files/directories. You may also let users in other groups or whole other groups access (read, execute) your files using this same mechanism. | * You may use access control list ('''ACL''') to allow your supervisor (or another user within your group) to manage files for you (i.e., create, move, rename, delete), while still retaining your access and permission as the original owner of the files/directories. You may also let users in other groups or whole other groups access (read, execute) your files using this same mechanism. | ||
| Line 200: | Line 200: | ||
<!--T:34--> | <!--T:34--> | ||
Then issue the following | Then issue the following two commands: | ||
<pre> | <pre> | ||
1) $ mmputacl -i /tmp/supervisor.acl /project/g/group/[owner] | 1) $ mmputacl -i /tmp/supervisor.acl /project/g/group/[owner] | ||