Bureaucrats, cc_docs_admin, cc_staff
2,915
edits
Scratch purging policy: Difference between revisions
Jump to navigation
Jump to search
remove draft tag
No edit summary |
(remove draft tag) |
||
| Line 1: | Line 1: | ||
<langauges /> | |||
<translate> | |||
=Overview= | =Overview= | ||
The scratch filesystem on Compute Canada clusters is intended as temporary, fast storage for data being used during job execution. Data needed for long term storage and reference should be kept in either [[Project layout|<tt>/project</tt>]] or other archival storage areas. In order to ensure adequate space on scratch older files are periodically deleted according to the policy outlined in this page. The threshold for purging is 60 days, which is a little more than twice the maximum duration of a job on the cluster. | The scratch filesystem on Compute Canada clusters is intended as temporary, fast storage for data being used during job execution. Data needed for long term storage and reference should be kept in either [[Project layout|<tt>/project</tt>]] or other archival storage areas. In order to ensure adequate space on scratch older files are periodically deleted according to the policy outlined in this page. The threshold for purging is 60 days, which is a little more than twice the maximum duration of a job on the cluster. | ||
| Line 27: | Line 27: | ||
=Abuse= | =Abuse= | ||
This method of tracking file age does allow for potential abuse the system by periodically running a recursive <tt>touch</tt> command on your files, preventing them from being flagged for expiration. Compute Canada staff have methods for detecting this and similar tactics to circumvent the purging policy. Users who employ such techniques will be contacted and asked to modify their behaviour, in particular to move the "retouched" data from scratch to a more appropriate location. | This method of tracking file age does allow for potential abuse the system by periodically running a recursive <tt>touch</tt> command on your files, preventing them from being flagged for expiration. Compute Canada staff have methods for detecting this and similar tactics to circumvent the purging policy. Users who employ such techniques will be contacted and asked to modify their behaviour, in particular to move the "retouched" data from scratch to a more appropriate location. | ||
</translate> | |||