Bureaucrats, cc_docs_admin, cc_staff
2,306
edits
Scratch purging policy: Difference between revisions
Jump to navigation
Jump to search
→Expiration Procedure
(→Abuse) |
|||
| Line 17: | Line 17: | ||
The scratch filesystem is checked at the end of the month for files which will be candidates for expiry on the 15th of the following month. On the first day of the month, a notification e-mail is sent to all users who have at least one file which is a candidate for purging and containing the location of a file which lists all the candidates for purging. You will thus have two weeks to make arrangements to move data to your project space or some other location if you wish to save the data in question. | The scratch filesystem is checked at the end of the month for files which will be candidates for expiry on the 15th of the following month. On the first day of the month, a notification e-mail is sent to all users who have at least one file which is a candidate for purging and containing the location of a file which lists all the candidates for purging. You will thus have two weeks to make arrangements to move data to your project space or some other location if you wish to save the data in question. | ||
On the 12th of the month, a final notification e-mail will be sent with an updated assessment of candidate files for expiration on the 15th, giving you 72 hours to make arrangements for moving these files. At the end of day on the 15th any remaining files on the scratch filesystem with the property that the minimum of their <tt>ctime</tt> and <tt>atime</tt> is greater than 60 days will be deleted. | On the 12th of the month, a final notification e-mail will be sent with an updated assessment of candidate files for expiration on the 15th, giving you 72 hours to make arrangements for moving these files. At the end of day on the 15th any remaining files on the scratch filesystem with the property that the minimum of their <tt>ctime</tt> and <tt>atime</tt> is greater than 60 days will be deleted. | ||
Note that simplying copying or using the <tt>rsync</tt> command to displace your files will update the <tt>atime</tt> for the original data on scratch, so that once you have put the data in another location be sure to delete the original files and directories in scratch ahead of the automatic purging. | |||
=Abuse= | =Abuse= | ||
It should be noted that this method of tracking file age does allow for potential abuse the system by periodically running a recursive <tt>touch</tt> command on your files, preventing them from being flagged for expiration. However Compute Canada staff have methods for detecting this and similar tactics to defeat the purging policy and users who employ such techniques will be contacted and asked to modify their behaviour, in particular to move the "retouched" data from scratch to a more appropriate location. | It should be noted that this method of tracking file age does allow for potential abuse the system by periodically running a recursive <tt>touch</tt> command on your files, preventing them from being flagged for expiration. However Compute Canada staff have methods for detecting this and similar tactics to defeat the purging policy and users who employ such techniques will be contacted and asked to modify their behaviour, in particular to move the "retouched" data from scratch to a more appropriate location. | ||