Bureaucrats, cc_docs_admin, cc_staff
2,320
edits
Storage and file management: Difference between revisions
Jump to navigation
Jump to search
no edit summary
No edit summary |
No edit summary |
||
| Line 31: | Line 31: | ||
== Best practices == <!--T:9--> | == Best practices == <!--T:9--> | ||
* Only use text format for files that are smaller than a few megabytes. | * Only use text format for files that are smaller than a few megabytes. | ||
* As far as possible, use scratch local storage for temporary files. | * As far as possible, use scratch and local storage for temporary files. For local storage you can use the temporary directory created by the [[Running jobs|job scheduler]] for this, named <code>$SLURM_TMPDIR</code>. | ||
* If your program must search within a file, it is fastest to do it by first reading it completely before searching, or to use a RAM disk. | * If your program must search within a file, it is fastest to do it by first reading it completely before searching, or to use a RAM disk. | ||
* Regularly clean up your data in the scratch and project spaces, because those filesystems are used for huge data collections. | * Regularly clean up your data in the scratch and project spaces, because those filesystems are used for huge data collections. | ||