Bureaucrats, cc_docs_admin, cc_staff
2,306
edits
Translations:Tutoriel Apprentissage machine/34/en: Difference between revisions
Jump to navigation
Jump to search
Translations:Tutoriel Apprentissage machine/34/en (view source)
Revision as of 15:00, 7 October 2019
, 5 years agono edit summary
(Created page with "The filesystems on Compute Canada clusters are designed for a small number of extremely large files. Make sure that the data set which you need for your training is an archive...") |
No edit summary |
||
| Line 1: | Line 1: | ||
The filesystems on Compute Canada clusters are designed for a small number of extremely large files. Make sure that the data set which you need for your training is an archive format like ''tar'', which you can then transfer to your job's compute node when the job starts. '''If you do not respect these rules, you risk causing enormous numbers of I/O operations on the shared filesystem, leading to performance issues on the cluster for all of its users.''' If you want to learn more about how to handle collections of large number of files, we recommend that you spend some time reading [[Handling_large_collections_of_files this page]]. | The filesystems on Compute Canada clusters are designed for a small number of extremely large files. Make sure that the data set which you need for your training is an archive format like ''tar'', which you can then transfer to your job's compute node when the job starts. '''If you do not respect these rules, you risk causing enormous numbers of I/O operations on the shared filesystem, leading to performance issues on the cluster for all of its users.''' If you want to learn more about how to handle collections of large number of files, we recommend that you spend some time reading [[Handling_large_collections_of_files|this page]]. | ||