rsnt_translations
56,420
edits
AI and Machine Learning: Difference between revisions
Jump to navigation
Jump to search
no edit summary
(Remove draft tag) |
No edit summary |
||
| Line 1: | Line 1: | ||
<translate> | |||
To get the most out of our clusters for machine learning applications, special care must be taken. A cluster is a complicated beast that is very different from your local machine that you use for prototyping. Notably, a cluster uses a distributed filesystem, linking many storage devices seamlessly. Accessing a file on <tt>/project</tt> ''feels the same'' as accessing one from the current node; but under the hood, these two IO operations have very different performance implications. In short, you need to [[#Managing_your_datasets|choose wisely where to put your data]]. | To get the most out of our clusters for machine learning applications, special care must be taken. A cluster is a complicated beast that is very different from your local machine that you use for prototyping. Notably, a cluster uses a distributed filesystem, linking many storage devices seamlessly. Accessing a file on <tt>/project</tt> ''feels the same'' as accessing one from the current node; but under the hood, these two IO operations have very different performance implications. In short, you need to [[#Managing_your_datasets|choose wisely where to put your data]]. | ||
| Line 48: | Line 49: | ||
... you should consider grouping many jobs into one. [[GLOST]] and [[GNU Parallel]] are available to help you with this. | ... you should consider grouping many jobs into one. [[GLOST]] and [[GNU Parallel]] are available to help you with this. | ||
</translate> | |||