AI and Machine Learning: Difference between revisions

AI and Machine Learning (view source)

11 bytes removed , 5 years ago

Remove draft tag

cc_staff

353

edits

Revision as of 20:53, 18 July 2019 (view source) Lemc2220 (talk \| contribs) (Long running computations; Running many similar jobs) ← Older edit		Revision as of 15:32, 23 July 2019 (view source) Lemc2220 (talk \| contribs) (Remove draft tag) Newer edit →
Line 1:		Line 1:
	~~{{Draft}}~~

	To get the most out of our clusters for machine learning applications, special care must be taken. A cluster is a complicated beast that is very different from your local machine that you use for prototyping. Notably, a cluster uses a distributed filesystem, linking many storage devices seamlessly. Accessing a file on <tt>/project</tt> ''feels the same'' as accessing one from the current node; but under the hood, these two IO operations have very different performance implications. In short, you need to [[#Managing_your_datasets\|choose wisely where to put your data]].		To get the most out of our clusters for machine learning applications, special care must be taken. A cluster is a complicated beast that is very different from your local machine that you use for prototyping. Notably, a cluster uses a distributed filesystem, linking many storage devices seamlessly. Accessing a file on <tt>/project</tt> ''feels the same'' as accessing one from the current node; but under the hood, these two IO operations have very different performance implications. In short, you need to [[#Managing_your_datasets\|choose wisely where to put your data]].