cc_staff
282
edits
Large Scale Machine Learning (Big Data): Difference between revisions
Jump to navigation
Jump to search
Large Scale Machine Learning (Big Data) (view source)
Revision as of 18:57, 15 March 2022
, 2 years agono edit summary
No edit summary |
No edit summary |
||
| Line 136: | Line 136: | ||
=Snap ML= <!--T:34--> | =Snap ML= <!--T:34--> | ||
[https://snapml.readthedocs.io/en/latest/ Snap ML] is a closed-source Machine Learning library being developed by IBM. <code>Snap ML</code> currently supports a number of classical machine learning models and scales gracefully to data sets with billions of examples and/or features. It offers distributed training, GPU acceleration and supports sparse data structures. It features an API very similar to <code>scikit-learn</code> and can be used as a replacement for that library when dealing with massive datasets. | [https://snapml.readthedocs.io/en/latest/ Snap ML] is a closed-source Machine Learning library being developed by IBM. <code>Snap ML</code> currently supports a number of classical machine learning models and scales gracefully to data sets with billions of examples and/or features. It offers distributed training, GPU acceleration and supports sparse data structures. It features an API very similar to <code>scikit-learn</code> and can be used as a replacement for that library when dealing with massive datasets. | ||
== Installation == | |||
===Latest available wheels=== | |||
To see the latest version of PyTorch that we have built: | |||
{{Command|avail_wheels "snapml"}} | |||
For more information, see [[Python#Available_wheels |Available wheels]]. | |||
===Installing the Compute Canada wheel=== | |||
The preferred option is to install it using the Python [https://pythonwheels.com/ wheel] as follows: | |||
:1. Load a Python [[Utiliser_des_modules/en#Sub-command_load|module]], thus <tt>module load python</tt> | |||
:2. Create and start a [[Python#Creating_and_using_a_virtual_environment|virtual environment]]. | |||
:3. Install PyTorch in the virtual environment with <code>pip install</code>. | |||
:{{Command|prompt=(venv) [name@server ~]|pip install --no-index snapml }} | |||
==Multithreading== <!--T:35--> | ==Multithreading== <!--T:35--> | ||