cc_staff
353
edits
Python: Difference between revisions
Jump to navigation
Jump to search
Add section about custom wheelhouse
No edit summary |
(Add section about custom wheelhouse) |
||
| Line 169: | Line 169: | ||
<!--T:43--> | <!--T:43--> | ||
Note that the above instructions require all of the packages you need to be available in the python wheels that we provide (see below). If | Note that the above instructions require all of the packages you need to be available in the python wheels that we provide (see "Listing available wheels" below). If the wheel is not available in our wheelhouse, you could also create your own (see "Pre-downloading packages" section below). If you think that the missing wheel should be included in the Compute Canada wheelhouse, please contact [[Technical support]] to make a request. | ||
=== Listing available wheels === <!--T:23--> | === Listing available wheels === <!--T:23--> | ||
| Line 222: | Line 222: | ||
* List the wheels specifically for GPU and display only ''name'', ''version'', ''python'' columns: <tt>avail_wheels --column name version python --all_versions --name "*gpu"</tt> | * List the wheels specifically for GPU and display only ''name'', ''version'', ''python'' columns: <tt>avail_wheels --column name version python --all_versions --name "*gpu"</tt> | ||
* Display usage and help: <tt>avail_wheels --help</tt> | * Display usage and help: <tt>avail_wheels --help</tt> | ||
=== Pre-downloading packages === | |||
You can make your own wheelhouse in your home. A wheelhouse is a folder containing downloaded Python packages (or "wheels"). Compute Canada maintains a wheelhouse that is available on all nodes; that is why you can install many packages without internet access. Here is how to create your own: | |||
# Create a folder in your home, e.g.: <tt>mkdir ~/wheelhouse</tt>. | |||
# Go inside this folder, and run <tt>pip download thepackageyouneed</tt>. This will download the package and its dependencies in the working directory. The syntax of <tt>pip download</tt> is the same as <tt>pip install</tt>. | |||
# Then, when installing packages on a compute node, specify the path to your wheelhouse: <tt>pip install thepackageyouneed -f ~/wheelhouse</tt>. | |||
== Parallel programming with Python <tt>multiprocessing</tt> module == <!--T:45--> | == Parallel programming with Python <tt>multiprocessing</tt> module == <!--T:45--> | ||