PyTorch

PyTorch is a Python package that provides two high-level features:

• Tensor computation (like NumPy) with strong GPU acceleration
• Deep neural networks built on a tape-based autograd system

Installation

There are two options to install PyTorch.

• You can install PyTorch using Anaconda. First install Anaconda, and then install PyTorch in a conda environment as follows:

1. Load the Miniconda 2 or Miniconda 3 module.

[name@server ~]$ module load miniconda3

2. Create a new conda virtual environment.

[name@server ~]$ conda create --name pytorch

3. When conda asks you to proceed, type y.

4. Activate the newly created conda virtual environment.

[name@server ~]$ source activate pytorch

5. Install PyTorch in the conda virtual environment.

[name@server ~]$ conda install pytorch torchvision cuda80  -c soumith

Here, we instruct conda to use the soumith channel to retrieve the packages from the release channel belonging to the main PyTorch developer, Soumith Chintala. This guarantees you will have the latest version.

• You can install PyTorch from a Python wheel, as follows:

1. Load a SciPy-stack environment module in order to access NumPy. For Python 2,

[name@server ~]$ module load python27-scipy-stack/2017a

For Python 3,

[name@server ~]$ module load python35-scipy-stack/2017a

2. Create and start a virtual environment.

3. Install PyTorch in the virtual environment with pip install. For both GPU and CPU support,

[name@server ~]$ pip install torch_gpu

If you only need CPU support,

[name@server ~]$ pip install torch_cpu

The current default wheel provides PyTorch version 0.2

Job submission

Once the setup is completed, you can submit a PyTorch job with

[name@server ~]$ sbatch pytorch-test.sh

Here is an example of a job submission script:

File : pytorch-test.sh

#!/bin/bash
#SBATCH --gres=gpu:1       # Request GPU "generic resources"
#SBATCH --cpus-per-task=6  # Cores proportional to GPUs: 6 on Cedar, 16 on Graham.
#SBATCH --mem=32000M       # Memory proportional to GPUs: 32000 Cedar, 64000 Graham.
#SBATCH --time=0-03:00
#SBATCH --output=%N-%j.out
module load miniconda3
source activate pytorch
python ./pytorch-test.py

The Python script pytorch-test.py has the form

File : pytorch-test.py

import torch
x = torch.Tensor(5, 3)
print(x)
y = torch.rand(5, 3)
print(y)
# let us run the following only if CUDA is available
if torch.cuda.is_available():
    x = x.cuda()
    y = y.cuda()
    print(x + y)