PyTorch/fr: Difference between revisions
No edit summary |
(Updating to match new version of source page) |
||
Line 7: | Line 7: | ||
Il y a une certaine ressemblance entre PyTorch et [[Torch/fr|Torch]], mais pour des raisons pratiques vous pouvez considérer que ce sont des projets différents. | Il y a une certaine ressemblance entre PyTorch et [[Torch/fr|Torch]], mais pour des raisons pratiques vous pouvez considérer que ce sont des projets différents. | ||
PyTorch developers also offer [[LibTorch]], which allows one to implement extensions to PyTorch using C++, and to implement pure C++ machine learning applications. Models written in Python using PyTorch can be converted and used in pure C++ through [https://pytorch.org/tutorials/advanced/cpp_export.html TorchScript]. | PyTorch developers also offer [[#LibTorch|LibTorch]], which allows one to implement extensions to PyTorch using C++, and to implement pure C++ machine learning applications. Models written in Python using PyTorch can be converted and used in pure C++ through [https://pytorch.org/tutorials/advanced/cpp_export.html TorchScript]. | ||
= Installation = | = Installation = | ||
Line 30: | Line 30: | ||
En plus de <tt>torch</tt>, vous pouvez aussi installer <tt>torchvision</tt>, <tt>torchtext</tt> et <tt>torchaudio</tt>. | En plus de <tt>torch</tt>, vous pouvez aussi installer <tt>torchvision</tt>, <tt>torchtext</tt> et <tt>torchaudio</tt>. | ||
{{Command|prompt=(venv) [name@server ~]|pip install torch torchvision torchtext torchaudio --no-index}} | {{Command|prompt=(venv) [name@server ~]|pip install torch torchvision torchtext torchaudio --no-index}} | ||
=Soumettre une tâche= | =Soumettre une tâche= | ||
Line 104: | Line 101: | ||
== Fuites de mémoire == | == Fuites de mémoire == | ||
Sur le matériel AVX512 (nœuds V100, Skylake ou Béluga), les versions PyTorch antérieures à v1.0.1 qui utilisent des bibliothèques moins récentes (cuDNN < v7.5 ou MAGMA < v2.5) peuvent avoir des fuites de mémoire importantes et créer des exceptions de mémoire insuffisante et terminer vos tâches. Pour contrer ceci, utilisez la plus récente version de <tt>torch</tt>. | Sur le matériel AVX512 (nœuds V100, Skylake ou Béluga), les versions PyTorch antérieures à v1.0.1 qui utilisent des bibliothèques moins récentes (cuDNN < v7.5 ou MAGMA < v2.5) peuvent avoir des fuites de mémoire importantes et créer des exceptions de mémoire insuffisante et terminer vos tâches. Pour contrer ceci, utilisez la plus récente version de <tt>torch</tt>. | ||
= LibTorch = | |||
LibTorch allows one to implement both C++ extensions to PyTorch and '''pure C++ machine learning applications'''. It contains "all headers, libraries and CMake configuration files required to depend on PyTorch" (as mentioned in the [https://pytorch.org/cppdocs/installing.html docs]). | |||
=== How to use LibTorch === | |||
==== Get the library ==== | |||
<syntaxhighlight> | |||
wget https://download.pytorch.org/libtorch/cu100/libtorch-shared-with-deps-latest.zip | |||
unzip libtorch-shared-with-deps-latest.zip | |||
cd libtorch | |||
export LIBTORCH_ROOT=$(pwd) # this variable is used in the example below | |||
</syntaxhighlight> | |||
Patch the library (this workaround is needed for compiling on Compute Canada clusters): | |||
<syntaxhighlight> | |||
sed -i -e 's/\/usr\/local\/cuda\/lib64\/libculibos.a;dl;\/usr\/local\/cuda\/lib64\/libculibos.a;//g' share/cmake/Caffe2/Caffe2Targets.cmake | |||
</syntaxhighlight> | |||
The library is also included in the PyTorch wheel, but this is not the recommended way to acquire it. Once Pytorch is installed in a virtual environment, you can find it at: <tt>$VIRTUAL_ENV/lib/python3.6/site-packages/torch/lib/libtorch.so</tt>. | |||
==== Compile a minimal example ==== | |||
Create the following two files: | |||
{{File | |||
|name=example-app.cpp | |||
|lang="cpp" | |||
|contents= | |||
#include <torch/torch.h> | |||
#include <iostream> | |||
int main() { | |||
torch::Device device(torch::kCPU); | |||
if (torch::cuda::is_available()) { | |||
std::cout << "CUDA is available! Using GPU." << std::endl; | |||
device = torch::Device(torch::kCUDA); | |||
} | |||
torch::Tensor tensor = torch::rand({2, 3}).to(device); | |||
std::cout << tensor << std::endl; | |||
} | |||
}} | |||
{{File | |||
|name=CMakeLists.txt | |||
|lang="txt" | |||
|contents= | |||
cmake_minimum_required(VERSION 3.0 FATAL_ERROR) | |||
project(example-app) | |||
find_package(Torch REQUIRED) | |||
add_executable(example-app example-app.cpp) | |||
target_link_libraries(example-app "${TORCH_LIBRARIES}") | |||
set_property(TARGET example-app PROPERTY CXX_STANDARD 11) | |||
}} | |||
Load the necessary modules: | |||
<syntaxhighlight> | |||
module load cmake intel/2018.3 cuda/10 cudnn | |||
</syntaxhighlight> | |||
Compile the program: | |||
<syntaxhighlight> | |||
mkdir build | |||
cd build | |||
cmake -DCMAKE_PREFIX_PATH="$LIBTORCH_ROOT;$EBROOTCUDA;$EBROOTCUDNN" .. | |||
make | |||
</syntaxhighlight> | |||
Run the program: | |||
<syntaxhighlight> | |||
./example-app | |||
</syntaxhighlight> | |||
To test an application with CUDA, request an [[Running_jobs#Interactive_jobs|interactive job]] with a [[Using_GPUs_with_Slurm|GPU]]. | |||
=== Resources === | |||
https://pytorch.org/cppdocs/ |
Revision as of 17:51, 23 July 2019
PyTorch est un paquet Python qui offre deux fonctionnalités de haut niveau :
- le calcul tensoriel (semblable à celui effectué par NumPy) avec grande accélération de GPU,
- des réseaux de neurones d’apprentissage profond dans un système de gradients conçu sur le modèle d’un magnétophone.
Il y a une certaine ressemblance entre PyTorch et Torch, mais pour des raisons pratiques vous pouvez considérer que ce sont des projets différents.
PyTorch developers also offer LibTorch, which allows one to implement extensions to PyTorch using C++, and to implement pure C++ machine learning applications. Models written in Python using PyTorch can be converted and used in pure C++ through TorchScript.
Installation
Wheels récemment ajoutés
Pour connaître la dernière version de PyTorch, utilisez
[name@server ~]$ avail_wheels "torch*"
Voyez aussi Lister les wheels disponibles.
Installation de la wheel Calcul Canada
La meilleure option est d'installer avec Python wheels comme suit :
- 1. Chargez un module Python, soit python/2.7, python/3.5, python/3.6 ou python/3.7.
- 2. Créez et démarrez un environnement virtuel.
- 3. Installez PyTorch dans l'environnement virtuel avec
pip install
.
GPU et CPU
-
(venv) [name@server ~] pip install numpy torch --no-index
En supplément
En plus de torch, vous pouvez aussi installer torchvision, torchtext et torchaudio.
(venv) [name@server ~] pip install torch torchvision torchtext torchaudio --no-index
Soumettre une tâche
Le script suivant est un exemple de soumission d'une tâche utilisant le wheel Python dans un environnement virtuel de $HOME/pytorch.
#!/bin/bash
#SBATCH --gres=gpu:1 # Request GPU "generic resources"
#SBATCH --cpus-per-task=6 # Cores proportional to GPUs: 6 on Cedar, 16 on Graham.
#SBATCH --mem=32000M # Memory proportional to GPUs: 32000 Cedar, 64000 Graham.
#SBATCH --time=0-03:00
#SBATCH --output=%N-%j.out
module load python/3.6
virtualenv --no-download $SLURM_TMPDIR/env
source $SLURM_TMPDIR/env/bin/activate
pip install torch --no-index
python pytorch-test.py
Le script Python pytorch-test.py
est semblable à
import torch
x = torch.Tensor(5, 3)
print(x)
y = torch.rand(5, 3)
print(y)
# let us run the following only if CUDA is available
if torch.cuda.is_available():
x = x.cuda()
y = y.cuda()
print(x + y)
Vous pouvez alors soumettre une tâche PyTorch avec
[name@server ~]$ sbatch pytorch-test.sh
Dépannage
Fuites de mémoire
Sur le matériel AVX512 (nœuds V100, Skylake ou Béluga), les versions PyTorch antérieures à v1.0.1 qui utilisent des bibliothèques moins récentes (cuDNN < v7.5 ou MAGMA < v2.5) peuvent avoir des fuites de mémoire importantes et créer des exceptions de mémoire insuffisante et terminer vos tâches. Pour contrer ceci, utilisez la plus récente version de torch.
LibTorch
LibTorch allows one to implement both C++ extensions to PyTorch and pure C++ machine learning applications. It contains "all headers, libraries and CMake configuration files required to depend on PyTorch" (as mentioned in the docs).
How to use LibTorch
Get the library
wget https://download.pytorch.org/libtorch/cu100/libtorch-shared-with-deps-latest.zip
unzip libtorch-shared-with-deps-latest.zip
cd libtorch
export LIBTORCH_ROOT=$(pwd) # this variable is used in the example below
Patch the library (this workaround is needed for compiling on Compute Canada clusters):
sed -i -e 's/\/usr\/local\/cuda\/lib64\/libculibos.a;dl;\/usr\/local\/cuda\/lib64\/libculibos.a;//g' share/cmake/Caffe2/Caffe2Targets.cmake
The library is also included in the PyTorch wheel, but this is not the recommended way to acquire it. Once Pytorch is installed in a virtual environment, you can find it at: $VIRTUAL_ENV/lib/python3.6/site-packages/torch/lib/libtorch.so.
Compile a minimal example
Create the following two files:
#include <torch/torch.h>
#include <iostream>
int main() {
torch::Device device(torch::kCPU);
if (torch::cuda::is_available()) {
std::cout << "CUDA is available! Using GPU." << std::endl;
device = torch::Device(torch::kCUDA);
}
torch::Tensor tensor = torch::rand({2, 3}).to(device);
std::cout << tensor << std::endl;
}
cmake_minimum_required(VERSION 3.0 FATAL_ERROR)
project(example-app)
find_package(Torch REQUIRED)
add_executable(example-app example-app.cpp)
target_link_libraries(example-app "${TORCH_LIBRARIES}")
set_property(TARGET example-app PROPERTY CXX_STANDARD 11)
Load the necessary modules:
module load cmake intel/2018.3 cuda/10 cudnn
Compile the program:
mkdir build
cd build
cmake -DCMAKE_PREFIX_PATH="$LIBTORCH_ROOT;$EBROOTCUDA;$EBROOTCUDNN" ..
make
Run the program:
./example-app
To test an application with CUDA, request an interactive job with a GPU.