PyKeOps/fr: Difference between revisions

From Alliance Doc
Jump to navigation Jump to search
(Created page with "PyKeOps")
 
No edit summary
 
(27 intermediate revisions by 2 users not shown)
Line 1: Line 1:
<languages />
<languages />
<div lang="en" dir="ltr" class="mw-content-ltr">
__FORCETOC__
__FORCETOC__
The [https://www.kernel-operations.io/keops/index.html KeOps] library lets you compute reductions of large arrays whose entries are given by a mathematical formula or a neural network. It combines efficient C++ routines with an automatic differentiation engine and can be used with Python (NumPy, PyTorch), Matlab and R..
La bibliothèque [https://www.kernel-operations.io/keops/index.html KeOps] permet de calculer des réductions de tableaux de grande taille dont les entrées sont des formules mathématiques ou des réseaux de neurones. Elle combine des routines C++ efficaces avec un moteur de différentiation automatique et peut être utilisée avec [[Python/fr|Python]] ([https://numpy.org/doc/stable/ NumPy], [[PyTorch/fr|PyTorch]]), [[MATLAB/fr|MATLAB]] et [[R/fr|R]].
</div>


<div lang="en" dir="ltr" class="mw-content-ltr">
= Versions disponibles =  
= Available versions =
Les versions disponibles sur nos grappes sont des wheels Python. Voyez la liste en lançant <code>avail_wheels</code>.
PyKeOps is available on our clusters as prebuilt Python packages (wheels). You can list available versions with <code>avail_wheels</code>.
{{Command
{{Command
|avail_wheels pykeops
|avail_wheels pykeops
Line 15: Line 12:
pykeops  2.2.3      py3      generic
pykeops  2.2.3      py3      generic
}}
}}
</div>


<div lang="en" dir="ltr" class="mw-content-ltr">
= Installation dans un environnement virtuel Python =
= Installing PyKeOps in a Python virtual environment =
1. Chargez les dépendances.
1. Load runtime dependencies.
{{Command|module load StdEnv/2023 python/3.11}}
{{Command|module load StdEnv/2023 python/3.11}}
</div>




<div lang="en" dir="ltr" class="mw-content-ltr">
2. Créez et activez un [[Python/fr#Créer_et_utiliser_un_environnement_virtuel | environnement virtuel Python]].
2. Create and activate a Python virtual environment.
{{Commands
{{Commands
|virtualenv --no-download ~/pykeops_env
|virtualenv --no-download ~/pykeops_env
|source ~/pykeops_env/bin/activate
|source ~/pykeops_env/bin/activate
}}
}}
</div>




<div lang="en" dir="ltr" class="mw-content-ltr">
3. Installez une version de PyKeOps avec ses dépendances Python.
3. Install a specific version of PyKeOps and its Python dependencies.
{{Commands
{{Commands
|prompt=(pykeops_env) [name@server ~]
|prompt=(pykeops_env) [name@server ~]
Line 40: Line 31:
|pip install --no-index pykeops{{=}}{{=}}X.Y.Z
|pip install --no-index pykeops{{=}}{{=}}X.Y.Z
}}
}}
where <code>X.Y.Z</code> is the exact desired version, for instance <code>2.2.3</code>.  
où <code>X.Y.Z</code> est le numéro de la version, par exemple <code>2.2.3</code>.  
You can omit to specify the version in order to install the latest one available from the wheelhouse.
Pour installer la plus récente version, n'entrez pas de numéro.
</div>




<div lang="en" dir="ltr" class="mw-content-ltr">
4. Validez.
4. Validate it.
{{Commands
{{Commands
|prompt=(pykeops_env) [name@server ~]
|prompt=(pykeops_env) [name@server ~]
|python -c 'import pykeops; pykeops.test_numpy_bindings()'
|python -c 'import pykeops; pykeops.test_numpy_bindings()'
}}
}}
</div>




<div lang="en" dir="ltr" class="mw-content-ltr">
5. Gelez l'environnement et l'ensemble des exigences.
5. Freeze the environment and requirements set.
{{Command
{{Command
|prompt=(pykeops_env) [name@server ~]
|prompt=(pykeops_env) [name@server ~]
|pip freeze --local > ~/pykeops-2.2.3-requirements.txt
|pip freeze --local > ~/pykeops-2.2.3-requirements.txt
}}
}}
</div>




<div lang="en" dir="ltr" class="mw-content-ltr">
6. Supprimez l'environnement virtuel local.
6. Remove the local virtual environment.
{{Command
{{Command
|prompt=(pykeops_env) [name@server ~]
|prompt=(pykeops_env) [name@server ~]
|deactivate && rm -r ~/pykeops_env
|deactivate && rm -r ~/pykeops_env
}}
}}
</div>


<div lang="en" dir="ltr" class="mw-content-ltr">
= Exécution =  
= Running KeOps =
Vous pouvez exécuter PyKeOps sur un CPU ou un GPU.  
You can run PyKeOps on CPU or GPU.
</div>


<div lang="en" dir="ltr" class="mw-content-ltr">
1. Préparez votre script d'exécution.
1. Write your job submission script.
<tabs>
<tabs>
<tab name="CPU">
<tab name="CPU">
Line 85: Line 66:
|contents=
|contents=
#!/bin/bash
#!/bin/bash
</div>


#SBATCH --account=def-someprof    # adjust this to match the accounting group you are using to submit jobs
#SBATCH --account=def-someprof    # adjust this to match the accounting group you are using to submit jobs
Line 92: Line 72:
#SBATCH --mem-per-cpu=4G          # adjust this according to the memory you need per cpu
#SBATCH --mem-per-cpu=4G          # adjust this according to the memory you need per cpu


<div lang="en" dir="ltr" class="mw-content-ltr">
# Load modules dependencies.
# Load modules dependencies.
module load StdEnv/2023 python/3.11
module load StdEnv/2023 python/3.11
</div>


<div lang="en" dir="ltr" class="mw-content-ltr">
# create the virtual environment on the compute node:  
# create the virtual environment on the compute node:  
virtualenv --no-download $SLURM_TMPDIR/env
virtualenv --no-download $SLURM_TMPDIR/env
source $SLURM_TMPDIR/env/bin/activate
source $SLURM_TMPDIR/env/bin/activate
</div>


<div lang="en" dir="ltr" class="mw-content-ltr">
pip install --no-index --upgrade pip
pip install --no-index --upgrade pip
pip install --no-index -r pykeops-2.2.3-requirements.txt
pip install --no-index -r pykeops-2.2.3-requirements.txt
</div>


<div lang="en" dir="ltr" class="mw-content-ltr">
# test that everything is OK
# test that everything is OK
python -c 'import pykeops; pykeops.test_numpy_bindings()'
python -c 'import pykeops; pykeops.test_numpy_bindings()'
Line 119: Line 92:
|contents=
|contents=
#!/bin/bash
#!/bin/bash
</div>


<div lang="en" dir="ltr" class="mw-content-ltr">
#SBATCH --account=def-someprof    # adjust this to match the accounting group you are using to submit jobs
#SBATCH --account=def-someprof    # adjust this to match the accounting group you are using to submit jobs
#SBATCH --time=08:00:00          # adjust this to match the walltime of your job
#SBATCH --time=08:00:00          # adjust this to match the walltime of your job
Line 127: Line 98:
#SBATCH --mem-per-cpu=4G          # adjust this according to the memory you need per cpu
#SBATCH --mem-per-cpu=4G          # adjust this according to the memory you need per cpu
#SBATCH --gpus=1
#SBATCH --gpus=1
</div>


<div lang="en" dir="ltr" class="mw-content-ltr">
# Load modules dependencies. The custom-ctypes is critical here.
# Load modules dependencies. The custom-ctypes is critical here.
module load StdEnv/2023 python/3.11 cuda/12 custom-ctypes
module load StdEnv/2023 python/3.11 cuda/12 custom-ctypes
</div>


<div lang="en" dir="ltr" class="mw-content-ltr">
# create the virtual environment on the compute node:  
# create the virtual environment on the compute node:  
virtualenv --no-download $SLURM_TMPDIR/env
virtualenv --no-download $SLURM_TMPDIR/env
source $SLURM_TMPDIR/env/bin/activate
source $SLURM_TMPDIR/env/bin/activate
</div>


<div lang="en" dir="ltr" class="mw-content-ltr">
pip install --no-index --upgrade pip
pip install --no-index --upgrade pip
pip install --no-index -r pykeops-2.2.3-requirements.txt
pip install --no-index -r pykeops-2.2.3-requirements.txt
</div>


<div lang="en" dir="ltr" class="mw-content-ltr">
# test that nvrtc binding are also found
# test that nvrtc binding are also found
python -c 'import pykeops; pykeops.test_numpy_bindings()'
python -c 'import pykeops; pykeops.test_numpy_bindings()'
Line 151: Line 114:
</tab>
</tab>
</tabs>
</tabs>
</div>


<div lang="en" dir="ltr" class="mw-content-ltr">
2. Pour savoir si votre script contient des erreurs, testez-le avec une  [[Running_jobs/fr#Tâches_interactives|tâche interactive]].
2. Submit your job to the scheduler.
 
Before submitting your job, it is important to test that your submission script will start without errors.
3. Soumettez la tâche à l'ordonnanceur.
You can do a quick test in an [[Running_jobs#Interactive_jobs|interactive job]].
{{Command
{{Command
|sbatch submit-keops.sh
|sbatch submit-keops.sh
}}
}}
</div>

Latest revision as of 19:03, 30 September 2024

Other languages:

La bibliothèque KeOps permet de calculer des réductions de tableaux de grande taille dont les entrées sont des formules mathématiques ou des réseaux de neurones. Elle combine des routines C++ efficaces avec un moteur de différentiation automatique et peut être utilisée avec Python (NumPy, PyTorch), MATLAB et R.

Versions disponibles

Les versions disponibles sur nos grappes sont des wheels Python. Voyez la liste en lançant avail_wheels.

Question.png 
[name@server ~]$ avail_wheels pykeops
name     version    python    arch
-------  ---------  --------  -------
pykeops  2.2.3      py3       generic

Installation dans un environnement virtuel Python

1. Chargez les dépendances.

Question.png 
[name@server ~]$ module load StdEnv/2023 python/3.11


2. Créez et activez un environnement virtuel Python. 

[name@server ~]$ virtualenv --no-download ~/pykeops_env
[name@server ~]$ source ~/pykeops_env/bin/activate


3. Installez une version de PyKeOps avec ses dépendances Python. 

(pykeops_env) [name@server ~] pip install --no-index --upgrade pip
(pykeops_env) [name@server ~] pip install --no-index pykeops==X.Y.Z

où X.Y.Z est le numéro de la version, par exemple 2.2.3. Pour installer la plus récente version, n'entrez pas de numéro.


4. Validez. 

(pykeops_env) [name@server ~] python -c 'import pykeops; pykeops.test_numpy_bindings()'


5. Gelez l'environnement et l'ensemble des exigences.

Question.png 
(pykeops_env) [name@server ~] pip freeze --local > ~/pykeops-2.2.3-requirements.txt


6. Supprimez l'environnement virtuel local.

Question.png 
(pykeops_env) [name@server ~] deactivate && rm -r ~/pykeops_env

Exécution

Vous pouvez exécuter PyKeOps sur un CPU ou un GPU.

1. Préparez votre script d'exécution.

File : submit-pykeops-cpu.sh

#!/bin/bash

#SBATCH --account=def-someprof    # adjust this to match the accounting group you are using to submit jobs
#SBATCH --time=08:00:00           # adjust this to match the walltime of your job
#SBATCH --cpus-per-task=4         # adjust this to match the number of cores to use
#SBATCH --mem-per-cpu=4G          # adjust this according to the memory you need per cpu

# Load modules dependencies.
module load StdEnv/2023 python/3.11

# create the virtual environment on the compute node: 
virtualenv --no-download $SLURM_TMPDIR/env
source $SLURM_TMPDIR/env/bin/activate

pip install --no-index --upgrade pip
pip install --no-index -r pykeops-2.2.3-requirements.txt

# test that everything is OK
python -c 'import pykeops; pykeops.test_numpy_bindings()'


File : submit-pykeops-gpu.sh

#!/bin/bash

#SBATCH --account=def-someprof    # adjust this to match the accounting group you are using to submit jobs
#SBATCH --time=08:00:00           # adjust this to match the walltime of your job
#SBATCH --cpus-per-task=4         # adjust this to match the number of cores to use
#SBATCH --mem-per-cpu=4G          # adjust this according to the memory you need per cpu
#SBATCH --gpus=1

# Load modules dependencies. The custom-ctypes is critical here.
module load StdEnv/2023 python/3.11 cuda/12 custom-ctypes

# create the virtual environment on the compute node: 
virtualenv --no-download $SLURM_TMPDIR/env
source $SLURM_TMPDIR/env/bin/activate

pip install --no-index --upgrade pip
pip install --no-index -r pykeops-2.2.3-requirements.txt

# test that nvrtc binding are also found
python -c 'import pykeops; pykeops.test_numpy_bindings()'


2. Pour savoir si votre script contient des erreurs, testez-le avec une tâche interactive.

3. Soumettez la tâche à l'ordonnanceur.

Question.png 
[name@server ~]$ sbatch submit-keops.sh
Retrieved from "https://docs.alliancecan.ca/mediawiki/index.php?title=PyKeOps/fr&oldid=161086"