cc_staff
284
edits
PyKeOps: Difference between revisions
Marked this version for translation
(Added PyKeOps page) |
(Marked this version for translation) |
||
| Line 1: | Line 1: | ||
<languages /> | <languages /> | ||
<translate> | <translate> | ||
<!--T:1--> | |||
__FORCETOC__ | __FORCETOC__ | ||
The [https://www.kernel-operations.io/keops/index.html KeOps] library lets you compute reductions of large arrays whose entries are given by a mathematical formula or a neural network. It combines efficient C++ routines with an automatic differentiation engine and can be used with Python (NumPy, PyTorch), Matlab and R.. | The [https://www.kernel-operations.io/keops/index.html KeOps] library lets you compute reductions of large arrays whose entries are given by a mathematical formula or a neural network. It combines efficient C++ routines with an automatic differentiation engine and can be used with Python (NumPy, PyTorch), Matlab and R.. | ||
= Available versions = | = Available versions = <!--T:2--> | ||
PyKeOps is available on our clusters as prebuilt Python packages (wheels). You can list available versions with <code>avail_wheels</code>. | PyKeOps is available on our clusters as prebuilt Python packages (wheels). You can list available versions with <code>avail_wheels</code>. | ||
{{Command | {{Command | ||
| Line 14: | Line 15: | ||
}} | }} | ||
= Installing PyKeOps in a Python virtual environment = | = Installing PyKeOps in a Python virtual environment = <!--T:3--> | ||
1. Load runtime dependencies. | 1. Load runtime dependencies. | ||
{{Command|module load StdEnv/2023 python/3.11}} | {{Command|module load StdEnv/2023 python/3.11}} | ||
<!--T:4--> | |||
2. Create and activate a Python virtual environment. | 2. Create and activate a Python virtual environment. | ||
{{Commands | {{Commands | ||
| Line 26: | Line 28: | ||
<!--T:5--> | |||
3. Install a specific version of PyKeOps and its Python dependencies. | 3. Install a specific version of PyKeOps and its Python dependencies. | ||
{{Commands | {{Commands | ||
| Line 36: | Line 39: | ||
<!--T:6--> | |||
4. Validate it. | 4. Validate it. | ||
{{Commands | {{Commands | ||
| Line 43: | Line 47: | ||
<!--T:7--> | |||
5. Freeze the environment and requirements set. | 5. Freeze the environment and requirements set. | ||
{{Command | {{Command | ||
| Line 50: | Line 55: | ||
<!--T:8--> | |||
6. Remove the local virtual environment. | 6. Remove the local virtual environment. | ||
{{Command | {{Command | ||
| Line 56: | Line 62: | ||
}} | }} | ||
= Running KeOps = | = Running KeOps = <!--T:9--> | ||
You can run PyKeOps on CPU or GPU. | You can run PyKeOps on CPU or GPU. | ||
<!--T:10--> | |||
1. Write your job submission script. | 1. Write your job submission script. | ||
<tabs> | <tabs> | ||
| Line 68: | Line 75: | ||
#!/bin/bash | #!/bin/bash | ||
<!--T:11--> | |||
#SBATCH --account=def-someprof # adjust this to match the accounting group you are using to submit jobs | #SBATCH --account=def-someprof # adjust this to match the accounting group you are using to submit jobs | ||
#SBATCH --time=08:00:00 # adjust this to match the walltime of your job | #SBATCH --time=08:00:00 # adjust this to match the walltime of your job | ||
| Line 73: | Line 81: | ||
#SBATCH --mem-per-cpu=4G # adjust this according to the memory you need per cpu | #SBATCH --mem-per-cpu=4G # adjust this according to the memory you need per cpu | ||
<!--T:12--> | |||
# Load modules dependencies. | # Load modules dependencies. | ||
module load StdEnv/2023 python/3.11 | module load StdEnv/2023 python/3.11 | ||
<!--T:13--> | |||
# create the virtual environment on the compute node: | # create the virtual environment on the compute node: | ||
virtualenv --no-download $SLURM_TMPDIR/env | virtualenv --no-download $SLURM_TMPDIR/env | ||
source $SLURM_TMPDIR/env/bin/activate | source $SLURM_TMPDIR/env/bin/activate | ||
<!--T:14--> | |||
pip install --no-index --upgrade pip | pip install --no-index --upgrade pip | ||
pip install --no-index -r pykeops-2.2.3-requirements.txt | pip install --no-index -r pykeops-2.2.3-requirements.txt | ||
<!--T:15--> | |||
# test that everything is OK | # test that everything is OK | ||
python -c 'import pykeops; pykeops.test_numpy_bindings()' | python -c 'import pykeops; pykeops.test_numpy_bindings()' | ||
| Line 94: | Line 106: | ||
#!/bin/bash | #!/bin/bash | ||
<!--T:16--> | |||
#SBATCH --account=def-someprof # adjust this to match the accounting group you are using to submit jobs | #SBATCH --account=def-someprof # adjust this to match the accounting group you are using to submit jobs | ||
#SBATCH --time=08:00:00 # adjust this to match the walltime of your job | #SBATCH --time=08:00:00 # adjust this to match the walltime of your job | ||
| Line 100: | Line 113: | ||
#SBATCH --gpus=1 | #SBATCH --gpus=1 | ||
<!--T:17--> | |||
# Load modules dependencies. The custom-ctypes is critical here. | # Load modules dependencies. The custom-ctypes is critical here. | ||
module load StdEnv/2023 python/3.11 cuda/12 custom-ctypes | module load StdEnv/2023 python/3.11 cuda/12 custom-ctypes | ||
<!--T:18--> | |||
# create the virtual environment on the compute node: | # create the virtual environment on the compute node: | ||
virtualenv --no-download $SLURM_TMPDIR/env | virtualenv --no-download $SLURM_TMPDIR/env | ||
source $SLURM_TMPDIR/env/bin/activate | source $SLURM_TMPDIR/env/bin/activate | ||
<!--T:19--> | |||
pip install --no-index --upgrade pip | pip install --no-index --upgrade pip | ||
pip install --no-index -r pykeops-2.2.3-requirements.txt | pip install --no-index -r pykeops-2.2.3-requirements.txt | ||
<!--T:20--> | |||
# test that nvrtc binding are also found | # test that nvrtc binding are also found | ||
python -c 'import pykeops; pykeops.test_numpy_bindings()' | python -c 'import pykeops; pykeops.test_numpy_bindings()' | ||
| Line 116: | Line 133: | ||
</tabs> | </tabs> | ||
<!--T:21--> | |||
2. Submit your job to the scheduler. | 2. Submit your job to the scheduler. | ||
Before submitting your job, it is important to test that your submission script will start without errors. | Before submitting your job, it is important to test that your submission script will start without errors. | ||