MPI4py: Difference between revisions

<languages />

<translate>

[https://mpi4py.readthedocs.io/en/stable/ MPI for Python] provides Python bindings for the Message Passing Interface (MPI) standard, allowing Python applications to exploit multiple processors on workstations, clusters and supercomputers.

__FORCETOC__

<code>mpi4py</code> is available as a module, and not from the [https://docs.alliancecan.ca/wiki/Available_Python_wheels wheelhouse] as typical Python packages do.

One can find available version using:

{{Command|module spider mpi4py}}

and look for more information on a specific version by using:

{{Command|module spider mpi4py/X.Y.Z}}

where <code>X.Y.Z</code> is the exact desired version, for instance <code>4.0.0</code>.  

1. Run a short [https://docs.alliancecan.ca/wiki/Running_jobs#Interactive_jobs interactive job] :

{{Command|salloc --account{{=}}<your account> --ntasks{{=}}5}}

2. Load the module:

{{Command|module load mpi4py/4.0.0 python/3.12}}

3. Run a Hello World test:

{{Command

In the case above, two nodes (<code>node1</code> and <code>node3</code>) were allocated, and the tasks were distributed accross the available ressources.

Often <code>mpi4py</code> is a dependency for another package. In order to fulfill this dependency :

1. Deactivate any Python virtual environment:

{{Command|test $VIRTUAL_ENV && deactivate}}

'''Note:''' If you had a virtual environment activated, it is important to deactivate it first, then load the module, before re-activating your virtual environment.

2. Load the module:

{{Command|module load mpi4py/4.0.0 python/3.12}}

3. Check that it is visible by <code>pip</code>:

{{Command

If no errors are raised, then everything is ok!

4. [https://docs.alliancecan.ca/wiki/Python#Creating_and_using_a_virtual_environment Create a virtual env. and install your packages]

You can run mpi jobs distributed across multiple nodes or cores.  

For efficient MPI scheduling, please see:

* [[Advanced MPI scheduling]]

1. Write your python code, for instance broadcasting a numpy array:

{{File

import numpy as np

comm = MPI.COMM_WORLD

rank = comm.Get_rank()

if rank == 0:

     data = np.arange(100, dtype='i')

     data = np.empty(100, dtype='i')

comm.Bcast(data, root=0)

for i in range(100):

     assert data[i] == i

The example above is based on [https://mpi4py.readthedocs.io/en/stable/tutorial.html#running-python-scripts-with-mpi mpi4py tutorial].

2. Write your submission script:

<tabs>

#!/bin/bash

#SBATCH --account=def-someprof    # adjust this to match the accounting group you are using to submit jobs

#SBATCH --time=08:00:00          # adjust this to match the walltime of your job

#SBATCH --mem-per-cpu=4G          # adjust this according to the memory you need per process

# Run on cores across the system : https://docs.alliancecan.ca/wiki/Advanced_MPI_scheduling#Few_cores,_any_number_of_nodes

# Load modules dependencies.

module load StdEnv/2023 gcc mpi4py/4.0.0 python/3.12

# create the virtual environment on each allocated node:  

srun --ntasks $SLURM_NNODES --tasks-per-node=1 bash << EOF

source $SLURM_TMPDIR/env/bin/activate

pip install --no-index --upgrade pip

pip install --no-index numpy==2.1.1

EOF

# activate only on main node

source $SLURM_TMPDIR/env/bin/activate;

# srun exports the current env, which contains $VIRTUAL_ENV and $PATH variables

srun python mpi4py-np-bc.py;

</tab>

<tab name="Whole nodes">

{{File

#!/bin/bash

#SBATCH --account=def-someprof    # adjust this to match the accounting group you are using to submit jobs

#SBATCH --time=01:00:00          # adjust this to match the walltime of your job

#SBATCH --mem-per-cpu=1G          # adjust this according to the memory you need per process

# Run on N whole nodes : https://docs.alliancecan.ca/wiki/Advanced_MPI_scheduling#Whole_nodes

# Load modules dependencies.

module load StdEnv/2023 gcc openmpi mpi4py/4.0.0 python/3.12

# create the virtual environment on each allocated node:  

srun --ntasks $SLURM_NNODES --tasks-per-node=1 bash << EOF

source $SLURM_TMPDIR/env/bin/activate

pip install --no-index --upgrade pip

pip install --no-index numpy==2.1.1

EOF

# activate only on main node

source $SLURM_TMPDIR/env/bin/activate;

# srun exports the current env, which contains $VIRTUAL_ENV and $PATH variables

srun python mpi4py-np-bc.py;

</tabs>

2. Submit your job to the scheduler.

2.1 Test your script.

Before submitting your job, it is important to test that your submission script will start without errors.

You can do a quick test in an [[Running_jobs#Interactive_jobs|interactive job]].

2.2 Submit your job

{{Command|sbatch submit-mpi4py-distributed.sh}}

1. From a login node, download the demo example:

{{Command

The example above and others, can be found in the [https://github.com/mpi4py/mpi4py/tree/master/demo demo folder].

2. Write your submission script:

{{File

#!/bin/bash

#SBATCH --account=def-someprof    # adjust this to match the accounting group you are using to submit jobs

#SBATCH --time=08:00:00          # adjust this to match the walltime of your job

#SBATCH --gpus=1

# Load modules dependencies.

module load StdEnv/2023 gcc cuda/12 mpi4py/4.0.0 python/3.11

# create the virtual environment on each allocated node:

virtualenv --no-download $SLURM_TMPDIR/env

source $SLURM_TMPDIR/env/bin/activate

pip install --no-index --upgrade pip

pip install --no-index cupy numba

srun python use_cupy.py;

}}

2. Submit your job to the scheduler.

2.1 Test your script.

Before submitting your job, it is important to test that your submission script will start without errors.

You can do a quick test in an [[Running_jobs#Interactive_jobs|interactive job]].

2.2 Submit your job

{{Command|sbatch submit-mpi4py-gpu.sh}}

== ModuleNotFoundError: No module named 'mpi4py' ==

If <code>mpi4py</code> is not accessible, one may get the following error when importing it:

</code>

Possible solution to fix this error:

* check compatible python version with <code>module spider mpi4py/X.Y.Z</code> and that <code>python -c 'import mpi4py'</code> works.

* load the module before activating a virtual environment: please see the above section [[MPI4py#mpi4py as a package dependency]]

And also see [[Python#ModuleNotFoundError:_No_module_named_'X']]

</translate>