Dedalus: Difference between revisions

From Alliance Doc
Jump to navigation Jump to search
(Added Dedalus page)
 
(Marked this version for translation)
Line 1: Line 1:
<languages />
<languages />
<translate>
<translate>
<!--T:1-->
[https://dedalus-project.org/ Dedalus] is a flexible framework for solving partial differential equations using modern spectral methods.
[https://dedalus-project.org/ Dedalus] is a flexible framework for solving partial differential equations using modern spectral methods.


= Available versions =
= Available versions = <!--T:2-->
Dedalus is available on our clusters as prebuilt Python packages (wheels). You can list available versions with <code>avail_wheels</code>.
Dedalus is available on our clusters as prebuilt Python packages (wheels). You can list available versions with <code>avail_wheels</code>.
{{Command
{{Command
Line 15: Line 16:
}}
}}


= Installing Dedalus in a Python virtual environment =
= Installing Dedalus in a Python virtual environment = <!--T:3-->
1. Load Dedalus runtime dependencies.
1. Load Dedalus runtime dependencies.
{{Command|module load StdEnv/2023 gcc openmpi mpi4py/3.1.4 fftw-mpi/3.3.10 hdf5-mpi/1.14.2 python/3.11}}
{{Command|module load StdEnv/2023 gcc openmpi mpi4py/3.1.4 fftw-mpi/3.3.10 hdf5-mpi/1.14.2 python/3.11}}


<!--T:4-->
2. Create and activate a Python virtual environment.
2. Create and activate a Python virtual environment.
{{Commands
{{Commands
Line 25: Line 27:
}}
}}


<!--T:5-->
3. Install a specific version of Dedalus and its Python dependencies.
3. Install a specific version of Dedalus and its Python dependencies.
{{Commands
{{Commands
Line 34: Line 37:
You can omit to specify the version in order to install the latest one available from the wheelhouse.
You can omit to specify the version in order to install the latest one available from the wheelhouse.


<!--T:6-->
4. Validate it.
4. Validate it.
{{Command
{{Command
Line 40: Line 44:
}}
}}


<!--T:7-->
5. Freeze the environment and requirements set.
5. Freeze the environment and requirements set.
{{Command
{{Command
Line 46: Line 51:
}}
}}


<!--T:8-->
6. Remove the local virtual environment.
6. Remove the local virtual environment.
{{Command
{{Command
Line 52: Line 58:
}}
}}


= Running Dedalus =
= Running Dedalus = <!--T:9-->
You can run dedalus distributed accross multiple nodes or cores.  
You can run dedalus distributed accross multiple nodes or cores.  
For efficient MPI scheduling, please see:
For efficient MPI scheduling, please see:
Line 58: Line 64:
* [[Advanced_MPI_scheduling]]
* [[Advanced_MPI_scheduling]]


<!--T:10-->
1. Write your job submission script.
1. Write your job submission script.
<tabs>
<tabs>
Line 67: Line 74:
#!/bin/bash
#!/bin/bash


<!--T:11-->
#SBATCH --account=def-someprof    # adjust this to match the accounting group you are using to submit jobs
#SBATCH --account=def-someprof    # adjust this to match the accounting group you are using to submit jobs
#SBATCH --time=08:00:00          # adjust this to match the walltime of your job
#SBATCH --time=08:00:00          # adjust this to match the walltime of your job
Line 72: Line 80:
#SBATCH --mem-per-cpu=4G          # adjust this according to the memory you need per process
#SBATCH --mem-per-cpu=4G          # adjust this according to the memory you need per process


<!--T:12-->
# Run on cores accross the system : https://docs.alliancecan.ca/wiki/Advanced_MPI_scheduling#Few_cores,_any_number_of_nodes
# Run on cores accross the system : https://docs.alliancecan.ca/wiki/Advanced_MPI_scheduling#Few_cores,_any_number_of_nodes


<!--T:13-->
# Load modules dependencies.
# Load modules dependencies.
module load StdEnv/2023 gcc openmpi mpi4py/3.1.4 fftw-mpi/3.3.10 hdf5-mpi/1.14.2 python/3.11
module load StdEnv/2023 gcc openmpi mpi4py/3.1.4 fftw-mpi/3.3.10 hdf5-mpi/1.14.2 python/3.11


<!--T:14-->
# create the virtual environment on each allocated node:  
# create the virtual environment on each allocated node:  
srun --ntasks $SLURM_NNODES --tasks-per-node=1 bash << EOF
srun --ntasks $SLURM_NNODES --tasks-per-node=1 bash << EOF
Line 82: Line 93:
source $SLURM_TMPDIR/env/bin/activate
source $SLURM_TMPDIR/env/bin/activate


<!--T:15-->
pip install --no-index --upgrade pip
pip install --no-index --upgrade pip
pip install --no-index -r dedalus-3.0.2-requirements.txt
pip install --no-index -r dedalus-3.0.2-requirements.txt
EOF
EOF


<!--T:16-->
# activate only on main node
# activate only on main node
source $SLURM_TMPDIR/env/bin/activate;
source $SLURM_TMPDIR/env/bin/activate;


<!--T:17-->
export OMP_NUM_THREADS=1
export OMP_NUM_THREADS=1


<!--T:18-->
# srun exports the current env, which contains $VIRTUAL_ENV and $PATH variables
# srun exports the current env, which contains $VIRTUAL_ENV and $PATH variables
srun python $SCRATCH/myscript.py;
srun python $SCRATCH/myscript.py;
Line 96: Line 111:
</tab>
</tab>


<!--T:19-->
<tab name="Whole nodes">
<tab name="Whole nodes">
{{File
{{File
Line 103: Line 119:
#!/bin/bash
#!/bin/bash


<!--T:20-->
#SBATCH --account=def-someprof    # adjust this to match the accounting group you are using to submit jobs
#SBATCH --account=def-someprof    # adjust this to match the accounting group you are using to submit jobs
#SBATCH --time=08:00:00          # adjust this to match the walltime of your job
#SBATCH --time=08:00:00          # adjust this to match the walltime of your job
Line 109: Line 126:
#SBATCH --mem-per-cpu=4G          # adjust this according to the memory you need per process
#SBATCH --mem-per-cpu=4G          # adjust this according to the memory you need per process


<!--T:21-->
# Run on N whole nodes : https://docs.alliancecan.ca/wiki/Advanced_MPI_scheduling#Whole_nodes
# Run on N whole nodes : https://docs.alliancecan.ca/wiki/Advanced_MPI_scheduling#Whole_nodes


<!--T:22-->
# Load modules dependencies.
# Load modules dependencies.
module load StdEnv/2023 gcc openmpi mpi4py/3.1.4 fftw-mpi/3.3.10 hdf5-mpi/1.14.2 python/3.11
module load StdEnv/2023 gcc openmpi mpi4py/3.1.4 fftw-mpi/3.3.10 hdf5-mpi/1.14.2 python/3.11


<!--T:23-->
# create the virtual environment on each allocated node:  
# create the virtual environment on each allocated node:  
srun --ntasks $SLURM_NNODES --tasks-per-node=1 bash << EOF
srun --ntasks $SLURM_NNODES --tasks-per-node=1 bash << EOF
Line 119: Line 139:
source $SLURM_TMPDIR/env/bin/activate
source $SLURM_TMPDIR/env/bin/activate


<!--T:24-->
pip install --no-index --upgrade pip
pip install --no-index --upgrade pip
pip install --no-index -r dedalus-3.0.2-requirements.txt
pip install --no-index -r dedalus-3.0.2-requirements.txt
EOF
EOF


<!--T:25-->
# activate only on main node
# activate only on main node
source $SLURM_TMPDIR/env/bin/activate;
source $SLURM_TMPDIR/env/bin/activate;


<!--T:26-->
export OMP_NUM_THREADS=1
export OMP_NUM_THREADS=1


<!--T:27-->
# srun exports the current env, which contains $VIRTUAL_ENV and $PATH variables
# srun exports the current env, which contains $VIRTUAL_ENV and $PATH variables
srun python $SCRATCH/myscript.py;
srun python $SCRATCH/myscript.py;
Line 134: Line 158:
</tabs>
</tabs>


<!--T:28-->
2. Submit your job to the scheduler.
2. Submit your job to the scheduler.
{{Command
{{Command

Revision as of 11:18, 24 April 2024

Other languages:

Dedalus is a flexible framework for solving partial differential equations using modern spectral methods.

Available versions[edit]

Dedalus is available on our clusters as prebuilt Python packages (wheels). You can list available versions with avail_wheels.

Question.png 
[name@server ~]$ avail_wheels dedalus --all-versions
$ avail_wheels dedalus
name     version    python    arch
-------  ---------  --------  ---------
dedalus  3.0.2      cp311     x86-64-v3
dedalus  3.0.2      cp310     x86-64-v3

Installing Dedalus in a Python virtual environment[edit]

1. Load Dedalus runtime dependencies.

Question.png 
[name@server ~]$ module load StdEnv/2023 gcc openmpi mpi4py/3.1.4 fftw-mpi/3.3.10 hdf5-mpi/1.14.2 python/3.11

2. Create and activate a Python virtual environment. 

[name@server ~]$ virtualenv --no-download ~/dedalus_env
[name@server ~]$ source ~/dedalus_env/bin/activate


3. Install a specific version of Dedalus and its Python dependencies. 

(dedalus_env) [name@server ~] pip install --no-index --upgrade pip
(dedalus_env) [name@server ~] pip install --no-index dedalus==X.Y.Z

where X.Y.Z is the exact desired version, for instance 3.0.2. You can omit to specify the version in order to install the latest one available from the wheelhouse.

4. Validate it.

Question.png 
(dedalus_env) [name@server ~] python -c 'import dedalus'

5. Freeze the environment and requirements set.

Question.png 
(dedalus_env) [name@server ~] pip freeze --local > ~/dedalus-3.0.2-requirements.txt

6. Remove the local virtual environment.

Question.png 
(dedalus_env) [name@server ~] deactivate && rm -r ~/dedalus_env

Running Dedalus[edit]

You can run dedalus distributed accross multiple nodes or cores. For efficient MPI scheduling, please see:

1. Write your job submission script.

File : submit-dedalus-distributed.sh

#!/bin/bash

#SBATCH --account=def-someprof    # adjust this to match the accounting group you are using to submit jobs
#SBATCH --time=08:00:00           # adjust this to match the walltime of your job
#SBATCH --ntasks=4                # adjust this to match the number of tasks/processes to run
#SBATCH --mem-per-cpu=4G          # adjust this according to the memory you need per process

# Run on cores accross the system : https://docs.alliancecan.ca/wiki/Advanced_MPI_scheduling#Few_cores,_any_number_of_nodes

# Load modules dependencies.
module load StdEnv/2023 gcc openmpi mpi4py/3.1.4 fftw-mpi/3.3.10 hdf5-mpi/1.14.2 python/3.11

# create the virtual environment on each allocated node: 
srun --ntasks $SLURM_NNODES --tasks-per-node=1 bash << EOF
virtualenv --no-download $SLURM_TMPDIR/env
source $SLURM_TMPDIR/env/bin/activate

pip install --no-index --upgrade pip
pip install --no-index -r dedalus-3.0.2-requirements.txt
EOF

# activate only on main node
source $SLURM_TMPDIR/env/bin/activate;

export OMP_NUM_THREADS=1

# srun exports the current env, which contains $VIRTUAL_ENV and $PATH variables
srun python $SCRATCH/myscript.py;


File : submit-dedalus-whole-nodes.sh

#!/bin/bash

#SBATCH --account=def-someprof    # adjust this to match the accounting group you are using to submit jobs
#SBATCH --time=08:00:00           # adjust this to match the walltime of your job
#SBATCH --nodes=2                 # adjust this to match the number of whole node
#SBATCH --ntasks-per-node=4       # adjust this to match the number of tasks/processes to run per node
#SBATCH --mem-per-cpu=4G          # adjust this according to the memory you need per process

# Run on N whole nodes : https://docs.alliancecan.ca/wiki/Advanced_MPI_scheduling#Whole_nodes

# Load modules dependencies.
module load StdEnv/2023 gcc openmpi mpi4py/3.1.4 fftw-mpi/3.3.10 hdf5-mpi/1.14.2 python/3.11

# create the virtual environment on each allocated node: 
srun --ntasks $SLURM_NNODES --tasks-per-node=1 bash << EOF
virtualenv --no-download $SLURM_TMPDIR/env
source $SLURM_TMPDIR/env/bin/activate

pip install --no-index --upgrade pip
pip install --no-index -r dedalus-3.0.2-requirements.txt
EOF

# activate only on main node
source $SLURM_TMPDIR/env/bin/activate;

export OMP_NUM_THREADS=1

# srun exports the current env, which contains $VIRTUAL_ENV and $PATH variables
srun python $SCRATCH/myscript.py;


2. Submit your job to the scheduler.

Question.png 
[name@server ~]$ sbatch submit-dedalus.sh

Message Passing Interface

Retrieved from "https://docs.alliancecan.ca/mediawiki/index.php?title=Dedalus&oldid=152347"