AMBER: Difference between revisions

From Alliance Doc
Jump to navigation Jump to search
(Marked this version for translation)
 
(29 intermediate revisions by 3 users not shown)
Line 9: Line 9:


<!--T:21-->
<!--T:21-->
* The [https://ambermd.org/AmberTools.php AmberTools] (module <code>ambertools</code>) contain a number of tools for preparing and analysing simulations, as well as <code>sander</code> to perform molecular dynamics simulations, all of which are free and open source.
* The [https://ambermd.org/AmberTools.php AmberTools] (module <code>ambertools</code>) contains a number of tools for preparing and analyzing simulations, as well as <code>sander</code> to perform molecular dynamics simulations, all of which are free and open source.
* [https://ambermd.org/AmberMD.php Amber] (module <code>amber</code>) contains everything that is included in <code>ambertools</code>, but adds the advanced <code>pmemd</code> program for molecular dynamics simulations.
* [https://ambermd.org/AmberMD.php Amber] (module <code>amber</code>) contains everything that is included in <code>ambertools</code>, but adds the advanced <code>pmemd</code> program for molecular dynamics simulations.


Line 15: Line 15:
To see a list of installed versions and which other modules they depend on, you can use the <code>module spider</code> [[Using modules#Sub-command_spider|command]] or check the [[Available software]] page.
To see a list of installed versions and which other modules they depend on, you can use the <code>module spider</code> [[Using modules#Sub-command_spider|command]] or check the [[Available software]] page.


== Loading modules == <!--T:42-->
== Loading modules == <!--T:42-->
<tabs>
<tabs>
<tab name="StdEnv/2023">
{| class="wikitable sortable"
|-
! AMBER version !! modules for running on CPUs !! modules for running on GPUs (CUDA) !! Notes
|-
| amber/22.5-23.5 || <code> StdEnv/2023 gcc/12.3 openmpi/4.1.5 amber/22.5-23.5</code> || <code>StdEnv/2023 gcc/12.3 openmpi/4.1.5 cuda/12.2 amber/22.5-23.5</code> || GCC, FlexiBLAS & FFTW
|-
| ambertools/23.5 || <code> StdEnv/2023 gcc/12.3 openmpi/4.1.5 ambertools/23.5</code> || <code>StdEnv/2023 gcc/12.3 openmpi/4.1.5 cuda/12.2 ambertools/23.5</code> || GCC, FlexiBLAS & FFTW
|-
|}</tab>
<tab name="StdEnv/2020">
<tab name="StdEnv/2020">
{| class="wikitable sortable"
{| class="wikitable sortable"
Line 56: Line 66:


===Amber 20=== <!--T:30-->
===Amber 20=== <!--T:30-->
There are two versions of amber/20 modules: 20.9-20.15 and 20.12-20.15. The first one uses MKL and cuda/11.0, while the second uses FlexiBLAS and cuda/11.4. MKL libraries do not perform well on AMD CPU, and FlexiBLAS solves this problem. It detects CPU type and uses libraries optimized for the hardware. CUDA/11.4 is required for running simulations on A100 GPUs installed on Narval.  
There are two versions of amber/20 modules: 20.9-20.15 and 20.12-20.15. The first one uses MKL and cuda/11.0, while the second uses FlexiBLAS and cuda/11.4. MKL libraries do not perform well on AMD CPU, and FlexiBLAS solves this problem. It detects CPU type and uses libraries optimized for the hardware. cuda/11.4 is required for running simulations on A100 GPUs installed on Narval.  


<!--T:45-->
<!--T:45-->
CPU-only modules provide all MD programs available in AmberTools/20 plus pmemd (serial) and pmemd.MPI (parallel). GPU modules add pmemd.cuda (single GPU), and pmemd.cuda.MPI (multi - GPU).
CPU-only modules provide all MD programs available in AmberTools/20 plus pmemd (serial) and pmemd.MPI (parallel). GPU modules add pmemd.cuda (single GPU), and pmemd.cuda.MPI (multi - GPU).


=== Known issues ===  
=== Known issues === <!--T:41-->
<!--T:41-->
1. Module amber/20.12-20.15 does not have MMPBSA.py.MPI executable.
1. Module amber/20.12-20.15 does not have MMPBSA.py.MPI executable.


<!--T:46-->
<!--T:46-->
2. MMPBSA.py from amber/18-10-18.11 and amber/18.14-18.17 modules can not perform PB calculations. Use more recent amber/20 modules for this type of calculations.
2. MMPBSA.py from amber/18-10-18.11 and amber/18.14-18.17 modules cannot perform PB calculations. Use more recent amber/20 modules for this type of calculations.


==Job submission examples==
==Job submission examples== <!--T:37-->
=== Single GPU job === <!--T:37-->
=== Single GPU job ===
For GPU-accelerated simulations on Narval, use amber/20.12-20.15. Modules compiled with cuda version < 11.4 do not work on A100 GPUs. Below is an example submission script for a single-GPU job with amber/20.12-20.15.
For GPU-accelerated simulations on Narval, use amber/20.12-20.15. Modules compiled with CUDA version < 11.4 do not work on A100 GPUs. Below is an example submission script for a single-GPU job with amber/20.12-20.15.
{{File
{{File
   |name=pmemd_cuda.sh
   |name=pmemd_cuda_job.sh
   |lang="bash"
   |lang="bash"
   |contents=
   |contents=
#!/bin/bash
#!/bin/bash
#SBATCH --cpus-per-task=1  
#SBATCH --ntasks=1  
#SBATCH --gpus-per-node=1  
#SBATCH --gpus-per-node=1  
#SBATCH --mem-per-cpu=2000  
#SBATCH --mem-per-cpu=2000  
#SBATCH --time=10:0:0 
#SBATCH --time=10:00:00
 
<!--T:54-->
module purge
module purge
module load StdEnv/2020 gcc/9.3.0 cuda/11.4 openmpi/4.0.3 amber/20.12-20.15
module load StdEnv/2023 gcc/12.3 openmpi/4.1.5 cuda/12.2 amber/22
 
<!--T:55-->
pmemd.cuda -O -i input.in -p topol.parm7 -c coord.rst7 -o output.mdout -r restart.rst7
pmemd.cuda -O -i input.in -p topol.parm7 -c coord.rst7 -o output.mdout -r restart.rst7
}}
}}


=== CPU-only parallel MPI job ===
=== CPU-only parallel MPI job === <!--T:47-->
The example below requests four full nodes on Narval (64 tasks per node). If --nodes=4 is omitted SLURM will decide how many nodes to use based on availability.
 
<!--T:56-->
<tabs>
<tab name="Graham">
{{File
{{File
   |name=pmemd_MPI.sh
   |name=pmemd_MPI_job_graham.sh
   |lang="bash"
   |lang="sh"
  |contents=
#!/bin/bash
#SBATCH --nodes=4
#SBATCH --ntasks-per-node=32
#SBATCH --mem-per-cpu=2000
#SBATCH --time=1:00:00
 
<!--T:57-->
module purge
module load StdEnv/2023 gcc/12.3 openmpi/4.1.5 amber/22
 
<!--T:58-->
srun pmemd.MPI -O -i input.in -p topol.parm7 -c coord.rst7 -o output.mdout -r restart.rst7
}}</tab>
<tab name="Cedar">
{{File
  |name=pmemd_MPI_job_cedar.sh
  |lang="sh"
  |contents=
#!/bin/bash
#SBATCH --nodes=4
#SBATCH --ntasks-per-node=48
#SBATCH --mem-per-cpu=2000
#SBATCH --time=1:00:00
 
<!--T:59-->
module purge
module load StdEnv/2023 gcc/12.3 openmpi/4.1.5 amber/22
 
<!--T:60-->
srun pmemd.MPI -O -i input.in -p topol.parm7 -c coord.rst7 -o output.mdout -r restart.rst7
}}</tab>
<tab name="Béluga">
{{File
  |name=pmemd_MPI_job_beluga.sh
  |lang="sh"
  |contents=
#!/bin/bash
#SBATCH --nodes=4
#SBATCH --ntasks-per-node=40
#SBATCH --mem-per-cpu=2000
#SBATCH --time=1:00:00
 
<!--T:61-->
module purge
module load StdEnv/2023 gcc/12.3 openmpi/4.1.5 amber/22
 
<!--T:62-->
srun pmemd.MPI -O -i input.in -p topol.parm7 -c coord.rst7 -o output.mdout -r restart.rst7
}}</tab>
<tab name="Narval">
{{File
  |name=pmemd_MPI_job_narval.sh
  |lang="sh"
  |contents=
#!/bin/bash
#SBATCH --nodes=4
#SBATCH --ntasks-per-node=64
#SBATCH --mem-per-cpu=2000
#SBATCH --time=1:00:00
 
<!--T:63-->
module purge
module load StdEnv/2023 gcc/12.3 openmpi/4.1.5 amber/22
 
<!--T:64-->
srun pmemd.MPI -O -i input.in -p topol.parm7 -c coord.rst7 -o output.mdout -r restart.rst7
}}</tab>
<tab name="Niagara">
{{File
  |name=pmemd_MPI_job_narval.sh
  |lang="sh"
   |contents=
   |contents=
#!/bin/bash
#!/bin/bash
#SBATCH --nodes=4
#SBATCH --nodes=4
#SBATCH --ntasks=512
#SBATCH --ntasks-per-node=40
#SBATCH --mem-per-cpu=2000
#SBATCH --mem-per-cpu=2000
#SBATCH --time=1:0:0
#SBATCH --time=1:00:00
 
<!--T:65-->
module purge
module purge
module load StdEnv/2020 gcc/9.3.0 cuda/11.4 openmpi/4.0.3 amber/20.12-20.15
module load StdEnv/2023 gcc/12.3 openmpi/4.1.5 amber/22
 
<!--T:66-->
srun pmemd.MPI -O -i input.in -p topol.parm7 -c coord.rst7 -o output.mdout -r restart.rst7
srun pmemd.MPI -O -i input.in -p topol.parm7 -c coord.rst7 -o output.mdout -r restart.rst7
}}
}}</tab>
</tabs>


=== QM/MM distributed multi-GPU job ===
=== QM/MM distributed multi-GPU job === <!--T:48-->
The example below requests eight GPUs.
The example below requests eight GPUs.
{{File
{{File
   |name=pmemd_MPI.sh
   |name=quick_MPI_job.sh
   |lang="bash"
   |lang="bash"
   |contents=
   |contents=
#!/bin/bash
#!/bin/bash
#SBATCH --ntasks=8  
#SBATCH --ntasks=8 --cpus-per-task=1
#SBATCH --cpus-per-task=1  
#SBATCH --gpus-per-task=1  
#SBATCH --gpus-per-task=1
#SBATCH --mem-per-cpu=4000  
#SBATCH --mem-per-cpu=4000  
#SBATCH --time=1:00:00
#SBATCH --time=02:00:00
 
<!--T:52-->
module purge
module purge
module load StdEnv/2020 gcc/9.3.0 cuda/11.4 openmpi/4.0.3 ambertools/21
module load StdEnv/2023 gcc/12.3 openmpi/4.1.5 cuda/12.2 ambertools/23.5
source $EBROOTAMBERTOOLS/amber.sh
 
<!--T:53-->
srun sander.quick.cuda.MPI -O -i input.in -p topol.parm7 -c coord.rst7 -o output.mdout -r restart.rst7
srun sander.quick.cuda.MPI -O -i input.in -p topol.parm7 -c coord.rst7 -o output.mdout -r restart.rst7
}}
}}


=== Parallel MMPBSA job ===
=== Parallel MMPBSA job === <!--T:6-->
The example below uses 32 MPI processes. MMPBSA scales linearly because each trajectory frame is processed independently.  
The example below uses 32 MPI processes. MMPBSA scales linearly because each trajectory frame is processed independently.  
{{File
{{File
   |name=pmemd_MPI.sh
   |name=mmpbsa_job.sh
   |lang="bash"
   |lang="bash"
   |contents=
   |contents=
Line 129: Line 224:
#SBATCH --mem-per-cpu=4000  
#SBATCH --mem-per-cpu=4000  
#SBATCH --time=1:00:00
#SBATCH --time=1:00:00
<!--T:67-->
module purge
module purge
module load StdEnv/2020 gcc/9.3.0 openmpi/4.0.3 amber/20.9-20.15 scipy-stack
module load module load StdEnv/2023 gcc/12.3 openmpi/4.1.5 amber/22
 
<!--T:68-->
srun MMPBSA.py.MPI -O -i mmpbsa.in -o mmpbsa.dat -sp solvated_complex.parm7 -cp complex.parm7 -rp receptor.parm7 -lp ligand.parm7 -y trajectory.nc
srun MMPBSA.py.MPI -O -i mmpbsa.in -o mmpbsa.dat -sp solvated_complex.parm7 -cp complex.parm7 -rp receptor.parm7 -lp ligand.parm7 -y trajectory.nc
}}
}}
<!--T:6-->
You can modify scripts to fit your simulation requirements for computing resources. See [[Running jobs]] for more details.
You can modify scripts to fit your simulation requirements for computing resources. See [[Running jobs]] for more details.
</translate>


==Performance==
==Performance and benchmarking== <!--T:69-->
For a quick estimate of the time and resources required for a simulation, visit our new Molecular Dynamics Performance Guide[https://mdbench.ace-net.ca]. or follow direct links to AMBER benchmarks below.
 
<!--T:70-->
A team at [https://www.ace-net.ca/ ACENET] has created a [https://mdbench.ace-net.ca/mdbench/ Molecular Dynamics Performance Guide] for Alliance clusters.
It can help you determine optimal conditions for AMBER, GROMACS, NAMD, and OpenMM jobs. The present section focuses on AMBER performance.


View benchmarks of simulations with PMEMD[http://localhost:8000/mdbench/bform/?software_contains=PMEMD&software_id=&module_contains=&module_version=&site_contains=&gpu_model=&cpu_model=&arch=&dataset=6n4o]  
<!--T:50-->
View benchmarks of simulations with PMEMD[http://mdbench.ace-net.ca/mdbench/bform/?software_contains=PMEMD&software_id=&module_contains=&module_version=&site_contains=&gpu_model=&cpu_model=&arch=&dataset=6n4o]  


View benchmarks of QM/MM simulations with SANDER.QUICK [http://localhost:8000/mdbench/bform/?software_contains=&software_id=&module_contains=&module_version=&site_contains=&gpu_model=&cpu_model=&arch=&dataset=4cg1].
<!--T:51-->
View benchmarks of QM/MM simulations with SANDER.QUICK [http://mdbench.ace-net.ca/mdbench/bform/?software_contains=&software_id=&module_contains=&module_version=&site_contains=&gpu_model=&cpu_model=&arch=&dataset=4cg1].
</translate>

Latest revision as of 16:55, 16 October 2024

Other languages:

Introduction[edit]

Amber is the collective name for a suite of programs that allow users to perform molecular dynamics simulations, particularly on biomolecules. None of the individual programs carry this name, but the various parts work reasonably well together, and provide a powerful framework for many common calculations.

Amber vs. AmberTools[edit]

We have modules for both Amber and AmberTools available in our software stack.

  • The AmberTools (module ambertools) contains a number of tools for preparing and analyzing simulations, as well as sander to perform molecular dynamics simulations, all of which are free and open source.
  • Amber (module amber) contains everything that is included in ambertools, but adds the advanced pmemd program for molecular dynamics simulations.

To see a list of installed versions and which other modules they depend on, you can use the module spider command or check the Available software page.

Loading modules[edit]

AMBER version modules for running on CPUs modules for running on GPUs (CUDA) Notes
amber/22.5-23.5 StdEnv/2023 gcc/12.3 openmpi/4.1.5 amber/22.5-23.5 StdEnv/2023 gcc/12.3 openmpi/4.1.5 cuda/12.2 amber/22.5-23.5 GCC, FlexiBLAS & FFTW
ambertools/23.5 StdEnv/2023 gcc/12.3 openmpi/4.1.5 ambertools/23.5 StdEnv/2023 gcc/12.3 openmpi/4.1.5 cuda/12.2 ambertools/23.5 GCC, FlexiBLAS & FFTW
AMBER version modules for running on CPUs modules for running on GPUs (CUDA) Notes
ambertools/21 StdEnv/2020 gcc/9.3.0 openmpi/4.0.3 scipy-stack ambertools/21 StdEnv/2020 gcc/9.3.0 cuda/11.4 openmpi/4.0.3 scipy-stack ambertools/21 GCC, FlexiBLAS & FFTW
amber/20.12-20.15 StdEnv/2020 gcc/9.3.0 openmpi/4.0.3 amber/20.12-20.15 StdEnv/2020 gcc/9.3.0 cuda/11.4 openmpi/4.0.3 amber/20.12-20.15 GCC, FlexiBLAS & FFTW
amber/20.9-20.15 StdEnv/2020 gcc/9.3.0 openmpi/4.0.3 amber/20.9-20.15 StdEnv/2020 gcc/9.3.0 cuda/11.0 openmpi/4.0.3 amber/20.9-20.15 GCC, MKL & FFTW
amber/18.14-18.17 StdEnv/2020 gcc/9.3.0 openmpi/4.0.3 amber/18.14-18.17 StdEnv/2020 gcc/8.4.0 cuda/10.2 openmpi/4.0.3 GCC, MKL
AMBER version modules for running on CPUs modules for running on GPUs (CUDA) Notes
amber/18 StdEnv/2016 gcc/5.4.0 openmpi/2.1.1 scipy-stack/2019a amber/18 StdEnv/2016 gcc/5.4.0 openmpi/2.1.1 cuda/9.0.176 scipy-stack/2019a amber/18 GCC, MKL
amber/18.10-18.11 StdEnv/2016 gcc/5.4.0 openmpi/2.1.1 scipy-stack/2019a amber/18.10-18.11 StdEnv/2016 gcc/5.4.0 openmpi/2.1.1 cuda/9.0.176 scipy-stack/2019a amber/18.10-18.11 GCC, MKL
amber/18.10-18.11 StdEnv/2016 gcc/7.3.0 openmpi/3.1.2 scipy-stack/2019a amber/18.10-18.11 StdEnv/2016 gcc/7.3.0 cuda/9.2.148 openmpi/3.1.2 scipy-stack/2019a amber/18.10-18.11 GCC, MKL
amber/16 StdEnv/2016.4 amber/16 Available only on Graham. Some Python functionality is not supported

Using modules[edit]

AmberTools 21[edit]

Currently, AmberTools 21 module is available on all clusters. AmberTools provide the following MD engines: sander, sander.LES, sander.LES.MPI, sander.MPI, sander.OMP, sander.quick.cuda, and sander.quick.cuda.MPI. After loading the module set AMBER environment variables: 

source $EBROOTAMBERTOOLS/amber.sh

Amber 20[edit]

There are two versions of amber/20 modules: 20.9-20.15 and 20.12-20.15. The first one uses MKL and cuda/11.0, while the second uses FlexiBLAS and cuda/11.4. MKL libraries do not perform well on AMD CPU, and FlexiBLAS solves this problem. It detects CPU type and uses libraries optimized for the hardware. cuda/11.4 is required for running simulations on A100 GPUs installed on Narval.

CPU-only modules provide all MD programs available in AmberTools/20 plus pmemd (serial) and pmemd.MPI (parallel). GPU modules add pmemd.cuda (single GPU), and pmemd.cuda.MPI (multi - GPU).

Known issues[edit]

1. Module amber/20.12-20.15 does not have MMPBSA.py.MPI executable.

2. MMPBSA.py from amber/18-10-18.11 and amber/18.14-18.17 modules cannot perform PB calculations. Use more recent amber/20 modules for this type of calculations.

Job submission examples[edit]

Single GPU job[edit]

For GPU-accelerated simulations on Narval, use amber/20.12-20.15. Modules compiled with CUDA version < 11.4 do not work on A100 GPUs. Below is an example submission script for a single-GPU job with amber/20.12-20.15.

File : pmemd_cuda_job.sh

#!/bin/bash
#SBATCH --ntasks=1 
#SBATCH --gpus-per-node=1 
#SBATCH --mem-per-cpu=2000 
#SBATCH --time=10:00:00

module purge
module load StdEnv/2023 gcc/12.3 openmpi/4.1.5 cuda/12.2 amber/22

pmemd.cuda -O -i input.in -p topol.parm7 -c coord.rst7 -o output.mdout -r restart.rst7


CPU-only parallel MPI job[edit]

File : pmemd_MPI_job_graham.sh

#!/bin/bash
#SBATCH --nodes=4
#SBATCH --ntasks-per-node=32
#SBATCH --mem-per-cpu=2000
#SBATCH --time=1:00:00

module purge
module load StdEnv/2023 gcc/12.3 openmpi/4.1.5 amber/22

srun pmemd.MPI -O -i input.in -p topol.parm7 -c coord.rst7 -o output.mdout -r restart.rst7
File : pmemd_MPI_job_cedar.sh

#!/bin/bash
#SBATCH --nodes=4
#SBATCH --ntasks-per-node=48
#SBATCH --mem-per-cpu=2000
#SBATCH --time=1:00:00

module purge
module load StdEnv/2023 gcc/12.3 openmpi/4.1.5 amber/22

srun pmemd.MPI -O -i input.in -p topol.parm7 -c coord.rst7 -o output.mdout -r restart.rst7
File : pmemd_MPI_job_beluga.sh

#!/bin/bash
#SBATCH --nodes=4
#SBATCH --ntasks-per-node=40
#SBATCH --mem-per-cpu=2000
#SBATCH --time=1:00:00

module purge
module load StdEnv/2023 gcc/12.3 openmpi/4.1.5 amber/22

srun pmemd.MPI -O -i input.in -p topol.parm7 -c coord.rst7 -o output.mdout -r restart.rst7
File : pmemd_MPI_job_narval.sh

#!/bin/bash
#SBATCH --nodes=4
#SBATCH --ntasks-per-node=64
#SBATCH --mem-per-cpu=2000
#SBATCH --time=1:00:00

module purge
module load StdEnv/2023 gcc/12.3 openmpi/4.1.5 amber/22

srun pmemd.MPI -O -i input.in -p topol.parm7 -c coord.rst7 -o output.mdout -r restart.rst7
File : pmemd_MPI_job_narval.sh

#!/bin/bash
#SBATCH --nodes=4
#SBATCH --ntasks-per-node=40
#SBATCH --mem-per-cpu=2000
#SBATCH --time=1:00:00

module purge
module load StdEnv/2023 gcc/12.3 openmpi/4.1.5 amber/22

srun pmemd.MPI -O -i input.in -p topol.parm7 -c coord.rst7 -o output.mdout -r restart.rst7

QM/MM distributed multi-GPU job[edit]

The example below requests eight GPUs.

File : quick_MPI_job.sh

#!/bin/bash
#SBATCH --ntasks=8 --cpus-per-task=1
#SBATCH --gpus-per-task=1 
#SBATCH --mem-per-cpu=4000 
#SBATCH --time=02:00:00

module purge
module load StdEnv/2023 gcc/12.3 openmpi/4.1.5 cuda/12.2 ambertools/23.5

srun sander.quick.cuda.MPI -O -i input.in -p topol.parm7 -c coord.rst7 -o output.mdout -r restart.rst7


Parallel MMPBSA job[edit]

The example below uses 32 MPI processes. MMPBSA scales linearly because each trajectory frame is processed independently.

File : mmpbsa_job.sh

#!/bin/bash
#SBATCH --ntasks=32 
#SBATCH --mem-per-cpu=4000 
#SBATCH --time=1:00:00

module purge
module load module load StdEnv/2023 gcc/12.3 openmpi/4.1.5 amber/22

srun MMPBSA.py.MPI -O -i mmpbsa.in -o mmpbsa.dat -sp solvated_complex.parm7 -cp complex.parm7 -rp receptor.parm7 -lp ligand.parm7 -y trajectory.nc


You can modify scripts to fit your simulation requirements for computing resources. See Running jobs for more details.

Performance and benchmarking[edit]

A team at ACENET has created a Molecular Dynamics Performance Guide for Alliance clusters. It can help you determine optimal conditions for AMBER, GROMACS, NAMD, and OpenMM jobs. The present section focuses on AMBER performance.

View benchmarks of simulations with PMEMD[1]

View benchmarks of QM/MM simulations with SANDER.QUICK [2].

GNU Compiler Collection, an open source compiler collection

Intel Math Kernel Library, a software library of optimized math routines

Message Passing Interface

Retrieved from "https://docs.alliancecan.ca/mediawiki/index.php?title=AMBER&oldid=162446"