NAMD: Difference between revisions

From Alliance Doc
Jump to navigation Jump to search
(remove empty sections)
(use slurm_hl2hl.py script to get list of nodes for charm)
Line 71: Line 71:
#SBATCH --ntasks 64            # number of tasks
#SBATCH --ntasks 64            # number of tasks
#SBATCH --nodes=2
#SBATCH --nodes=2
#SBATCH --ntasks-per-node=32
#SBATCH --mem 0            # memory per node, 0 means all memory
#SBATCH --mem 0            # memory per node, 0 means all memory
#SBATCH -o slurm.%N.%j.out    # STDOUT
#SBATCH -o slurm.%N.%j.out    # STDOUT
Line 76: Line 77:
#SBATCH --account=def-specifyaccount
#SBATCH --account=def-specifyaccount


cat << EOF > nodefile.py
slurm_hl2hl.py --format CHARM > nodefile.dat
#!/usr/bin/python
import sys
a=sys.argv[1]
nodefile=open("nodefile.dat","w")
 
cluster=a[0:3]
for st in a.lstrip(cluster+"[").rstrip("]").split(","):
    d=st.split("-")
    start=int(d[0])
    finish=start
    if(len(d)==2):
        finish=int(d[1])
 
    for i in range(start,finish+1):
        nodefile.write("host "+cluster+str(i)+"\n")
 
nodefile.close()
 
EOF
 
python nodefile.py $SLURM_NODELIST
NODEFILE=nodefile.dat
NODEFILE=nodefile.dat
P=$SLURM_NTASKS
P=$SLURM_NTASKS
Line 161: Line 141:
#SBATCH --ntasks 64            # number of tasks
#SBATCH --ntasks 64            # number of tasks
#SBATCH --nodes=2
#SBATCH --nodes=2
#SBATCH --ntasks-per-node=32
#SBATCH --mem 1024            # memory pool per process
#SBATCH --mem 1024            # memory pool per process
#SBATCH --gres=gpu:2
#SBATCH --gres=gpu:2
Line 167: Line 148:
#SBATCH --account=def-specifyaccount
#SBATCH --account=def-specifyaccount


cat << EOF > nodefile.py
slurm_hl2hl.py --format CHARM > nodefile.dat
#!/usr/bin/python
import sys
a=sys.argv[1]
nodefile=open("nodefile.dat","w")
 
cluster=a[0:3]
for st in a.lstrip(cluster+"[").rstrip("]").split(","):
    d=st.split("-")
    start=int(d[0])
    finish=start
    if(len(d)==2):
        finish=int(d[1])
 
    for i in range(start,finish+1):
        nodefile.write("host "+cluster+str(i)+"\n")
 
nodefile.close()
 
EOF
 
python nodefile.py $SLURM_NODELIST
NODEFILE=nodefile.dat
NODEFILE=nodefile.dat
OMP_NUM_THREADS=32
OMP_NUM_THREADS=32

Revision as of 19:29, 5 January 2018


This article is a draft

This is not a complete article: This is a draft, a work in progress that is intended to be published into an article, which may or may not be ready for inclusion in the main wiki. It should not necessarily be considered factual or authoritative.

General[edit]

NAMD is a parallel, object-oriented molecular dynamics code designed for high-performance simulation of large biomolecular systems. Simulation preparation and analysis is integrated into the visualization package VMD.

A registration required to download software.

Quickstart Guide[edit]

This section summarizes configuration details.

Environment Modules[edit]

The following modules providing NAMD are available on graham and cedar.

Compiled without CUDA support:

  • namd-multicore/2.12
  • namd-verbs/2.12

Compiled with CUDA support:

  • namd-multicore/2.12
  • namd-verbs-smp/2.12

To access these modules which require CUDA, first execute:

module load cuda/8.0.44

Note: using verbs library is more efficient than using OpenMPI, hence only verbs versions are provided.

Submission Scripts[edit]

Please refer to the page "Running jobs" for help on using the SLURM workload manager.

Serial Job[edit]

Here's a simple job script for serial simulation:

File : serial_namd_job.sh

#!/bin/bash
#
#SBATCH --ntasks 1            # number of tasks
#SBATCH --mem 1024            # memory pool per process
#SBATCH -o slurm.%N.%j.out    # STDOUT
#SBATCH -t 0:20:00            # time (D-HH:MM)
#SBATCH --account=def-specifyaccount


module load namd-multicore/2.12
namd2 +p1 +idlepoll apoa1.namd


Verbs Job[edit]

These provisional Instructions will be refined further once this configuration can be fully tested on the new clusters. This example uses 64 processes in total on 2 nodes, each node running 32 processes, thus fully utilizing its 32 cores. This script assumes full nodes are used, thus ntasks/nodes should be 32 (on graham). For best performance, NAMD jobs should use full nodes.

NOTE: The verbs version will not run on cedar because of its different interconnect. Use the MPI version instead.

File : verbs_namd_job.sh

#!/bin/bash
#
#SBATCH --ntasks 64            # number of tasks
#SBATCH --nodes=2
#SBATCH --ntasks-per-node=32
#SBATCH --mem 0            # memory per node, 0 means all memory
#SBATCH -o slurm.%N.%j.out    # STDOUT
#SBATCH -t 0:05:00            # time (D-HH:MM)
#SBATCH --account=def-specifyaccount

slurm_hl2hl.py --format CHARM > nodefile.dat
NODEFILE=nodefile.dat
P=$SLURM_NTASKS

module load namd-verbs/2.12
CHARMRUN=`which charmrun`
NAMD2=`which namd2`
$CHARMRUN ++p $P ++nodelist $NODEFILE  $NAMD2  +idlepoll apoa1.namd


MPI Job[edit]

NOTE: Use this only on cedar, where verbs version will not work.

File : mpi_namd_job.sh

#!/bin/bash
#
#SBATCH --ntasks 64            # number of tasks
#SBATCH --nodes=2
#SBATCH --mem 4024            # memory pool per process
#SBATCH -o slurm.%N.%j.out    # STDOUT
#SBATCH -t 0:05:00            # time (D-HH:MM)
#SBATCH --account=def-specifyaccount

module load namd-mpi/2.12
NAMD2=`which namd2`
srun $NAMD2 apoa1.namd


GPU Job[edit]

This example uses 8 CPU cores and 1 GPU on a single node.

File : multicore_gpu_namd_job.sh

#!/bin/bash
#
#SBATCH --ntasks 8            # number of tasks
#SBATCH --mem 1024            # memory pool per process
#SBATCH -o slurm.%N.%j.out    # STDOUT
#SBATCH -t 0:05:00            # time (D-HH:MM)
#SBATCH --gres=gpu:1
#SBATCH --account=def-specifyaccount


module load cuda/8.0.44
module load namd-multicore/2.12
namd2 +p8 +idlepoll apoa1.namd


Verbs-GPU Job[edit]

These provisional Instructions will be refined further once this configuration can be fully tested on the new clusters. This example uses 64 processes in total on 2 nodes, each node running 32 processes, thus fully utilizing its 32 cores. Each node uses 2 GPUs, so job uses 4 GPUs in total. This script assumes full nodes are used, thus ntasks/nodes should be 32 (on graham). For best performance, NAMD jobs should use full nodes.

NOTE: The verbs version will not run on cedar because of its different interconnect.

File : verbsgpu_namd_job.sh

#!/bin/bash
#
#SBATCH --ntasks 64            # number of tasks
#SBATCH --nodes=2
#SBATCH --ntasks-per-node=32
#SBATCH --mem 1024            # memory pool per process
#SBATCH --gres=gpu:2
#SBATCH -o slurm.%N.%j.out    # STDOUT
#SBATCH -t 0:05:00            # time (D-HH:MM)
#SBATCH --account=def-specifyaccount

slurm_hl2hl.py --format CHARM > nodefile.dat
NODEFILE=nodefile.dat
OMP_NUM_THREADS=32
P=$SLURM_NTASKS

module load cuda/8.0.44
module load namd-verbs-smp/2.12
CHARMRUN=`which charmrun`
NAMD2=`which namd2`
$CHARMRUN ++p $P ++ppn $OMP_NUM_THREADS ++nodelist $NODEFILE  $NAMD2  +idlepoll apoa1.namd


Installation[edit]

NAMD is installed by the Compute Canada software team and is available as a module. If a new version is required, please email tech support and request it. Also, if for some reason you need to do your own installation, please contact tech support for advice and help with that. You can also ask for details of how our NAMD modules were compiled.

Links[edit]