User:Stuekero/StarCCM

StarCCM+

Cluster Specific Notes

Cedar

At Cedar you need to add the option -mpi intel so that StarCCM+ uses Intel MPI and can therefore use the OmniPath fibre network.

Niagara

At Niagara the compute noded mount the $HOME filesystem as "read-only". Therefore it is important to define the environment variable $STARCCM_TMP and point it to a location on $SCRATCH, which is unique to the version of StarCCM+. Otherwise StarCCM+ will try to create such a directory in the $HOME and crash in the process.

GrahamCedarNiagara

File : starccm_job.sh

#!/bin/bash
#SBATCH --time=0-01:00        # Time limit: d-hh:mm
#SBATCH --nodes=2
#SBATCH --ntasks-per-node=1
#SBATCH --cpus-per-task=32
#SBATCH --mem=0               # Request all available MEM on full nodes
module load starccm/12.04.011-R8

export LM_PROJECT='YOUR ADAPCO PROJECT ID GOES HERE'
export CDLMD_LICENSE_FILE="1999@flex.cd-adapco.com"

slurm_hl2hl.py --format STAR-CCM+ > machinefile

NCORE=$((SLURM_NTASKS * SLURM_CPUS_PER_TASK))

starccm+ -power -np $NCORE -machinefile machinefile -batch  /path/to/your/simulation/file

File : starccm_job.sh

#!/bin/bash
#SBATCH --time=0-01:00        # Time limit: d-hh:mm
#SBATCH --nodes=2
#SBATCH --ntasks-per-node=1
#SBATCH --cpus-per-task=48
#SBATCH --mem=0               # Request all available MEM on full nodes
module load starccm/12.04.011-R8

export LM_PROJECT='YOUR ADAPCO PROJECT ID GOES HERE'
export CDLMD_LICENSE_FILE="1999@flex.cd-adapco.com"

slurm_hl2hl.py --format STAR-CCM+ > machinefile

NCORE=$((SLURM_NTASKS * SLURM_CPUS_PER_TASK))

starccm+ -power -np $NCORE -machinefile machinefile -batch -mpi intel /path/to/your/simulation/file

File : starccm_job.sh

#!/bin/bash
#SBATCH --time=0-01:00        # Time limit: d-hh:mm
#SBATCH --nodes=2
#SBATCH --ntasks-per-node=1
#SBATCH --cpus-per-task=40    # or 80 to use HyperThreading
#SBATCH --mem=0               # Request all available MEM on full nodes
module purge --force
module load CCEnv
module load StdEnv
module load starccm/12.04.011-R8

export LM_PROJECT='YOUR ADAPCO PROJECT ID GOES HERE'
export CDLMD_LICENSE_FILE="1999@localhost"
ssh nia-gw -L 1999:flex.cd-adapco.com:1999 -L 2099:flex.cd-adapco.com:2099 -N -f

export STARCCM_TMP="${SCRATCH}/.starccm-${EBVERSIONSTARCCM}"
mkdir -p "$STARCCM_TMP"

slurm_hl2hl.py --format STAR-CCM+ > machinefile

NCORE=$((SLURM_NTASKS * SLURM_CPUS_PER_TASK))

starccm+ -power -np $NCORE -machinefile machinefile -batch /path/to/your/simulation/file

Message Passing Interface