R: Difference between revisions

For more on submitting jobs, see the [[Running jobs]] page.

====Usage====  

Foreach can be considered as a unified interface for all backends (i.e. doMC, doMPI, doParallel, doRedis, etc.). It works on all platforms, assuming that the backend works. doParallel acts as an interface between foreach and the parallel package and can be loaded alone. There are some known efficiency issues when using foreach to run a very large number of very small tasks. Therefore, keep in mind that the following code is not the best example of an optimized use of the foreach() call but rather that the function chosen was kept at a minimum for demonstration purposes.

You must register the backend by feeding it the number of cores available. If the backend is not registered, foreach will assume that the number of cores is 1 and will proceed to go through the iterations serially.  

# to register the backend;

# to call foreach() by keeping it on the same line as the %do% (serial) or %dopar% operator.

1. Place your R code in a script file, in this case the file is called ''test_foreach.R''.

library(doParallel) #

# a very simple function

test_func <- function(var1, var2) {

}

# we will iterate over two sets of values, you can modify this to explore the mechanism of foreach

var1.v = c(1:8)

var2.v = seq(0.1, 1, length.out = 8)

# Use the environment variable SLURM_NTASKS to set the number of cores.

# This is for SLURM. Replace SLURM_NTASKS by the proper variable for your system.

ncores = Sys.getenv("SLURM_NTASKS")  

registerDoParallel(cores=ncores)# Shows the number of Parallel Workers to be used

print(ncores) # this how many cores are available, and how many you have requested.

getDoParWorkers()# you can compare with the number of actual workers

# be careful! foreach() and %dopar% must be on the same line!

foreach(var1=var1.v, .combine=rbind) %:% foreach(var2=var2.v, .combine=rbind) %dopar% {test_func(var1=var1, var2=var2)}

#SBATCH --mail-type=ALL          # send an email in all cases (job started, job ended, job aborted)

module load r/3.4.3

export R_LIBS=~/local/R_libs/