Standard software environments
What are standard software environments ?[edit]
Compute Canada's software environment is provided through a set of modules which allow you to switch between different versions of software packages you may want to use. These modules are organized in a tree, with the main branches being determined by a set of tools such as compilers, MPI distributions and CUDA. At the root of this tree are modules which we call "standard environments", named StdEnv. As of 2020, there are three such standard environments, versioned 2016.4, 2018.3, and 2020. Some major improvements happened between each of these versions. This page is describing these changes, and why you should upgrade to more recent versions. In general, new versions of software packages will get installed with the newest version of our software environment.
StdEnv/2016.4[edit]
This was the initial version of our software environment, and was released in 2016, with the deployment of Cedar and Graham. This branch features the compilers GCC 5.4.0 and Intel 2016.4 as default compilers, and Open MPI 2.1.1 as its default implementation of MPI. Most of the software compiled with this environment does not support AVX512 instructions provided by the Skylake processors on Béluga, Niagara, as well as the most recent additions to Cedar and Graham.
To activate this environment, use the command
[name@server ~]$ module load StdEnv/2016.4
StdEnv/2018.3[edit]
This was the second revision of our software environment, and was released in 2018, with the deployment of Béluga, and shortly after the deployment of Niagara. Defaults were upgraded to GCC 7.3.0, Intel 2018.3, and Open MPI 3.1.2. This was also the first version to really support AVX512 instructions.
To activate this environment, use the command
[name@server ~]$ module load StdEnv/2018.3
StdEnv/2020[edit]
This is the most recent iteration of our software environment, and the largest change so far. It uses GCC 9.3.0, Intel 2020.1, and Open MPI 4.0.3 as defaults. Several changes were made with this release, most of which result in performance improvements.
To activate this environment, use the command
[name@server ~]$ module load StdEnv/2020
Performance improvements[edit]
Binaries compiled with the Intel compiler now automatically support both AVX2 and AVX512 instruction sets. In technical terms, we say that they are multi-architecture binaries, also known as fat binaries. This means that when running on a cluster which has multiple generations of processors, such as Cedar and Graham, you don't have to manually load one of the arch modules if you use software packages compiled with the Intel compiler.
Many software packages which were previously installed either with GCC or with Intel are now installed at a lower level of the software hierarchy, which makes the same module visible irrespective of which compiler is loaded. For example, this is the case for the R modules, which previously required loading the gcc module. This is also the case for many bioinformatics software packages. This could be done because we introduced optimizations specific to CPU architectures at a level of the software hierarchy lower than the compiler level.
We also installed a more recent version of the GNU C Library, which introduces optimizations in some mathematical functions. This has increased the requirement on the version of the Linux Kernel (see below).
Change in the compatibility layer[edit]
Another major change introduced in the 2020 release is a switch between two tools for our compatibility layer. This is a layer of the software hierarchy which we provide to isolate it from the underlying operating system. For example, this ensures that our software packages will work whether it is run on CentOS, Ubuntu, or Fedora systems. For the 2016.4 and 2018.3 versions, we used the Nix package manager, while for the 2020 version, we use Gentoo Prefix.
Change in kernel requirement[edit]
Versions 2016.4 and 2018.3 required a Linux kernel version 2.6.32 or more recent. This supported CentOS versions starting at CentOS 6. With the 2020 version, we require a Linux kernel 3.10 or better. This means it no longer supports CentOS 6, but requires CentOS 7 instead. Other distributions usually have kernels which are much more recent, so you probably do not need to change your distribution if you are using this standard environment on something other than CentOS.
How can I change which version of StdEnv is my default?[edit]
Our clusters use different versions of StdEnv as their default version. As of August 2020, Cedar and Graham use StdEnv/2016.4, while Béluga uses StdEnv/2018.3. Niagara also defaults to StdEnv/2018.3 if you module load CCEnv StdEnv. In the future, we will probably switch all of them to use StdEnv/2020, but that time hasn't come yet. Users can however specify their own default by running the following command (example provided for the 2020 version):
[name@server ~]$ echo "module-version StdEnv/2020 default" >> $HOME/.modulerc
Do I need to reinstall/recompile my code if the StdEnv version changes?[edit]
If you compile your own code, or install R or Python packages, yes, you should recompile or reinstall the packages you need using the newer version of the standard environment.