Apptainer/en: Difference between revisions

** While Podman is a Linux container technology there are [https://github.com/containers/podman/blob/main/docs/tutorials/mac_win_client.md instructions for installing Podman on Windows/Mac machines].

** Using Docker on an multi-user cluster creates security risks, therefore we do not make Docker available on our HPC clusters.

Another important option is the <code>-W</code> or <code>--workdir</code> option. On Alliance clusters and on most Linux systems, <code>/tmp</code> and similar filesystems use RAM, not disk space. Since jobs typically run on our clusters with limited RAM amounts, this can result in jobs getting killed because they consume too much RAM relative to what was requested for the job. A suitable work-around for this is to tell Apptainer to use a real disk location for its working directory ("workdir"). This is done by passing the <code>-W</code> option followed by a path to a disk location where Apptainer can read/write temporary files, etc. For example, suppose one wanted to run a command called <code>myprogram</code> in a using an Apptainer container image called <code>myimage.sif</code> with its "workdir" set to <code>/path/to/a/workdir</code> in the filesystem:

If you need to run MPI programs inside a container there are things that need to be done in the host environment in order for such to work. Please see the [[#Running_MPI_Programs_Inside_an_Apptainer Container|Running MPI Programs section below]] for an example of how to run MPI programs inside a container. The [http://apptainer.org/docs/user/main/mpi.html official Apptainer documentation] has more information concerning how MPI programs can be run inside a container.

For example, suppose you want to run the <code>g++</code> compiler inside your container to compile a C++ program called <code>myprog.cpp</code> and then run that program. To this this you might use this command:

On our clusters, you will want to use a number of additional options (that appear after <code>run</code> but before <code>your-container-name.sif</code>). These options will include <code>-C</code>, <code>-c</code>, <code>-e</code>, <code>-W</code> as well as various bind mount options to make your disk space available to the programs that run in your container. For example:

where <code>./my_data_file.txt</code> is is a file in the current directory on the host, i.e., the file is not stored in the container at all. Because of the <code>-C</code> option, this file will not be accessible to the <code>wc</code> program inside the container --and so an access error will result. The fix is to bind mount the current directory, e.g.,

Unfortunately some instructions for packages only provide a <code>Dockerfile</code> without a container image. A <code>Dockerfile</code> contains the instructions necessary for the Docker software to build that container. Our clusters do not have the Docker software installed. That said, if you've access to a system with both Docker and Apptainer installed, and, sufficient access to Docker (e.g., <code>sudo</code> or root access, or, you are in that system's <code>docker</code> group) and if needed Apptainer (e.g., <code>sudo</code> or root access, or, you have <code>--fakeroot</code> access), then you can follow the instructions below to use Docker and then Apptainer to build an Apptainer image on that system.

After this is done, the SIF file is an Apptainer container for the <code>Dockerfile</code>. Transfer the SIF to the approprate cluster(s) in order to use such.

followed by runninng <code>docker image rm ID</code> (where ID is the image ID output from the <code>docker images</code> command) in order to free up the disk space associated with those other image layers on the system you are using.