Apptainer: Difference between revisions

** You can install and use Docker on your own computer and use it to create an Apptainer image, which can then be uploaded to an HPC cluster as outlined in '''[[#Creating_an_Apptainer_Container_From_a_Dockerfile|this section]]''' later on this page.

* In order to use Apptainer you must have a container '''image''', e.g., a <code>.sif</code> file or a "sandbox" directory created previously. If you don't already have an image or a sandbox, see the section on '''[[#Building_an_Apptainer_Container/Image|building an image]]''' below.

* While Apptainer is installed and available for use, using Apptainer will require you to install and/or build all software you will need to make use of in your container. In many instances, '''[[Available_software|we already have such software installed on our clusters]]''' so there is often no need to create a container with the same installed in it.

Software that is run inside a container runs in a different environment using different libraries and tools than what is installed on the host system. It is, therefore, important to run programs within containers by '''not''' using any environment settings or software defined outside of the container. Unfortunately, by default, Apptainer will run adopting the shell environment of the host and this can result in issues when running programs. To avoid such issues, when using <code>apptainer run</code>, <code>apptainer shell</code>, <code>apptainer exec</code>, and/or </code>apptainer instance</code>, use one of these options (with more preference to those options listed earlier in the table below):

** ASIDE: One should '''not''' be running programs (including Apptainer) on a login node.  Use an interactive <code>salloc</code> job.

'''IMPORTANT:''' In addition to choose to use the above options, if you are making use of a persistent overlay image (as a separate file or contained within the SIF file) and want changes to be written to that image, it is extremely important to pass the <code>-w</code> or <code>--writable</code> option to your container. If this option is not passed to it, any changes you make to the image in the <code>apptainer shell</code> session will not be saved!

'''NOTE 1:''' Don't run daemons manually without using <code>apptainer instance</code> and related commands. Apptainer works properly with other tools such as the Slurm scheduler that run on our clusters. When a job is cancelled, killed, crashes, or is otherwise finished, daemons run using <code>apptainer instance</code> will not hang or result in defunct processes. Additionally by using the <code>apptainer instance</code> command you will be able to control the daemons and programs running in the same container.

'''NOTE 2:''' Running daemons in your job will only run those daemons while your job runs. The scheduler will kill all processes attached to a job when the granted job time expires. If you need to run daemons continuously for longer than a job, submit a ticket asking to discuss such with staff persons. Such may require creating a cloud virtual machine to achieve such.

'''NOTE:''' When all MPI processes are running on a single shared-memory node, there is no need to use interconnection hardware and there will be no issues running MPI programs within an Apptainer container when all MPI processes run on a single cluster node, e.g., when the slurm option <code>--nodes=1</code> is used with an <code>sbatch</code> script. Unless one '''explicitly''' sets the maximum number of cluster nodes used to <code>1</code>, the scheduler can choose to run an MPI program over multiple nodes. If such will run from within an Apptainer container and has not been set up to properly run, then it is possible it will fail to run.

=Miscellaneous Items= <!--T:131-->