38,760
edits
Apptainer/en: Difference between revisions
Updating to match new version of source page
(Updating to match new version of source page) |
(Updating to match new version of source page) |
||
| Line 22: | Line 22: | ||
* Renamed all environment variables having <code>SINGULARITY</code> in their names to have <code>APPTAINER</code> in them. | * Renamed all environment variables having <code>SINGULARITY</code> in their names to have <code>APPTAINER</code> in them. | ||
Should you need to port scripts | Should you need to port scripts to Apptainer, know Apptainer version 1 is backwards compatible with Singularity so switching to Apptainer can be done incrementally. | ||
==Other Linux container technologies== | ==Other Linux container technologies== | ||
| Line 45: | Line 45: | ||
* Install Linux, Apptainer, and <code>sudo</code> in a virtual machine on a system you control so you will be able to have <code>sudo</code> access within such. Build your image(s) on that machine and upload them in order to use them on Alliance systems. | * Install Linux, Apptainer, and <code>sudo</code> in a virtual machine on a system you control so you will be able to have <code>sudo</code> access within such. Build your image(s) on that machine and upload them in order to use them on Alliance systems. | ||
* If appropriate, [[Technical Support|submit a ticket]] asking if Alliance staff would be able to help build the image(s) | * If appropriate, [[Technical Support|submit a ticket]] asking if Alliance staff would be able to help build the image(s) required needing <code>sudo</code>. This may or may not be possible, but feel free to ask in a ticket if what you wish to achieve is beyond your means. Additionally, we may respond with other ways to achieve such which may or may not involve Apptainer. | ||
* Apptainer version 1.1.x and newer has improved support for users using <code>--fakeroot</code> implicitly and explicitly so some things may be possible that were not with Apptainer version 1.0 and Singularity. This includes being able to build some images from <code>.def</code> definition files and building some images without needing to use <code>sudo</code>. That said, not all images will be able to be built without needing to use <code>sudo</code> or superuser/root. | * Apptainer version 1.1.x and newer has improved support for users using <code>--fakeroot</code> implicitly and explicitly so some things may be possible that were not with Apptainer version 1.0 and Singularity. This includes being able to build some images from <code>.def</code> definition files and building some images without needing to use <code>sudo</code>. That said, not all images will be able to be built without needing to use <code>sudo</code> or superuser/root. | ||
| Line 250: | Line 250: | ||
=Building an Apptainer image= | =Building an Apptainer image= | ||
<b>NOTE:</b> Please note and heed the advice given in section <b>[[#Building_images_or_overlays|Building images and overlays]]</b>. | <b>NOTE:</b> Please note and heed the advice given in section <b>[[#Building_images_or_overlays|Building images and overlays]]</b>. | ||
| Line 261: | Line 260: | ||
<b><code>SIF</code> files</b> can contain multiple parts where each part is typically a <code>squashfs</code> filesystem (which are read-only and compressed). It is possible for <code>SIF</code> files to contain read-write filesystems and overlay images as well, but such is beyond the scope of this page; see [http://apptainer.org/docs Apptainer's official documentation] on how to do such. Unless more advanced methods of building an image were used, the Apptainer <code>build</code> command produces a <code>SIF</code> file with a read-only <code>squashfs</code> filesystem when building images. This is the preferred option since the resulting image remains as-is since it is read-only, and the image is much smaller because it is compressed. Know that disk reads from that image are done very quickly. | <b><code>SIF</code> files</b> can contain multiple parts where each part is typically a <code>squashfs</code> filesystem (which are read-only and compressed). It is possible for <code>SIF</code> files to contain read-write filesystems and overlay images as well, but such is beyond the scope of this page; see [http://apptainer.org/docs Apptainer's official documentation] on how to do such. Unless more advanced methods of building an image were used, the Apptainer <code>build</code> command produces a <code>SIF</code> file with a read-only <code>squashfs</code> filesystem when building images. This is the preferred option since the resulting image remains as-is since it is read-only, and the image is much smaller because it is compressed. Know that disk reads from that image are done very quickly. | ||
<b>A sandbox directory</b> is a normal directory in the filesystem that starts out as empty. As Apptainer builds the image, it adds to it the files | <b>A sandbox directory</b> is a normal directory in the filesystem that starts out as empty. As Apptainer builds the image, it adds to it the files and directories needed in the image. The contents of a sandbox directory should only be accessed or updated through the use of Apptainer. One might need to use a sandbox directory in situations where one needs to have read-write access to the image itself in order to be able to update the container image. That said, if updates are infrequent, it is typically easier and better to use a <code>SIF</code> file and when updates need to be done, build a sandbox image from the <code>SIF</code> file, make the required changes, and then build a new <code>SIF</code> file, e.g., | ||
<source lang="console">$ cd $HOME | <source lang="console">$ cd $HOME | ||
| Line 272: | Line 271: | ||
$ rm -rf mynewimage.dir</source> | $ rm -rf mynewimage.dir</source> | ||
Using an <code>SIF</code> image is recommended as disk performance (from the container image) will be faster than storing each file | Using an <code>SIF</code> image is recommended as disk performance (from the container image) will be faster than storing each file separately on our cluster filesystems, which are set up to handle large files and parallel I/O. Using an <code>SIF</code> file instead of a sandbox image will also only use a quota file count amount of 1 instead of thousands (some images will typically contain thousands of files and directories). | ||
Many Linux distribution package managers require root permissions in order to use them. This implies that Apptainer version 1.0.x and the older Singularity cannot be used on compute clusters to build images as a normal user. Should such occur, [[Technical support|submit a ticket]] asking for help to create that image or use a computer with Apptainer installed where you have root permissions. | Many Linux distribution package managers require root permissions in order to use them. This implies that Apptainer version 1.0.x and the older Singularity cannot be used on compute clusters to build images as a normal user. Should such occur, [[Technical support|submit a ticket]] asking for help to create that image or use a computer with Apptainer installed where you have root permissions. | ||
| Line 333: | Line 332: | ||
<b>NOTE:</b> Using Docker may fail if you are not in the <code>docker</code> group. Similarly, building some containers may fail with Apptainer without appropriate <code>sudo</code>, root, or <code>--fakeroot</code> permissions. It is your responsibility to ensure you've such access on the system you are running the commands below. | <b>NOTE:</b> Using Docker may fail if you are not in the <code>docker</code> group. Similarly, building some containers may fail with Apptainer without appropriate <code>sudo</code>, root, or <code>--fakeroot</code> permissions. It is your responsibility to ensure you've such access on the system you are running the commands below. | ||
If one only has a Dockerfile and wishes to create an Apptainer image, run the following on a computer with Docker and Apptainer installed (where you've sufficient permissions | If one only has a Dockerfile and wishes to create an Apptainer image, run the following on a computer with Docker and Apptainer installed (where you've sufficient permissions): | ||
docker build -f Dockerfile -t your-tag-name | docker build -f Dockerfile -t your-tag-name | ||