rsnt_translations
56,430
edits
Singularity: Difference between revisions
Jump to navigation
Jump to search
no edit summary
(Marked this version for translation) |
No edit summary |
||
| Line 2: | Line 2: | ||
<translate> | <translate> | ||
=Overview= <!--T:1--> | =Overview= <!--T:1--> | ||
Singularity<ref>Singularity Software | Singularity<ref>Singularity Software website: https://www.sylabs.io/docs/</ref> is open source software created by Berkeley Lab: | ||
* as a '''secure way''' to use Linux containers on Linux multi-user clusters, | * as a '''secure way''' to use Linux containers on Linux multi-user clusters, | ||
* as a way to enable users to have '''full control of their environment,''' and, | * as a way to enable users to have '''full control of their environment,''' and, | ||
| Line 21: | Line 21: | ||
<!--T:4--> | <!--T:4--> | ||
Singularity is similar to other container solutions such as Docker<ref>Docker Software | Singularity is similar to other container solutions such as Docker<ref>Docker Software website: https://www.docker.com/</ref> except Singularity was specifically designed to enable containers to be used securely without requiring any special permissions especially on multi-user compute clusters.<ref>Singularity Security Documentation: https://www.sylabs.io/guides/2.5.1/admin-guide/security.html</ref> | ||
= | =Availability= <!--T:5--> | ||
<!--T:6--> | <!--T:6--> | ||
| Line 31: | Line 31: | ||
Should you wish to use Singularity on your own computer systems, you will need to download and install Singularity per its documentation.<ref>Singularity Documentation: https://www.sylabs.io/docs/</ref> You should be using a relatively recent version of some Linux distribution (e.g., ideally your kernel is v3.10.0 or newer). | Should you wish to use Singularity on your own computer systems, you will need to download and install Singularity per its documentation.<ref>Singularity Documentation: https://www.sylabs.io/docs/</ref> You should be using a relatively recent version of some Linux distribution (e.g., ideally your kernel is v3.10.0 or newer). | ||
=Singularity | =Singularity on Compute Canada systems= <!--T:8--> | ||
== | ==Loading a module== <!--T:9--> | ||
<!--T:10--> | <!--T:10--> | ||
| Line 43: | Line 43: | ||
<source lang="console">$ module spider singularity</source> | <source lang="console">$ module spider singularity</source> | ||
==Creating | ==Creating images== <!--T:12--> | ||
<!--T:13--> | <!--T:13--> | ||
'''Before''' using Singularity, you will first need to '''create a (container) image'''. A Singularity image is either a file or a directory '''containing an installation of Linux'''. One can create a Singularity image by any of the following: | '''Before''' using Singularity, you will first need to '''create a (container) image'''. A Singularity image is either a file or a directory '''containing an installation of Linux'''. One can create a Singularity image by any of the following: | ||
* downloading a container from '''Singularity Hub'''<ref>Singularity Hub | * downloading a container from '''Singularity Hub'''<ref>Singularity Hub website: https://singularity-hub.org/</ref> | ||
* downloading a container from '''Docker Hub'''<ref>Docker Hub | * downloading a container from '''Docker Hub'''<ref>Docker Hub website: https://hub.docker.com/</ref> | ||
* from a container you already have, | * from a container you already have, | ||
* from a '''tarball''' or a '''directory''' containing an installation of Linux, or, | * from a '''tarball''' or a '''directory''' containing an installation of Linux, or, | ||
* from a '''Singularity recipe file'''. | * from a '''Singularity recipe file'''. | ||
===Creating an | ===Creating an image using Singularity Hub=== <!--T:14--> | ||
<!--T:15--> | <!--T:15--> | ||
[https://singularity-hub.org/ Singularity Hub] provides a search interface for pre-built images. | [https://singularity-hub.org/ Singularity Hub] provides a search interface for pre-built images. Suppose you find one you want to use, for instance [https://singularity-hub.org/collections/543 Ubuntu], | ||
then you would download the image by running: | then you would download the image by running: | ||
<source lang="console">$ singularity pull shub://singularityhub/ubuntu</source> | <source lang="console">$ singularity pull shub://singularityhub/ubuntu</source> | ||
===Creating an | ===Creating an image using Docker Hub=== <!--T:16--> | ||
<!--T:17--> | <!--T:17--> | ||
| Line 70: | Line 70: | ||
<source lang="console">$ singularity pull docker://ubuntu</source> | <source lang="console">$ singularity pull docker://ubuntu</source> | ||
===Creating a | ===Creating a tarball of your own Linux system=== <!--T:19--> | ||
<!--T:20--> | <!--T:20--> | ||
| Line 80: | Line 80: | ||
The created tarball will need to be converted into a Singularity image which is discussed [[#Creating an Image From a Tarball|later on this page]]. | The created tarball will need to be converted into a Singularity image which is discussed [[#Creating an Image From a Tarball|later on this page]]. | ||
===Creating an | ===Creating an image from a tarball=== <!--T:22--> | ||
<!--T:23--> | <!--T:23--> | ||
| Line 101: | Line 101: | ||
Singularity single-file images filenames typically have a <code>.simg</code> extension. | Singularity single-file images filenames typically have a <code>.simg</code> extension. | ||
===Creating an | ===Creating an image from a Singularity recipe=== <!--T:27--> | ||
<!--T:28--> | <!--T:28--> | ||
'''NOTE:''' Singularity recipes require <code>root</code> permissions, thus, recipes can only be run on a computer where you can be the <code>root</code> user, e.g., your own Linux computer. | '''NOTE:''' Singularity recipes require <code>root</code> permissions, thus, recipes can only be run on a computer where you can be the <code>root</code> user, e.g., your own Linux computer. | ||
====Recipe: Creating a Singularity | ====Recipe: Creating a Singularity image of the local filesystem==== <!--T:29--> | ||
<!--T:30--> | <!--T:30--> | ||
| Line 136: | Line 136: | ||
<source lang="console">$ sudo singularity build new-ubuntu-image.simg update-existing-container-recipe</source> | <source lang="console">$ sudo singularity build new-ubuntu-image.simg update-existing-container-recipe</source> | ||
====Recipe: Creating a Singularity | ====Recipe: Creating a Singularity image from a Docker URL==== <!--T:34--> | ||
<!--T:35--> | <!--T:35--> | ||
| Line 214: | Line 214: | ||
and illustrates how one can easily make new images at later points in time. | and illustrates how one can easily make new images at later points in time. | ||
===Is sudo | ===Is sudo needed or not needed?=== <!--T:39--> | ||
<!--T:40--> | <!--T:40--> | ||
| Line 254: | Line 254: | ||
# Run a container instance in order to run '''daemons''' which may have '''backgrounded processes'''. | # Run a container instance in order to run '''daemons''' which may have '''backgrounded processes'''. | ||
===Running | ===Running commands interactively=== <!--T:46--> | ||
<!--T:47--> | <!--T:47--> | ||
| Line 318: | Line 318: | ||
and upload myimage-new.simg to a cluster in order to use it. | and upload myimage-new.simg to a cluster in order to use it. | ||
===Running a | ===Running a single command=== <!--T:65--> | ||
<!--T:66--> | <!--T:66--> | ||
| Line 357: | Line 357: | ||
Remember to [[#Bind Mounts|bind mount]] the directories you will need access to in order for your job to run successfully. | Remember to [[#Bind Mounts|bind mount]] the directories you will need access to in order for your job to run successfully. | ||
===Running | ===Running container instances=== <!--T:77--> | ||
<!--T:78--> | <!--T:78--> | ||
| Line 400: | Line 400: | ||
</source> | </source> | ||
===Bind | ===Bind mounts=== <!--T:88--> | ||
<!--T:89--> | <!--T:89--> | ||
| Line 423: | Line 423: | ||
==HPC | ==HPC issues== <!--T:93--> | ||
===Running MPI | ===Running MPI programs from within a container=== <!--T:94--> | ||
<!--T:95--> | <!--T:95--> | ||
| Line 437: | Line 437: | ||
Running jobs across nodes with Singularity+MPI has not been successfully done yet on Compute Canada systems. | Running jobs across nodes with Singularity+MPI has not been successfully done yet on Compute Canada systems. | ||
=See | =See also= <!--T:98--> | ||
* SHARCNET General Interest Webinar, "Singularity", presented by Paul Preney on Feb. 14, 2018. See this [https://www.youtube.com/watch?v=C4va7d7GxjM YouTube Video] as well as the [https://www.sharcnet.ca/help/index.php/Online_Seminars SHARCNET Online Seminars] page for slides. | * SHARCNET General Interest Webinar, "Singularity", presented by Paul Preney on Feb. 14, 2018. See this [https://www.youtube.com/watch?v=C4va7d7GxjM YouTube Video] as well as the [https://www.sharcnet.ca/help/index.php/Online_Seminars SHARCNET Online Seminars] page for slides. | ||