Bureaucrats, cc_docs_admin, cc_staff
2,879
edits
Biorepo containers: Difference between revisions
Jump to navigation
Jump to search
add some links
(Removed redirect to Biorepo containers) Tag: Removed redirect |
(add some links) |
||
| Line 3: | Line 3: | ||
=Introduction= | =Introduction= | ||
Containers as a method of distributing software along with their often complex dependencies has recently become commonplace thanks to the rising popularity of container managements systems and runtime environments such as Docker and Singularity, now called Apptainer. Users on the Alliance system are free to use containers of their own in order to bring their software to the Alliance systems. While many users will simply build containers and store them in their own project spaces, groups with many users, those who wish to run the same container on many systems, or those who need to preserve a complex workflow for some period of time may choose to deposit their container image into the CVMFS repository described below. For container authors and contributors, this article outlines the requirements containers need to meet to be hosted in the repository. For users of these containers, this article will outline available containers and some information on their use. | Containers as a method of distributing software along with their often complex dependencies has recently become commonplace thanks to the rising popularity of container managements systems and runtime environments such as Docker and Singularity, now called [[Apptainer]]. Users on the Alliance system are free to use containers of their own in order to bring their software to the Alliance systems. While many users will simply build containers and store them in their own project spaces, groups with many users, those who wish to run the same container on many systems, or those who need to preserve a complex workflow for some period of time may choose to deposit their container image into the [[CVMFS]] repository described below. For container authors and contributors, this article outlines the requirements containers need to meet to be hosted in the repository. For users of these containers, this article will outline available containers and some information on their use. | ||
=Container repository= | =Container repository= | ||
The Research Software National Team (RSNT) and the Bioinformatics National Team (BNT) operate a container repository to store Apptainer images in CVMFS to make these containers available on all Alliance systems. The BNT maintains the portion of that repository of interest to bioinformatics-focused users. This not only reduces the effort to run workflows across our infrastructure, but allows for the hard work of creating workflows to be leveraged by others in the community while enabling a high degree of reproducibility. Worth noting is the fact these container images will be available to all users on Alliance systems and container authors must take this into account when considering this service for workflows they wish to keep private. | The Research Software National Team (RSNT) and the Bioinformatics National Team (BNT) operate a container repository to store [[Apptainer]] images in [[CVMFS]] to make these containers available on all Alliance systems. The BNT maintains the portion of that repository of interest to bioinformatics-focused users. This not only reduces the effort to run workflows across our infrastructure, but allows for the hard work of creating workflows to be leveraged by others in the community while enabling a high degree of reproducibility. Worth noting is the fact these container images will be available to all users on Alliance systems and container authors must take this into account when considering this service for workflows they wish to keep private. | ||
We support Apptainer | We support [[Apptainer]] (formerly [[Singularity]]) containers on our systems. Some examples of supported use cases are: | ||
# Containers with mature workflows, especially for the clinical studies, that are needed for long-term projects (3-5 years). These types of containers can provide reproducible workflows and tools commonly used in this type of research. | # Containers with mature workflows, especially for the clinical studies, that are needed for long-term projects (3-5 years). These types of containers can provide reproducible workflows and tools commonly used in this type of research. | ||
# Containers that require Conda and can not easily be ported to | # Containers that require [[Anaconda/en|Conda]] and can not easily be ported to use [[Python#Creating_and_using_a_virtual_environment|pip]] and our [[Available_software|modules]]. This type of container can be used when the desired tools must be or are much more easily installed through Conda or a similar running environment. | ||
# Containers with most of the tools, packages or libraries installed for a certain workflow, such as scRNA-seq. This type of container has the commonly used tools as well as the prerequisite libraries and system files which make it easier to update the version of the desired tools or install new ones. | # Containers with most of the tools, packages or libraries installed for a certain workflow, such as scRNA-seq. This type of container has the commonly used tools as well as the prerequisite libraries and system files which make it easier to update the version of the desired tools or install new ones. | ||
Additionally, all applications in a container must be open source and governed by a license that allows us to distribute the software in this manner. | Additionally, all applications in a container must be open source and governed by a license that allows us to distribute the software in this manner. | ||
| Line 19: | Line 19: | ||
If you have a container image you would like to incorporate into this system, please provide us with the following information: | If you have a container image you would like to incorporate into this system, please provide us with the following information: | ||
# An Apptainer | # An Apptainer or Singularity recipe file along with a brief description of the container to be made available on the wiki page. Ensure that the recipe is reproducible: pin the version of software being installed as much as possible. | ||
# Instructions and/or examples of usage if your container implements a complex workflow rather than simply making a piece of software available. | # Instructions and/or examples of usage if your container implements a complex workflow rather than simply making a piece of software available. | ||
# The audience for this image: Containers that we are intended to have a reach beyond their originating contributor. What is the number of research groups/PIs involved and the number of users who would be using the image? | # The audience for this image: Containers that we are intended to have a reach beyond their originating contributor. What is the number of research groups/PIs involved and the number of users who would be using the image? | ||