QIIME
QIIME (pronounced chime) stands for Quantitative Insights Into Microbial Ecology, is an open-source bioinformatics pipeline for performing microbiome analysis from raw DNA sequencing data. QIIME is designed to take users from raw sequencing data generated on Illumina or other platforms to publication-quality graphics and statistics. This includes demultiplexing and quality filtering, OTU picking, taxonomic assignment, phylogenetic reconstruction, diversity analyses and visualizations. QIIME has been applied to studies based on billions of sequences from tens of thousands of samples.
Note: QIIME 2 has replaced QIIME 1 as of January 1, 2018; version 1 is no longer supported.
Note: As of February 2020, due to various issues generated by Conda environments on our HPC systems, installation using Anaconda or Miniconda is no longer supported.
QIIME2 as a module
QIIME2 is available in our environment via a module which wraps around a container. To know which versions are available, run
[name@server ~]$ module spider qiime2
After loading the module, you can run
[name@server ~]$ qiime --help
Since the qiime command is in fact calling a container, you may have to define the environment variable APPTAINER_BIND to bind specific folders within the container to access your data. For example: APPTAINER_BIND=/home qiime ...
Minimal Example
A minimal example of a job submission script:
#!/bin/bash
#SBATCH --account=def-someprof # adjust this to match the accounting group you are using to submit jobs
#SBATCH --mem-per-cpu=4096M # adjust this according to the memory you need
#SBATCH --cpus-per-task=1 # adjust this according to the number of core you need
#SBATCH --time=08:00:00 # adjust this to match the walltime of your job
module load StdEnv/2023 qiime2/2024.5
# Change directory to where our data live
cd $SCRATCH
# https://apptainer.org/docs/user/main/appendix.html
# Bind source:destination, bind current directory to /data
export APPTAINER_BIND=$PWD:/data
# Set the working directory to be used for /tmp, /var/tmp and $HOME
export APPTAINER_WORKDIR=$SLURM_TMPDIR
# run with /data mounted
qiime tools import --input-path /data/input.fasta --output-path /data/output.fasta.qza --type 'FeatureData[Sequence]'
Installation
QIIME2 can be installed using Apptainer or EasyBuild. Apptainer is strongly preferred since it does not generate many thousands of files in your /home directory, potentially causing you to exceed the disk quota limit on the number of files.
This page or section contains obsolete information and some statements may not be valid. The technical documentation is currently being updated by our support team.
Using Apptainer
The QIIME2 developers publish images on Quay.io. In order to use one of these images on our systems, you must first build an Apptainer image:
[name@server ~]$ module load apptainer
[name@server ~]$ apptainer build qiime2-2021.11.sif docker://quay.io/qiime2/core:2021.11
This build step may take over an hour, but you only need to do this once. Save the image file (qiime2-2021.11.sif in this example) for later re-use.
Then run your code as described in the Apptainer page. You will typically run each QIIME command in a apptainer exec statement:
[name@server ~]$ apptainer exec qiime2-2021.11.sif <your QIIME command>
So your SBATCH script might look something like this:
#!/bin/bash #SBATCH --time=15:00:00 #SBATCH --account=def-someuser apptainer exec -B $PWD:/home -B /scratch/someuser:/outputs \ -B /project/def-somePI/someuser/path/to/inputs:/inputs qiime2-2021.11.sif \ qiime tools import --type 'FeatureData[Sequence]' \ --input-path /inputs/some_fastafile.fa \ --output-path /outputs/some_output_feature.qza apptainer exec -B $PWD:/home -B /scratch/someuser:/outputs \ -B /project/def-somePI/someuser/path/to/inputs:/inputs qiime2-2021.11.sif \ qiime tools import \ --type 'FeatureData[Taxonomy]' \ --input-format HeaderlessTSVTaxonomyFormat \ --input-path /inputs/some_taxonomy_file.tax \ --output-path /outputs/some_output_ref-taxonomy.qza apptainer exec -B $PWD:/home -B /scratch/someuser:/outputs \ -B /project/def-somePI/someuser/path/to/inputs:/inputs qiime2-2021.11.sif \ qiime feature-classifier fit-classifier-naive-bayes \ --i-reference-reads /outputs/some_output_feature.qza \ --i-reference-taxonomy /outputs/some_output_ref-taxonomy.qza \ --o-classifier /outputs/some_output_classifier.qza
Note that it is important to use the bind option (-B) with each folder you want to work with when you run programs in your container. For more information about Apptainer, you can watch this Apptainer webinar.
On first importing data into QIIME format, you may receive an error ending with a message like this:
Timezone offset does not match system offset: 0 != -18000. Please, check your config files.
This can be worked around by setting a time zone before invoking Apptainer:
[name@server ~]$ export TZ='UTC'
[name@server ~]$ apptainer exec qiime2-2021.11.sif qiime tools import ...