QIIME

QIIME (pronounced chime) stands for Quantitative Insights Into Microbial Ecology, is an open-source bioinformatics pipeline for performing microbiome analysis from raw DNA sequencing data. QIIME is designed to take users from raw sequencing data generated on Illumina or other platforms to publication-quality graphics and statistics. This includes demultiplexing and quality filtering, OTU picking, taxonomic assignment, phylogenetic reconstruction, diversity analyses and visualizations. QIIME has been applied to studies based on billions of sequences from tens of thousands of samples.

Note: QIIME 2 has replaced QIIME 1 as of January 1, 2018; version 1 is no longer supported.

Note: As of February 18, 2020, and given the issues generated by conda environments in our HPC systems, installation using Ananconda or Miniconda is no longer supported.

Installation

QIIME 2 can be installed through a docker/singularity container or using easy build wheel. The former is strongly preferred since it does not generate thousands of files in your home directory, potentially preventing you from running your jobs.

Using Singularity (preferred)

It is possible to run QIIME2 through singularity using preexisting images. A docker image is available at qiime2, but first you need to build a singularity image from docker:

[name@server ~]$ module load singularity
[name@server ~]$ singularity build qiime2-2019.10.sif docker://qiime2/core:2019.10

Then run your code as any other singularity job. For most QIIME usage, you want to run each QIIME command within a singularity exec statement:

[name@server ~]$ singularity exec qiime2-2019.10.sif <your QIIME command>

So your SBATCH script might look something like this:

#!/bin/bash
#SBATCH --time=15:00:00
#SBATCH --account=def-someuser

singularity exec -B /home -B /project -B /scratch \
  qiime2-2019.10.sif qiime tools import \
  --type 'FeatureData[Sequence]' \
  --input-path /path/to/some_fastafile.fa \
  --output-path /path/to/some_output_feature.qza

singularity exec -B /home -B /project -B /scratch \
  qiime tools import \
  --type 'FeatureData[Taxonomy]' \
  --input-format HeaderlessTSVTaxonomyFormat \
  --input-path /path/to/some_taxonomy_file.tax \
  --output-path /path/to/some_output_ref-taxonomy.qza

singularity exec -B /home -B /project -B /scratch \
  qiime feature-classifier fit-classifier-naive-bayes \
  --i-reference-reads  /path/to/some_output_feature.qza \
  --i-reference-taxonomy /path/to/some_output_ref-taxonomy.qza \
  --o-classifier /path/to/some_output_classifier.qza

Note that it is important to bind the folders you want to work with to the execution of the container. For more information about singularity, you can watch the recorded singularity webminar.

On first importing data into QIIME format you may receive an error ending with a message like this:

Timezone offset does not match system offset: 0 != -18000. Please, check your config files.

This can be worked around by setting a time zone before invoking Singularity:

[name@server ~]$ export TZ='UTC'
[name@server ~]$ singularity exec qiime2-2019.10.sif qiime tools import ...

With the EasyBuild wheel

In case the singularity solution isn't working for you, we also provide an EasyBuild package that can be used with the eb command. To install version 2019.7, use

[name@server ~]$ eb --rebuild Miniconda3-4.7.10.eb QIIME2-2019.7.eb

This takes a while, but will provide you with the qiime/2019.7 module that you need to load before using QIIME 2.

[name@server ~]$ module load qiime2/2019.7

Because this creates all the packages required for QIIME, it generates too many files in your home directory. We recommend that you remove those files once you are done with all computations since it uses almost half of your total allocation. Even better, we recommend you use our singularity solution, that creates a big file instead of thousand of smaller files.

References

QIIME homepage
EasyBuild documentation