Alliance Doc

QIIME/en: Difference between revisions

Page Discussion

QIIME/en (view source)

Revision as of 19:49, 18 February 2020

1,146 bytes added ,  4 years ago
Updating to match new version of source page
(Updating to match new version of source page)
(Updating to match new version of source page)
Line 3: Line 3:


'''Note''': QIIME 2 has replaced QIIME 1 as of January 1, 2018; version 1 is no longer supported.
'''Note''': QIIME 2 has replaced QIIME 1 as of January 1, 2018; version 1 is no longer supported.
'''Note''': As of February, 2020, due to various issues generated by Conda environments in our HPC systems, '''installation using Anaconda or Miniconda is no longer supported'''.


==Installation ==
==Installation ==
QIIME 2 can be installed in your home directory using conda.  It is not possible to install this software using pip at this time.
QIIME2 can be installed using [[Singularity]] or using EasyBuild. Singularity is strongly preferred since it does not generate many thousands of files in your home directory, potentially causing you to exceed the disk quota limit on the number of files.
 
=== Using Singularity ===
 
The QIIME2 developers publish images on [https://hub.docker.com/u/qiime2 Docker Hub].  In order to use one of these images on our systems you must first [[Singularity/en#Creating_an_image_using_Docker_Hub|build a Singularity image]]:
 
{{Commands
|module load singularity
|singularity build qiime2-2019.10.sif docker://qiime2/core:2019.10
}}


The installation procedure depends on whether or not you have a [https://www.anaconda.com/ conda] environment.
This build step may take over an hour, but you only need do this once.  Save the image file (<code>qiime2-2019.10.sif</code> in this example)
for later re-use.  


===With a conda environment===
Then run your code as described at [[Singularity]]. Typically you will run each QIIME command in a <code>singularity exec</code> statement:
Although we '''do not recommend''' using conda environments in an HPC environment, we provide the option to do so at your own peril.


General installation instructions by the QIIME 2 developers are found [https://docs.qiime2.org/ here]. Instructions adapted for Compute Canada systems are given here for version 2019.7.  If a new version of the <tt>.yml</tt> file is released (check the link), use the newer version by updating the instructions below.
{{Commands
<pre>
|singularity exec qiime2-2019.10.sif <your QIIME command>
eb Miniconda3-4.5.12.eb
}}
module load miniconda3
conda update conda
wget https://data.qiime2.org/distro/core/qiime2-2019.7-py36-linux-conda.yml
conda env create -n qiime2-2019.7 --file qiime2-2019.7-py36-linux-conda.yml
conda init bash
conda activate qiime2-2019.7
</pre>
Remember to log out and log back in after the "conda init bash" step.


To run QIIME 2, use
So your [[Running jobs|SBATCH]] script might look something like this:
<pre>
module load miniconda3
conda activate qiime2-2019.7
</pre>


===Without a conda environment (preferred)===
Since we know that conda environments can create more problems that they solve in HPC systems, we provide EasyBuild packages that can be used with the <tt>eb</tt> command. To install version 2019.7, use
<pre>
eb --rebuild Miniconda3-4.7.10.eb QIIME2-2019.7.eb
</pre>
This takes a while, but will provide you with the qiime/2019.7 module that you need to load before using QIIME 2.
<pre>
<pre>
module load qiime2/2019.7
#!/bin/bash
</pre>
#SBATCH --time=15:00:00
#SBATCH --account=def-someuser


=== With singularity ===
singularity exec -B /home -B /project -B /scratch qiime2-2019.10.sif \
  qiime tools import \
  --type 'FeatureData[Sequence]' \
  --input-path /path/to/some_fastafile.fa \
  --output-path /path/to/some_output_feature.qza


It is also possible to run QIIME2 through singularity. A docker image is available at [https://hub.docker.com/u/qiime2 qiime2]. First, you need to build a singularity image from [[Singularity/en#Creating_an_image_using_Docker_Hub|docker]]:
singularity exec -B /home -B /project -B /scratch qiime2-2019.10.sif \
  qiime tools import \
  --type 'FeatureData[Taxonomy]' \
  --input-format HeaderlessTSVTaxonomyFormat \
  --input-path /path/to/some_taxonomy_file.tax \
  --output-path /path/to/some_output_ref-taxonomy.qza


{{Commands
singularity exec -B /home -B /project -B /scratch qiime2-2019.10.sif \
|module load singularity
  qiime feature-classifier fit-classifier-naive-bayes \
|singularity build qiime2-2019.10.sif docker://qiime2/core:2019.10
  --i-reference-reads  /path/to/some_output_feature.qza \
}}
  --i-reference-taxonomy /path/to/some_output_ref-taxonomy.qza \
  --o-classifier /path/to/some_output_classifier.qza
</pre>


Then run your code as any other singularity [[Singularity/en|job]].
Note that it is important to [[Singularity#Bind_mounts|bind]] the folders you want to work with to the execution of the container. For more information about Singularity, you can watch the recorded [https://www.youtube.com/watch?v=kYb0aXS5DEE Singularity webinar].


On first importing data into QIIME format you may receive an error ending with a message like this:
On first importing data into QIIME format you may receive an error ending with a message like this:
Line 61: Line 66:
|singularity exec qiime2-2019.10.sif qiime tools import ...
|singularity exec qiime2-2019.10.sif qiime tools import ...
}}
}}
=== Using an EasyBuild wheel ===
In case the Singularity solution isn't working for you, we also provide an EasyBuild package that can be used with the <tt>eb</tt> command. To install version 2019.7, use
{{Commands
|eb --rebuild Miniconda3-4.7.10.eb QIIME2-2019.7.eb
}}
This takes a while, but will provide you with the qiime/2019.7 module that you need to load before using QIIME 2.
{{Commands
|module load qiime2/2019.7
}}
Because this creates all the packages required for QIIME, it generates too many files in your home directory. We recommend that you remove those files once you are done with all computations since it uses almost half of your total allocation.  The Singularity solution described above creates one big file instead of thousand of smaller files, which is why we recommend it.


=References =
=References =
Line 66: Line 83:
[http://qiime.org/ QIIME homepage]<br>
[http://qiime.org/ QIIME homepage]<br>
[https://easybuild.readthedocs.io/en/latest/ EasyBuild documentation]<br>
[https://easybuild.readthedocs.io/en/latest/ EasyBuild documentation]<br>
[https://docs.conda.io/projects/conda/en/latest/user-guide/getting-started.html Getting started with Conda]<br>
<!--[https://docs.conda.io/projects/conda/en/latest/user-guide/getting-started.html Getting started with Conda]<br>-->
[[Category:Bioinformatics]]
[[Category:Bioinformatics]]
[[Category:User Installed Software]]
[[Category:User Installed Software]]
FuzzyBot
Bots
38,760

edits

Retrieved from "https://docs.alliancecan.ca/wiki/Special:MobileDiff/79995"