Anaconda

Attention: While Conda works well in a desktop environment, it tends to create more problems than it solves on a cluster. For example, Conda very often installs software (compilers, scientific libraries etc.) which is already available on the Compute Canada clusters in the form of modules but with a far from ideal configuration. With the installation of all of this additional software by Conda, you also risk exceeding the quota on the number of files in your home directory.

Do not install Anaconda on our clusters

We are aware of the fact that Anaconda is widely used in several domains, such as data science, AI, bioinformatics etc. Anaconda is a useful solution for simplifying the management of Python and scientific libraries on a personal computer. However, on a cluster like those supported by Compute Canada, the management of these libraries and dependencies should be done by our staff, in order to ensure compatibility and optimal performance. Here is a list of reasons:

• Anaconda very often installs software (compilers, scientific libraries etc.) which already exist on Compute Canada clusters as modules, with a configuration that is not optimal.
• It installs binaries which are not optimized for the processor architecture on our clusters.
• It makes incorrect assumptions about the location of various system libraries.
• Anaconda uses the $HOME directory for its installtion, where it writes an enormous number of files. A single Anaconda installation can easily absorb almost half of your quota for the number of files in your home directory.
• Anaconda is slower than the installation of packages via Python wheels.
• Anaconda modifies the $HOME/.bashrc file, which can easily cause conflicts.

How to transition from Conda to Virtualenv

A virtual environment offers you all the functionality which you need to use Python on our clusters. Here is how to convert to the use of virtual environments if you use Anaconda on your personal computer:

1. Listez les dépendances (requirements) de l'application que vous voulez utiliser.
2. Trouvez quelles dépendances sont des paquets Python, et lesquelles sont des librairies fournies par Anaconda. Par exemple, CUDA et CuDNN sont des librairies disponible sur l'Anaconda Cloud, mais que vous ne devez pas installer vous-même sur nos grappes. Elles sont déjà installées.
3. Retirez de la liste de dépendance tout ce qui n'est pas un paquet Python (par exemple, retirez cudatoolkit et cudnn).
4. Utilisez un virtualenv, dans lequel vous installerez ces dépendances.

Votre application devrait fonctionner. Si ce n'est pas le cas, n'hésitez pas à contacter notre soutien technique.