Anaconda

Attention: While Conda works well in a desktop environment, it tends to create more problems than it solves on a cluster. For example, Conda very often installs software (compilers, scientific libraries etc.) which is already available on the Compute Canada clusters in the form of modules but with a far from ideal configuration. With the installation of all of this additional software by Conda, you also risk exceeding the quota on the number of files in your home directory.

Do not install Anaconda on our clusters

We are aware of the fact that Anaconda is widely used in several domains, such as data science, AI, bioinformatics etc. Anaconda is a useful solution for simplifying the management of Python and scientific libraries on a personal computer. However, on a cluster like those supported by Compute Canada, the management of these libraries and dependencies should be done by our staff, in order to ensure compatibility and optimal performance. Here is a list of reasons:

• Anaconda very often installs software (compilers, scientific libraries etc.) which already exist on Compute Canada clusters as modules, with a configuration that is not optimal.
• It installs binaries which are not optimized for the processor architecture on our clusters.
• It makes incorrect assumptions about the location of various system libraries.
• Anaconda uses the $HOME directory for its installtion, where it writes an enormous number of files. A single Anaconda installation can easily absorb almost half of your quota for the number of files in your home directory.
• Anaconda is slower than the installation of packages via Python wheels.
• Anaconda modifies the $HOME/.bashrc file, which can easily cause conflicts.

How to transition from Conda to Virtualenv

A virtual environment offers you all the functionality which you need to use Python on our clusters. Here is how to convert to the use of virtual environments if you use Anaconda on your personal computer:

1. List the dependencies (requirements) of the application you want to use.
2. Find which dependencies are Python modules and which are libraries provided by Anaconda. For example, CUDA and CuDNN are libraries which are available on Anaconda Cloud but which you should not install yourself on our clusters - they are already installed.
3. Remove from the list of dependencies everything which is not a Python module (e.g. cudatoolkit and cudnn).
4. Use a virtual environment in which you will install your dependencies.

Votre application devrait fonctionner. Si ce n'est pas le cas, n'hésitez pas à contacter notre soutien technique.