SpaCy: Difference between revisions
Jump to navigation
Jump to search
m (Added section for gpu fix on cuda libraries) |
No edit summary |
||
| (5 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
<languages /> | <languages /> | ||
[[Category:Software]] | [[Category:Software]][[Category:AI and Machine Learning]] | ||
<translate> | <translate> | ||
| Line 6: | Line 6: | ||
[https://spacy.io/ spaCy] is a Python package that provides industrial-strength natural language processing. | [https://spacy.io/ spaCy] is a Python package that provides industrial-strength natural language processing. | ||
= Installation = | = Installation = <!--T:26--> | ||
==Latest available wheels== | |||
==Latest available wheels== <!--T:31--> | |||
<!--T:32--> | |||
To see the latest version of <tt>spaCy</tt> that we have built: | To see the latest version of <tt>spaCy</tt> that we have built: | ||
{{Command|avail_wheels spacy thinc thinc_gpu_ops}} | {{Command|avail_wheels spacy thinc thinc_gpu_ops}} | ||
| Line 16: | Line 19: | ||
<!--T:28--> | <!--T:28--> | ||
The preferred option is to install it using the python [https://pythonwheels.com/ wheel] that we compile, as follows: | The preferred option is to install it using the python [https://pythonwheels.com/ wheel] that we compile, as follows: | ||
:1. Load | :1. Load python 3.6 module: <tt>python/3.6</tt> | ||
:2. Create and | :2. Create and activate a [[Python#Creating_and_using_a_virtual_environment|virtual environment]]. | ||
:3. Install <tt>spaCy</tt> in the virtual environment with <code>pip install</code>. For both GPU and CPU support: | :3. Install <tt>spaCy</tt> in the virtual environment with <code>pip install</code>. For both GPU and CPU support: | ||
:{{Command|prompt=(venv) [name@server ~]|pip install spacy[cuda] --no-index}} | :{{Command|prompt=(venv) [name@server ~]|pip install spacy[cuda] --no-index}} | ||
| Line 23: | Line 26: | ||
:{{Command|prompt=(venv) [name@server ~]|pip install spacy --no-index}} | :{{Command|prompt=(venv) [name@server ~]|pip install spacy --no-index}} | ||
'''GPU version''': | <!--T:30--> | ||
'''GPU version''': At the present time, in order to use the GPU version you need to add the CUDA libraries to <tt>LD_LIBRARY_PATH</tt>: | |||
{{Commands | {{Commands | ||
|prompt=(venv) [name@server ~] | |prompt=(venv) [name@server ~] | ||
| Line 31: | Line 35: | ||
<!--T:29--> | <!--T:29--> | ||
If you want to use the [https://docs.computecanada.ca/wiki/PyTorch Pytorch] wrapper with <tt>thinc</tt>, you'll also need to install the <tt>torch_cpu</tt> or <tt>torch_gpu</tt> wheel. | |||
</translate> | </translate> | ||
Latest revision as of 13:02, 19 July 2022
spaCy is a Python package that provides industrial-strength natural language processing.
Installation
Latest available wheels
To see the latest version of spaCy that we have built:
[name@server ~]$ avail_wheels spacy thinc thinc_gpu_ops
For more information on listing wheels, see listing available wheels.
Pre-build
The preferred option is to install it using the python wheel that we compile, as follows:
- 1. Load python 3.6 module: python/3.6
- 2. Create and activate a virtual environment.
- 3. Install spaCy in the virtual environment with
pip install. For both GPU and CPU support: -
(venv) [name@server ~] pip install spacy[cuda] --no-index
- If you only need CPU support:
-
(venv) [name@server ~] pip install spacy --no-index
GPU version: At the present time, in order to use the GPU version you need to add the CUDA libraries to LD_LIBRARY_PATH:
(venv) [name@server ~] module load gcc/5.4.0 cuda/9
(venv) [name@server ~] export LD_LIBRARY_PATH=$CUDA_HOME/lib64:$LD_LIBRARY_PATH
If you want to use the Pytorch wrapper with thinc, you'll also need to install the torch_cpu or torch_gpu wheel.