Faiss: Difference between revisions
(Added page on Faiss) |
No edit summary |
||
| (5 intermediate revisions by 2 users not shown) | |||
| Line 2: | Line 2: | ||
<translate> | <translate> | ||
[https://github.com/facebookresearch/faiss/wiki Faiss] is a library for efficient similarity search and clustering of dense vectors. It contains algorithms that search in sets of vectors of any size, up to ones that possibly do not fit in RAM. It also contains supporting code for evaluation and parameter tuning. Faiss is written in C++ with complete wrappers for Python (versions 2 and 3). Some of the most useful algorithms are implemented on | <!--T:1--> | ||
[https://github.com/facebookresearch/faiss/wiki Faiss] is a library for efficient similarity search and clustering of dense vectors. It contains algorithms that search in sets of vectors of any size, up to ones that possibly do not fit in RAM. It also contains supporting code for evaluation and parameter tuning. Faiss is written in C++ with complete wrappers for Python (versions 2 and 3). Some of the most useful algorithms are implemented on GPU. It is developed primarily at [https://research.facebook.com/ Meta AI Research] with help from external contributors. | |||
<!--T:2--> | |||
__TOC__ | __TOC__ | ||
== Python bindings == | == Python bindings == <!--T:3--> | ||
The module contains bindings for multiple Python versions. | The module contains bindings for multiple Python versions. | ||
To discover which are the compatible Python versions, run | To discover which are the compatible Python versions, run | ||
| Line 14: | Line 16: | ||
where <TT>X.Y.Z</TT> represent the desired version. | where <TT>X.Y.Z</TT> represent the desired version. | ||
=== Usage === | === Usage === <!--T:4--> | ||
1. Load the required modules. | 1. Load the required modules. | ||
{{Command|module load StdEnv/2023 gcc cuda faiss/X.Y.Z python/3.11}} | {{Command|module load StdEnv/2023 gcc cuda faiss/X.Y.Z python/3.11}} | ||
where <TT>X.Y.Z</TT> represent the desired version. | where <TT>X.Y.Z</TT> represent the desired version. | ||
2. Import Faiss | <!--T:5--> | ||
2. Import Faiss. | |||
{{Command|python -c "import faiss"}} | {{Command|python -c "import faiss"}} | ||
<!--T:6--> | |||
If the command displays nothing, the import was successful. | If the command displays nothing, the import was successful. | ||
==== Available Python packages ==== | ==== Available Python packages ==== <!--T:7--> | ||
Other Python packages depend on <tt>faiss-cpu</tt> or <tt>faiss-gpu</tt> bindings in order to be installed. | Other Python packages depend on <tt>faiss-cpu</tt> or <tt>faiss-gpu</tt> bindings in order to be installed. | ||
faiss module provides: | The <code>faiss</code> module provides: | ||
* <code>faiss</code> | * <code>faiss</code> | ||
* <code>faiss-gpu</code> | * <code>faiss-gpu</code> | ||
* <code>faiss-cpu</code> | * <code>faiss-cpu</code> | ||
<!--T:8--> | |||
{{Command | {{Command | ||
|pip list {{!}} fgrep faiss | |pip list {{!}} fgrep faiss | ||
| Line 39: | Line 44: | ||
}} | }} | ||
With the <code>faiss</code> module loaded, | <!--T:9--> | ||
With the <code>faiss</code> module loaded, package dependency for the above extensions will be satisfied. | |||
</translate> | </translate> | ||
Latest revision as of 21:15, 1 May 2024
Faiss is a library for efficient similarity search and clustering of dense vectors. It contains algorithms that search in sets of vectors of any size, up to ones that possibly do not fit in RAM. It also contains supporting code for evaluation and parameter tuning. Faiss is written in C++ with complete wrappers for Python (versions 2 and 3). Some of the most useful algorithms are implemented on GPU. It is developed primarily at Meta AI Research with help from external contributors.
Python bindings
The module contains bindings for multiple Python versions. To discover which are the compatible Python versions, run
[name@server ~]$ module spider faiss/X.Y.Z
Or search directly faiss-cpu, by running
[name@server ~]$ module spider faiss-cpu/X.Y.Z
where X.Y.Z represent the desired version.
Usage
1. Load the required modules.
[name@server ~]$ module load StdEnv/2023 gcc cuda faiss/X.Y.Z python/3.11
where X.Y.Z represent the desired version.
2. Import Faiss.
[name@server ~]$ python -c "import faiss"
If the command displays nothing, the import was successful.
Available Python packages
Other Python packages depend on faiss-cpu or faiss-gpu bindings in order to be installed.
The faiss module provides:
faissfaiss-gpufaiss-cpu
[name@server ~]$ pip list | fgrep faiss
faiss-gpu 1.7.4
faiss-cpu 1.7.4
faiss 1.7.4
With the faiss module loaded, package dependency for the above extensions will be satisfied.