Huggingface

Hugging Face is an organization that builds and maintains several popular open-source software packages widely used in Artificial Intelligence research. In this article, you will find information and tutorials on how to use packages from the Hugging Face ecosystem on our clusters.

Transformers

Transformers is a python package that provides APIs and tools to easily download and train state-of-the-art models, pre-trained on various tasks in multiple domains.

Installing Transformers

Our recommendation is to install it using our provided Python wheel as follows:

1. Load a Python module, thus module load python

2. Create and start a virtual environment.

3. Install Transformers in the virtual environment with pip install.

(venv) [name@server ~] pip install --no-index transformers

Downloading pre-trained models

To download a pre-trained model from the Hugging Face model hub, choose one of the options below and follow the instructions on the login node of the cluster you are working on.

Using git lfs

Pre-trained models are usually made up of fairly large binary files. The Hugging Face makes these files available for download via Git Large File Storage. To download a model, load the git-lfs module and clone your chosen model repository from the model hub:

module load git-lfs/3.3.0
git clone https://huggingface.co/bert-base-uncased

Now that you have a copy of the pre-trained model saved locally in the cluster's filesystem, you can load it with a python script inside a job with the local_files_only option to avoid attempts to download it from the web:

from transformers import AutoModel, AutoTokenizer
model = AutoModel.from_pretrained("/path/to/where/you/cloned/the/model", local_files_only=True)
tokenizer = AutoTokenizer.from_pretrained("/path/to/where/you/cloned/the/model", local_files_only=True)

Using python

It is also possible to download pre-trained models using Python instead of Git. The following must be executed on a login node as an internet connection is required to download the model files:

from transformers import AutoModel, AutoTokenizer
model = AutoModel.from_pretrained("bert-base-uncased")
tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")

This will store the pre-trained model files in a cache directory, which defaults to $HOME/.cache/huggingface/hub. You can change the cache directory by setting the environment variable TRANSFORMERS_CACHE before you import anything from the transformers package in your Python script. For example, the following will store model files in the current working directory:

import os
os.environ['TRANSFORMERS_CACHE']="./"
from transformers import AutoModel, AutoTokenizer
model = AutoModel.from_pretrained("bert-base-uncased")
tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")

Whether you change the default cache directory location or not, you can load the pre-trained model from disk in a job by using the local_files_only option:

from transformers import AutoModel, AutoTokenizer
model = AutoModel.from_pretrained("/path/to/where/model/is/saved", local_files_only=True)
tokenizer = AutoTokenizer.from_pretrained("/path/to/where/model/is/saved", local_files_only=True)

Using a pipeline

Another frequently used way of loading a pre-trained model is via a pipeline. On a login node you can simply pass a model name or a type of task as an argument to pipeline. This will download and store the model at the default cache location:

from transformers import pipeline
pipe = pipeline("text-classification")

In an environment without internet connection however, such as inside a job, you must specify the location of the model as well as its tokenizer when calling pipeline:

 from transformers import pipeline, AutoModel, AutoTokenizer
 model = AutoModel.from_pretrained("/path/to/where/model/is/saved", local_files_only=True)
 tokenizer = AutoTokenizer.from_pretrained("/path/to/where/model/is/saved", local_files_only=True)
 pipe = pipeline(task = "text-classification", model = model, tokenizer = tokenizer)

Failing to do so will result in pipeline attempting to download models from the internet, which will result in a connection timeout error during a job.

Datasets

Datasets is a python package for easily accessing and sharing datasets for Audio, Computer Vision, and Natural Language Processing (NLP) tasks.

Installing Datasets

Our recommendation is to install it using our provided Python wheel as follows:

1. Load a Python module, thus module load python

2. Create and start a virtual environment.

3. Load the Arrow module. This will make the pyarrow package (a dependency of Datasets) available inside your virtualenv.

3. Install Datasets in the virtual environment with pip install.

(venv) [name@server ~] module load gcc/9.3.0 arrow/11.0.0

(venv) [name@server ~] pip install --no-index datasets

Note: you will need to load the arrow module you every time intend to import the Datasets package in your Python script.

Downloading Datasets

The exact method to download and use a dataset from the Hugging Face hub depends on a number of factors such as format and the type of task for which the data will be used. Regardless of the exact method used, any download must be performed on a login node. See the package's official documentation for details on how to download different types of dataset.

Once the dataset has been downloaded, it will be stored locally in a cache directory, which defaults to $HOME/.cache/huggingface/datasets. It is possible to change the default cache location by setting the environment variable HF_DATASETS_CACHE before you import anything from the Datasets package in your python script.

To load a dataset in a job where there is no internet connection, set the environment variable HF_DATASETS_OFFLINE=1 and specify the location of the cache directory where the dataset is stored when calling load_dataset():

import os
os.environ['HF_DATASETS_OFFLINE'] = '1'
from datasets import load_dataset
dataset = load_dataset('name_of_the_loading_script', cache_dir="/path/to/where/dataset/is/saved")