cc_staff
282
edits
Huggingface: Difference between revisions
Jump to navigation
Jump to search
m
→Using git lfs
(Created page with "[http://huggingface.co Hugging Face] is an organization that builds and maintains several popular open-source software packages widely used in Artificial Intelligence research. In this article, you will find information and tutorials on how to use packages from the Hugging Face ecosystem on our clusters. =Transformers= Transformers is a python package that provides APIs and tools to easily download and train state-of-the-art pre-trained models on various tasks in multi...") |
m (→Using git lfs) |
||
| Line 20: | Line 20: | ||
==Using git lfs== | ==Using git lfs== | ||
Pre-trained models are usually made up of fairly large binary files. The Hugging Face makes these files available for download via [https://git-lfs.com/ Git Large File Storage]. To download a model, load the <tt>git-lfs</tt> module and clone your chosen model repository from | Pre-trained models are usually made up of fairly large binary files. The Hugging Face makes these files available for download via [https://git-lfs.com/ Git Large File Storage]. To download a model, load the <tt>git-lfs</tt> module and clone your chosen model repository from the model hub: | ||
module load git-lfs/3.3.0 | module load git-lfs/3.3.0 | ||
git clone https://huggingface.co/bert-base-uncased | git clone https://huggingface.co/bert-base-uncased | ||
| Line 28: | Line 28: | ||
model = AutoModel.from_pretrained("/path/to/where/you/cloned/the/model", local_files_only=True) | model = AutoModel.from_pretrained("/path/to/where/you/cloned/the/model", local_files_only=True) | ||
tokenizer = AutoTokenizer.from_pretrained("/path/to/where/you/cloned/the/model", local_files_only=True) | tokenizer = AutoTokenizer.from_pretrained("/path/to/where/you/cloned/the/model", local_files_only=True) | ||
==Using python== | ==Using python== | ||