cc_staff
282
edits
Huggingface: Difference between revisions
Jump to navigation
Jump to search
no edit summary
m (→Using git lfs) |
No edit summary |
||
| Line 18: | Line 18: | ||
To download a pre-trained model from the Hugging Face model hub, choose one of the options below and follow the instructions on the '''login node''' of the cluster you are working on. | To download a pre-trained model from the Hugging Face model hub, choose one of the options below and follow the instructions on the '''login node''' of the cluster you are working on. | ||
==Using git lfs== | ===Using git lfs=== | ||
Pre-trained models are usually made up of fairly large binary files. The Hugging Face makes these files available for download via [https://git-lfs.com/ Git Large File Storage]. To download a model, load the <tt>git-lfs</tt> module and clone your chosen model repository from the model hub: | Pre-trained models are usually made up of fairly large binary files. The Hugging Face makes these files available for download via [https://git-lfs.com/ Git Large File Storage]. To download a model, load the <tt>git-lfs</tt> module and clone your chosen model repository from the model hub: | ||
| Line 29: | Line 29: | ||
tokenizer = AutoTokenizer.from_pretrained("/path/to/where/you/cloned/the/model", local_files_only=True) | tokenizer = AutoTokenizer.from_pretrained("/path/to/where/you/cloned/the/model", local_files_only=True) | ||
==Using python== | ===Using python=== | ||
It is also possible to download pre-trained models using Python instead of Git. The following must be executed '''on a login node''' as an internet connection is required to download the model files: | It is also possible to download pre-trained models using Python instead of Git. The following must be executed '''on a login node''' as an internet connection is required to download the model files: | ||