cc_staff
282
edits
Huggingface: Difference between revisions
m
→Downloading Datasets
| Line 83: | Line 83: | ||
Once the dataset has been downloaded, it will be stored locally in a cache directory, which defaults to <tt>$HOME/.cache/huggingface/datasets</tt>. It is possible to change the default cache location by setting the environment variable <tt>HF_DATASETS_CACHE</tt> '''before''' you import anything from the Datasets package in your python script. | Once the dataset has been downloaded, it will be stored locally in a cache directory, which defaults to <tt>$HOME/.cache/huggingface/datasets</tt>. It is possible to change the default cache location by setting the environment variable <tt>HF_DATASETS_CACHE</tt> '''before''' you import anything from the Datasets package in your python script. | ||
To load a dataset in a job | To load a dataset in a job where there is no internet connection, set the environment variable <tt>HF_DATASETS_OFFLINE=1</tt> and specify the location of the cache directory where the dataset is stored when calling <tt>load_dataset()</tt>: | ||
import os | import os | ||