Huggingface: Difference between revisions

Jump to navigation Jump to search
m
Line 83: Line 83:
Once the dataset has been downloaded, it will be stored locally in a cache directory, which defaults to <tt>$HOME/.cache/huggingface/datasets</tt>. It is possible to change the default cache location by setting the environment variable <tt>HF_DATASETS_CACHE</tt> '''before''' you import anything from the Datasets package in your python script.
Once the dataset has been downloaded, it will be stored locally in a cache directory, which defaults to <tt>$HOME/.cache/huggingface/datasets</tt>. It is possible to change the default cache location by setting the environment variable <tt>HF_DATASETS_CACHE</tt> '''before''' you import anything from the Datasets package in your python script.


To load a dataset in a job, where there is no internet connection, set the environment variable <tt>HF_DATASETS_OFFLINE=1</tt> and specify the location of the cache directory where the dataset is stored when calling <tt>load_dataset()</tt>:
To load a dataset in a job where there is no internet connection, set the environment variable <tt>HF_DATASETS_OFFLINE=1</tt> and specify the location of the cache directory where the dataset is stored when calling <tt>load_dataset()</tt>:


  import os
  import os
cc_staff
282

edits

Navigation menu