记录一个奇葩的huggingface数据加载问题
使用dataset = datasets.load_dataset("beyond/rlhf-reward-single-round-trans_chinese")下载数据集会报错:
FileNotFoundError: [Errno 2] No such file or directory: 'C:/Users/Chenxm/.cache/huggingface/datasets/beyond___rlhf-reward-single-round-trans_chinese/default-56c83c4a1ab39cac/0.0.0/e58c486e4bad3c9cf8d969f920449d1103bbdf069a7150db2cf96c695aeca990.incomplete/rlhf-reward-single-round-trans_chinese-train-00000-00000-of-NNNNN.arrow'
打开该路径确实啥都没有。
只用设置个缓存参数就可以下载
dataset = datasets.load_dataset("beyond/rlhf-reward-single-round-trans_chinese", cache_dir="./dataset")