记录一个奇葩的huggingface数据加载问题

使用dataset = datasets.load_dataset("beyond/rlhf-reward-single-round-trans_chinese")下载数据集会报错:

FileNotFoundError: [Errno 2] No such file or directory: 'C:/Users/Chenxm/.cache/huggingface/datasets/beyond___rlhf-reward-single-round-trans_chinese/default-56c83c4a1ab39cac/0.0.0/e58c486e4bad3c9cf8d969f920449d1103bbdf069a7150db2cf96c695aeca990.incomplete/rlhf-reward-single-round-trans_chinese-train-00000-00000-of-NNNNN.arrow'

打开该路径确实啥都没有。

只用设置个缓存参数就可以下载

dataset = datasets.load_dataset("beyond/rlhf-reward-single-round-trans_chinese", cache_dir="./dataset")


posted @ 2023-08-10 17:45  sunshine丶23  阅读(528)  评论(0编辑  收藏  举报