书生开源大模型训练营-第6讲-作业
基础作业
- 使用 OpenCompass 评测 InternLM2-Chat-7B 模型在 C-Eval 数据集上的性能
进阶作业
- 使用 OpenCompass 评测 InternLM2-Chat-7B 模型使用 LMDeploy 0.2.0 部署后在 C-Eval 数据集上的性能
============================基础作业=========================
截图：
1、创建虚拟环境、在虚拟环境中安装opencompass
conda create --name opencompass --clone=/root/share/conda_envs/internlm-base
conda activate opencompass
屏幕输出：
(base) root@intern-studio-069640:~# conda create --name opencompass --clone=/root/share/conda_envs/internlm-base Source: /root/share/conda_envs/internlm-base Destination: /root/.conda/envs/opencompass Packages: 96 Files: 0 Downloading and Extracting Packages: Downloading and Extracting Packages: Preparing transaction: done Verifying transaction: done Executing transaction: done # # To activate this environment, use # # $ conda activate opencompass # # To deactivate an active environment, use # # $ conda deactivate (base) root@intern-studio-069640:~# conda activate opencompass (opencompass) root@intern-studio-069640:~#
从源码中安装opencompass
git clone https://github.com/open-compass/opencompass cd opencompass pip install -e .
屏幕输出：
(opencompass) root@intern-studio-069640:~/opencompass# pip install -e . Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple Obtaining file:///root/opencompass Preparing metadata (setup.py) ... done Collecting absl-py (from opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/a2/ad/e0d3c824784ff121c03cc031f944bc7e139a8f1870ffd2845cc2dd76f6c4/absl_py-2.1.0-py3-none-any.whl (133 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.7/133.7 kB 1.0 MB/s eta 0:00:00 Collecting accelerate>=0.19.0 (from opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/1b/da/24a54b9205fce3bdbaad521c35944d0b0a2d292ac5ae921e484b76312b43/accelerate-0.27.2-py3-none-any.whl (279 kB) Collecting boto3 (from opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/15/1e/cbec55e05c0577429945d785cce8e16eebf2a8bd9c5ccda2b9c6e2a51ab4/boto3-1.34.44-py3-none-any.whl (139 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 139.3/139.3 kB 398.2 kB/s eta 0:00:00 Collecting cn2an (from opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/1c/3d/3e04a822b8615904269f7126d8b019ae5c3b5c3c78397ec8bab056b02099/cn2an-0.5.22-py3-none-any.whl (224 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 225.0/225.0 kB 2.4 MB/s eta 0:00:00 Collecting cpm_kernels (from opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/af/84/1831ce6ffa87b8fd4d9673c3595d0fc4e6631c0691eb43f406d3bf89b951/cpm_kernels-1.0.11-py3-none-any.whl (416 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 416.6/416.6 kB 1.1 MB/s eta 0:00:00 Collecting datasets>=2.12.0 (from opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/74/4d/63b033169534f0742b7fe13957118cae08c83b04bfde46511f397872e2e7/datasets-2.17.0-py3-none-any.whl (536 kB) Collecting einops==0.5.0 (from opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/18/d7/ed1ce1d5e00b0cd0e1ca46a710eb00822add013048c733d5b82db490e643/einops-0.5.0-py3-none-any.whl (36 kB) Collecting evaluate>=0.3.0 (from opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/70/63/7644a1eb7b0297e585a6adec98ed9e575309bb973c33b394dae66bc35c69/evaluate-0.4.1-py3-none-any.whl (84 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 84.1/84.1 kB 458.4 kB/s eta 0:00:00 Collecting fairscale (from opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/c1/08/b3334d7b543ac10dcb129cef4f84723ab696725512f18d69ab3a784b0bf5/fairscale-0.4.13.tar.gz (266 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 266.3/266.3 kB 900.3 kB/s eta 0:00:00 Installing build dependencies ... done Getting requirements to build wheel ... done Installing backend dependencies ... done Preparing metadata (pyproject.toml) ... done Collecting func_timeout (from opencompass==0.2.2) Using cached func_timeout-4.3.5-py3-none-any.whl Collecting fuzzywuzzy (from opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/43/ff/74f23998ad2f93b945c0309f825be92e04e0348e062026998b5eefef4c33/fuzzywuzzy-0.18.0-py2.py3-none-any.whl (18 kB) Collecting jieba (from opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/c6/cb/18eeb235f833b726522d7ebed54f2278ce28ba9438e3135ab0278d9792a2/jieba-0.42.1.tar.gz (19.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 19.2/19.2 MB 2.6 MB/s eta 0:00:00 Preparing metadata (setup.py) ... done Collecting ltp (from opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/ba/9f/0b3471ebc33e27ff452f8b06ada19b7bb4a810cd6c9573d43943de1ca157/ltp-4.2.13-py3-none-any.whl (20 kB) Collecting mmengine-lite (from opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/94/23/e81857ffc29602341a506d106e0cbfc4f90f5dd29bfeb3d0e011ba375fa1/mmengine_lite-0.10.3-py3-none-any.whl (451 kB) Collecting nltk==3.8 (from opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/35/45/64f4abaa5b36b698aaeb556ae6dc533e57a6b9e72ac6fc7f0d7f9cb15bb4/nltk-3.8-py3-none-any.whl (1.5 MB) Collecting numpy==1.23.4 (from opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/0c/83/78ae18fffc185d0d57097610d5a97473ef11dbdca95f16739ee96b158087/numpy-1.23.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 17.1/17.1 MB 3.3 MB/s eta 0:00:00 Collecting openai (from opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/26/a1/75474477af2a1dae3a25f80b72bbaf20e8296191ece7fff2f67984206f33/openai-1.12.0-py3-none-any.whl (226 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 226.7/226.7 kB 502.4 kB/s eta 0:00:00 Collecting OpenCC (from opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/df/27/3d4652dcf73d1ddde83348ab167dc33372822f96eac76fd6235d5144868a/OpenCC-1.1.7-cp310-cp310-manylinux1_x86_64.whl (779 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 779.8/779.8 kB 2.8 MB/s eta 0:00:00 Collecting opencv-python-headless (from opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/71/19/3c65483a80a1d062d46ae20faf5404712d25cb1dfdcaf371efbd67c38544/opencv_python_headless-4.9.0.80-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (49.6 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 49.6/49.6 MB 1.9 MB/s eta 0:00:00 Collecting pandas<2.0.0 (from opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/49/e2/79e46612dc25ebc7603dc11c560baa7266c90f9e48537ecf1a02a0dd6bff/pandas-1.5.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.1/12.1 MB 2.3 MB/s eta 0:00:00 Collecting prettytable (from opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/4d/81/316b6a55a0d1f327d04cc7b0ba9d04058cb62de6c3a4d4b0df280cbe3b0b/prettytable-3.9.0-py3-none-any.whl (27 kB) Collecting pypinyin (from opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/f6/a2/13adff7046a0913917a30cf5a8d8524f1e49b039aa0e6ab6826ad263b176/pypinyin-0.50.0-py2.py3-none-any.whl (1.4 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.4/1.4 MB 2.0 MB/s eta 0:00:00 Collecting python-Levenshtein (from opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/c9/79/eaa5f632f10be7b9ff85673be2246926e5a6a83fc489a228a22a95b5dcf0/python_Levenshtein-0.25.0-py3-none-any.whl (9.4 kB) Collecting rank_bm25==0.2.2 (from opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/2a/21/f691fb2613100a62b3fa91e9988c991e9ca5b89ea31c0d3152a3210344f9/rank_bm25-0.2.2-py3-none-any.whl (8.6 kB) Collecting rapidfuzz (from opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/02/39/3f94121e21b78e0a2699b272a8906ee5eb6f9d70082d90784464b0a4fcc8/rapidfuzz-3.6.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.4 MB) Requirement already satisfied: requests==2.31.0 in /root/.conda/envs/opencompass/lib/python3.10/site-packages (from opencompass==0.2.2) (2.31.0) Collecting rich (from opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/be/be/1520178fa01eabe014b16e72a952b9f900631142ccd03dc36cf93e30c1ce/rich-13.7.0-py3-none-any.whl (240 kB) Collecting rouge (from opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/32/7c/650ae86f92460e9e8ef969cc5008b24798dcf56a9a8947d04c78f550b3f5/rouge-1.0.1-py3-none-any.whl (13 kB) Collecting rouge_chinese (from opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/03/0f/394cf877be7b903881020ef7217f7dc644dad158d52a9353fcab22e3464d/rouge_chinese-1.0.3-py3-none-any.whl (21 kB) Collecting rouge_score (from opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/e2/c5/9136736c37022a6ad27fea38f3111eb8f02fe75d067f9a985cc358653102/rouge_score-0.1.2.tar.gz (17 kB) Preparing metadata (setup.py) ... done Collecting sacrebleu (from opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/de/ea/025db0a39337b63d4728a900d262c39c3029b0fe76a9876ce6297b1aa6a0/sacrebleu-2.4.0-py3-none-any.whl (106 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 106.3/106.3 kB 201.1 kB/s eta 0:00:00 Collecting scikit_learn==1.2.1 (from opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/80/5e/f095ccdf24860a7548b39f93d2df03017ad3218f90a0406feb5e5661d0c7/scikit_learn-1.2.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (9.6 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 9.6/9.6 MB 1.0 MB/s eta 0:00:00 Collecting seaborn (from opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/83/11/00d3c3dfc25ad54e731d91449895a79e4bf2384dc3ac01809010ba88f6d5/seaborn-0.13.2-py3-none-any.whl (294 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 294.9/294.9 kB 546.2 kB/s eta 0:00:00 Collecting sentence_transformers==2.2.2 (from opencompass==0.2.2) Using cached sentence_transformers-2.2.2-py3-none-any.whl Collecting tabulate (from opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/40/44/4a5f08c96eb108af5cb50b41f76142f0afa346dfa99d5296fe7202a11854/tabulate-0.9.0-py3-none-any.whl (35 kB) Collecting tiktoken (from opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/16/05/5efbd91252ffb1301ea393d88ef736b33d41e75d4bcf0bd31d660050e400/tiktoken-0.6.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.8 MB) Collecting timeout_decorator (from opencompass==0.2.2) Using cached timeout_decorator-0.5.0-py3-none-any.whl Collecting tokenizers>=0.13.3 (from opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/1c/5d/cf5e122ce4f1a29f165b2a69dc33d1ff30bce303343d58a54775ddba5d51/tokenizers-0.15.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB) Requirement already satisfied: torch>=1.13.1 in /root/.conda/envs/opencompass/lib/python3.10/site-packages (from opencompass==0.2.2) (2.0.1) Collecting tqdm==4.64.1 (from opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/47/bb/849011636c4da2e44f1253cd927cfb20ada4374d8b3a4e425416e84900cc/tqdm-4.64.1-py2.py3-none-any.whl (78 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 78.5/78.5 kB 1.2 MB/s eta 0:00:00 Collecting transformers>=4.29.1 (from opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/85/f6/c5065913119c41ecad148c34e3a861f719e16b89a522287213698da911fc/transformers-4.37.2-py3-none-any.whl (8.4 MB) Collecting typer (from opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/bf/0e/c68adf10adda05f28a6ed7b9f4cd7b8e07f641b44af88ba72d9c89e4de7a/typer-0.9.0-py3-none-any.whl (45 kB) Collecting click (from nltk==3.8->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/00/2e/d53fa4befbf2cfa713304affc7ca780ce4fc1fd8710527771b58311a3229/click-8.1.7-py3-none-any.whl (97 kB) Collecting joblib (from nltk==3.8->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/10/40/d551139c85db202f1f384ba8bcf96aca2f329440a844f924c8a0040b6d02/joblib-1.3.2-py3-none-any.whl (302 kB) Collecting regex>=2021.8.3 (from nltk==3.8->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/81/8a/96a62ce98e8ff1b16db56fde3debc8a571f6b7ea42ee137eb0d995cdfa26/regex-2023.12.25-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (773 kB) Requirement already satisfied: charset-normalizer<4,>=2 in /root/.conda/envs/opencompass/lib/python3.10/site-packages (from requests==2.31.0->opencompass==0.2.2) (2.0.4) Requirement already satisfied: idna<4,>=2.5 in /root/.conda/envs/opencompass/lib/python3.10/site-packages (from requests==2.31.0->opencompass==0.2.2) (3.4) Requirement already satisfied: urllib3<3,>=1.21.1 in /root/.conda/envs/opencompass/lib/python3.10/site-packages (from requests==2.31.0->opencompass==0.2.2) (1.26.18) Requirement already satisfied: certifi>=2017.4.17 in /root/.conda/envs/opencompass/lib/python3.10/site-packages (from requests==2.31.0->opencompass==0.2.2) (2023.11.17) Collecting scipy>=1.3.2 (from scikit_learn==1.2.1->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/f5/aa/8e6071a5e4dca4ec68b5b22e4991ee74c59c5d372112b9c236ec1faff57d/scipy-1.12.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (38.4 MB) Collecting threadpoolctl>=2.0.0 (from scikit_learn==1.2.1->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/b1/2c/f504e55d98418f2fcf756a56877e6d9a45dd5ed28b3d7c267b300e85ad5b/threadpoolctl-3.3.0-py3-none-any.whl (17 kB) Requirement already satisfied: torchvision in /root/.conda/envs/opencompass/lib/python3.10/site-packages (from sentence_transformers==2.2.2->opencompass==0.2.2) (0.15.2) Collecting sentencepiece (from sentence_transformers==2.2.2->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/7f/e5/323dc813b3e1339305f888d035e2f3725084fc4dcf051995b366dd26cc90/sentencepiece-0.1.99-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB) Collecting huggingface-hub>=0.4.0 (from sentence_transformers==2.2.2->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/28/03/7d3c7153113ec59cfb31e3b8ee773f5f420a0dd7d26d40442542b96675c3/huggingface_hub-0.20.3-py3-none-any.whl (330 kB) Collecting packaging>=20.0 (from accelerate>=0.19.0->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/ec/1a/610693ac4ee14fcdf2d9bf3c493370e4f2ef7ae2e19217d7a237ff42367d/packaging-23.2-py3-none-any.whl (53 kB) Collecting psutil (from accelerate>=0.19.0->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/c5/4f/0e22aaa246f96d6ac87fe5ebb9c5a693fbe8877f537a1022527c47ca43c5/psutil-5.9.8-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (288 kB) Collecting pyyaml (from accelerate>=0.19.0->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/29/61/bf33c6c85c55bc45a29eee3195848ff2d518d84735eb0e2d8cb42e0d285e/PyYAML-6.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (705 kB) Collecting safetensors>=0.3.1 (from accelerate>=0.19.0->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/d0/ba/b2254fafc7f5fdc98a2fa4d5a5eeb029fbf9589ec87f2c230c3ac0a1dd53/safetensors-0.4.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB) Requirement already satisfied: filelock in /root/.conda/envs/opencompass/lib/python3.10/site-packages (from datasets>=2.12.0->opencompass==0.2.2) (3.13.1) Collecting pyarrow>=12.0.0 (from datasets>=2.12.0->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/d4/ca/ef67abb77f9dd51a0d3ff7fcebff58296068a046d7da352b9548070005ed/pyarrow-15.0.0-cp310-cp310-manylinux_2_28_x86_64.whl (38.3 MB) Collecting pyarrow-hotfix (from datasets>=2.12.0->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/e4/f4/9ec2222f5f5f8ea04f66f184caafd991a39c8782e31f5b0266f101cb68ca/pyarrow_hotfix-0.6-py3-none-any.whl (7.9 kB) Collecting dill<0.3.9,>=0.3.0 (from datasets>=2.12.0->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/c9/7a/cef76fd8438a42f96db64ddaa85280485a9c395e7df3db8158cfec1eee34/dill-0.3.8-py3-none-any.whl (116 kB) Collecting xxhash (from datasets>=2.12.0->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/80/8a/1dd41557883b6196f8f092011a5c1f72d4d44cf36d7b67d4a5efe3127949/xxhash-3.4.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (194 kB) Collecting multiprocess (from datasets>=2.12.0->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/bc/f7/7ec7fddc92e50714ea3745631f79bd9c96424cb2702632521028e57d3a36/multiprocess-0.70.16-py310-none-any.whl (134 kB) Collecting fsspec<=2023.10.0,>=2023.1.0 (from fsspec[http]<=2023.10.0,>=2023.1.0->datasets>=2.12.0->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/e8/f6/3eccfb530aac90ad1301c582da228e4763f19e719ac8200752a4841b0b2d/fsspec-2023.10.0-py3-none-any.whl (166 kB) Collecting aiohttp (from datasets>=2.12.0->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/93/40/d3decda219ebd5410eba627601d537ec3782efbcadba308e9ce381cc0b71/aiohttp-3.9.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB) Collecting responses<0.19 (from evaluate>=0.3.0->opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/79/f3/2b3a6dc5986303b3dd1bbbcf482022acb2583c428cd23f0b6d37b1a1a519/responses-0.18.0-py3-none-any.whl (38 kB) Collecting python-dateutil>=2.8.1 (from pandas<2.0.0->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/36/7a/87837f39d0296e723bb9b62bbb257d0355c7f6128853c78955f57342a56d/python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB) Collecting pytz>=2020.1 (from pandas<2.0.0->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/9c/3d/a121f284241f08268b21359bd425f7d4825cffc5ac5cd0e1b3d82ffd2b10/pytz-2024.1-py2.py3-none-any.whl (505 kB) Requirement already satisfied: typing-extensions in /root/.conda/envs/opencompass/lib/python3.10/site-packages (from torch>=1.13.1->opencompass==0.2.2) (4.7.1) Requirement already satisfied: sympy in /root/.conda/envs/opencompass/lib/python3.10/site-packages (from torch>=1.13.1->opencompass==0.2.2) (1.11.1) Requirement already satisfied: networkx in /root/.conda/envs/opencompass/lib/python3.10/site-packages (from torch>=1.13.1->opencompass==0.2.2) (3.1) Requirement already satisfied: jinja2 in /root/.conda/envs/opencompass/lib/python3.10/site-packages (from torch>=1.13.1->opencompass==0.2.2) (3.1.2) Collecting botocore<1.35.0,>=1.34.44 (from boto3->opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/aa/3a/5b08bc151e45ffe8c661af1a587cf2ac6ad9410e7d341e343ca46bfca83e/botocore-1.34.44-py3-none-any.whl (12.0 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.0/12.0 MB 1.7 MB/s eta 0:00:00 Collecting jmespath<2.0.0,>=0.7.1 (from boto3->opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/31/b4/b9b800c45527aadd64d5b442f9b932b00648617eb5d63d2c7a6587b7cafc/jmespath-1.0.1-py3-none-any.whl (20 kB) Collecting s3transfer<0.11.0,>=0.10.0 (from boto3->opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/12/bb/7e7912e18cd558e7880d9b58ffc57300b2c28ffba9882b3a54ba5ce3ebc4/s3transfer-0.10.0-py3-none-any.whl (82 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 82.1/82.1 kB 160.7 kB/s eta 0:00:00 Requirement already satisfied: setuptools>=47.3.1 in /root/.conda/envs/opencompass/lib/python3.10/site-packages (from cn2an->opencompass==0.2.2) (68.0.0) Collecting proces>=0.1.3 (from cn2an->opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/6f/88/06cc0c7d890ed8d7e16ef0e56880dea516a21643fb1f3a69a50f4cc6f716/proces-0.1.7-py3-none-any.whl (137 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 137.7/137.7 kB 716.7 kB/s eta 0:00:00 Collecting ltp-core>=0.1.3 (from ltp->opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/73/55/bb880fd459976e5bc95a75e83b29b156e4b3acf2b97acc9b9cdeb694440e/ltp_core-0.1.4-py3-none-any.whl (66 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 66.5/66.5 kB 145.5 kB/s eta 0:00:00 Collecting ltp-extension>=0.1.9 (from ltp->opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/f9/ef/5bf08c654b412dff0c0229bff542a2914da1e15ec061982a8436420ee535/ltp_extension-0.1.11-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.4 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.4/1.4 MB 1.1 MB/s eta 0:00:00 Collecting addict (from mmengine-lite->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/6a/00/b08f23b7d7e1e14ce01419a467b583edbb93c6cdb8654e54a9cc579cd61f/addict-2.4.0-py3-none-any.whl (3.8 kB) Collecting termcolor (from mmengine-lite->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/d9/5f/8c716e47b3a50cbd7c146f45881e11d9414def768b7cd9c5e6650ec2a80a/termcolor-2.4.0-py3-none-any.whl (7.7 kB) Collecting yapf (from mmengine-lite->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/66/c9/d4b03b2490107f13ebd68fe9496d41ae41a7de6275ead56d0d4621b11ffd/yapf-0.40.2-py3-none-any.whl (254 kB) Collecting anyio<5,>=3.5.0 (from openai->opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/14/fd/2f20c40b45e4fb4324834aea24bd4afdf1143390242c0b33774da0e2e34f/anyio-4.3.0-py3-none-any.whl (85 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 85.6/85.6 kB 356.6 kB/s eta 0:00:00 Collecting distro<2,>=1.7.0 (from openai->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/12/b3/231ffd4ab1fc9d679809f356cebee130ac7daa00d6d6f3206dd4fd137e9e/distro-1.9.0-py3-none-any.whl (20 kB) Collecting httpx<1,>=0.23.0 (from openai->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/39/9b/4937d841aee9c2c8102d9a4eeb800c7dad25386caabb4a1bf5010df81a57/httpx-0.26.0-py3-none-any.whl (75 kB) Collecting pydantic<3,>=1.9.0 (from openai->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/db/dc/afecbd9650f486889181c6d1a0d675b580c06253ea7e304588e4c7485bdb/pydantic-2.6.1-py3-none-any.whl (394 kB) Collecting sniffio (from openai->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/c3/a0/5dba8ed157b0136607c7f2151db695885606968d1fae123dc3391e0cfdbf/sniffio-1.3.0-py3-none-any.whl (10 kB) Collecting wcwidth (from prettytable->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/fd/84/fd2ba7aafacbad3c4201d395674fc6348826569da3c0937e75505ead3528/wcwidth-0.2.13-py2.py3-none-any.whl (34 kB) Collecting Levenshtein==0.25.0 (from python-Levenshtein->opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/bc/4b/b21cef6f195a241aa72176ebb47f9d879cafcee097ac9205b63cbc76101b/Levenshtein-0.25.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (177 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 177.4/177.4 kB 510.3 kB/s eta 0:00:00 Collecting markdown-it-py>=2.2.0 (from rich->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/42/d7/1ec15b46af6af88f19b8e5ffea08fa375d433c998b8a7639e76935c14f1f/markdown_it_py-3.0.0-py3-none-any.whl (87 kB) Collecting pygments<3.0.0,>=2.13.0 (from rich->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/97/9c/372fef8377a6e340b1704768d20daaded98bf13282b5327beb2e2fe2c7ef/pygments-2.17.2-py3-none-any.whl (1.2 MB) Collecting six (from rouge->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/d9/5a/e7c31adbe875f2abbb91bd84cf2dc52d792b5a01506781dbcf25c91daf11/six-1.16.0-py2.py3-none-any.whl (11 kB) Collecting portalocker (from sacrebleu->opencompass==0.2.2) Downloading https://pypi.tuna.tsinghua.edu.cn/packages/17/9e/87671efcca80ba6203811540ed1f9c0462c1609d2281d7b7f53cef05da3d/portalocker-2.8.2-py3-none-any.whl (17 kB) Collecting colorama (from sacrebleu->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl (25 kB) Collecting lxml (from sacrebleu->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/25/5c/979167df4ca5a1c308105bb1590412c54bd1b0baa1883212f39cb42d4fcd/lxml-5.1.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (8.0 MB) Collecting matplotlib!=3.6.1,>=3.4 (from seaborn->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/c1/f2/325897d6c498278b0f8b460d44b516f5db865ddb4ba9018e9fe58a3e4633/matplotlib-3.8.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.6 MB) Collecting exceptiongroup>=1.0.2 (from anyio<5,>=3.5.0->openai->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/b8/9a/5028fd52db10e600f1c4674441b968cf2ea4959085bfb5b99fb1250e5f68/exceptiongroup-1.2.0-py3-none-any.whl (16 kB) Collecting aiosignal>=1.1.2 (from aiohttp->datasets>=2.12.0->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/76/ac/a7305707cb852b7e16ff80eaf5692309bde30e2b1100a1fcacdc8f731d97/aiosignal-1.3.1-py3-none-any.whl (7.6 kB) Collecting attrs>=17.3.0 (from aiohttp->datasets>=2.12.0->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/e0/44/827b2a91a5816512fcaf3cc4ebc465ccd5d598c45cefa6703fcf4a79018f/attrs-23.2.0-py3-none-any.whl (60 kB) Collecting frozenlist>=1.1.1 (from aiohttp->datasets>=2.12.0->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/ec/25/0c87df2e53c0c5d90f7517ca0ff7aca78d050a8ec4d32c4278e8c0e52e51/frozenlist-1.4.1-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (239 kB) Collecting multidict<7.0,>=4.5 (from aiohttp->datasets>=2.12.0->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/33/62/2c9085e571318d51212a6914566fe41dd0e33d7f268f7e2f23dcd3f06c56/multidict-6.0.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (124 kB) Collecting yarl<2.0,>=1.0 (from aiohttp->datasets>=2.12.0->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/c3/a0/0ade1409d184cbc9e85acd403a386a7c0563b92ff0f26d138ff9e86e48b4/yarl-1.9.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (301 kB) Collecting async-timeout<5.0,>=4.0 (from aiohttp->datasets>=2.12.0->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/a7/fa/e01228c2938de91d47b307831c62ab9e4001e747789d0b05baf779a6488c/async_timeout-4.0.3-py3-none-any.whl (5.7 kB) Collecting httpcore==1.* (from httpx<1,>=0.23.0->openai->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/11/a6/24139fa27831cf2127fcf578d6d0a852a611f10cefecd800b1c557333d7a/httpcore-1.0.3-py3-none-any.whl (77 kB) Collecting h11<0.15,>=0.13 (from httpcore==1.*->httpx<1,>=0.23.0->openai->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/95/04/ff642e65ad6b90db43e668d70ffb6736436c7ce41fcc549f4e9472234127/h11-0.14.0-py3-none-any.whl (58 kB) Collecting mdurl~=0.1 (from markdown-it-py>=2.2.0->rich->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/b3/38/89ba8ad64ae25be8de66a6d463314cf1eb366222074cfda9ee839c56a4b4/mdurl-0.1.2-py3-none-any.whl (10.0 kB) Collecting contourpy>=1.0.1 (from matplotlib!=3.6.1,>=3.4->seaborn->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/58/56/e2c43dcfa1f9c7db4d5e3d6f5134b24ed953f4e2133a4b12f0062148db58/contourpy-1.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (310 kB) Collecting cycler>=0.10 (from matplotlib!=3.6.1,>=3.4->seaborn->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/e7/05/c19819d5e3d95294a6f5947fb9b9629efb316b96de511b418c53d245aae6/cycler-0.12.1-py3-none-any.whl (8.3 kB) Collecting fonttools>=4.22.0 (from matplotlib!=3.6.1,>=3.4->seaborn->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/a6/ba/5eac3e9c9bbc2dea3606e46de08bcef0908d74e7ccf89a71701b95a16747/fonttools-4.49.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.6 MB) Collecting kiwisolver>=1.3.1 (from matplotlib!=3.6.1,>=3.4->seaborn->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/6f/40/4ab1fdb57fced80ce5903f04ae1aed7c1d5939dda4fd0c0aa526c12fe28a/kiwisolver-1.4.5-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.6 MB) Requirement already satisfied: pillow>=8 in /root/.conda/envs/opencompass/lib/python3.10/site-packages (from matplotlib!=3.6.1,>=3.4->seaborn->opencompass==0.2.2) (10.0.1) Collecting pyparsing>=2.3.1 (from matplotlib!=3.6.1,>=3.4->seaborn->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl (103 kB) Collecting annotated-types>=0.4.0 (from pydantic<3,>=1.9.0->openai->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/28/78/d31230046e58c207284c6b2c4e8d96e6d3cb4e52354721b944d3e1ee4aa5/annotated_types-0.6.0-py3-none-any.whl (12 kB) Collecting pydantic-core==2.16.2 (from pydantic<3,>=1.9.0->openai->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/50/5e/2978d9f0e8d0cfd78e22115c028a41e0599e3d684e5aef7ed9bd18fcbd0c/pydantic_core-2.16.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB) Requirement already satisfied: MarkupSafe>=2.0 in /root/.conda/envs/opencompass/lib/python3.10/site-packages (from jinja2->torch>=1.13.1->opencompass==0.2.2) (2.1.1) Requirement already satisfied: mpmath>=0.19 in /root/.conda/envs/opencompass/lib/python3.10/site-packages (from sympy->torch>=1.13.1->opencompass==0.2.2) (1.3.0) Collecting importlib-metadata>=6.6.0 (from yapf->mmengine-lite->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/c0/8b/d8427f023c081a8303e6ac7209c16e6878f2765d5b59667f3903fbcfd365/importlib_metadata-7.0.1-py3-none-any.whl (23 kB) Collecting platformdirs>=3.5.1 (from yapf->mmengine-lite->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/55/72/4898c44ee9ea6f43396fbc23d9bfaf3d06e01b83698bdf2e4c919deceb7c/platformdirs-4.2.0-py3-none-any.whl (17 kB) Collecting tomli>=2.0.1 (from yapf->mmengine-lite->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/97/75/10a9ebee3fd790d20926a90a2547f0bf78f371b2f13aa822c759680ca7b9/tomli-2.0.1-py3-none-any.whl (12 kB) Collecting zipp>=0.5 (from importlib-metadata>=6.6.0->yapf->mmengine-lite->opencompass==0.2.2) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/d9/66/48866fc6b158c81cc2bfecc04c480f105c6040e8b077bc54c634b4a67926/zipp-3.17.0-py3-none-any.whl (7.4 kB) Building wheels for collected packages: fairscale, jieba, rouge_score Building wheel for fairscale (pyproject.toml) ... done Created wheel for fairscale: filename=fairscale-0.4.13-py3-none-any.whl size=332104 sha256=0282663d1d5b201e8351d424b98d7fc6494b8dec660574f6251d68600ec877de Stored in directory: /root/.cache/pip/wheels/37/e2/5d/327c36dc18dd27b5b93e1a3ab3c10173da5e44c5f5837db8e3 Building wheel for jieba (setup.py) ... done Created wheel for jieba: filename=jieba-0.42.1-py3-none-any.whl size=19314458 sha256=27da23d86063c363700121aaf46501efe47198bc54dd65a54c77c7252a17bfff Stored in directory: /root/.cache/pip/wheels/b2/9b/80/7537177f75993c29af08e0d00c753724c7f06c646352be50a3 Building wheel for rouge_score (setup.py) ... done Created wheel for rouge_score: filename=rouge_score-0.1.2-py3-none-any.whl size=24932 sha256=e37b63fb3bda04e62a9cd9134998ae477d9d6e9a14176df7919d4127e7b2f6ea Stored in directory: /root/.cache/pip/wheels/81/78/38/125dd7761f58c20d80190e182ee76c29247621549f51e25329 Successfully built fairscale jieba rouge_score Installing collected packages: wcwidth, timeout_decorator, sentencepiece, pytz, OpenCC, ltp-extension, jieba, fuzzywuzzy, func_timeout, cpm_kernels, addict, zipp, xxhash, tqdm, tomli, threadpoolctl, termcolor, tabulate, sniffio, six, safetensors, regex, rapidfuzz, pyyaml, pypinyin, pyparsing, pygments, pydantic-core, pyarrow-hotfix, psutil, proces, prettytable, portalocker, platformdirs, packaging, numpy, multidict, mdurl, lxml, kiwisolver, joblib, jmespath, h11, fsspec, frozenlist, fonttools, exceptiongroup, einops, distro, dill, cycler, colorama, click, attrs, async-timeout, annotated-types, absl-py, yarl, typer, tiktoken, scipy, sacrebleu, rouge_chinese, rouge, responses, rank_bm25, python-dateutil, pydantic, pyarrow, opencv-python-headless, nltk, multiprocess, markdown-it-py, Levenshtein, importlib-metadata, huggingface-hub, httpcore, contourpy, cn2an, anyio, aiosignal, yapf, tokenizers, scikit_learn, rouge_score, rich, python-Levenshtein, pandas, matplotlib, httpx, fairscale, botocore, aiohttp, accelerate, transformers, seaborn, s3transfer, openai, mmengine-lite, sentence_transformers, ltp-core, datasets, boto3, ltp, evaluate, opencompass Attempting uninstall: numpy Found existing installation: numpy 1.26.2 Uninstalling numpy-1.26.2: Successfully uninstalled numpy-1.26.2 Running setup.py develop for opencompass Successfully installed Levenshtein-0.25.0 OpenCC-1.1.7 absl-py-2.1.0 accelerate-0.27.2 addict-2.4.0 aiohttp-3.9.3 aiosignal-1.3.1 annotated-types-0.6.0 anyio-4.3.0 async-timeout-4.0.3 attrs-23.2.0 boto3-1.34.44 botocore-1.34.44 click-8.1.7 cn2an-0.5.22 colorama-0.4.6 contourpy-1.2.0 cpm_kernels-1.0.11 cycler-0.12.1 datasets-2.17.0 dill-0.3.8 distro-1.9.0 einops-0.5.0 evaluate-0.4.1 exceptiongroup-1.2.0 fairscale-0.4.13 fonttools-4.49.0 frozenlist-1.4.1 fsspec-2023.10.0 func_timeout-4.3.5 fuzzywuzzy-0.18.0 h11-0.14.0 httpcore-1.0.3 httpx-0.26.0 huggingface-hub-0.20.3 importlib-metadata-7.0.1 jieba-0.42.1 jmespath-1.0.1 joblib-1.3.2 kiwisolver-1.4.5 ltp-4.2.13 ltp-core-0.1.4 ltp-extension-0.1.11 lxml-5.1.0 markdown-it-py-3.0.0 matplotlib-3.8.3 mdurl-0.1.2 mmengine-lite-0.10.3 multidict-6.0.5 multiprocess-0.70.16 nltk-3.8 numpy-1.23.4 openai-1.12.0 opencompass-0.2.2 opencv-python-headless-4.9.0.80 packaging-23.2 pandas-1.5.3 platformdirs-4.2.0 portalocker-2.8.2 prettytable-3.9.0 proces-0.1.7 psutil-5.9.8 pyarrow-15.0.0 pyarrow-hotfix-0.6 pydantic-2.6.1 pydantic-core-2.16.2 pygments-2.17.2 pyparsing-3.1.1 pypinyin-0.50.0 python-Levenshtein-0.25.0 python-dateutil-2.8.2 pytz-2024.1 pyyaml-6.0.1 rank_bm25-0.2.2 rapidfuzz-3.6.1 regex-2023.12.25 responses-0.18.0 rich-13.7.0 rouge-1.0.1 rouge_chinese-1.0.3 rouge_score-0.1.2 s3transfer-0.10.0 sacrebleu-2.4.0 safetensors-0.4.2 scikit_learn-1.2.1 scipy-1.12.0 seaborn-0.13.2 sentence_transformers-2.2.2 sentencepiece-0.1.99 six-1.16.0 sniffio-1.3.0 tabulate-0.9.0 termcolor-2.4.0 threadpoolctl-3.3.0 tiktoken-0.6.0 timeout_decorator-0.5.0 tokenizers-0.15.2 tomli-2.0.1 tqdm-4.64.1 transformers-4.37.2 typer-0.9.0 wcwidth-0.2.13 xxhash-3.4.1 yapf-0.40.2 yarl-1.9.4 zipp-3.17.0 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv (opencompass) root@intern-studio-069640:~/opencompass#
2、准备测试数据：
cp /share/temp/datasets/OpenCompassData-core-20231110.zip /root/opencompass/ unzip OpenCompassData-core-20231110.zip
3、启动测评
先以debug模型启动测评：
python run.py --datasets ceval_gen --hf-path /share/model_repos/internlm2-chat-7b/ --tokenizer-path /share/model_repos/internlm2-chat-7b/ --tokenizer-kwargs padding_side='left' truncation='left' trust_remote_code=True --model-kwargs trust_remote_code=True device_map='auto' --max-seq-len 2048 --max-out-len 16 --batch-size 4 --num-gpus 1 --debug
屏幕输出：
(opencompass) root@intern-studio-069640:~/opencompass# python run.py --datasets ceval_gen --hf-path /share/model_repos/internlm2-chat-7b/ --tokenizer-path /share/model_repos/internlm2-chat-7b/ --tokenizer-kwargs padding_side='left' truncation='left' trust_remote_code=True --model-kwargs trust_remote_code=True device_map='auto' --max-seq-len 2048 --max-out-len 16 --batch-size 4 --num-gpus 1 --debug 02/19 17:56:07 - OpenCompass - INFO - Loading ceval_gen: configs/datasets/ceval/ceval_gen.py 02/19 17:56:07 - OpenCompass - INFO - Loading example: configs/summarizers/example.py 02/19 17:56:07 - OpenCompass - WARNING - SlurmRunner is not used, so the partition argument is ignored. 02/19 17:56:07 - OpenCompass - DEBUG - Modules of opencompass's partitioner registry have been automatically imported from opencompass.partitioners 02/19 17:56:07 - OpenCompass - DEBUG - Get class `SizePartitioner` from "partitioner" registry in "opencompass" 02/19 17:56:07 - OpenCompass - DEBUG - An `SizePartitioner` instance is built from registry, and its implementation can be found in opencompass.partitioners.size 02/19 17:56:07 - OpenCompass - DEBUG - Key eval.runner.task.judge_cfg not found in config, ignored. 02/19 17:56:07 - OpenCompass - DEBUG - Key eval.runner.task.dump_details not found in config, ignored. 02/19 17:56:07 - OpenCompass - DEBUG - Additional config: {} 02/19 17:56:07 - OpenCompass - DEBUG - Modules of opencompass's load_dataset registry have been automatically imported from opencompass.datasets 02/19 17:56:07 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:07 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:07 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:08 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:08 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:08 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:08 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:08 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:08 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:08 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:08 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:08 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:08 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:08 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:08 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:08 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:08 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:08 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:08 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:08 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:08 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:08 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:08 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:08 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:08 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:08 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:08 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:08 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:08 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:08 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:08 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:08 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:08 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:08 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:08 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:08 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:08 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:08 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:08 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:08 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:08 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:08 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:08 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:08 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:08 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:08 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:08 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:08 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:08 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:09 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:09 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:09 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:09 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:09 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:09 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:09 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:09 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:09 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:09 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:09 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:09 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:09 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:09 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:09 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:09 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:09 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:09 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:09 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:09 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:09 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:09 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:09 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:09 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:09 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:09 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:09 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:09 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:09 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:09 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:09 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:09 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:09 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:09 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:09 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:09 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:09 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:09 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:09 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:09 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:09 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:09 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:09 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:09 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:09 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:09 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:09 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:09 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:09 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:09 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:10 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:10 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:10 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:10 - OpenCompass - DEBUG - Get class `CEvalDataset` from "load_dataset" registry in "opencompass" 02/19 17:56:10 - OpenCompass - DEBUG - An `CEvalDataset` instance is built from registry, and its implementation can be found in opencompass.datasets.ceval 02/19 17:56:10 - OpenCompass - INFO - Partitioned into 1 tasks. 02/19 17:56:10 - OpenCompass - DEBUG - Task 0: [opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-college_economics,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-accountant,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-tax_accountant,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-physician,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-civil_servant,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-urban_and_rural_planner,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-teacher_qualification,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-college_programming,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-electrical_engineer,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-business_administration,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-art_studies,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-fire_engineer,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-environmental_impact_assessment_engineer,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-education_science,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-professional_tour_guide,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-college_chemistry,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-metrology_engineer,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-mao_zedong_thought,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-law,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-veterinary_medicine,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-modern_chinese_history,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-chinese_language_and_literature,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-legal_professional,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-logic,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-middle_school_history,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-plant_protection,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-clinical_medicine,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-computer_architecture,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-middle_school_biology,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-middle_school_politics,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-middle_school_chemistry,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-high_school_history,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-computer_network,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-operating_system,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-college_physics,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-advanced_mathematics,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-high_school_physics,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-high_school_chemistry,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-high_school_biology,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-middle_school_mathematics,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-middle_school_physics,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-marxism,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-high_school_politics,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-high_school_geography,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-ideological_and_moral_cultivation,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-high_school_chinese,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-sports_science,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-basic_medicine,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-probability_and_statistics,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-high_school_mathematics,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-discrete_mathematics,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-middle_school_geography] 02/19 17:56:10 - OpenCompass - DEBUG - Modules of opencompass's runner registry have been automatically imported from opencompass.runners 02/19 17:56:10 - OpenCompass - DEBUG - Get class `LocalRunner` from "runner" registry in "opencompass" 02/19 17:56:10 - OpenCompass - DEBUG - An `LocalRunner` instance is built from registry, and its implementation can be found in opencompass.runners.local 02/19 17:56:10 - OpenCompass - DEBUG - Modules of opencompass's task registry have been automatically imported from opencompass.tasks 02/19 17:56:10 - OpenCompass - DEBUG - Get class `OpenICLInferTask` from "task" registry in "opencompass" 02/19 17:56:10 - OpenCompass - DEBUG - An `OpenICLInferTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_infer 02/19 17:56:37 - OpenCompass - INFO - Task [opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-college_economics,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-accountant,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-tax_accountant,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-physician,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-civil_servant,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-urban_and_rural_planner,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-teacher_qualification,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-college_programming,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-electrical_engineer,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-business_administration,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-art_studies,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-fire_engineer,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-environmental_impact_assessment_engineer,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-education_science,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-professional_tour_guide,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-college_chemistry,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-metrology_engineer,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-mao_zedong_thought,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-law,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-veterinary_medicine,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-modern_chinese_history,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-chinese_language_and_literature,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-legal_professional,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-logic,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-middle_school_history,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-plant_protection,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-clinical_medicine,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-computer_architecture,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-middle_school_biology,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-middle_school_politics,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-middle_school_chemistry,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-high_school_history,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-computer_network,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-operating_system,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-college_physics,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-advanced_mathematics,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-high_school_physics,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-high_school_chemistry,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-high_school_biology,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-middle_school_mathematics,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-middle_school_physics,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-marxism,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-high_school_politics,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-high_school_geography,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-ideological_and_moral_cultivation,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-high_school_chinese,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-sports_science,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-basic_medicine,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-probability_and_statistics,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-high_school_mathematics,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-discrete_mathematics,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-middle_school_geography] Loading checkpoint shards: 0%| | 0/8 [00:00<?, ?it/s]/root/.conda/envs/opencompass/lib/python3.10/site-packages/torch/_utils.py:776: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage() return self.fget.__get__(instance, owner)() Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:09<00:00, 1.18s/it] 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 55/55 [00:00<00:00, 1246955.24it/s] [2024-02-19 17:57:25,945] [opencompass.openicl.icl_inferencer.icl_gen_inferencer] [INFO] Starting inference process... 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 14/14 [00:45<00:00, 3.23s/it] 02/19 17:58:11 - OpenCompass - INFO - Start inferencing [opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-accountant] 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 49/49 [00:00<00:00, 1447330.25it/s] [2024-02-19 17:58:11,436] [opencompass.openicl.icl_inferencer.icl_gen_inferencer] [INFO] Starting inference process... 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 13/13 [00:57<00:00, 4.40s/it] 02/19 17:59:08 - OpenCompass - INFO - Start inferencing [opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-tax_accountant] 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 49/49 [00:00<00:00, 1457595.01it/s] [2024-02-19 17:59:08,800] [opencompass.openicl.icl_inferencer.icl_gen_inferencer] [INFO] Starting inference process... 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 13/13 [00:47<00:00, 3.64s/it] 02/19 17:59:56 - OpenCompass - INFO - Start inferencing [opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-physician] 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 49/49 [00:00<00:00, 1447330.25it/s] [2024-02-19 17:59:56,335] [opencompass.openicl.icl_inferencer.icl_gen_inferencer] [INFO] Starting inference process... 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 13/13 [00:31<00:00, 2.42s/it] 02/19 18:00:27 - OpenCompass - INFO - Start inferencing [opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-civil_servant] 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 47/47 [00:00<00:00, 1493426.42it/s] [2024-02-19 18:00:28,041] [opencompass.openicl.icl_inferencer.icl_gen_inferencer] [INFO] Starting inference process...
正式测评，去掉debug
python run.py --datasets ceval_gen --hf-path /share/model_repos/internlm2-chat-7b/ --tokenizer-path /share/model_repos/internlm2-chat-7b/ --tokenizer-kwargs padding_side='left' truncation='left' trust_remote_code=True --model-kwargs trust_remote_code=True device_map='auto' --max-seq-len 2048 --max-out-len 16 --batch-size 2 --num-gpus 1
(opencompass) root@intern-studio-069640:~/opencompass# python run.py --datasets ceval_gen --hf-path /share/model_repos/internlm2-chat-7b/ --tokenizer-path /share/model_repos/internlm2-chat-7b/ --tokenizer-kwargs padding_side='left' truncation='left' trust_remote_code=True --model-kwargs trust_remote_code=True device_map='auto' --max-seq-len 2048 --max-out-len 16 --batch-size 2 --num-gpus 1 02/19 18:18:32 - OpenCompass - INFO - Loading ceval_gen: configs/datasets/ceval/ceval_gen.py 02/19 18:18:32 - OpenCompass - INFO - Loading example: configs/summarizers/example.py 02/19 18:18:32 - OpenCompass - WARNING - SlurmRunner is not used, so the partition argument is ignored. 02/19 18:18:32 - OpenCompass - INFO - Partitioned into 1 tasks. launch OpenICLInfer[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-college_economics,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-accountant,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-tax_accountant,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-physician,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-civil_servant,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-urban_and_rural_planner,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-teacher_qualification,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-college_programming,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-electrical_engineer,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-business_administration,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-art_studies,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-fire_engineer,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-environmental_impact_assessment_engineer,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-education_science,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-professional_tour_guide,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-college_chemistry,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-metrology_engineer,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-mao_zedong_thought,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-law,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-veterinary_medicine,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-modern_chinese_history,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-chinese_language_and_literature,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-legal_professional,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-logic,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-middle_school_history,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-plant_protection,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-clinical_medicine,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-computer_architecture,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-middle_school_biology,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-middle_school_politics,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-middle_school_chemistry,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-high_school_history,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-computer_network,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-operating_system,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-college_physics,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-advanced_mathematics,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-high_school_physics,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-high_school_chemistry,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-high_school_biology,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-middle_school_mathematics,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-middle_school_physics,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-marxism,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-high_school_politics,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-high_school_geography,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-ideological_and_moral_cultivation,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-high_school_chinese,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-sports_science,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-basic_medicine,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-probability_and_statistics,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-high_school_mathematics,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-discrete_mathematics,opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-middle_school_geography] on GPU 0 0%| | 0/1 [00:00<?, ?it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [21:55<00:00, 1315.42s/it] 02/19 18:40:27 - OpenCompass - INFO - Partitioned into 52 tasks. launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-computer_network] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-operating_system] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-computer_architecture] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-college_programming] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-college_physics] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-college_chemistry] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-advanced_mathematics] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-probability_and_statistics] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-discrete_mathematics] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-electrical_engineer] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-metrology_engineer] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-high_school_mathematics] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-high_school_physics] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-high_school_biology] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-high_school_chemistry] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-middle_school_mathematics] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-middle_school_physics] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-middle_school_biology] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-veterinary_medicine] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-middle_school_chemistry] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-college_economics] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-business_administration] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-mao_zedong_thought] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-marxism] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-education_science] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-teacher_qualification] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-high_school_politics] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-high_school_geography] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-middle_school_politics] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-middle_school_geography] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-modern_chinese_history] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-ideological_and_moral_cultivation] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-logic] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-law] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-chinese_language_and_literature] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-art_studies] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-legal_professional] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-professional_tour_guide] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-high_school_chinese] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-high_school_history] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-middle_school_history] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-civil_servant] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-sports_science] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-plant_protection] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-basic_medicine] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-clinical_medicine] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-urban_and_rural_planner] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-accountant] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-fire_engineer] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-environmental_impact_assessment_engineer] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-tax_accountant] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b/ceval-physician] on CPU 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 52/52 [03:07<00:00, 3.61s/it] dataset version metric mode opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b ---------------------------------------------- --------- ------------- ------ -------------------------------------------------------------------------- ceval-computer_network db9ce2 accuracy gen 47.37 ceval-operating_system 1c2571 accuracy gen 57.89 ceval-computer_architecture a74dad accuracy gen 47.62 ceval-college_programming 4ca32a accuracy gen 51.35 ceval-college_physics 963fa8 accuracy gen 36.84 ceval-college_chemistry e78857 accuracy gen 33.33 ceval-advanced_mathematics ce03e2 accuracy gen 21.05 ceval-probability_and_statistics 65e812 accuracy gen 27.78 ceval-discrete_mathematics e894ae accuracy gen 18.75 ceval-electrical_engineer ae42b9 accuracy gen 43.24 ceval-metrology_engineer ee34ea accuracy gen 58.33 ceval-high_school_mathematics 1dc5bf accuracy gen 50 ceval-high_school_physics adf25f accuracy gen 47.37 ceval-high_school_chemistry 2ed27f accuracy gen 52.63 ceval-high_school_biology 8e2b9a accuracy gen 26.32 ceval-middle_school_mathematics bee8d5 accuracy gen 31.58 ceval-middle_school_biology 86817c accuracy gen 66.67 ceval-middle_school_physics 8accf6 accuracy gen 63.16 ceval-middle_school_chemistry 167a15 accuracy gen 95 ceval-veterinary_medicine b4e08d accuracy gen 39.13 ceval-college_economics f3f4e6 accuracy gen 47.27 ceval-business_administration c1614e accuracy gen 51.52 ceval-marxism cf874c accuracy gen 84.21 ceval-mao_zedong_thought 51c7a4 accuracy gen 70.83 ceval-education_science 591fee accuracy gen 72.41 ceval-teacher_qualification 4e4ced accuracy gen 77.27 ceval-high_school_politics 5c0de2 accuracy gen 21.05 ceval-high_school_geography 865461 accuracy gen 42.11 ceval-middle_school_politics 5be3e7 accuracy gen 42.86 ceval-middle_school_geography 8a63be accuracy gen 50 ceval-modern_chinese_history fc01af accuracy gen 65.22 ceval-ideological_and_moral_cultivation a2aa4a accuracy gen 89.47 ceval-logic f5b022 accuracy gen 54.55 ceval-law a110a1 accuracy gen 41.67 ceval-chinese_language_and_literature 0f8b68 accuracy gen 60.87 ceval-art_studies 2a1300 accuracy gen 69.7 ceval-professional_tour_guide 4e673e accuracy gen 82.76 ceval-legal_professional ce8787 accuracy gen 34.78 ceval-high_school_chinese 315705 accuracy gen 68.42 ceval-high_school_history 7eb30a accuracy gen 75 ceval-middle_school_history 48ab4a accuracy gen 63.64 ceval-civil_servant 87d061 accuracy gen 53.19 ceval-sports_science 70f27b accuracy gen 73.68 ceval-plant_protection 8941f9 accuracy gen 77.27 ceval-basic_medicine c409d6 accuracy gen 63.16 ceval-clinical_medicine 49e82d accuracy gen 45.45 ceval-urban_and_rural_planner 95b885 accuracy gen 58.7 ceval-accountant 002837 accuracy gen 46.94 ceval-fire_engineer bc23f5 accuracy gen 35.48 ceval-environmental_impact_assessment_engineer c64e2d accuracy gen 51.61 ceval-tax_accountant 3a5e3c accuracy gen 48.98 ceval-physician 6e277d accuracy gen 51.02 ceval-stem - naive_average gen 45.77 ceval-social-science - naive_average gen 55.95 ceval-humanities - naive_average gen 64.19 ceval-other - naive_average gen 55.04 ceval-hard - naive_average gen 35.97 ceval - naive_average gen 53.59 02/19 18:43:35 - OpenCompass - INFO - write summary to /root/opencompass/outputs/default/20240219_181832/summary/summary_20240219_181832.txt 02/19 18:43:35 - OpenCompass - INFO - write csv to /root/opencompass/outputs/default/20240219_181832/summary/summary_20240219_181832.csv (opencompass) root@intern-studio-069640:~/opencompass# (opencompass) root@intern-studio-069640:~/opencompass# (opencompass) root@intern-studio-069640:~/opencompass#
posted on 2024-02-19 18:24 littlesuccess 阅读(44) 评论(0) 编辑 收藏 举报
