书生开源大模型-第2讲-作业

公告

View Post

基础作业：

使用 InternLM-Chat-7B 模型生成 300 字的小故事（需截图）

熟悉 hugging face 下载功能，使用 huggingface_hub python 包，下载 InternLM-20B 的 config.json 文件到本地（需截图下载过程）。

首先在自己的macbook上安装依赖：

Last login: Tue Feb 13 19:50:19 on ttys000
(base) gongshaocheng@192 ~ % pip install -U huggingface_hub
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Collecting huggingface_hub
  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/7f/c4/adcbe9a696c135578cabcbdd7331332daad4d49b7c43688bc2d36b3a47d2/huggingface_hub-0.16.4-py3-none-any.whl (268 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 268.8/268.8 kB 2.6 MB/s eta 0:00:00
Collecting packaging>=20.9
  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/ec/1a/610693ac4ee14fcdf2d9bf3c493370e4f2ef7ae2e19217d7a237ff42367d/packaging-23.2-py3-none-any.whl (53 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 53.0/53.0 kB 1.9 MB/s eta 0:00:00
Requirement already satisfied: typing-extensions>=3.7.4.3 in ./opt/anaconda3/lib/python3.7/site-packages (from huggingface_hub) (4.1.1)
Requirement already satisfied: filelock in ./opt/anaconda3/lib/python3.7/site-packages (from huggingface_hub) (3.0.12)
Requirement already satisfied: tqdm>=4.42.1 in ./opt/anaconda3/lib/python3.7/site-packages (from huggingface_hub) (4.42.1)
Requirement already satisfied: importlib-metadata in ./opt/anaconda3/lib/python3.7/site-packages (from huggingface_hub) (4.11.2)
Requirement already satisfied: requests in ./opt/anaconda3/lib/python3.7/site-packages (from huggingface_hub) (2.27.1)
Requirement already satisfied: pyyaml>=5.1 in ./opt/anaconda3/lib/python3.7/site-packages (from huggingface_hub) (5.4.1)
Requirement already satisfied: fsspec in ./opt/anaconda3/lib/python3.7/site-packages (from huggingface_hub) (0.6.2)
Requirement already satisfied: zipp>=0.5 in ./opt/anaconda3/lib/python3.7/site-packages (from importlib-metadata->huggingface_hub) (3.7.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in ./opt/anaconda3/lib/python3.7/site-packages (from requests->huggingface_hub) (1.25.8)
Requirement already satisfied: charset-normalizer~=2.0.0 in ./opt/anaconda3/lib/python3.7/site-packages (from requests->huggingface_hub) (2.0.12)
Requirement already satisfied: idna<4,>=2.5 in ./opt/anaconda3/lib/python3.7/site-packages (from requests->huggingface_hub) (2.9)
Requirement already satisfied: certifi>=2017.4.17 in ./opt/anaconda3/lib/python3.7/site-packages (from requests->huggingface_hub) (2019.11.28)
Installing collected packages: packaging, huggingface_hub
  Attempting uninstall: packaging
    Found existing installation: packaging 20.1
    Uninstalling packaging-20.1:
      Successfully uninstalled packaging-20.1
Successfully installed huggingface_hub-0.16.4 packaging-23.2

[notice] A new release of pip available: 22.3 -> 24.0
[notice] To update, run: pip install --upgrade pip
(base) gongshaocheng@192 ~ %

View Code

然后新建 python 文件，填入以下代码，运行即可

import os 
from huggingface_hub import hf_hub_download  # Load model directly 

hf_hub_download(repo_id="internlm/internlm-7b", filename="config.json")

运行该脚本，出错

(internlm-camp) gongshaocheng@192 internlm-camp % python ex2-2.py
Consider using `hf_transfer` for faster downloads. This solution comes with some limitations. See https://huggingface.co/docs/huggingface_hub/hf_transfer for more details.
Traceback (most recent call last):
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/urllib3/connection.py", line 198, in _new_conn
    sock = connection.create_connection(
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/urllib3/util/connection.py", line 85, in create_connection
    raise err
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
TimeoutError: [Errno 60] Operation timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/urllib3/connectionpool.py", line 793, in urlopen
    response = self._make_request(
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/urllib3/connectionpool.py", line 491, in _make_request
    raise new_e
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/urllib3/connectionpool.py", line 467, in _make_request
    self._validate_conn(conn)
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1099, in _validate_conn
    conn.connect()
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/urllib3/connection.py", line 616, in connect
    self.sock = sock = self._new_conn()
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/urllib3/connection.py", line 207, in _new_conn
    raise ConnectTimeoutError(
urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPSConnection object at 0x7fb5ac757a90>, 'Connection to huggingface.co timed out. (connect timeout=None)')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/urllib3/connectionpool.py", line 847, in urlopen
    retries = retries.increment(
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/urllib3/util/retry.py", line 515, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/internlm/internlm-chat-20b/revision/main (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7fb5ac757a90>, 'Connection to huggingface.co timed out. (connect timeout=None)'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/huggingface_hub/_snapshot_download.py", line 179, in snapshot_download
    repo_info = api.repo_info(repo_id=repo_id, repo_type=repo_type, revision=revision, token=token)
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/huggingface_hub/hf_api.py", line 2275, in repo_info
    return method(
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/huggingface_hub/hf_api.py", line 2084, in model_info
    r = get_session().get(path, headers=headers, timeout=timeout, params=params)
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/requests/sessions.py", line 602, in get
    return self.request("GET", url, **kwargs)
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/huggingface_hub/utils/_http.py", line 67, in send
    return super().send(request, *args, **kwargs)
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/requests/adapters.py", line 507, in send
    raise ConnectTimeout(e, request=request)
requests.exceptions.ConnectTimeout: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/internlm/internlm-chat-20b/revision/main (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7fb5ac757a90>, 'Connection to huggingface.co timed out. (connect timeout=None)'))"), '(Request ID: fd53a057-2f6d-4ca2-a974-7c9263160d55)')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/bin/huggingface-cli", line 8, in <module>
    sys.exit(main())
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/huggingface_hub/commands/huggingface_cli.py", line 49, in main
    service.run()
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/huggingface_hub/commands/download.py", line 160, in run
    print(self._download())  # Print path to downloaded files
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/huggingface_hub/commands/download.py", line 201, in _download
    return snapshot_download(
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/huggingface_hub/_snapshot_download.py", line 251, in snapshot_download
    raise LocalEntryNotFoundError(
huggingface_hub.utils._errors.LocalEntryNotFoundError: An error happened while trying to locate the files on the Hub and we cannot find the appropriate snapshot folder for the specified revision on the local disk. Please check your internet connection and try again.
Traceback (most recent call last):
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/urllib3/connection.py", line 198, in _new_conn
    sock = connection.create_connection(
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/urllib3/util/connection.py", line 85, in create_connection
    raise err
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
TimeoutError: timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/urllib3/connectionpool.py", line 793, in urlopen
    response = self._make_request(
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/urllib3/connectionpool.py", line 491, in _make_request
    raise new_e
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/urllib3/connectionpool.py", line 467, in _make_request
    self._validate_conn(conn)
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1099, in _validate_conn
    conn.connect()
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/urllib3/connection.py", line 616, in connect
    self.sock = sock = self._new_conn()
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/urllib3/connection.py", line 207, in _new_conn
    raise ConnectTimeoutError(
urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPSConnection object at 0x7ff5f08c9270>, 'Connection to huggingface.co timed out. (connect timeout=10)')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/urllib3/connectionpool.py", line 847, in urlopen
    retries = retries.increment(
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/urllib3/util/retry.py", line 515, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /internlm/internlm-20b/resolve/main/config.json (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7ff5f08c9270>, 'Connection to huggingface.co timed out. (connect timeout=10)'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1238, in hf_hub_download
    metadata = get_hf_file_metadata(
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1631, in get_hf_file_metadata
    r = _request_wrapper(
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 385, in _request_wrapper
    response = _request_wrapper(
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 408, in _request_wrapper
    response = get_session().request(method=method, url=url, **params)
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/huggingface_hub/utils/_http.py", line 67, in send
    return super().send(request, *args, **kwargs)
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/requests/adapters.py", line 507, in send
    raise ConnectTimeout(e, request=request)
requests.exceptions.ConnectTimeout: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /internlm/internlm-20b/resolve/main/config.json (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7ff5f08c9270>, 'Connection to huggingface.co timed out. (connect timeout=10)'))"), '(Request ID: d7004a93-fd11-48ec-bcd1-278e8a725a6d)')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/gongshaocheng/internlm-camp/ex2-2.py", line 8, in <module>
    hf_hub_download(repo_id="internlm/internlm-20b", filename="config.json")
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
  File "/Users/gongshaocheng/opt/anaconda3/envs/internlm-camp/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1371, in hf_hub_download
    raise LocalEntryNotFoundError(
huggingface_hub.utils._errors.LocalEntryNotFoundError: An error happened while trying to locate the file on the Hub and we cannot find the requested files in the local cache. Please check your connection and try again or make sure your Internet connection is on.

View Code

原因可能是外网的原因。改用开发机：

进阶作业（可选做）

完成浦语·灵笔的图文理解及创作部署（需截图）

完成 Lagent 工具调用 Demo 创作部署（需截图）

posted on 2024-02-14 08:14 littlesuccess 阅读(142) 评论(0) 编辑收藏举报

刷新页面返回顶部