InternLM2 Demo实操-书生浦语大模型实战营第二期第2节作业&大语言模型3
大语言模型-3.InternLM2 Demo实操
书生浦语大模型实战营第二期第二节作业
本文包括第二期实战营的第2课作业的相关内容。本来是想在学习笔记中给InetrnLM官方教程做做补充的，没想到官方教程的质量还是相当高的，跟着一步一步做没啥坑。所以这篇随笔主要记录一下作业中的步骤，另外拆解一下 InternStudio 封装的一些东西，防止在本地复现时出现各种问题。
作业要求
- 基础作业
- 进阶作业
对话Demo：InternLM2-Chat-1.8B 智能对话（使用 InternLM2-Chat-1.8B 模型生成 300 字的小故事）
搭建环境
首先是搭建环境这里，官方教程说：
进入开发机后，在 `terminal` 中输入环境配置命令 (配置环境时间较长，需耐心等待)：
studio-conda -o internlm-base -t demo
# 与 studio-conda 等效的配置方案
# conda create -n demo python==3.10 -y
# conda activate demo
# conda install pytorch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 pytorch-cuda=11.7 -c pytorch -c nvidia
studio-conda 命令
那么，这句studio-conda -o internlm-base -t demo究竟是什么呢？我们直接查看一下/root/.bashrc，发现里面就一句：
source /share/.aide/config/bashrc
继续查看/share/.aide/config/bashrc，这个可长了，这里给出最后两句：
export HF_ENDPOINT='https://hf-mirror.com'
alias studio-conda="/share/install_conda_env.sh"
alias studio-smi="/share/studio-smi"
点击查看/share/.aide/config/bashrc的全部代码
#! /bin/bash
# ~/.bashrc: executed by bash(1) for non-login shells.
# see /usr/share/doc/bash/examples/startup-files (in the package bash-doc)
# for examples
# If not running interactively, don't do anything
case $- in
*i*) ;;
*) return;;
esac
# don't put duplicate lines or lines starting with space in the history.
# See bash(1) for more options
HISTCONTROL=ignoreboth
# append to the history file, don't overwrite it
shopt -s histappend
# for setting history length see HISTSIZE and HISTFILESIZE in bash(1)
HISTSIZE=1000
HISTFILESIZE=2000
# check the window size after each command and, if necessary,
# update the values of LINES and COLUMNS.
shopt -s checkwinsize
# If set, the pattern "**" used in a pathname expansion context will
# match all files and zero or more directories and subdirectories.
#shopt -s globstar
# make less more friendly for non-text input files, see lesspipe(1)
[ -x /usr/bin/lesspipe ] && eval "$(SHELL=/bin/sh lesspipe)"
# set variable identifying the chroot you work in (used in the prompt below)
if [ -z "${debian_chroot:-}" ] && [ -r /etc/debian_chroot ]; then
debian_chroot=$(cat /etc/debian_chroot)
fi
# set a fancy prompt (non-color, unless we know we "want" color)
case "$TERM" in
xterm-color|*-256color) color_prompt=yes;;
esac
# uncomment for a colored prompt, if the terminal has the capability; turned
# off by default to not distract the user: the focus in a terminal window
# should be on the output of commands, not on the prompt
#force_color_prompt=yes
if [ -n "$force_color_prompt" ]; then
if [ -x /usr/bin/tput ] && tput setaf 1 >&/dev/null; then
# We have color support; assume it's compliant with Ecma-48
# (ISO/IEC-6429). (Lack of such support is extremely rare, and such
# a case would tend to support setf rather than setaf.)
color_prompt=yes
else
color_prompt=
fi
fi
if [ "$color_prompt" = yes ]; then
PS1='${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ '
else
PS1='${debian_chroot:+($debian_chroot)}\u@\h:\w\$ '
fi
unset color_prompt force_color_prompt
# If this is an xterm set the title to user@host:dir
case "$TERM" in
xterm*|rxvt*)
PS1="\[\e]0;${debian_chroot:+($debian_chroot)}\u@\h: \w\a\]$PS1"
;;
*)
;;
esac
# enable color support of ls and also add handy aliases
if [ -x /usr/bin/dircolors ]; then
test -r ~/.dircolors && eval "$(dircolors -b ~/.dircolors)" || eval "$(dircolors -b)"
alias ls='ls --color=auto'
#alias dir='dir --color=auto'
#alias vdir='vdir --color=auto'
alias grep='grep --color=auto'
alias fgrep='fgrep --color=auto'
alias egrep='egrep --color=auto'
fi
# colored GCC warnings and errors
#export GCC_COLORS='error=01;31:warning=01;35:note=01;36:caret=01;32:locus=01:quote=01'
# some more ls aliases
alias ll='ls -alF'
alias la='ls -A'
alias l='ls -CF'
# Add an "alert" alias for long running commands. Use like so:
# sleep 10; alert
alias alert='notify-send --urgency=low -i "$([ $? = 0 ] && echo terminal || echo error)" "$(history|tail -n1|sed -e '\''s/^\s*[0-9]\+\s*//;s/[;&|]\s*alert$//'\'')"'
# Alias definitions.
# You may want to put all your additions into a separate file like
# ~/.bash_aliases, instead of adding them here directly.
# See /usr/share/doc/bash-doc/examples in the bash-doc package.
if [ -f ~/.bash_aliases ]; then
. ~/.bash_aliases
fi
# enable programmable completion features (you don't need to enable
# this, if it's already enabled in /etc/bash.bashrc and /etc/profile
# sources /etc/bash.bashrc).
if ! shopt -oq posix; then
if [ -f /usr/share/bash-completion/bash_completion ]; then
. /usr/share/bash-completion/bash_completion
elif [ -f /etc/bash_completion ]; then
. /etc/bash_completion
fi
fi
# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/root/.conda/condabin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
eval "$__conda_setup"
else
if [ -f "/root/.conda/etc/profile.d/conda.sh" ]; then
. "/root/.conda/etc/profile.d/conda.sh"
else
export PATH="/root/.conda/condabin:$PATH"
fi
fi
unset __conda_setup
# <<< conda initialize <<<
if [ -d "/root/.conda/envs/xtuner" ]; then
CONDA_ENV=xtuner
else
CONDA_ENV=base
fi
source activate $CONDA_ENV
cat /share/.aide/config/welcome_vgpu
#if [ $CONDA_ENV != "xtuner" ]; then
# echo -e """
# \033[31m 检测到您尚未初始化xtuner环境, 建议执行> source init_xtuner_env.sh \033[0m
# """
#fi
export https_proxy=http://proxy.intern-ai.org.cn:50000
export http_proxy=http://proxy.intern-ai.org.cn:50000
export no_proxy='localhost,127.0.0.1,0.0.0.0,172.18.47.140'
export PATH=/root/.local/bin:$PATH
export HF_ENDPOINT='https://hf-mirror.com'
alias studio-conda="/share/install_conda_env.sh"
alias studio-smi="/share/studio-smi"
注意到倒数第二行：alias studio-conda="/share/install_conda_env.sh"，也就是说studio-conda是/share/install_conda_env.sh的别名。我们在执行studio-conda -o internlm-base -t demo的时候，实际上调用的是/share/install_conda_env.sh这个脚本。我们进一步查看/share/install_conda_env.sh：
HOME_DIR=/root
CONDA_HOME=$HOME_DIR/.conda
SHARE_CONDA_HOME=/share/conda_envs
SHARE_HOME=/share
echo -e "\033[34m [1/2] 开始安装conda环境: <$target>. \033[0m"
sleep 3
tar --skip-old-files -xzvf /share/pkgs.tar.gz -C ${CONDA_HOME}
wait_echo&
wait_pid=$!
conda create -n $target --clone ${SHARE_CONDA_HOME}/${source}
if [ $? -ne 0 ]; then
echo -e "\033[31m 初始化conda环境: ${target}失败 \033[0m"
exit 10
fi
kill $wait_pid
# for xtuner, re-install dependencies
case "$source" in
xtuner)
source_install_xtuner $target
;;
esac
echo -e "\033[34m [2/2] 同步当前conda环境至jupyterlab kernel \033[0m"
lab add $target
source $CONDA_HOME/bin/activate $target
cd $HOME_DIR
点击查看/share/install_conda_env.sh的全部代码
#!/bin/bash
# clone internlm-base conda env to user's conda env
# created by xj on 01.07.2024
# modifed by xj on 01.19.2024 to fix bug of conda env clone
# modified by ljy on 01.26.2024 to extend
XTUNER_UPDATE_DATE=`cat /share/repos/UPDATE | grep xtuner |awk -F= '{print $2}'`
HOME_DIR=/root
CONDA_HOME=$HOME_DIR/.conda
SHARE_CONDA_HOME=/share/conda_envs
SHARE_HOME=/share
list() {
cat <<-EOF
预设环境 描述
internlm-base pytorch:2.0.1, pytorch-cuda:11.7
xtuner Xtuner(源码安装: main $(echo -e "\033[4mhttps://github.com/InternLM/xtuner/tree/main\033[0m"), 更新日期：$XTUNER_UPDATE_DATE)
pytorch-2.1.2 pytorch:2.1.2, pytorch-cuda:11.8
EOF
}
help() {
cat <<-EOF
说明: 用于快速clone预设的conda环境
使用:
1. studio-conda env -l/list 打印预设的conda环境列表
2. studio-conda <target-conda-name> 快速clone: 默认拷贝internlm-base conda环境
3. studio-conda -t <target-conda-name> -o <origin-conda-name> 将预设的conda环境拷贝到指定的conda环境
EOF
}
clone() {
source=$1
target=$2
if [[ -z "$source" || -z "$target" ]]; then
echo -e "\033[31m 输入不符合规范 \033[0m"
help
exit 1
fi
if [ ! -d "${SHARE_CONDA_HOME}/$source" ]; then
echo -e "\033[34m 指定的预设环境: $source不存在\033[0m"
list
exit 1
fi
if [ -d "${CONDA_HOME}/envs/$target" ]; then
echo -e "\033[34m 指定conda环境的目录: ${CONDA_HOME}/envs/$target已存在, 将清空原目录安装 \033[0m"
wait_echo&
wait_pid=$!
rm -rf "${CONDA_HOME}/envs/$target"
kill $wait_pid
fi
echo -e "\033[34m [1/2] 开始安装conda环境: <$target>. \033[0m"
sleep 3
tar --skip-old-files -xzvf /share/pkgs.tar.gz -C ${CONDA_HOME}
wait_echo&
wait_pid=$!
conda create -n $target --clone ${SHARE_CONDA_HOME}/${source}
if [ $? -ne 0 ]; then
echo -e "\033[31m 初始化conda环境: ${target}失败 \033[0m"
exit 10
fi
kill $wait_pid
# for xtuner, re-install dependencies
case "$source" in
xtuner)
source_install_xtuner $target
;;
esac
echo -e "\033[34m [2/2] 同步当前conda环境至jupyterlab kernel \033[0m"
lab add $target
source $CONDA_HOME/bin/activate $target
cd $HOME_DIR
echo -e "\033[32m conda环境: $target安装成功! \033[0m"
echo """
============================================
ALL DONE!
============================================
"""
}
source_install_xtuner() {
conda_env=$1
echo -e "\033[34m 源码安装xtuner... \033[0m"
sleep 2
pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple
install=0
if [ -d "${HOME_DIR}/xtuner" ]; then
read -r -p "$HOME_DIR中已存在目录xtuner: 是否清空目录? [Y/N][yes/no]" input
case $input in
[yY][eE][sS]|[yY])
echo -e "\033[34m 清空目录: $HOME_DIR/xtuner, 并同步源码至该目录进行源码安装... \033[0m"
install=1
;;
*)
echo -e "\033[34m 尝试使用: $HOME_DIR/xtuner目录进行源码安装... \033[0m"
;;
esac
else
install=1
fi
if [ $install -eq 1 ]; then
rm -rf $HOME_DIR/xtuner
mkdir -p $HOME_DIR/xtuner
cp -rf $SHARE_HOME/repos/xtuner/* $HOME_DIR/xtuner/
fi
cd $HOME_DIR/xtuner
$CONDA_HOME/envs/$conda_env/bin/pip install -e '.[all]'
if [ $? -ne 0 ]; then
echo -e "\033[31m 源码安装xtuner失败 \033[0m"
exit 10
fi
$CONDA_HOME/envs/$conda_env/bin/pip install cchardet
$CONDA_HOME/envs/$conda_env/bin/pip install -U datasets
}
wait_echo() {
local i=0
local sp='/-\|'
local n=${#sp}
printf ' '
while sleep 0.1; do
printf '\b%s' "${sp:i++%n:1}"
done
}
dispatch() {
if [ $# -lt 1 ]; then
help
exit -2
fi
if [ $1 == "env" ]; then
list
exit 0
fi
if [[ $1 == "-h" || $1 == "help" ]]; then
help
exit 0
fi
origin_env=
target_env=
if [ $# -eq 1 ]; then
origin_env=internlm-base
target_env=$1
else
while getopts t:o: flag; do
case "${flag}" in
t) target_env=${OPTARG} ;;
o) origin_env=${OPTARG} ;;
esac
done
fi
echo -e "\033[32m 预设环境: $origin_env \033[0m"
echo -e "\033[32m 目标conda环境名称: $target_env \033[0m"
sleep 3
clone $origin_env $target_env
}
dispatch $@
这个文件就是它设置代码环境的了。脚本里面定义了几个变量和函数，之后就直接调用dispatch函数了。之后的流程如下：
- 因为我们给的参数是
-o internlm-base -t demo，所以会直接从dispatch这里执行脚本中的clone函数，参数是internlm-base demo。 CONDA_HOME会通过HOME_DIR=/root; CONDA_HOME=$HOME_DIR/.conda指定为/root/.conda，即工作区下的文件夹。- 然后，将
/share/pkgs.tar.gz解压至目录，再通过conda create clone的方式克隆环境完成环境的搭建。
所以这个命令实际上是将预配置好的环境打包解压克隆了一遍，和教程中的等效代码还是有较大不同的。
然后需要我们执行以下代码配置环境。轻轻吐槽一下既然都是直接解压并conda clone了，为什么不直接做一个装好这些库的conda环境压缩包。
conda activate demo
pip install huggingface-hub==0.17.3
pip install transformers==4.34
pip install psutil==5.9.8
pip install accelerate==0.24.1
pip install streamlit==1.32.2
pip install matplotlib==3.8.3
pip install modelscope==1.9.5
pip install sentencepiece==0.1.99
查看环境
base环境下面还是啥都没有的，我们来看看demo环境下面都有些什么：
python 3.10.13 h955ad1f_0 defaults
pytorch 2.0.1 py3.10_cuda11.7_cudnn8.5.0_0 pytorch
pytorch-cuda 11.7 h778d358_5 pytorch
pytorch-mutex 1.0 cuda pytorch
torchaudio 2.0.2 py310_cu117 pytorch
torchtriton 2.0.0 py310 pytorch
torchvision 0.15.2 py310_cu117 pytorch
点击查看全部库
# packages in environment at /root/.conda/envs/demo:
#
# Name Version Build Channel
_libgcc_mutex 0.1 main defaults
_openmp_mutex 5.1 1_gnu defaults
accelerate 0.24.1 pypi_0 pypi
addict 2.4.0 pypi_0 pypi
aiohttp 3.9.3 pypi_0 pypi
aiosignal 1.3.1 pypi_0 pypi
aliyun-python-sdk-core 2.15.0 pypi_0 pypi
aliyun-python-sdk-kms 2.16.2 pypi_0 pypi
altair 5.2.0 pypi_0 pypi
asttokens 2.4.1 pypi_0 pypi
async-timeout 4.0.3 pypi_0 pypi
attrs 23.2.0 pypi_0 pypi
blas 1.0 mkl defaults
blinker 1.7.0 pypi_0 pypi
brotli-python 1.0.9 py310h6a678d5_7 defaults
bzip2 1.0.8 h7b6447c_0 defaults
ca-certificates 2023.08.22 h06a4308_0 defaults
cachetools 5.3.3 pypi_0 pypi
certifi 2023.11.17 py310h06a4308_0 defaults
cffi 1.16.0 py310h5eee18b_0 defaults
charset-normalizer 2.0.4 pyhd3eb1b0_0 defaults
click 8.1.7 pypi_0 pypi
comm 0.2.2 pypi_0 pypi
contourpy 1.2.0 pypi_0 pypi
crcmod 1.7 pypi_0 pypi
cryptography 41.0.3 py310hdda0065_0 defaults
cuda-cudart 11.7.99 0 nvidia
cuda-cupti 11.7.101 0 nvidia
cuda-libraries 11.7.1 0 nvidia
cuda-nvrtc 11.7.99 0 nvidia
cuda-nvtx 11.7.91 0 nvidia
cuda-runtime 11.7.1 0 nvidia
cycler 0.12.1 pypi_0 pypi
datasets 2.13.0 pypi_0 pypi
debugpy 1.8.1 pypi_0 pypi
decorator 5.1.1 pypi_0 pypi
dill 0.3.6 pypi_0 pypi
einops 0.7.0 pypi_0 pypi
exceptiongroup 1.2.0 pypi_0 pypi
executing 2.0.1 pypi_0 pypi
ffmpeg 4.3 hf484d3e_0 pytorch
filelock 3.13.1 py310h06a4308_0 defaults
fonttools 4.50.0 pypi_0 pypi
freetype 2.12.1 h4a9f257_0 defaults
frozenlist 1.4.1 pypi_0 pypi
fsspec 2024.3.1 pypi_0 pypi
gast 0.5.4 pypi_0 pypi
giflib 5.2.1 h5eee18b_3 defaults
gitdb 4.0.11 pypi_0 pypi
gitpython 3.1.42 pypi_0 pypi
gmp 6.2.1 h295c915_3 defaults
gmpy2 2.1.2 py310heeb90bb_0 defaults
gnutls 3.6.15 he1e5248_0 defaults
huggingface-hub 0.17.3 pypi_0 pypi
idna 3.4 py310h06a4308_0 defaults
importlib-metadata 7.1.0 pypi_0 pypi
intel-openmp 2023.1.0 hdb19cb5_46306 defaults
ipykernel 6.29.4 pypi_0 pypi
ipython 8.22.2 pypi_0 pypi
jedi 0.19.1 pypi_0 pypi
jinja2 3.1.2 py310h06a4308_0 defaults
jmespath 0.10.0 pypi_0 pypi
jpeg 9e h5eee18b_1 defaults
jsonschema 4.21.1 pypi_0 pypi
jsonschema-specifications 2023.12.1 pypi_0 pypi
jupyter-client 8.6.1 pypi_0 pypi
jupyter-core 5.7.2 pypi_0 pypi
kiwisolver 1.4.5 pypi_0 pypi
lame 3.100 h7b6447c_0 defaults
lcms2 2.12 h3be6417_0 defaults
ld_impl_linux-64 2.38 h1181459_1 defaults
lerc 3.0 h295c915_0 defaults
libcublas 11.10.3.66 0 nvidia
libcufft 10.7.2.124 h4fbf590_0 nvidia
libcufile 1.8.1.2 0 nvidia
libcurand 10.3.4.101 0 nvidia
libcusolver 11.4.0.1 0 nvidia
libcusparse 11.7.4.91 0 nvidia
libdeflate 1.17 h5eee18b_1 defaults
libffi 3.4.4 h6a678d5_0 defaults
libgcc-ng 11.2.0 h1234567_1 defaults
libgomp 11.2.0 h1234567_1 defaults
libiconv 1.16 h7f8727e_2 defaults
libidn2 2.3.4 h5eee18b_0 defaults
libnpp 11.7.4.75 0 nvidia
libnvjpeg 11.8.0.2 0 nvidia
libpng 1.6.39 h5eee18b_0 defaults
libstdcxx-ng 11.2.0 h1234567_1 defaults
libtasn1 4.19.0 h5eee18b_0 defaults
libtiff 4.5.1 h6a678d5_0 defaults
libunistring 0.9.10 h27cfd23_0 defaults
libuuid 1.41.5 h5eee18b_0 defaults
libwebp 1.3.2 h11a3e52_0 defaults
libwebp-base 1.3.2 h5eee18b_0 defaults
lz4-c 1.9.4 h6a678d5_0 defaults
markdown-it-py 3.0.0 pypi_0 pypi
markupsafe 2.1.1 py310h7f8727e_0 defaults
matplotlib 3.8.3 pypi_0 pypi
matplotlib-inline 0.1.6 pypi_0 pypi
mdurl 0.1.2 pypi_0 pypi
mkl 2023.1.0 h213fc3f_46344 defaults
mkl-service 2.4.0 py310h5eee18b_1 defaults
mkl_fft 1.3.8 py310h5eee18b_0 defaults
mkl_random 1.2.4 py310hdb19cb5_0 defaults
modelscope 1.9.5 pypi_0 pypi
mpc 1.1.0 h10f8cd9_1 defaults
mpfr 4.0.2 hb69a4c5_1 defaults
mpmath 1.3.0 py310h06a4308_0 defaults
multidict 6.0.5 pypi_0 pypi
multiprocess 0.70.14 pypi_0 pypi
ncurses 6.4 h6a678d5_0 defaults
nest-asyncio 1.6.0 pypi_0 pypi
nettle 3.7.3 hbbd107a_1 defaults
networkx 3.1 py310h06a4308_0 defaults
numpy 1.26.2 py310h5f9d8c6_0 defaults
numpy-base 1.26.2 py310hb5e798b_0 defaults
openh264 2.1.1 h4ff587b_0 defaults
openjpeg 2.4.0 h3ad879b_0 defaults
openssl 3.0.12 h7f8727e_0 defaults
oss2 2.18.4 pypi_0 pypi
packaging 23.2 pypi_0 pypi
pandas 2.2.1 pypi_0 pypi
parso 0.8.3 pypi_0 pypi
pexpect 4.9.0 pypi_0 pypi
pillow 10.0.1 py310ha6cbd5a_0 defaults
pip 23.3.1 py310h06a4308_0 defaults
platformdirs 4.2.0 pypi_0 pypi
prompt-toolkit 3.0.43 pypi_0 pypi
protobuf 4.25.3 pypi_0 pypi
psutil 5.9.8 pypi_0 pypi
ptyprocess 0.7.0 pypi_0 pypi
pure-eval 0.2.2 pypi_0 pypi
pyarrow 15.0.2 pypi_0 pypi
pycparser 2.21 pyhd3eb1b0_0 defaults
pycryptodome 3.20.0 pypi_0 pypi
pydeck 0.8.1b0 pypi_0 pypi
pygments 2.17.2 pypi_0 pypi
pyopenssl 23.2.0 py310h06a4308_0 defaults
pyparsing 3.1.2 pypi_0 pypi
pysocks 1.7.1 py310h06a4308_0 defaults
python 3.10.13 h955ad1f_0 defaults
python-dateutil 2.9.0.post0 pypi_0 pypi
pytorch 2.0.1 py3.10_cuda11.7_cudnn8.5.0_0 pytorch
pytorch-cuda 11.7 h778d358_5 pytorch
pytorch-mutex 1.0 cuda pytorch
pytz 2024.1 pypi_0 pypi
pyyaml 6.0.1 pypi_0 pypi
pyzmq 25.1.2 pypi_0 pypi
readline 8.2 h5eee18b_0 defaults
referencing 0.34.0 pypi_0 pypi
regex 2023.12.25 pypi_0 pypi
requests 2.31.0 py310h06a4308_0 defaults
rich 13.7.1 pypi_0 pypi
rpds-py 0.18.0 pypi_0 pypi
safetensors 0.4.2 pypi_0 pypi
scipy 1.12.0 pypi_0 pypi
sentencepiece 0.1.99 pypi_0 pypi
setuptools 68.0.0 py310h06a4308_0 defaults
simplejson 3.19.2 pypi_0 pypi
six 1.16.0 pypi_0 pypi
smmap 5.0.1 pypi_0 pypi
sortedcontainers 2.4.0 pypi_0 pypi
sqlite 3.41.2 h5eee18b_0 defaults
stack-data 0.6.3 pypi_0 pypi
streamlit 1.32.2 pypi_0 pypi
sympy 1.11.1 py310h06a4308_0 defaults
tbb 2021.8.0 hdb19cb5_0 defaults
tenacity 8.2.3 pypi_0 pypi
tk 8.6.12 h1ccaba5_0 defaults
tokenizers 0.14.1 pypi_0 pypi
toml 0.10.2 pypi_0 pypi
tomli 2.0.1 pypi_0 pypi
toolz 0.12.1 pypi_0 pypi
torchaudio 2.0.2 py310_cu117 pytorch
torchtriton 2.0.0 py310 pytorch
torchvision 0.15.2 py310_cu117 pytorch
tornado 6.4 pypi_0 pypi
tqdm 4.66.2 pypi_0 pypi
traitlets 5.14.2 pypi_0 pypi
transformers 4.34.0 pypi_0 pypi
typing_extensions 4.7.1 py310h06a4308_0 defaults
tzdata 2024.1 pypi_0 pypi
urllib3 1.26.18 py310h06a4308_0 defaults
watchdog 4.0.0 pypi_0 pypi
wcwidth 0.2.13 pypi_0 pypi
wheel 0.41.2 py310h06a4308_0 defaults
xxhash 3.4.1 pypi_0 pypi
xz 5.4.2 h5eee18b_0 defaults
yapf 0.40.2 pypi_0 pypi
yarl 1.9.4 pypi_0 pypi
zipp 3.18.1 pypi_0 pypi
zlib 1.2.13 h5eee18b_0 defaults
zstd 1.5.5 hc292b87_0 defaults
其实里面确实也没啥，这里列出了包的版本信息
下载模型
使用modelscope下载模型
按照教程，通过调用modelscope.hub.snapshot_download从modelscope下载模型：
import os
from modelscope.hub.snapshot_download import snapshot_download
os.system("mkdir /root/models")
save_dir="/root/models"
snapshot_download("Shanghai_AI_Laboratory/internlm2-chat-1_8b",
cache_dir=save_dir, revision='v1.1.0')
有一说一，官方教程新建文件夹这里不调用os.mkdir而是直接os.system("mkdir /root/models")真是个bad practice，别学。
使用 huggingface_hub下载模型
如果要使用huggingface_hub下载模型，首先要安装huggingface_hub包。实际上官方教程中已经把这一步夹在了2.1 配置环境中，这一步不需要我们操作了。
然后还要注意的是我们需要设置好huggingface镜像，这是因为huggingface是需要梯子才能进的网站，目前用的最多的镜像站是hf-mirror。
这一步可以在终端中通过设置终端环境变量进行：
export HF_ENDPOINT='https://hf-mirror.com'
也可以设定为单条命令的环境：
# HF_ENDPOINT='https://hf-mirror.com' 后面接你要执行的命令，如：
HF_ENDPOINT='https://hf-mirror.com' python download_hf.py
还可以在Python脚本中设定：
# 配置镜像环境变量
import os
os.environ["HF_ENDPOINT"] = 'https://hf-mirror.com'
# 从 huggingface 下载模型
from huggingface_hub import hf_hub_download # Load model directly
hf_hub_download(repo_id="internlm/internlm2-7b", filename="config.json")
官方也帮我们配置在.bashrc里了。如果使用开发机是不需要这个操作的。
执行一下这个代码：
现在问题来了：我们的config.json文件下载到哪了呢？
我们看一下hf_hub_download的函数原型：
@validate_hf_hub_args
def hf_hub_download(
repo_id: str,
filename: str,
*,
subfolder: Optional[str] = None,
repo_type: Optional[str] = None,
revision: Optional[str] = None,
endpoint: Optional[str] = None,
library_name: Optional[str] = None,
library_version: Optional[str] = None,
cache_dir: Union[str, Path, None] = None,
local_dir: Union[str, Path, None] = None,
local_dir_use_symlinks: Union[bool, Literal["auto"]] = "auto",
user_agent: Union[Dict, str, None] = None,
force_download: bool = False,
force_filename: Optional[str] = None,
proxies: Optional[Dict] = None,
etag_timeout: float = 10,
resume_download: bool = False,
token: Union[bool, str, None] = None,
local_files_only: bool = False,
legacy_cache_layout: bool = False,
) -> str:
"""Download a given file if it's not already present in the local cache.
The new cache file layout looks like this:
- The cache directory contains one subfolder per repo_id (namespaced by repo type)
- inside each repo folder:
- refs is a list of the latest known revision => commit_hash pairs
- blobs contains the actual file blobs (identified by their git-sha or sha256, depending on
whether they're LFS files or not)
- snapshots contains one subfolder per commit, each "commit" contains the subset of the files
that have been resolved at that particular commit. Each filename is a symlink to the blob
at that particular commit.
If `local_dir` is provided, the file structure from the repo will be replicated in this location. You can configure
how you want to move those files:
- If `local_dir_use_symlinks="auto"` (default), files are downloaded and stored in the cache directory as blob
files. Small files (<5MB) are duplicated in `local_dir` while a symlink is created for bigger files. The goal
is to be able to manually edit and save small files without corrupting the cache while saving disk space for
binary files. The 5MB threshold can be configured with the `HF_HUB_LOCAL_DIR_AUTO_SYMLINK_THRESHOLD`
environment variable.
- If `local_dir_use_symlinks=True`, files are downloaded, stored in the cache directory and symlinked in `local_dir`.
This is optimal in term of disk usage but files must not be manually edited.
- If `local_dir_use_symlinks=False` and the blob files exist in the cache directory, they are duplicated in the
local dir. This means disk usage is not optimized.
- Finally, if `local_dir_use_symlinks=False` and the blob files do not exist in the cache directory, then the
files are downloaded and directly placed under `local_dir`. This means if you need to download them again later,
they will be re-downloaded entirely.
Args:
repo_id (`str`):
A user or an organization name and a repo name separated by a `/`.
filename (`str`):
The name of the file in the repo.
subfolder (`str`, *optional*):
An optional value corresponding to a folder inside the model repo.
repo_type (`str`, *optional*):
Set to `"dataset"` or `"space"` if downloading from a dataset or space,
`None` or `"model"` if downloading from a model. Default is `None`.
revision (`str`, *optional*):
An optional Git revision id which can be a branch name, a tag, or a
commit hash.
endpoint (`str`, *optional*):
Hugging Face Hub base url. Will default to https://huggingface.co/. Otherwise, one can set the `HF_ENDPOINT`
environment variable.
library_name (`str`, *optional*):
The name of the library to which the object corresponds.
library_version (`str`, *optional*):
The version of the library.
cache_dir (`str`, `Path`, *optional*):
Path to the folder where cached files are stored.
local_dir (`str` or `Path`, *optional*):
If provided, the downloaded file will be placed under this directory, either as a symlink (default) or
a regular file (see description for more details).
local_dir_use_symlinks (`"auto"` or `bool`, defaults to `"auto"`):
To be used with `local_dir`. If set to "auto", the cache directory will be used and the file will be either
duplicated or symlinked to the local directory depending on its size. It set to `True`, a symlink will be
created, no matter the file size. If set to `False`, the file will either be duplicated from cache (if
already exists) or downloaded from the Hub and not cached. See description for more details.
user_agent (`dict`, `str`, *optional*):
The user-agent info in the form of a dictionary or a string.
force_download (`bool`, *optional*, defaults to `False`):
Whether the file should be downloaded even if it already exists in
the local cache.
proxies (`dict`, *optional*):
Dictionary mapping protocol to the URL of the proxy passed to
`requests.request`.
etag_timeout (`float`, *optional*, defaults to `10`):
When fetching ETag, how many seconds to wait for the server to send
data before giving up which is passed to `requests.request`.
resume_download (`bool`, *optional*, defaults to `False`):
If `True`, resume a previously interrupted download.
token (`str`, `bool`, *optional*):
A token to be used for the download.
- If `True`, the token is read from the HuggingFace config
folder.
- If a string, it's used as the authentication token.
local_files_only (`bool`, *optional*, defaults to `False`):
If `True`, avoid downloading the file and return the path to the
local cached file if it exists.
legacy_cache_layout (`bool`, *optional*, defaults to `False`):
If `True`, uses the legacy file cache layout i.e. just call [`hf_hub_url`]
then `cached_download`. This is deprecated as the new cache layout is
more powerful.
Returns:
Local path (string) of file or if networking is off, last version of
file cached on disk.
<Tip>
Raises the following errors:
- [`EnvironmentError`](https://docs.python.org/3/library/exceptions.html#EnvironmentError)
if `token=True` and the token cannot be found.
- [`OSError`](https://docs.python.org/3/library/exceptions.html#OSError)
if ETag cannot be determined.
- [`ValueError`](https://docs.python.org/3/library/exceptions.html#ValueError)
if some parameter value is invalid
- [`~utils.RepositoryNotFoundError`]
If the repository to download from cannot be found. This may be because it doesn't exist,
or because it is set to `private` and you do not have access.
- [`~utils.RevisionNotFoundError`]
If the revision to download from cannot be found.
- [`~utils.EntryNotFoundError`]
If the file to download cannot be found.
- [`~utils.LocalEntryNotFoundError`]
If network is disabled or unavailable and file is not found in cache.
</Tip>
"""
所以，文件会被下载至我们指定的 local_dir 里。如果不指定local_dir，就会下载至/root/.cache/huggingface内：
当然，这样找太费劲了。我们修改代码，重新运行：
from huggingface_hub import hf_hub_download # Load model directly
download_path = "/root/download/"
hf_hub_download(repo_id="internlm/internlm2-7b", filename="config.json", local_dir=download_path)
就能在download_path 看到 config.json了：
模型推理
输入命令，执行 Demo 程序：
conda activate demo
python /root/demo/cli_demo.py
运行结果为：
基础作业还是轻轻又松松啊哈哈哈哈。。。不过其实之前模型输出崩坏过一次：
对的，模型直接给了30个故事的名字。我直接掐断了模型的输出。
智能体Demo：使用 Lagent 运行 InternLM2-Chat-7B 模型为内核的智能体（完成 Lagent 工具调用 数据分析 Demo 部署）
搭建环境
我们需要从源码构建 Lagent：
git clone https://gitee.com/internlm/lagent.git
cd /root/demo/lagent
git checkout 581d9fb8987a5d9b72bb9ebd37a95efd47d479ac # 指定源码分支以固定版本
pip install -e . # 源码安装
搭建的环境请展开查看：
点击查看环境中安装的全部库
# packages in environment at /root/.conda/envs/demo:
#
# Name Version Build Channel
_libgcc_mutex 0.1 main defaults
_openmp_mutex 5.1 1_gnu defaults
accelerate 0.24.1 pypi_0 pypi
addict 2.4.0 pypi_0 pypi
aiohttp 3.9.3 pypi_0 pypi
aiosignal 1.3.1 pypi_0 pypi
aliyun-python-sdk-core 2.15.0 pypi_0 pypi
aliyun-python-sdk-kms 2.16.2 pypi_0 pypi
altair 5.2.0 pypi_0 pypi
anyio 4.3.0 pypi_0 pypi
argon2-cffi 23.1.0 pypi_0 pypi
argon2-cffi-bindings 21.2.0 pypi_0 pypi
arrow 1.3.0 pypi_0 pypi
arxiv 2.1.0 pypi_0 pypi
asttokens 2.4.1 pypi_0 pypi
async-lru 2.0.4 pypi_0 pypi
async-timeout 4.0.3 pypi_0 pypi
attrs 23.2.0 pypi_0 pypi
babel 2.14.0 pypi_0 pypi
beautifulsoup4 4.12.3 pypi_0 pypi
blas 1.0 mkl defaults
bleach 6.1.0 pypi_0 pypi
blinker 1.7.0 pypi_0 pypi
brotli-python 1.0.9 py310h6a678d5_7 defaults
bzip2 1.0.8 h7b6447c_0 defaults
ca-certificates 2023.08.22 h06a4308_0 defaults
cachetools 5.3.3 pypi_0 pypi
certifi 2023.11.17 py310h06a4308_0 defaults
cffi 1.16.0 py310h5eee18b_0 defaults
charset-normalizer 2.0.4 pyhd3eb1b0_0 defaults
click 8.1.7 pypi_0 pypi
colorama 0.4.6 pypi_0 pypi
comm 0.2.2 pypi_0 pypi
contourpy 1.2.0 pypi_0 pypi
crcmod 1.7 pypi_0 pypi
cryptography 41.0.3 py310hdda0065_0 defaults
cuda-cudart 11.7.99 0 nvidia
cuda-cupti 11.7.101 0 nvidia
cuda-libraries 11.7.1 0 nvidia
cuda-nvrtc 11.7.99 0 nvidia
cuda-nvtx 11.7.91 0 nvidia
cuda-runtime 11.7.1 0 nvidia
cycler 0.12.1 pypi_0 pypi
datasets 2.13.0 pypi_0 pypi
debugpy 1.8.1 pypi_0 pypi
decorator 5.1.1 pypi_0 pypi
defusedxml 0.7.1 pypi_0 pypi
dill 0.3.6 pypi_0 pypi
distro 1.9.0 pypi_0 pypi
einops 0.7.0 pypi_0 pypi
exceptiongroup 1.2.0 pypi_0 pypi
executing 2.0.1 pypi_0 pypi
fastjsonschema 2.19.1 pypi_0 pypi
feedparser 6.0.10 pypi_0 pypi
ffmpeg 4.3 hf484d3e_0 pytorch
filelock 3.13.1 py310h06a4308_0 defaults
fonttools 4.50.0 pypi_0 pypi
fqdn 1.5.1 pypi_0 pypi
freetype 2.12.1 h4a9f257_0 defaults
frozenlist 1.4.1 pypi_0 pypi
fsspec 2024.3.1 pypi_0 pypi
func-timeout 4.3.5 pypi_0 pypi
gast 0.5.4 pypi_0 pypi
giflib 5.2.1 h5eee18b_3 defaults
gitdb 4.0.11 pypi_0 pypi
gitpython 3.1.42 pypi_0 pypi
gmp 6.2.1 h295c915_3 defaults
gmpy2 2.1.2 py310heeb90bb_0 defaults
gnutls 3.6.15 he1e5248_0 defaults
griffe 0.42.1 pypi_0 pypi
h11 0.14.0 pypi_0 pypi
httpcore 1.0.5 pypi_0 pypi
httpx 0.27.0 pypi_0 pypi
huggingface-hub 0.17.3 pypi_0 pypi
idna 3.4 py310h06a4308_0 defaults
importlib-metadata 7.1.0 pypi_0 pypi
intel-openmp 2023.1.0 hdb19cb5_46306 defaults
ipykernel 6.29.4 pypi_0 pypi
ipython 8.22.2 pypi_0 pypi
ipywidgets 8.1.2 pypi_0 pypi
isoduration 20.11.0 pypi_0 pypi
jedi 0.19.1 pypi_0 pypi
jinja2 3.1.2 py310h06a4308_0 defaults
jmespath 0.10.0 pypi_0 pypi
jpeg 9e h5eee18b_1 defaults
json5 0.9.24 pypi_0 pypi
jsonpointer 2.4 pypi_0 pypi
jsonschema 4.21.1 pypi_0 pypi
jsonschema-specifications 2023.12.1 pypi_0 pypi
jupyter 1.0.0 pypi_0 pypi
jupyter-client 8.6.1 pypi_0 pypi
jupyter-console 6.6.3 pypi_0 pypi
jupyter-core 5.7.2 pypi_0 pypi
jupyter-events 0.10.0 pypi_0 pypi
jupyter-lsp 2.2.4 pypi_0 pypi
jupyter-server 2.13.0 pypi_0 pypi
jupyter-server-terminals 0.5.3 pypi_0 pypi
jupyterlab 4.1.5 pypi_0 pypi
jupyterlab-pygments 0.3.0 pypi_0 pypi
jupyterlab-server 2.25.4 pypi_0 pypi
jupyterlab-widgets 3.0.10 pypi_0 pypi
kiwisolver 1.4.5 pypi_0 pypi
lagent 0.2.2 dev_0 <develop>
lame 3.100 h7b6447c_0 defaults
lcms2 2.12 h3be6417_0 defaults
ld_impl_linux-64 2.38 h1181459_1 defaults
lerc 3.0 h295c915_0 defaults
libcublas 11.10.3.66 0 nvidia
libcufft 10.7.2.124 h4fbf590_0 nvidia
libcufile 1.8.1.2 0 nvidia
libcurand 10.3.4.101 0 nvidia
libcusolver 11.4.0.1 0 nvidia
libcusparse 11.7.4.91 0 nvidia
libdeflate 1.17 h5eee18b_1 defaults
libffi 3.4.4 h6a678d5_0 defaults
libgcc-ng 11.2.0 h1234567_1 defaults
libgomp 11.2.0 h1234567_1 defaults
libiconv 1.16 h7f8727e_2 defaults
libidn2 2.3.4 h5eee18b_0 defaults
libnpp 11.7.4.75 0 nvidia
libnvjpeg 11.8.0.2 0 nvidia
libpng 1.6.39 h5eee18b_0 defaults
libstdcxx-ng 11.2.0 h1234567_1 defaults
libtasn1 4.19.0 h5eee18b_0 defaults
libtiff 4.5.1 h6a678d5_0 defaults
libunistring 0.9.10 h27cfd23_0 defaults
libuuid 1.41.5 h5eee18b_0 defaults
libwebp 1.3.2 h11a3e52_0 defaults
libwebp-base 1.3.2 h5eee18b_0 defaults
lz4-c 1.9.4 h6a678d5_0 defaults
markdown-it-py 3.0.0 pypi_0 pypi
markupsafe 2.1.1 py310h7f8727e_0 defaults
matplotlib 3.8.3 pypi_0 pypi
matplotlib-inline 0.1.6 pypi_0 pypi
mdurl 0.1.2 pypi_0 pypi
mistune 3.0.2 pypi_0 pypi
mkl 2023.1.0 h213fc3f_46344 defaults
mkl-service 2.4.0 py310h5eee18b_1 defaults
mkl_fft 1.3.8 py310h5eee18b_0 defaults
mkl_random 1.2.4 py310hdb19cb5_0 defaults
modelscope 1.9.5 pypi_0 pypi
mpc 1.1.0 h10f8cd9_1 defaults
mpfr 4.0.2 hb69a4c5_1 defaults
mpmath 1.3.0 py310h06a4308_0 defaults
multidict 6.0.5 pypi_0 pypi
multiprocess 0.70.14 pypi_0 pypi
nbclient 0.10.0 pypi_0 pypi
nbconvert 7.16.3 pypi_0 pypi
nbformat 5.10.3 pypi_0 pypi
ncurses 6.4 h6a678d5_0 defaults
nest-asyncio 1.6.0 pypi_0 pypi
nettle 3.7.3 hbbd107a_1 defaults
networkx 3.1 py310h06a4308_0 defaults
notebook 7.1.2 pypi_0 pypi
notebook-shim 0.2.4 pypi_0 pypi
numpy 1.26.2 py310h5f9d8c6_0 defaults
numpy-base 1.26.2 py310hb5e798b_0 defaults
openh264 2.1.1 h4ff587b_0 defaults
openjpeg 2.4.0 h3ad879b_0 defaults
openssl 3.0.12 h7f8727e_0 defaults
oss2 2.18.4 pypi_0 pypi
overrides 7.7.0 pypi_0 pypi
packaging 23.2 pypi_0 pypi
pandas 2.2.1 pypi_0 pypi
pandocfilters 1.5.1 pypi_0 pypi
parso 0.8.3 pypi_0 pypi
pexpect 4.9.0 pypi_0 pypi
phx-class-registry 4.1.0 pypi_0 pypi
pillow 10.0.1 py310ha6cbd5a_0 defaults
pip 23.3.1 py310h06a4308_0 defaults
platformdirs 4.2.0 pypi_0 pypi
prometheus-client 0.20.0 pypi_0 pypi
prompt-toolkit 3.0.43 pypi_0 pypi
protobuf 4.25.3 pypi_0 pypi
psutil 5.9.8 pypi_0 pypi
ptyprocess 0.7.0 pypi_0 pypi
pure-eval 0.2.2 pypi_0 pypi
pyarrow 15.0.2 pypi_0 pypi
pycparser 2.21 pyhd3eb1b0_0 defaults
pycryptodome 3.20.0 pypi_0 pypi
pydeck 0.8.1b0 pypi_0 pypi
pygments 2.17.2 pypi_0 pypi
pyopenssl 23.2.0 py310h06a4308_0 defaults
pyparsing 3.1.2 pypi_0 pypi
pysocks 1.7.1 py310h06a4308_0 defaults
python 3.10.13 h955ad1f_0 defaults
python-dateutil 2.9.0.post0 pypi_0 pypi
python-json-logger 2.0.7 pypi_0 pypi
pytorch 2.0.1 py3.10_cuda11.7_cudnn8.5.0_0 pytorch
pytorch-cuda 11.7 h778d358_5 pytorch
pytorch-mutex 1.0 cuda pytorch
pytz 2024.1 pypi_0 pypi
pyyaml 6.0.1 pypi_0 pypi
pyzmq 25.1.2 pypi_0 pypi
qtconsole 5.5.1 pypi_0 pypi
qtpy 2.4.1 pypi_0 pypi
readline 8.2 h5eee18b_0 defaults
referencing 0.34.0 pypi_0 pypi
regex 2023.12.25 pypi_0 pypi
requests 2.31.0 py310h06a4308_0 defaults
rfc3339-validator 0.1.4 pypi_0 pypi
rfc3986-validator 0.1.1 pypi_0 pypi
rich 13.7.1 pypi_0 pypi
rpds-py 0.18.0 pypi_0 pypi
safetensors 0.4.2 pypi_0 pypi
scipy 1.12.0 pypi_0 pypi
send2trash 1.8.2 pypi_0 pypi
sentencepiece 0.1.99 pypi_0 pypi
setuptools 68.0.0 py310h06a4308_0 defaults
sgmllib3k 1.0.0 pypi_0 pypi
simplejson 3.19.2 pypi_0 pypi
six 1.16.0 pypi_0 pypi
smmap 5.0.1 pypi_0 pypi
sniffio 1.3.1 pypi_0 pypi
sortedcontainers 2.4.0 pypi_0 pypi
soupsieve 2.5 pypi_0 pypi
sqlite 3.41.2 h5eee18b_0 defaults
stack-data 0.6.3 pypi_0 pypi
streamlit 1.32.2 pypi_0 pypi
sympy 1.11.1 py310h06a4308_0 defaults
tbb 2021.8.0 hdb19cb5_0 defaults
tenacity 8.2.3 pypi_0 pypi
terminado 0.18.1 pypi_0 pypi
tiktoken 0.6.0 pypi_0 pypi
tinycss2 1.2.1 pypi_0 pypi
tk 8.6.12 h1ccaba5_0 defaults
tokenizers 0.14.1 pypi_0 pypi
toml 0.10.2 pypi_0 pypi
tomli 2.0.1 pypi_0 pypi
toolz 0.12.1 pypi_0 pypi
torchaudio 2.0.2 py310_cu117 pytorch
torchtriton 2.0.0 py310 pytorch
torchvision 0.15.2 py310_cu117 pytorch
tornado 6.4 pypi_0 pypi
tqdm 4.66.2 pypi_0 pypi
traitlets 5.14.2 pypi_0 pypi
transformers 4.34.0 pypi_0 pypi
types-python-dateutil 2.9.0.20240316 pypi_0 pypi
typing_extensions 4.7.1 py310h06a4308_0 defaults
tzdata 2024.1 pypi_0 pypi
uri-template 1.3.0 pypi_0 pypi
urllib3 1.26.18 py310h06a4308_0 defaults
watchdog 4.0.0 pypi_0 pypi
wcwidth 0.2.13 pypi_0 pypi
webcolors 1.13 pypi_0 pypi
webencodings 0.5.1 pypi_0 pypi
websocket-client 1.7.0 pypi_0 pypi
wheel 0.41.2 py310h06a4308_0 defaults
widgetsnbextension 4.0.10 pypi_0 pypi
xxhash 3.4.1 pypi_0 pypi
xz 5.4.2 h5eee18b_0 defaults
yapf 0.40.2 pypi_0 pypi
yarl 1.9.4 pypi_0 pypi
zipp 3.18.1 pypi_0 pypi
zlib 1.2.13 h5eee18b_0 defaults
zstd 1.5.5 hc292b87_0 defaults
下载模型
这里教程直接创建了share的软链接，应该是怕大家都下一遍占用带宽和硬盘。本地部署的时候需要重新去 huggingface 或者 modelscope 下载模型 internlm2-chat-7b，步骤之前已经详细写过了
ln -s /root/share/new_models/Shanghai_AI_Laboratory/internlm2-chat-7b /root/models/internlm2-chat-7b
模型推理
我们要执行的demo位于/root/demo/lagent/examples/internlm2_agent_web_demo_hf.py，首先要更改代码中耦合的模型文件（其实不改也行，顶多会重新从huggingface中下载一遍模型）：
之后使用strealit运行该文件：
streamlit run /root/demo/lagent/examples/internlm2_agent_web_demo_hf.py --server.address 127.0.0.1 --server.port 6006
再转发端口：
# 从本地使用 ssh 连接 studio 端口，将下方端口号 38374 替换成自己的端口号
ssh -CNg -L 6006:127.0.0.1:6006 root@ssh.intern-ai.org.cn -p 38374
端口转发成功后会弹出“是否允许ssh.exe访问专用网络”的窗口，同意即可。在浏览器中输入http://localhost:6006/打开demo.。首次加载模型会比较慢：
出现底部文字输入框后即模型加载完毕，此时终端中模型检查点全部加载完成：
不要忘记勾选“数据分析”选项：
运行结果为：
避坑指北
这里我说一下我碰到的坑吧：
- 命令太长导致执行的命令与预期不一致。
创建软连接这一步，ln -s /root/share/new_models/Shanghai_AI_Laboratory/internlm2-chat-7b /root/models/internlm2-chat-7b这个命令太长了，在vscode的终端里会自动只执行了一段，实际上链接没有创建成功，导致streamlit报错
huggingface_hub.utils._validators.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/root/models/internlm2-chat-7b'. Use `repo_type` argument if needed.
- ssh连接不转发端口
这里是教程没有认真看，直接执行ssh root@ssh.intern-ai.org.cn了，该命令不能转发端口。需要执行教程里的ssh -CNg -L 6006:127.0.0.1:6006 root@ssh.intern-ai.org.cn才能将服务器的6006端口转发至本地。当然后面发现code-server实在是太强大了。我直接在它的终端里点击打开链接，它会打开https://a-aide-20240330-一串字母-160311.intern-ai.org.cn/proxy/6006/，是直接就能远程访问体验demo的。
当然，官方也说了更推荐使用ssh转发端口：
(使用 VSCode 等工具进行端口转发) 直接访问开发机内 http/https 服务可能会遇到代理问题，推荐在本地机器终端执行命令
图文多模态大模型Demo（完成 浦语·灵笔2 的 图文创作 及 视觉问答 部署（需截图））
搭建环境
conda activate demo
pip install timm==0.4.12 sentencepiece==0.1.99 markdown2==2.4.10 xlsxwriter==3.1.2 gradio==4.13.0 modelscope==1.9.5
cd /root/demo
git clone https://gitee.com/internlm/InternLM-XComposer.git
cd /root/demo/InternLM-XComposer
git checkout f31220eddca2cf6246ee2ddf8e375a40457ff626
搭建的环境请展开查看：
点击查看环境中安装的全部库
# packages in environment at /root/.conda/envs/demo:
#
# Name Version Build Channel
_libgcc_mutex 0.1 main defaults
_openmp_mutex 5.1 1_gnu defaults
accelerate 0.24.1 pypi_0 pypi
addict 2.4.0 pypi_0 pypi
aiofiles 23.2.1 pypi_0 pypi
aiohttp 3.9.3 pypi_0 pypi
aiosignal 1.3.1 pypi_0 pypi
aliyun-python-sdk-core 2.15.0 pypi_0 pypi
aliyun-python-sdk-kms 2.16.2 pypi_0 pypi
altair 5.2.0 pypi_0 pypi
annotated-types 0.6.0 pypi_0 pypi
anyio 4.3.0 pypi_0 pypi
argon2-cffi 23.1.0 pypi_0 pypi
argon2-cffi-bindings 21.2.0 pypi_0 pypi
arrow 1.3.0 pypi_0 pypi
arxiv 2.1.0 pypi_0 pypi
asttokens 2.4.1 pypi_0 pypi
async-lru 2.0.4 pypi_0 pypi
async-timeout 4.0.3 pypi_0 pypi
attrs 23.2.0 pypi_0 pypi
babel 2.14.0 pypi_0 pypi
beautifulsoup4 4.12.3 pypi_0 pypi
blas 1.0 mkl defaults
bleach 6.1.0 pypi_0 pypi
blinker 1.7.0 pypi_0 pypi
brotli-python 1.0.9 py310h6a678d5_7 defaults
bzip2 1.0.8 h7b6447c_0 defaults
ca-certificates 2023.08.22 h06a4308_0 defaults
cachetools 5.3.3 pypi_0 pypi
certifi 2023.11.17 py310h06a4308_0 defaults
cffi 1.16.0 py310h5eee18b_0 defaults
charset-normalizer 2.0.4 pyhd3eb1b0_0 defaults
click 8.1.7 pypi_0 pypi
colorama 0.4.6 pypi_0 pypi
comm 0.2.2 pypi_0 pypi
contourpy 1.2.0 pypi_0 pypi
crcmod 1.7 pypi_0 pypi
cryptography 41.0.3 py310hdda0065_0 defaults
cuda-cudart 11.7.99 0 nvidia
cuda-cupti 11.7.101 0 nvidia
cuda-libraries 11.7.1 0 nvidia
cuda-nvrtc 11.7.99 0 nvidia
cuda-nvtx 11.7.91 0 nvidia
cuda-runtime 11.7.1 0 nvidia
cycler 0.12.1 pypi_0 pypi
datasets 2.13.0 pypi_0 pypi
debugpy 1.8.1 pypi_0 pypi
decorator 5.1.1 pypi_0 pypi
defusedxml 0.7.1 pypi_0 pypi
dill 0.3.6 pypi_0 pypi
distro 1.9.0 pypi_0 pypi
einops 0.7.0 pypi_0 pypi
exceptiongroup 1.2.0 pypi_0 pypi
executing 2.0.1 pypi_0 pypi
fastapi 0.110.0 pypi_0 pypi
fastjsonschema 2.19.1 pypi_0 pypi
feedparser 6.0.10 pypi_0 pypi
ffmpeg 4.3 hf484d3e_0 pytorch
ffmpy 0.3.2 pypi_0 pypi
filelock 3.13.1 py310h06a4308_0 defaults
fonttools 4.50.0 pypi_0 pypi
fqdn 1.5.1 pypi_0 pypi
freetype 2.12.1 h4a9f257_0 defaults
frozenlist 1.4.1 pypi_0 pypi
fsspec 2024.3.1 pypi_0 pypi
func-timeout 4.3.5 pypi_0 pypi
gast 0.5.4 pypi_0 pypi
giflib 5.2.1 h5eee18b_3 defaults
gitdb 4.0.11 pypi_0 pypi
gitpython 3.1.42 pypi_0 pypi
gmp 6.2.1 h295c915_3 defaults
gmpy2 2.1.2 py310heeb90bb_0 defaults
gnutls 3.6.15 he1e5248_0 defaults
gradio 4.13.0 pypi_0 pypi
gradio-client 0.8.0 pypi_0 pypi
griffe 0.42.1 pypi_0 pypi
h11 0.14.0 pypi_0 pypi
httpcore 1.0.5 pypi_0 pypi
httpx 0.27.0 pypi_0 pypi
huggingface-hub 0.22.2 pypi_0 pypi
idna 3.4 py310h06a4308_0 defaults
importlib-metadata 7.1.0 pypi_0 pypi
importlib-resources 6.4.0 pypi_0 pypi
intel-openmp 2023.1.0 hdb19cb5_46306 defaults
ipykernel 6.29.4 pypi_0 pypi
ipython 8.22.2 pypi_0 pypi
ipywidgets 8.1.2 pypi_0 pypi
isoduration 20.11.0 pypi_0 pypi
jedi 0.19.1 pypi_0 pypi
jinja2 3.1.2 py310h06a4308_0 defaults
jmespath 0.10.0 pypi_0 pypi
jpeg 9e h5eee18b_1 defaults
json5 0.9.24 pypi_0 pypi
jsonpointer 2.4 pypi_0 pypi
jsonschema 4.21.1 pypi_0 pypi
jsonschema-specifications 2023.12.1 pypi_0 pypi
jupyter 1.0.0 pypi_0 pypi
jupyter-client 8.6.1 pypi_0 pypi
jupyter-console 6.6.3 pypi_0 pypi
jupyter-core 5.7.2 pypi_0 pypi
jupyter-events 0.10.0 pypi_0 pypi
jupyter-lsp 2.2.4 pypi_0 pypi
jupyter-server 2.13.0 pypi_0 pypi
jupyter-server-terminals 0.5.3 pypi_0 pypi
jupyterlab 4.1.5 pypi_0 pypi
jupyterlab-pygments 0.3.0 pypi_0 pypi
jupyterlab-server 2.25.4 pypi_0 pypi
jupyterlab-widgets 3.0.10 pypi_0 pypi
kiwisolver 1.4.5 pypi_0 pypi
lagent 0.2.2 dev_0 <develop>
lame 3.100 h7b6447c_0 defaults
lcms2 2.12 h3be6417_0 defaults
ld_impl_linux-64 2.38 h1181459_1 defaults
lerc 3.0 h295c915_0 defaults
libcublas 11.10.3.66 0 nvidia
libcufft 10.7.2.124 h4fbf590_0 nvidia
libcufile 1.8.1.2 0 nvidia
libcurand 10.3.4.101 0 nvidia
libcusolver 11.4.0.1 0 nvidia
libcusparse 11.7.4.91 0 nvidia
libdeflate 1.17 h5eee18b_1 defaults
libffi 3.4.4 h6a678d5_0 defaults
libgcc-ng 11.2.0 h1234567_1 defaults
libgomp 11.2.0 h1234567_1 defaults
libiconv 1.16 h7f8727e_2 defaults
libidn2 2.3.4 h5eee18b_0 defaults
libnpp 11.7.4.75 0 nvidia
libnvjpeg 11.8.0.2 0 nvidia
libpng 1.6.39 h5eee18b_0 defaults
libstdcxx-ng 11.2.0 h1234567_1 defaults
libtasn1 4.19.0 h5eee18b_0 defaults
libtiff 4.5.1 h6a678d5_0 defaults
libunistring 0.9.10 h27cfd23_0 defaults
libuuid 1.41.5 h5eee18b_0 defaults
libwebp 1.3.2 h11a3e52_0 defaults
libwebp-base 1.3.2 h5eee18b_0 defaults
lz4-c 1.9.4 h6a678d5_0 defaults
markdown-it-py 3.0.0 pypi_0 pypi
markdown2 2.4.10 pypi_0 pypi
markupsafe 2.1.1 py310h7f8727e_0 defaults
matplotlib 3.8.3 pypi_0 pypi
matplotlib-inline 0.1.6 pypi_0 pypi
mdurl 0.1.2 pypi_0 pypi
mistune 3.0.2 pypi_0 pypi
mkl 2023.1.0 h213fc3f_46344 defaults
mkl-service 2.4.0 py310h5eee18b_1 defaults
mkl_fft 1.3.8 py310h5eee18b_0 defaults
mkl_random 1.2.4 py310hdb19cb5_0 defaults
modelscope 1.9.5 pypi_0 pypi
mpc 1.1.0 h10f8cd9_1 defaults
mpfr 4.0.2 hb69a4c5_1 defaults
mpmath 1.3.0 py310h06a4308_0 defaults
multidict 6.0.5 pypi_0 pypi
multiprocess 0.70.14 pypi_0 pypi
nbclient 0.10.0 pypi_0 pypi
nbconvert 7.16.3 pypi_0 pypi
nbformat 5.10.3 pypi_0 pypi
ncurses 6.4 h6a678d5_0 defaults
nest-asyncio 1.6.0 pypi_0 pypi
nettle 3.7.3 hbbd107a_1 defaults
networkx 3.1 py310h06a4308_0 defaults
notebook 7.1.2 pypi_0 pypi
notebook-shim 0.2.4 pypi_0 pypi
numpy 1.26.2 py310h5f9d8c6_0 defaults
numpy-base 1.26.2 py310hb5e798b_0 defaults
openh264 2.1.1 h4ff587b_0 defaults
openjpeg 2.4.0 h3ad879b_0 defaults
openssl 3.0.12 h7f8727e_0 defaults
orjson 3.10.0 pypi_0 pypi
oss2 2.18.4 pypi_0 pypi
overrides 7.7.0 pypi_0 pypi
packaging 23.2 pypi_0 pypi
pandas 2.2.1 pypi_0 pypi
pandocfilters 1.5.1 pypi_0 pypi
parso 0.8.3 pypi_0 pypi
pexpect 4.9.0 pypi_0 pypi
phx-class-registry 4.1.0 pypi_0 pypi
pillow 10.0.1 py310ha6cbd5a_0 defaults
pip 23.3.1 py310h06a4308_0 defaults
platformdirs 4.2.0 pypi_0 pypi
prometheus-client 0.20.0 pypi_0 pypi
prompt-toolkit 3.0.43 pypi_0 pypi
protobuf 4.25.3 pypi_0 pypi
psutil 5.9.8 pypi_0 pypi
ptyprocess 0.7.0 pypi_0 pypi
pure-eval 0.2.2 pypi_0 pypi
pyarrow 15.0.2 pypi_0 pypi
pycparser 2.21 pyhd3eb1b0_0 defaults
pycryptodome 3.20.0 pypi_0 pypi
pydantic 2.6.4 pypi_0 pypi
pydantic-core 2.16.3 pypi_0 pypi
pydeck 0.8.1b0 pypi_0 pypi
pydub 0.25.1 pypi_0 pypi
pygments 2.17.2 pypi_0 pypi
pyopenssl 23.2.0 py310h06a4308_0 defaults
pyparsing 3.1.2 pypi_0 pypi
pysocks 1.7.1 py310h06a4308_0 defaults
python 3.10.13 h955ad1f_0 defaults
python-dateutil 2.9.0.post0 pypi_0 pypi
python-json-logger 2.0.7 pypi_0 pypi
python-multipart 0.0.9 pypi_0 pypi
pytorch 2.0.1 py3.10_cuda11.7_cudnn8.5.0_0 pytorch
pytorch-cuda 11.7 h778d358_5 pytorch
pytorch-mutex 1.0 cuda pytorch
pytz 2024.1 pypi_0 pypi
pyyaml 6.0.1 pypi_0 pypi
pyzmq 25.1.2 pypi_0 pypi
qtconsole 5.5.1 pypi_0 pypi
qtpy 2.4.1 pypi_0 pypi
readline 8.2 h5eee18b_0 defaults
referencing 0.34.0 pypi_0 pypi
regex 2023.12.25 pypi_0 pypi
requests 2.31.0 py310h06a4308_0 defaults
rfc3339-validator 0.1.4 pypi_0 pypi
rfc3986-validator 0.1.1 pypi_0 pypi
rich 13.7.1 pypi_0 pypi
rpds-py 0.18.0 pypi_0 pypi
safetensors 0.4.2 pypi_0 pypi
scipy 1.12.0 pypi_0 pypi
semantic-version 2.10.0 pypi_0 pypi
send2trash 1.8.2 pypi_0 pypi
sentencepiece 0.1.99 pypi_0 pypi
setuptools 68.0.0 py310h06a4308_0 defaults
sgmllib3k 1.0.0 pypi_0 pypi
shellingham 1.5.4 pypi_0 pypi
simplejson 3.19.2 pypi_0 pypi
six 1.16.0 pypi_0 pypi
smmap 5.0.1 pypi_0 pypi
sniffio 1.3.1 pypi_0 pypi
sortedcontainers 2.4.0 pypi_0 pypi
soupsieve 2.5 pypi_0 pypi
sqlite 3.41.2 h5eee18b_0 defaults
stack-data 0.6.3 pypi_0 pypi
starlette 0.36.3 pypi_0 pypi
streamlit 1.32.2 pypi_0 pypi
sympy 1.11.1 py310h06a4308_0 defaults
tbb 2021.8.0 hdb19cb5_0 defaults
tenacity 8.2.3 pypi_0 pypi
terminado 0.18.1 pypi_0 pypi
tiktoken 0.6.0 pypi_0 pypi
timm 0.4.12 pypi_0 pypi
tinycss2 1.2.1 pypi_0 pypi
tk 8.6.12 h1ccaba5_0 defaults
tokenizers 0.14.1 pypi_0 pypi
toml 0.10.2 pypi_0 pypi
tomli 2.0.1 pypi_0 pypi
tomlkit 0.12.0 pypi_0 pypi
toolz 0.12.1 pypi_0 pypi
torchaudio 2.0.2 py310_cu117 pytorch
torchtriton 2.0.0 py310 pytorch
torchvision 0.15.2 py310_cu117 pytorch
tornado 6.4 pypi_0 pypi
tqdm 4.66.2 pypi_0 pypi
traitlets 5.14.2 pypi_0 pypi
transformers 4.34.0 pypi_0 pypi
typer 0.12.0 pypi_0 pypi
typer-cli 0.12.0 pypi_0 pypi
typer-slim 0.12.0 pypi_0 pypi
types-python-dateutil 2.9.0.20240316 pypi_0 pypi
typing-extensions 4.10.0 pypi_0 pypi
tzdata 2024.1 pypi_0 pypi
uri-template 1.3.0 pypi_0 pypi
urllib3 1.26.18 py310h06a4308_0 defaults
uvicorn 0.29.0 pypi_0 pypi
watchdog 4.0.0 pypi_0 pypi
wcwidth 0.2.13 pypi_0 pypi
webcolors 1.13 pypi_0 pypi
webencodings 0.5.1 pypi_0 pypi
websocket-client 1.7.0 pypi_0 pypi
websockets 11.0.3 pypi_0 pypi
wheel 0.41.2 py310h06a4308_0 defaults
widgetsnbextension 4.0.10 pypi_0 pypi
xlsxwriter 3.1.2 pypi_0 pypi
xxhash 3.4.1 pypi_0 pypi
xz 5.4.2 h5eee18b_0 defaults
yapf 0.40.2 pypi_0 pypi
yarl 1.9.4 pypi_0 pypi
zipp 3.18.1 pypi_0 pypi
zlib 1.2.13 h5eee18b_0 defaults
zstd 1.5.5 hc292b87_0 defaults
下载模型
ln -s /root/share/new_models/Shanghai_AI_Laboratory/internlm-xcomposer2-7b /root/models/internlm-xcomposer2-7b
ln -s /root/share/new_models/Shanghai_AI_Laboratory/internlm-xcomposer2-vl-7b /root/models/internlm-xcomposer2-vl-7b
模型推理
图文创作
cd /root/demo/InternLM-XComposer
python /root/demo/InternLM-XComposer/examples/gradio_demo_composition.py \
--code_path /root/models/internlm-xcomposer2-7b \
--private \
--num_gpus 1 \
--port 6006
我不知道本地哪里出问题了，应该是/root/share/new_models/Shanghai_AI_Laboratory/internlm-xcomposer2-7b里面没有模型文件，所以它直接从远端下载了一遍模型，吐血：
出现如下提示即表明模型加载成功。此时再ssh转发端口查看：
浏览器打开http://localhost:6006/：
这里如果使用VSCode转发端口，页面就不正常了。必须使用SSH图片显示才是正常的：
也有说法说对/root/demo/InternLM-XComposer/examples/gradio_demo_composition.py稍做修改以后就能正常的：
demo.launch(share=True, server_name="0.0.0.0", server_port=6006, root_path=f'/proxy/6006/')
直接运行Demo，运行结束后页面如下：
看上去和视频里展示的文本和图片完全一样。估计是随机种子的问题。
改成1024以后，它只生成一张图了😂，太离谱了。最后随便调了个种子（11196），再将插图数量设为8，终于好了：
此时，终端的输出是：
但是仔细看，还是觉得离谱：为什么第一张图里有知乎的水印？这张图是从哪里扒下来的？
我们不妨看看gradio_demo_composition.py里面到底在做些什么：大致流程是：
- 短文生长文
- 生成适合插入配图的标题
- 图片标题生图
- 根据生成的4张图像选择一张图
- 图文合并
其中，1-5步全都需要使用模型进行推理。所以，这个demo本身在工程上就是有工作量的。它演示了如何使用一个文生图文的大模型生成一篇图文夹杂文章的工作流。具体代码见学习笔记。
视觉问答
python /root/demo/InternLM-XComposer/examples/gradio_demo_chat.py \
--code_path /root/models/internlm-xcomposer2-vl-7b \
--private \
--num_gpus 1 \
--port 6006
这张图……这么像美人鱼吗？😥
此时终端输出为：
换了张图试一试，舒服了：
这个demo的源码和第一个对话的demo并没有什么不同，具体请看学习笔记。
服务器显卡信息
出于好奇看了看显卡信息：
开发机使用的GPU是A100，通过使用GPU虚拟化控制单个开发机的显存开销为10%、30%、50%。
这里，这个环境也封装了studio-smi命令给我们查看现在的可用显存情况：
studio-smi 命令
if command -v vgpu-smi &> /dev/null
then
echo "Running studio-smi by vgpu-smi"
vgpu-smi
else
echo "Running studio-smi by nvidia-smi"
nvidia-smi
fi
所以它实际上就是调用了vgpu-smi。而位于/usr/bin/vgpu-smi的vgpu-smi命令会直接调用/usr/bin/vgpu-smi-go这个二进制文件
demo与显卡占用
- 图文创作demo占用35604M，25% A100
- 图像理解demo占用35514M，33% A100
