[Linux系统] 使用WSL2下的Ubuntu20.04部署AI环境

一、安装WSL2下的Ubuntu20.04

 1）打开windows WSL服务

# 以管理员身份运行 PowerShell
dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart
dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart
# 使用powershell将wsl2设置为默认版本
wsl --set-default-version 2

2）安装linux子系统

# 安装操作系统 wsl --install -d <可安装的版本名> 例如，我安装版本名为Ubuntu-18.04，命令为：wsl --install -d Ubuntu-20.04
wsl --install -d Ubuntu-20.04
# 查看可以安装的linux版本
wsl  -l -o

3）给Ubuntu换源

# 备份源列表
sudo cp /etc/apt/sources.list /etc/apt/sources.list.bak
# 命令行打开sources.list文件
sudo vim /etc/apt/sources.list
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal main restricted universe multiverse
# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal main restricted universe multiverse
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal-updates main restricted universe multiverse
# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal-updates main restricted universe multiverse
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal-backports main restricted universe multiverse
# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal-backports main restricted universe multiverse
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal-security main restricted universe multiverse
# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal-security main restricted universe multiverse multiverse
sudo apt-get update

二、迁移子系统到其他磁盘

wsl --shutdown Ubuntu-20.04
wsl --export Ubuntu-20.04 D:\WSL2\WSL2Ubuntu20.04.bak
wsl --unregister Ubuntu-20.04
wsl --import Ubuntu-20.04 D:\WSL2Ubuntu20.04LTS D:\WSL2\WSL2Ubuntu20.04.bak --version 2

三、修改为静态IP地址

在windows下，查看网卡，可以看到用于WSL的虚拟网卡：

 在 控制面板->程序与功能->启用或关闭Windows功能 下，勾选Hyper-V，启用Hyper-V管理器

 重启后，以管理员运行Hyper-V管理器，将WSL虚拟交换机桥接到物理网卡上：

完成windows操作后，在 /etc/netplan 下创建 01-network-manager-all.yaml 文件，包含以下内容：

# Let NetworkManager manage all devices on this system
network:
  version: 2
  renderer: NetworkManager
  ethernets:
    eth0:    #网卡名
      dhcp4: no    #关闭ipv4动态分配ip地址
      dhcp6: no    #关闭ipv6动态分配ip地址
      addresses: [192.168.1.100/24]    #前面是ip地址，子网掩码是255.255.255.0
      gateway4: 192.168.1.1    #网关
      nameservers:
        addresses: [114.114.114.114, 61.139.2.69] #DNS

在每次重启WSL后，DNS地址会被自动更新为172开头的内部地址，需要去修改/etc/resolv.conf里的nameserver：

# This file was automatically generated by WSL. To stop automatic generation of this file, add the following entry># [network]
# generateResolvConf = false
# 系统自动配置的DNS，没法使用（可以使用脚本覆盖掉）
nameserver 172.26.208.1
# 添加的DNS
nameserver 61.139.2.69
或者在~/.bashrc的最后添加以下内容（重新创建/etc/resolv.conf）： 
rm -rf /etc/resolv.conf
echo "nameserver 61.139.2.69" >> /etc/resolv.conf

四、在WSL下启用 systemctl 命令

参考：https://www.cnblogs.com/leokale-zz/p/17579564.html

五、安装 openssh 

sudo apt-get install openssh-server -y

修改 /etc/ssh/sshd_config 配置内容

port 22
AllowUsers root leo
PermitRootLogin yes
PasswordAuthentication yes

启动服务：

sudo service ssh restart
#或
sudo systemctl start ssh
# 开机启动
sudo systemctl enable ssh

六、安装anaconda

下载：https://mirrors.tuna.tsinghua.edu.cn/anaconda/archive/
版本：Anaconda3-2023.07-1-Linux-x86_64.sh  （Python3.11）

bash Anaconda3-2023.07-1-Linux-x86_64.sh

七、安装nginx

下载使用：http://tengine.taobao.org/download/tengine-2.3.0.tar.gz

wget http://tengine.taobao.org/download/tengine-2.3.0.tar.gz
tar zxf tengine-2.3.0.tar.gz

编译安装

cd tengine-2.3.0
./configure --prefix=/usr/local/nginx
make -j16
sudo make install
# 将nginx用systemctl管理
vi /usr/lib/systemd/system/nginx.service

[Unit]
Description=The nginx HTTP and reverse proxy server
After=syslog.target network.target remote-fs.target nss-lookup.target

[Service]
Type=forking
PIDFile=/opt/nginx/logs/nginx.pid
ExecStartPre=/opt/nginx/sbin/nginx -t
ExecStart=/opt/nginx/sbin/nginx -c /opt/nginx/conf/nginx.conf
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -s QUIT $MAINPID
PrivateTmp=true

[Install]
WantedBy=multi-user.target
systemctl enable nginx.service
systemctl start nginx.service

Nginx配置：

参考：https://www.cnblogs.com/leokale-zz/p/11773349.html

八、安装redis

 下载：http://download.redis.io/releases/redis-5.0.7.tar.gz

# 下载redis
wget http://download.redis.io/releases/redis-5.0.7.tar.gz
# 解压缩
tar xzf redis-5.0.7.tar.gz
# 移动文件夹到/usr/local中
sudo mv redis-5.0.7 /usr/local/
# 进入redis目录
cd /usr/local/redis-5.0.7
sudo make all
sudo make install
cd /etc
sudo mkdir redis
cd /usr/local/redis-5.0.7
sudo cp redis.conf /etc/redis/redis.conf

修改redis.conf内容：

cd /etc/redis/
sudo nano redis.conf
bind 172.19.222.100 127.0.0.1 # 服务IP地址
port 6379  # 服务端口
daemonize yes  # 是否以守护进程形式运行
dbfilename dump.rdb  # 持久化数据文件名称
dir /var/lib/redis  # 持久化数据文件存放位置
logfile /var/log/redis/redis-server.log  # 日志文件存放位置
database 16  # 一共16个数据库，编号1~15
slaveof  # 设置主从复制（用于分布式）
cd /var/lib
sudo mkdir redis
cd /var/log
sudo mkdir redis

设置redis服务：

# 编辑文件
vim /usr/lib/systemd/system/redis.service

# 在redis.service文件中加入以下内容 并保存

##########################

[Unit]
Description=Redis-5.0.7-6379
After=network.target

[Service]
Type=forking
PIDfile=/var/run/redis-6379.pid
ExecStart=/usr/local/bin/redis-server /etc/redis/redis.conf
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -s QUIT $MAINPID
PrivateTmp=true

[Install]
WantedBy=multi-user.target

###########################
sudo systemctl daemon-reload
sudo systemctl enable redis
sudo systemctl start redis

九、安装 cuda toolkit

CUDA toolkit版本12.1

通过deb方式安装，下载链接：https://developer.nvidia.com/cuda-12-1-0-download-archive?target_os=Linux&target_arch=x86_64&Distribution=WSL-Ubuntu&target_version=2.0&target_type=deb_local

wget https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/cuda-wsl-ubuntu.pin
sudo mv cuda-wsl-ubuntu.pin /etc/apt/preferences.d/cuda-repository-pin-600
wget https://developer.download.nvidia.com/compute/cuda/12.1.0/local_installers/cuda-repo-wsl-ubuntu-12-1-local_12.1.0-1_amd64.deb
sudo dpkg -i cuda-repo-wsl-ubuntu-12-1-local_12.1.0-1_amd64.deb
sudo cp /var/cuda-repo-wsl-ubuntu-12-1-local/cuda-*-keyring.gpg /usr/share/keyrings/
sudo apt-get update
sudo apt-get -y install cuda

配置cuda环境变量

export CUDA_HOME=/usr/local/cuda-12.1
export PATH=/usr/local/cuda-12.1/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-12.1/lib64:$LD_LIBRARY_PATH

十、安装cudnn

cudnn版本8.9.3.28

下载：cudnn-local-repo-ubuntu2004-8.9.3.28_1.0-1_amd64.deb

sudo dpkg -i cudnn-local-repo-ubuntu2004-8.9.3.28_1.0-1_amd64.deb
sudo cp /var/cudnn-local-repo-ubuntu2004-8.9.3.28/cudnn-local-AAD7FE56-keyring.gpg /usr/share/keyrings/
apt-get update
sudo apt-get install libcudnn8=8.9.3.28-1+cuda12.1
sudo apt-get install libcudnn8-dev=8.9.3.28-1+cuda12.1
sudo apt-get install libcudnn8-samples=8.9.3.28-1+cuda12.1

验证是否成功：

sudo apt-get install libfreeimage3 libfreeimage-dev -y
cp -r /usr/src/cudnn_samples_v8/ $HOME
cd  $HOME/cudnn_samples_v8/mnistCUDNN
make clean && make
./mnistCUDNN
#Test Passed！

十一、编译安装ffmpeg

1）安装pkg-config

sudo apt-get install pkg-config

2）安装nv-codec-headers

git clone https://git.videolan.org/git/ffmpeg/nv-codec-headers.git
cd nv-codec-headers 
sudo make install 
# 验证是否安装成功
pkg-config --modversion ffnvcodec

3）源码安装libx264（可选apt安装）

git clone https://code.videolan.org/videolan/x264.git
cd x264
./configure --disable-asm --enable-shared --enable-pic
# 编译 & 安装
make -j32
sudo make install

4）安装依赖库

sudo apt-get install -y aptitude 
sudo aptitude install -y build-essential
sudo aptitude install -y yasm nasm
sudo apt-get install -y libssl-dev librtmp-dev libvorbis-dev
sudo apt-get install -y libgmp3-dev libaom-dev libtheora-dev
sudo apt-get install -y libass-dev libbluray-dev libfdk-aac-dev
sudo apt-get install -y libmp3lame-dev libopencore-amrnb-dev
sudo apt-get install -y libopenmpt-dev libopus-dev libxvidcore-dev liblzma-dev
sudo apt-get install -y libshine-dev libsnappy-dev libsoxr-dev
sudo apt-get install -y libspeex-dev libtheora-dev libtwolame-dev
sudo apt-get install -y libvo-amrwbenc-dev libvpx-dev libwavpack-dev
sudo apt-get install -y libwebp-dev libx264-dev libx265-dev

5）下载ffmpeg源码

wget https://ffmpeg.org/releases/ffmpeg-4.4.tar.gz
tar zxvf ffmpeg-4.4.tar.gz

6）编译安装

./configure  --prefix="/usr/local/ffmpeg" \
--enable-static --enable-shared \
--enable-version3 --enable-gpl \
--enable-nonfree --enable-small --enable-libmp3lame \
--enable-libx264 --enable-libx265 \
--enable-libvpx --enable-libtheora --enable-libvorbis \
--enable-libopus --enable-libfdk-aac --enable-libass --enable-libwebp \
--enable-librtmp --enable-postproc --enable-libfreetype --enable-pthreads \
--enable-openssl --enable-shared --disable-debug --disable-doc --disable-ffplay \
--enable-cuvid --enable-cuda --enable-cuda-nvcc \
--extra-cflags="-I/usr/local/cuda-12.1/include" --extra-ldflags="-L/usr/local/cuda-12.1/lib64" \
--extra-cflags="-I/usr/local/include" --extra-ldflags="-L/usr/local/lib" --extra-libs="-lpthread -lm"
sudo make -j16
sudo make install

问题：

ERROR: failed checking for nvcc.
#将configure文件中的sm_30为sm_90（cuda12对应90）

 十二、安装AI环境

1）安装python依赖

# requirements.txt文件内容
requests
pyyaml
wget
numpy
prettytable
tqdm
thop
Pillow
seaborn
redis
tensorboard
soundfile
librosa
Polygon3
zhconv
pydub==0.25.1
scikit-learn==1.2.1
typeguard==2.13.3
opencv-python==4.4.0.46
Flask==2.2.3
gpustat==1.0.0
psutil==5.9.4
pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple

2）安装cudatoolkit

cudatoolkit版本11.8

conda install cudatoolkit=11.8 -c conda-forge

3）安装torch

torch版本2.0.0

下载链接：https://download.pytorch.org/whl/cu118/torch-2.0.0%2Bcu118-cp39-cp39-linux_x86_64.whl

pip install torch-2.0.0+cu118-cp39-cp39-linux_x86_64.whl -i https://pypi.tuna.tsinghua.edu.cn/simple

4）安装paddlepaddle-gpu

python -m pip install paddlepaddle-gpu==2.5.0.post117 -f https://www.paddlepaddle.org.cn/whl/linux/mkl/avx/stable.html

 

 

 

 

 

= = 

 

posted @ 2023-07-25 12:34  风间悠香  阅读(919)  评论(0)  编辑  收藏  举报