Ubuntu+Yolov5+TensortRT加速部署
Ubuntu+Yolov5+TensortRT加速部署
0.前序
- 测试机器Ubuntu18.04 GPU: 3060, CUDA11.4, CUDA10.2均可
1.软件下载
- Nvidia驱动安装 ， 历史版本下载
- CUDA下载安装
- CUDNN下载安装
- OpenCV3.4.4下载
- Anaconda下载Linux版本 or Python安装
- Pytorch安装
- Yolov5安装
- TensorRT安装
2.软件安装
2.1 驱动安装
2.2 CUDA安装
wget https://developer.download.nvidia.com/compute/cuda/11.4.2/loc
al_installers/cuda_11.4.2_470.57.02_linux.run
sudo sh cuda_11.4.2_470.57.02_linux.run
- 终端输入
gedit ~/.bashrc
- 文件后面追加
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-11.4/lib64
export PATH=$PATH:/usr/local/cuda-11.4/bin
export CUDA_HOME=$CUDA_HOME:/usr/local/cuda-11.4
- 更新环境变量配置
source ~/.bashrc
- 至此cuda安装完成，输入nvcc -V 命令查看cuda查看
2.3 CUDNN安装
-
下载如下
sudo dpkg -i libcudnn8_8.2.4.15-1+cuda11.4_amd64.deb
sudo dpkg -i libcudnn8-dev_8.2.4.15-1+cuda11.4_amd64.deb
sudo dpkg -i libcudnn8-samples_8.2.4.15-1+cuda11.4_amd64.deb
- 验证
ldconfig -v | grep cudnn
- 解压安装包
tar zxvf cudnn-11.4-linux-x64-v8.2.4.tgz
- 拷贝.h和libs文件到cuda安装目录
$ sudo cp cuda/include/cudnn.h /usr/local/cuda/include
$ sudo cp cuda/lib64/libcudnn* /usr/local/cuda/lib64
$ sudo chmod a+r /usr/local/cuda/include/cudnn.h /usr/local/cuda/lib64/libcudnn*
- 软连接创建
sudo ln -sf /usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudnn_ops_train.so.8.2.4 /usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudnn_ops_train.so.8
sudo ln -sf /usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudnn_adv_train.so.8.2.4 /usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudnn_adv_train.so.8
sudo ln -sf /usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudnn_cnn_infer.so.8.2.4 /usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudnn_cnn_infer.so.8
sudo ln -sf /usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudnn_ops_infer.so.8.2.4 /usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudnn_ops_infer.so.8
sudo ln -sf /usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudnn_cnn_train.so.8.2.4 /usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudnn_cnn_train.so.8
sudo ln -sf /usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudnn_adv_infer.so.8.2.4 /usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudnn_adv_infer.so.8
sudo ln -sf /usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudnn.so.8.2.4 /usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudnn.so.8
- 测试
# Copy the cuDNN sample to a writable path.
cp -r /usr/src/cudnn_samples_v7/ $HOME
# Go to the writable path.
cd $HOME/cudnn_samples_v7/mnistCUDNN
#Compile the mnistCUDNN sample.
make clean && make
#Run the mnistCUDNN sample.
./mnistCUDNN
If cuDNN is properly installed and running on your Linux system, you will see a message similar to the following:
Test passed!
2.4 Opencv安装
-
Opencv3.4.4解压
tar zxvf opencv-3.4.4.tar.gz
- 下载QT
apt-get install qt4-default
cmake -D CMAKE_BUILD_TYPE=RELEASE -D PYTHON_DEFAULT_EXECUTABLE=$(/home/xujunkai/test/python_env/bin/python3 -c "import sys; print(sys.executable)") -D PYTHON3_EXECUTABLE=$(/home/xujunkai/test/python_env/bin/python3 -c "import sys; print(sys.executable)") -D PYTHON3_NUMPY_INCLUDE_DIRS=$(/home/xujunkai/test/python_env/bin/python3 -c "import numpy; print(numpy.get_include())") -D PYTHON3_PACKAGES_PATH=$(/home/xujunkai/test/python_env/bin/python3 -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())") -D CMAKE_INSTALL_PREFIX=/usr/local -D WITH_TBB=ON -D WITH_V4L=ON -D WITH_OPENGL=ON -D WITH_CUDA=ON -D ENABLE_FAST_MATH=1 -D WITH_FFMPEG=ON -D CUDA_FAST_MATH=1 -D CUDA_NVCC_FLAGS="-D_FORCE_INLINES" -D CUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-11.4/ -D WITH_NVCUVID=ON -D CUDA_ARCH_BIN=5.3,6.0,6.1,7.0,7.5 -D CUDA_ARCH_PTX=7.5 -D WITH_CUBLAS=1 -D OPENCV_EXTRA_MODULES_PATH=/home/xujunkai/test/opencv_base/opencv_contrib/modules -D CUDA_nppicom_LIBRARY=stdc++ -D WITH_OPENMP=ON ..
- 错误：fatal error: linux/videodev.h: 没有那个文件或目录 参照
sudo apt-get install libv4l-dev
cd /usr/include/linux
sudo ln -s ../libv4l1-videodev.h videodev.h
-
错误: fatal error: dynlink_nvcuvid.h: 没有那个文件或目录 参照
-
其他错误可参照FFmpeg(GPU)-CUDA11.1-CUDNN8-OpenCV4.2(GPU)-Ubuntu18.04构建docker镜像 -Opencv安装错误解决办法
-
错误: modules/xfeatures2d/src/boostdesc.cpp:653:20: fatal error: boostdesc_bgm.i: 没有那个文件或目录#include "boostdesc_bgm.i"
boostdesc_bgm 链接 提取码：3xqi
将 boostdesc_bgm 内步所有文件放在 opencv_contrib/modules/xfeatures2d/src/ 下
- 安装
make -j8
make install
2.5 Python环境安装
- 推荐编译安装
2.6 pytorch安装
- 安装
pip3 install torch==1.10.2+cu113 torchvision==0.11.3+cu113 torchaudi
o==0.10.2+cu113 -f https://download.pytorch.org/whl/cu113/torch_sta
ble.html
2.7 yolov5下载
- 克隆项目
git clone https://github.com/ultralytics/yolov5.git
- 下载依赖
pip install -i https://pypi.tuna.tsinghua.edu.cn/simple -r requirements.txt
-
下载Yolov5s.pt v4.0直通车 提取码:vykq
-
测试
python detect.py --source ./data/images/ --weights weights/yolov5s.pt --conf 0.4
2.8 TensorRT安装
- 解压
tar zxvf TensorRT-8.2.1.8.Linux.x86_64-gnu.cuda-11.4.cudnn8.2.tar.gz
- 添加环境变量
export LD_LIBRARY_PATH=TensorRT解压路径/lib:$LD_LIBRARY_PAT
- CUDA环境指定
vi ~/.bashrc
export CUDA_INSTALL_DIR=/usr/local/cuda-11.4
export CUDNN_INSTALL_DIR=/usr/local/cuda-11.4
source ~/.bashrc
- 复制TensorRT路径下/lib, /include 文件夹对应系统文件夹
sudo cp -r ./lib/* /usr/lib
sudo cp -r ./include/* /usr/include
- 安装pycuda
pip install pycuda
- 报错: #include pyconfig.h
apt-get install -y libpython3.6-dev
export CPLUS_INCLUDE_PATH=/usr/local/python3.6m
- 测试
cd TensorRT-8.2.1.8/python
pip install tensorrt-8.2.1.8-cp36-none-linux_x86_64.whl
# 进入python解释器
import tensorrt
tensorrt.__version__
- TensorRT示例代码测试
# TensorRT 为7.X版本需要去data/mnist 执行 python download_pgms.py瞎子啊数据集
cd 路径/TensorRT-8.2.1.8/samples/sampleMNIST
# 编译
make clean
make
- 运行
cd 路径/TensorRT-8.2.1.8/bin
./sample_mnist
2.9 编译tensorrt/yolov5
- 下载tensorrtx
git clone https://github.com/wang-xinyu/tensorrtx.git
- 将
tensorrt/yolov5下的gen_wts.py复制yolov5项目目录下 - 生成yolov5s.wts文件
python gen_wts.py --weights weights/yolov5s.pt
- 编译
# 创建build
mkdir build
cd build
# 拷贝 yolov5s.wts 到tensorrtx/yolov5/build
# yololayer.h内CLASS_NUM 可以定义种类类型
# yolov5.cpp 内可以更改 #define BATCH_SIZE 4
cmake ..
make
- 执行TensorRT加速后的yolov5命令
yolov5 -s yolov5s.wts yolov5s.engine s
小技巧：Pycharm引入docker镜像环境
- 启动容器:
docker run <镜像名称> /bin/sh -c "while true;do echo hell xjk; sleep 20;done"
- 进入容器：
docker exec -it <容器ID> /bin/bash
- 容器执行
# 下载ssh server端
apt-get install openssh-server
# 启动ssh
service ssh start
# 设置开机自启动
echo 'service ssh start'>>~/.bashrc
# 设置密码
passwd
# 更改ssh配置
vim /etc/ssh/sshd_config
PermitRootLogin yes
UsePAM no
# vim修改:使用如下命令安装
apt-get install vim
echo ':set term=builtin_ansi' >> /usr/share/vim/vimrc
# 退出
exit
- 重新打包
docker commit <容器ID> 新镜像名:版本
- 新镜像启动容器
docker run -id -p 10001:22 新镜像名:版本 /bin/sh -c "while true;do echo hell xjk; sleep 20;done"
- Pycharm配置
File->setting->Add->SSH Interpreter ->
输入Host, Port Passowrd连接
- 设置python路径即可
