搭建 onnxruntime-gpu 运行环境
1. 采用 conda 安装
遇到错误
[E:onnxruntime:Default, provider_bridge_ort.cc:1480 TryGetProviderInfo_CUDA] /onnxruntime_src/onnxruntime/core/session/provider_bridge_ort.cc:1193 onnxruntime::Provider& onnxruntime::ProviderLibrary::Get() [ONNXRuntimeError] : 1 : FAIL : Failed to load library libonnxruntime_providers_cuda.so with error: libcublasLt.so.11: cannot open shared object file: No such file or directory
2023-12-07 16:06:22.975254616 [W:onnxruntime:Default, onnxruntime_pybind_state.cc:747 CreateExecutionProviderInstance] Failed to create CUDAExecutionProvider. Please reference https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements to ensure all dependencies are met.
原因：
onnxruntime 只支持 cuda-11.8及以下：https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html
解决方案：
重装 cuda-11.8
https://developer.nvidia.com/cuda-11-8-0-download-archive
wget https://developer.download.nvidia.com/compute/cuda/11.8.0/local_installers/cuda_11.8.0_520.61.05_linux.run
sudo sh cuda_11.8.0_520.61.05_linux.run
选择 continue, 并且后续的选择不要安装驱动，只安装 CUDA Toolkit 即可。
安装 cudnn
https://developer.nvidia.com/rdp/cudnn-download
https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html
我们选择的 Download cuDNN v8.9.7 (December 5th, 2023), for CUDA 11.x
Local Installer for Linux x86_64 (Tar)
2. 采用 dokcer 安装
第二种方法：在 docker 容器中修改
https://hub.docker.com/r/pytorch/pytorch/tags?page=1&name=1.8
拉一个镜像，我们选择 devel 版本, 会帮我们安装好 cuda、cudnn、nvcc 等：
sudo docker pull pytorch/pytorch:2.1.0-cuda11.8-cudnn8-devel
启动一个容器
sudo docker run --gpus all -it -v $PWD:/workspace pytorch/pytorch:2.1.0-cuda11.8-cudnn8-devel
在容器中安装 onnxruntime-gpu:
pip install opencv-contrib-python-headless==4.7.0.72 onnxruntime-gpu -i https://pypi.tuna.tsinghua.edu.cn/simple
注意：
不选 devel 版本，而是选 runtime 版本，会遇到以下错误：
遇到错误： [ONNXRuntimeError] : 1 : FAIL : Failed to load library libonnxruntime_providers_cuda.so with error: libcudnn.so.8: cannot open shared object file: No such file or directory
该错误需要安装好对应的 cudnn。
