基于 docker 环境的 LET-VINS 复现
来源：https://www.bilibili.com/video/BV1284y197qi/
代码：https://github.com/linyicheng1/LET-NET
论文：https://arxiv.org/pdf/2310.15655
作者知乎：https://www.zhihu.com/people/xiao-lin-tong-xue-76-22/posts
该文介绍了在Docker环境下实践LET-NET的步骤。首先，从GitHub克隆LET-NET代码，然后使用Docker运行vins-mono环境。接着，安装ncnn，配置代理，更新源，安装依赖并验证安装成功。在运行LET-NET时遇到opencv版本不兼容问题，因此将opencv从3.2升级到3.4.20，通过克隆特定版本的opencv，编译并安装。之后，解决Docker容器中的显示问题，通过xhost +命令运行LET-NET。在ROS中构建VINS-Mono时，遇到了cv_bridge找不到的问题，通过安装cv_bridge，设置ROS环境和工作空间，以及解决ncnn的配置问题，最终成功运行VINS-Mono。同时，还需要下载UMA-VI数据集并修改相关路径以运行演示。
基于 docker 实践
下载代码：
git clone https://github.com/linyicheng1/LET-NET.git
cd LET-NET/
安装思路：
先安装 vins-mono 环境：
我们先去 docker hub 找个镜像：
docker pull hyeonjaegil/vins-mono:melodic
docker run -it --privileged --network host --gpus all \
-v /tmp/.X11-unix:/tmp/.X11-unix \
-e DISPLAY -e NVIDIA_VISIBLE_DEVICES=all \
-e NVIDIA_DRIVER_CAPABILITIES=all \
-e FASTRTPS_DEFAULT_PROFILES_FILE=/usr/local/share/middleware_profiles/rtps_udp_profile.xml \
-v /dev/*:/dev/* \
-v /etc/localtime:/etc/localtime:ro \
-v ~/.Xauthority:/root/.Xauthority:rw \
-v /home/h/projects/LET-NET:/root/LET-NET \
--name vins-mono-v1 \
hyeonjaegil/vins-mono:melodic /bin/bash
lsb_release -a # 查看系统版本
安装 ncnn
https://github.com/Tencent/ncnn/wiki/how-to-build#build-for-linux
https://blog.csdn.net/weixin_44855366/article/details/130165967
# 配置代理
git config --global http.proxy http://127.0.0.1:7890
git config --global https.proxy http://127.0.0.1:7890
# 更新源
apt-get update
git clone https://github.com/Tencent/ncnn.git
cd ncnn
git submodule update --init
# 安装依赖
sudo apt install build-essential git cmake libprotobuf-dev protobuf-compiler libomp-dev libvulkan-dev vulkan-utils libopencv-dev
# 安装 ncnn
mkdir build
cd build
cmake ..
make -j$(nproc)
make install # ncnn 的文件都安装在 build 下的 install 里面
# 验证
cd ../examples
../build/examples/squeezenet ../images/256-ncnn.png
# 得到输出
532 = 0.165951
920 = 0.094098
716 = 0.062193
运行 LET-NET
cd /root/LET-NET/build
cmake .. && make -j4
报错：
/root/LET-NET/tracking.cpp:147:6: error: invalid initialization of reference of type ‘const cv::ParallelLoopBody&’ from expression of type ‘corner_tracking::extractFeature(const cv::Mat&, int, const std::vector<cv::Point_<float> >&)::<lambda(const cv::Range&)>’
});
^
原因：
opencv 版本不兼容，这个镜像是 opencv 3.2，我们给它升级下
更新 opencv 3.2 到 opencv 3.4.20
怎么 安装 指定版本的 opencv?
要使用 git clone 命令克隆特定版本的 OpenCV，可以按照以下步骤操作。这些步骤包括克隆 OpenCV 仓库、切换到指定版本的分支或标签，然后继续编译和安装。
0. 卸载 原来的 opencv
apt purge libopencv*
apt update
安装依赖：
sudo apt update
sudo apt install build-essential cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev libhdf5-dev
sudo apt install python3-dev python3-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libdc1394-22-dev
步骤 1：克隆 OpenCV 仓库
首先，克隆 OpenCV 和 OpenCV Contrib 仓库：
git clone https://github.com/opencv/opencv.git
git clone https://github.com/opencv/opencv_contrib.git
步骤 2：切换到指定版本
进入克隆的 OpenCV 仓库并列出所有可用的版本标签：
cd opencv
git tag
这将显示所有版本的标签。例如，您可能会看到以下标签：
2.4.13
3.4.0
4.5.1
4.5.2
4.5.3
4.5.4
4.5.5
4.6.0
...
假设您想切换到版本 3.4.20，可以使用以下命令：
git checkout 3.4.20
同样，对于 OpenCV Contrib 仓库：
cd ../opencv_contrib
git tag
git checkout 3.4.20
步骤 3：编译和安装 OpenCV
回到 OpenCV 仓库目录，创建一个 build 目录并进入：
cd ../opencv
mkdir build && cd build
然后使用 cmake 配置 OpenCV 构建系统，指定 OpenCV Contrib 模块路径：
cmake -D CMAKE_BUILD_TYPE=Release -D CMAKE_INSTALL_PREFIX=/usr/local -D OPENCV_EXTRA_MODULES_PATH=../../opencv_contrib/modules ..
卡住（clash）：
export http_proxy=http://127.0.0.1:7890
export https_proxy=http://127.0.0.1:7890
接着编译 OpenCV：
make clean
make -j$(nproc)
最后安装 OpenCV：
sudo make install
sudo ldconfig
通过这些步骤，您可以克隆并安装特定版本的 OpenCV。这将确保您使用的是所需的版本，无论是为了兼容性还是特定的功能需求。
查看 opencv 版本等：
opencv3:
pkg-config opencv --libs # 查看 lib
pkg-config opencv --modversion # 查看版本
opencv4:
pkg-config opencv4 --libs # 查看 lib
pkg-config opencv4 --modversion # 查看版本
参考：https://blog.csdn.net/wangmj_hdu/article/details/111507782
继续运行 LET-NET
cd /root/LET-NET/build
cmake .. && make -j4
cd ..
./build/demo ./model/model.param ./model/model.bin ./assets/nyu_snippet.mp4
在 docker 容器中，报错：
terminate called after throwing an instance of 'cv::Exception'
what(): OpenCV(3.4.20) /home/shaozu/opencv/modules/highgui/src/window_gtk.cpp:624: error: (-2:Unspecified error) Can't initialize GTK backend in function 'cvInitSystem'
运行 Docker 容器时，需要将主机的显示环境传递给容器。这样，GTK 可以在容器中正确初始化。
xhost +
然后再运行即可。
Build VINS-Mono on ROS
将 LET-NET/assets/VINS-Mono.zip copy 到 /root/catkin_ws/src 路径下
修改 ncnn 的路径：
cd ~/catkin_ws/src
vim VINS-Mono/feature_tracker/CMakeLists.txt
# set(ncnn_DIR "/root/ncnn/build/install/lib/cmake/ncnn" CACHE PATH "Directory that contains ncnnConfig.cmake")
cd ../
catkin_make
报错：
-- Could NOT find cv_bridge (missing: cv_bridge_DIR)
-- Could not find the required component 'cv_bridge'. The following CMake error indicates that you either need to install the package with the same name or change your environment so that it can be found.
CMake Error at /opt/ros/melodic/share/catkin/cmake/catkinConfig.cmake:83 (find_package):
Could not find a package configuration file provided by "cv_bridge" with
any of the following names:
cv_bridgeConfig.cmake
cv_bridge-config.cmake
Add the installation prefix of "cv_bridge" to CMAKE_PREFIX_PATH or set
"cv_bridge_DIR" to a directory containing one of the above files. If
"cv_bridge" provides a separate development package or SDK, be sure it has
been installed.
原因：
这个错误表明 CMake 无法找到 cv_bridge 包的配置文件，这通常发生在配置 ROS（机器人操作系统）项目时。cv_bridge 是一个用于在 ROS 中转换 OpenCV 图像与 ROS 图像消息的包。如果您遇到这个错误，请按照以下步骤进行排查和解决：
步骤 1：安装 cv_bridge
确保您已经安装了 cv_bridge。可以通过以下命令安装：
sudo apt-get update
sudo apt-get install ros-<distro>-cv-bridge
将
这里我们选择：melodic(18.04)
步骤 2：设置 ROS 环境
确保您已经设置了 ROS 环境变量。通常在 ~/.bashrc 中添加以下内容：
sudo vim ~/.bashrc
source /opt/ros/<distro>/setup.bash
同样，将
然后，重新加载 ~/.bashrc：
source ~/.bashrc
步骤 3：确保工作空间的正确性
如果您正在使用 ROS 工作空间，请确保您已经在工作空间中构建并安装了所有依赖项。
-
初始化工作空间：
cd ~/catkin_ws
catkin_make
报错：
CMake Error at VINS-Mono/feature_tracker/CMakeLists.txt:20 (find_package):
By not providing "Findncnn.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "ncnn", but
CMake did not find one.
Could not find a package configuration file provided by "ncnn" with any of
the following names:
ncnnConfig.cmake
ncnn-config.cmake
Add the installation prefix of "ncnn" to CMAKE_PREFIX_PATH or set
"ncnn_DIR" to a directory containing one of the above files. If "ncnn"
provides a separate development package or SDK, be sure it has been
installed.
原因：
需要设置环境变量：
export ncnn_DIR=/root/ncnn/build/install/lib/cmake/ncnn
然后再：
catkin_make
source devel/setup.bash
修改 /root/catkin_ws/src/VINS-Mono/feature_tracker/src/feature_tracker_illu.cpp：
void readImage() {
const std::string path = "/root/LET-NET/datasets/lab-module-csc_2019-02-01-14-28-51_InOut/";
修改 src/VINS-Mono/feature_tracker/src/feature_tracker.cpp
// net.load_param("/home/c211/lyc/ncnn/let_net/LET-NET/model/model.param");
// net.load_model("/home/c211/lyc/ncnn/let_net/LET-NET/model/model.bin");
net.load_param("/root/LET-NET/model/model.param");
net.load_model("/root/LET-NET/model/model.bin");
修改：src/VINS-Mono/feature_tracker/src/parameters.cpp
config_file = "/root/catkin_ws/src/VINS-Mono/config/uma-vi.yaml";//readParam<std::string>(n, "config_file");
std::string VINS_FOLDER_PATH = "/root/catkin_ws/src/VINS-Mono/";//readParam<std::string>(n, "vins_folder");
catkin_make
source devel/setup.bash
Download UMA-VI datasets
Download two sequence in https://mapir.isa.uma.es/mapirwebsite/?p=2108
lab-module-csc
lab-module-rev-csc
- Modified the path
In feature_tracker folder, change the path value in function readImage() to your path.
If you want run in orign VINS, you can just delete the define #define LET_NET in feature_tracker/src/feature_tracker.h
- Run Demo as in VINS
In terminal 1, runroslaunch vins_estimator uma_vi.launch
In terminal 2, runroslaunch vins_estimator vins_rviz.launch
In terminal 3, run./devel/lib/feature_tracker/feature_tracker_illustration
那么如何开启多个终端呢？请参考：
参考：docker容器启动多个终端,docker启动多个终端
docker ps
docker exec -it cd3b79a85d7e /bin/bash
即可看到效果。
