Paper Reading: Stereo DSO

开篇第一篇就写一个paper reading吧，用markdown+vim写东西切换中英文挺麻烦的，有些就偷懒都用英文写了。

Stereo DSO: Large-Scale Direct Sparse Visual Odometry with Stereo Cameras

Abstract

Optimization objectives:

1. intrinsic/extrinsic parameters of all keyframes
2. all selected pixels' depth

Integrate constraints from static stereo (左右两个相机的立体视觉约束是静态的) into the bundle adjustment pipeline of temporal multi-view stereo.
Fixed-baseline stereo resolves scale drift.

? It also reduces the sensitivities to large optical flow and to rolling shutter effect which are known shortcomings of direct image alignment methods.

1. Introduction

stem from: working in an effective way
heuristically: 启发式的
hallucinate: 出现幻觉
strip down: reduced to its simplest form

Strasdat et al. proposed to expand the concept of keyframes to integrate scale and proposed a double window optimization (Figure out what is it)

Direct methods aim at computing geometry and motion directly from the images thereby skipping the intermediate keypoint selection step.

The key idea of LSD SLAM is to incrementally track the camera and simultaneously perform a pose graph optimization in order to keep the entire camera trajectory globally consistent. 作者认为这种方式没有减少累计误差，只是把它扩散到整个轨迹中（ So the meaning of pose graph is？ ）。

Three drawbacks of DSO:

1. The mentioned performance was gained on a photometrically calibrated dataset, in its absense, the performance would degrade.
2. Scale drift
3. DSO is quite sensitive to geometric distortion as those induces by fast motion and rolling shutter. While techniques for calibrating rolling shutter exist for direct SLAM algorithm, these are often quite involved and far from real-time capable.

Contribution:

1. A stereo version of DSO. detail the proposed combination of temporal multi-view stereo and static stereo.
2. Stereo DSO is good.

2. Direct Sparse VO with Stereo Camera

• Absolute scale can be directly calculated from static stereo from the known baseline of the stereo camera
• Static stereo can provide initial depth estimation for multi-view stereo
• Static Stereo can only accurately triangulate 3D points within a limited depth range while this limit is resolved by temporal multi-view stereo.

New stereo frames are first tracked with respect to their reference keyframe in a coarse-to-fine mannar.

A joint optimization of their poses, affine brightness (两个参数:a和b) parameters, as well as the depts of all the observed 3D points and camera intrinsics, is performed.

2.1 Notation

Nothing important.

2.2 Direct Image Alignment Formulation

\[E_{ij}=\sum_{p\in P_i}\omega_p \left\| I_j[p'] - I_i[p] \right\|_\gamma \]

where \(\omega_p\) is the weight which is shown as follows.(梯度越大权重越小，不知道为啥)

\[\omega_p = \frac{c^2}{c^2+\left\| \nabla I_i(p) \right\| ^2_2} \]

光度误差对突然的光照变化非常敏感。

2.3 Tracking

All the potins inside the active window are projected into the new frame. Then the pose of the new frame is optimized by minimizing the energy function.
在之前的单目DSO中，用随机深度值来初始化,所以都会需要一个确定模式的移动来初始化。在本文中，因为这时候stereo image pair的affine brightness transfer factor是位置的，所以用NCC在水平极限上的3*5的领域中搜索。

2.4 Frame Management

The basic idea is to check if the scene or the illumination has sufficiently changed.

• scene change: 用mean square optical flow和 mean squared optical flow without rotation between the current frame and the last keyframe来衡量。
• illumination change: 用relative brightness factor \(|a_j - a_i|\) 来衡量。

-> 一个点如果是梯度大于一个阈值并且是一个block里最大的点，那么他会被选择。

-> Before a candidate point is activated and optimized in the windowed optimization, its inverse depth is constantly refined by the following non-keyframes. (找出来怎么做的)

-> 旧去新来:在边缘化点的时候把候选点加入到联合优化中。

-> The constraints from static stereo introduce scale information into the system, and they also provide good geometric priors to temporal multi-view stereo.

2.5 Windowed Optimization

-> Temporal Multi-View Stereo: 就一般的不同时刻的图片之间的立体视觉
-> Static Stereo:
-> Stereo Coupling: 为了平衡上两种约束的权重，我们引入了\(\lambda\)参数。
-> Margninalization: 在边缘化一个关键帧之前，我们首先会边缘化所有没有被过去两个关键帧看到所有active window中的点。

3. Evaluation

暂且略过不表

4. Conclusion

未来可以做的两件事:

• Loop closuring and a database for map maintenance　（LDSO半闲居士做过了）
• Dynamic object handling to further boost the VO accuracy and robustness.　（用深度学习做动态物体检测然后动的点不要了？）
  虽然自己在SLAM领域还有很多可以学习的，但是这样感觉直接法的东西也做完了？悲伤。。