Computer Vision 基础学习（6）

 

Overview

     This week we mainly talked about two parts. The difference between the fundamental matrix and the essential matrix. Using correspondences to estimate the fundamental matrix. Find the homography matrix from two images that are projected from a planer scene.

Knowledge and Thinking

Fundamental Matrix

      Before we talk about the fundamental matrix, we should clarify the defination of camera matrix.In general, the image point \( x^\prime \sim K\Pi_0 \begin{bmatrix} R & T \\ 0 &1\end{bmatrix}P \), where 

\( K=K_{s} K_{f}=\left[\begin{array}{ccc} f s_{x} & f s_{\theta} & o_{x} \\ 0 & f s_{y} & o_{y} \\ 0 & 0 & 1 \end{array}\right] \). 

We call  \(x^\prime\) point in uncalibrated camera.  The point \( x = K^{-1}x^\prime \) is called the point in calibrated camera. And we have the equation of essential matrix\( x_2^\top Ex_1 =0 \)  where \( E \) is specially for calibrated camera.  In the uncalibrated camera, the equation can be modified as

\( \begin{equation}\begin{aligned}x_2^ \top Ex_1 = 0\\ (K^{-1}x_2^\prime )^\top E(K^{-1}x1') = 0\\ x_2^{\prime\top} \underbrace{K^{-\top}E K^{-\top}}_{:F}x_1^\prime = 0\end{aligned}\end{equation} \)

The matrix F is defined as fundamental matrix. The fundamental matrix contains the calibration information in addition to the transformation and rotation in the essential matrix. The essential matrix can be drived into \(E = \hat TR\). In the fundamental matrix the transformation and rotation should be modiflied by \(K\)  \( F \sim \hat T^\prime KRK^{-1} \) and \( \hat T^\prime = K^{-\top}\hat T K^{-1} \). 

The procedure to estimate the fundamental matrix has a bit of difference from the essential matrix since the svd of the fundamental matrix contains two different singular values. After using 8 point algorithm to capture the F, we also need singular value decomposition to add the rank constraint for the matrix. But we just set the third singular value to zero without modifying the first two singular values. 

\( SVD(F) = U\begin{bmatrix} \sigma_1 & 0 & 0 \\ 0 & \sigma_2 & 0 \\0 & 0 & 0 \end{bmatrix} V^\top \)

We need to find the corresponding transfomation and rotation of the points for reconstruction of 3d point. Therefore, it is not enough to find the 3d point  only by using fundamental matrix, because we cannot estimate the calibration matrix \(K\).

Planer scene

       The best way to reconstruct the 3d object is using the images which contain the different information of the object, such as by using the camera with  different views. But in some situations, we can only obtain some images from only one view such as the satellite images. All of the object points are in the same flat. The reconstruction algorithm should be modified because the features of the planer scene have been changed. A general case can be shown in the figure1

Figure 1 the illustration of planer  scene

The planer flat \( \mathcal{H} \) is defined as \( \mathcal H =: \{X\in\mathbb R^3 | n^\top x = d\} \) where d donates the difference from the left origin to the flat. and n is the normalized vector. 

Since \(P_2\) contains the rotation and transformation of the first point  \(P_1\) . \( P_2 = RP_1+T \) 

\( \begin{equation}\begin{aligned}P_2 &= RP_1+T\frac{1}{d}n^\top P_1 \\ &= (R+T\frac{1}{d}n^\top )P_1 \end{aligned}\end{equation} \Rightarrow x_2 \sim (R+T\frac{1}{d}n^\top)x_1 \\H=:R+T\frac{1}{d}n^\top \)

\( H \) describe the transformation from the image point \(x_2\) to \(x_1\). 

The planer geometry matrix \(H\) can also be interpreted as a special situation of a two view geometry, which the object points are in the same plan, therefore a coordinate information can be canceled and set to zero.  We suppose the Z coordinate is zero.

\( \left[\begin{array}{l} x \\ y \\ 1 \end{array}\right] \sim\left[\begin{array} f & 0 & 0 & 0 \\ 0 & f & 0 & 0 \\ 0 & 0 & 1 & 0 \end{array}\right]\left[\begin{array} r_{11} & r_{12} & r_{13} & t_{x} \\ r_{21} & r_{22} & r_{23} & t_{y} \\ r_{31} & r_{32} & r_{33} & t_{z} \\ 0 & 0 & 0 & 1 \end{array}\right]\left[\begin{array}{c} p \\ q \\ 0 \\ 1 \end{array}\right] \)

The transformation matrix can be simplified as a \(3\times 3\) matrix

\( \left[\begin{array}{l} x \\ y \\ 1 \end{array}\right] \sim\left[\begin{array}{lll} f & 0 & 0 \\ 0 & f & 0 \\ 0 & 0 & 1 \end{array}\right]\left[\begin{array}{lll} r_{11} & r_{12} & t_{x} \\ r_{21} & r_{22} & t_{y} \\ r_{31} & r_{32} & t_{z} \end{array}\right]\left[\begin{array}{l} p \\ q \\ 1 \end{array}\right] \)

\( \left[\begin{array}{l} x \\ y \\ 1 \end{array}\right] \sim \left[\begin{array}{lll} h_{11} & h_{12} & h_{13} \\ h_{21} & h_{22} & h_{23} \\ h_{31} & h_{32} & h_{33} \end{array}\right]\left[\begin{array}{l} p \\ q \\ 1 \end{array}\right] \)

Form the view of the image points, we can say that  \( x_2 \) can be obtained by a rotation and transformation of \(x_1\). \(x_2 = Hx_1\) H donates a linear transformation and H contains rotation and transformation. \( H = \begin{bmatrix} R & T\\ u^\top & v\end{bmatrix} \)

\( x_2 \sim Hx_1 \) and \(x_2\) is orthogonal to \(u\times Hx_1\) for all \(u \in \mathbb R^3\)  \(x_2^\top u\times Hx_1 = 0\)  according to 8 points algorithm \(Ax=0\) in the situation \(dim(A) \leq 6\) we can not use it to solve the equation.

Because of the transformation matrix \(H\) we can say that for every \( x_1 \) there is a correspondence point \( x_2 \)