CV Stereo Vision

Stereo Vision; Stereo correspondence;  dense two-frame correspond

 we mainly focus on the dense match instead of sparse or feature based stereo match methods.
Application: view synthesis and imagebased rendering

problems: noise, ambiguous, occlusion, and lack of texture. 

assumptions:

• Common assumptions are Lambertian surfaces,i.e., surfaces whose appearance does not vary with viewpoint.
• Some algorithms also model specific kinds of camera noise, or differences in gain or bias.
• algorithms have built-in smoothness assumptions (often implicit) without which the correspondence problem w ouldbe underconstrained and ill-posed.
•  ……

Terms:  

•  reference image<-->matching image m
• a disparity space (x; y; d) ->disparity space image/DSI: represent the confidence or log likelihood (i.e., cost) of a particular match implied by d(x; y).
• a pixel (x; y)<-->3-D space (X; Y;Z)

Four steps:

 (1) matching cost computation; (2) cost (support) aggregation;(3) disparity computation / optimization; and (4) disparity refinement.

 (1) Matching cost computation: -->the initial disparity space image M0(x; y; d).

squared intensity differences (SSD) 

absolute intensity differences (SAD)

normalized cross-correlation

gradient-based measures, phase and filter-bank responses

truncated quadratics and contaminated Gaussians

 (2) cost (support) aggregation:

Local and window-based methods aggregate the matching cost by summing or averaging over a support region in the DSI M(x; y; d).

 Two-dimensional evidence aggregation has been implemented using square windows or Gaussian convolution (traditional), multiple windows anchored at different points (shiftable windows), windows with adaptive sizes, and windows based on connected components of constant disparity. Three -dimensional support functions that have been proposed include limited disparity difference, limited disparity gradient, and Prazdny's coherence principle.

 (3) disparity computation / optimization:

Local methods. a local “winner-tak e-all” ”(WTA) 

Global optimization.（often skip the aggregation step）

minimize the global energy: E(d)=Edata(d)+λEsmooth(d).

Edata(d) =sum(M(x,y,d(x,y)))

Esmooth(d) =sum(ρ(d(x,y)-d(x+1,y))+ρ(d(x,y)-d(x,y+1))), where ρ is some monotonically increasing function of disparity

poor results occur at object boundaries, and the energy function solving this problems is called discontinuity-preserving based on robust ρ functions.

dynamic programming--scanline optimization.(problems: occlusion,inter-scanline consistency.

(4) disparity refinement:

estimate sub-pixel disparity. 

•  iterative gradient descent
•  fitting a curve to the matching costs at discrete disparity levels