multiview geometry and stereopsis. inputs: two images of a scene (taken from 2 viewpoints). output:...
TRANSCRIPT
Multiview Geometry and Stereopsis
Inputs: two images of a scene (taken from 2 viewpoints) . Output: Depth map.
Inputs: multiple images of a scene. Output: 3D model.
Stereopsis
Parallax
Wikipedia
Compute Correspondences
Compute Correspondences
Correspondences
Geometric Constraints (given by epipolar geometry) make this 1D search problem instead of 2D.
Reconstruction / Triangulation
Reconstruction
• Linear Method: find P such that
• Non-Linear Method: find Q minimizing
Geometry of Two-View
• Epipolar geometry– Essential matrix– Fundamental matrix
Stereopsis Define a (linear) function F(p1, p2) such that it is zero if p1 and p2 are corresponding points in two images.
• Epipolar plane defined by P, O, O’, p and p’
• Epipoles e, e’
• Epipolar lines l, l’
• Baseline OO’
Epipolar geometry
p’ lies on l’ where the epipolar plane intersects with image plane π’
l’ is epipolar line associated with p and intersects baseline OO’ on e’
e’ is the projection of O observed from O’
• Potential matches for p have to lie on the corresponding epipolar line l’
• Potential matches for p’ have to lie on the corresponding epipolar line l
Epipolar constraint
Epipolar Constraint: Calibrated Case
Essential Matrix(Longuet-Higgins, 1981)3 ×3 skew-symmetric
matrix: rank=2
O’P’ = t + R p’
• E is defined by 5 parameters (scaling not relevant)
• E p’ is the epipolar line associated with p’
• E T p is the epipolar line associated with p
• Can write as l .p = 0
• The point p lies on the epipolar line associated with the vectorE p’
Properties of essential matrix
Properties of essential matrix
• E is defined by 5 parameters (scaling not relevant)
• E e’=0 and ETe=0 (E e’=-RT[tx]e=0 )
• E is singular• E has two equal non-zero singular values
Epipolar Constraint: Uncalibrated Case
Fundamental Matrix(Faugeras and Luong, 1992)are normalized image coordinate pp ˆ,ˆ
• F has rank 2 and is defined by 7 parameters
• F p’ is the epipolar line associated with p’ in the 1st image
• F T p is the epipolar line associated with p in the 2nd image
• F e’=0 and F T e=0
• F is singular
Properties of fundamental matrix
Weak calibration
• In theory: – E can be estimated with 5 point
correspondences– F can be estimated with 7 point
correspondences– Some methods estimate E and F matrices from
a minimal number of parameters
• Estimating epipolar geometry from a redundant set of point correspondences with unknown intrinsic parameters
The Eight-Point Algorithm (Longuet-Higgins, 1981)
|F | =1.
Minimize:
under the constraint2
Homogenous system, set F33=1
Enforcing Rank 2 Constraint
• Singular Value Decomposition of F (svd in MATLAB)
Sigma is diagonal (its (non-negative) values are called singular values of F). U, V are orthogonal.
Set the smallest singular value to zero.
The Normalized Eight-Point Algorithm (Hartley, 1995)
• Estimation of transformation parameters suffer from poor numerical condition problem
• Center the image data at the origin, and scale it so themean squared distance between the origin and the data points is 2 pixels: q = T p , q’ = T’ p’
• Use the eight-point algorithm to compute F from thepoints q and q’
• Enforce the rank-2 constraint (use singular value decomposition)
• Output T F T’
T
Weak calibration experiment
a) Linear Least Squares
b) Hartley