eecs 274 computer vision geometry of multiple views
TRANSCRIPT
![Page 1: EECS 274 Computer Vision Geometry of Multiple Views](https://reader036.vdocument.in/reader036/viewer/2022062722/56649f295503460f94c433f3/html5/thumbnails/1.jpg)
EECS 274 Computer Vision
Geometry of Multiple Views
![Page 2: EECS 274 Computer Vision Geometry of Multiple Views](https://reader036.vdocument.in/reader036/viewer/2022062722/56649f295503460f94c433f3/html5/thumbnails/2.jpg)
Geometry of Multiple Views
• Epipolar geometry– Essential matrix– Fundamental matrix
• Trifocal tensor• Quadrifocal tensor• Reading: FP Chapter 10, S Chapter 7
![Page 3: EECS 274 Computer Vision Geometry of Multiple Views](https://reader036.vdocument.in/reader036/viewer/2022062722/56649f295503460f94c433f3/html5/thumbnails/3.jpg)
• Epipolar plane defined by P, O, O’, p and p’
• Epipoles e, e’
• Epipolar lines l, l’
• Baseline OO’
Epipolar geometry
p’ lies on l’ where the epipolar plane intersects with image plane π’
l’ is epipolar line associated with p and intersects baseline OO’ on e’
e’ is the projection of O observed from O’
![Page 4: EECS 274 Computer Vision Geometry of Multiple Views](https://reader036.vdocument.in/reader036/viewer/2022062722/56649f295503460f94c433f3/html5/thumbnails/4.jpg)
• Potential matches for p have to lie on the corresponding epipolar line l’
• Potential matches for p’ have to lie on the corresponding epipolar line l
Epipolar constraint
![Page 5: EECS 274 Computer Vision Geometry of Multiple Views](https://reader036.vdocument.in/reader036/viewer/2022062722/56649f295503460f94c433f3/html5/thumbnails/5.jpg)
Epipolar Constraint: Calibrated Case
Essential Matrix(Longuet-Higgins, 1981)3 ×3 skew-symmetric
matrix: rank=2
M’=(R t)
![Page 6: EECS 274 Computer Vision Geometry of Multiple Views](https://reader036.vdocument.in/reader036/viewer/2022062722/56649f295503460f94c433f3/html5/thumbnails/6.jpg)
• E is defined by 5 parameters (3 for rotation and 2 for translation)
• E p’ is the epipolar line associated with p’
• E T p is the epipolar line associated with p
• Can write as l .p = 0
• The point p lies on the epipolar line associated with the vectorE p’
Properties of essential matrix
![Page 7: EECS 274 Computer Vision Geometry of Multiple Views](https://reader036.vdocument.in/reader036/viewer/2022062722/56649f295503460f94c433f3/html5/thumbnails/7.jpg)
Properties of essential matrix (cont’d)
• E e’=0 and ETe=0 (E e’=-RT[tx]e=0 )
• E is singular• E has two equal non-zero singular
values (Huang and Faugeras, 1989)
![Page 8: EECS 274 Computer Vision Geometry of Multiple Views](https://reader036.vdocument.in/reader036/viewer/2022062722/56649f295503460f94c433f3/html5/thumbnails/8.jpg)
Epipolar Constraint: Small MotionsTo First-Order:
Pure translation: Focus of Expansion e
The motion field at every point in the image points to focus of expansion
![Page 9: EECS 274 Computer Vision Geometry of Multiple Views](https://reader036.vdocument.in/reader036/viewer/2022062722/56649f295503460f94c433f3/html5/thumbnails/9.jpg)
Epipolar Constraint: Uncalibrated Case
Fundamental Matrix(Faugeras and Luong, 1992)are normalized image coordinate pp ˆ,ˆ
![Page 10: EECS 274 Computer Vision Geometry of Multiple Views](https://reader036.vdocument.in/reader036/viewer/2022062722/56649f295503460f94c433f3/html5/thumbnails/10.jpg)
• F has rank 2 and is defined by 7 parameters
• F p’ is the epipolar line associated with p’ in the 1st image
• F T p is the epipolar line associated with p in the 2nd image
• F e’=0 and F T e=0
• F is singular
Properties of fundamental matrix
![Page 11: EECS 274 Computer Vision Geometry of Multiple Views](https://reader036.vdocument.in/reader036/viewer/2022062722/56649f295503460f94c433f3/html5/thumbnails/11.jpg)
Rank-2 constraint
• F admits 7 independent parameter• Possible choice of parameterization
using e=(α,β)T and e’=(α’,β’)T and epipolar transformation
• Can be written with 4 parameters: a, b, c, d
''''''''
badcacbd
dccd
baab
F
![Page 12: EECS 274 Computer Vision Geometry of Multiple Views](https://reader036.vdocument.in/reader036/viewer/2022062722/56649f295503460f94c433f3/html5/thumbnails/12.jpg)
Weak calibration
• In theory: – E can be estimated with 5 point
correspondences– F can be estimated with 7 point
correspondences– Some methods estimate E and F matrices from
a minimal number of parameters
• Estimating epipolar geometry from a redundant set of point correspondences with unknown intrinsic parameters
![Page 13: EECS 274 Computer Vision Geometry of Multiple Views](https://reader036.vdocument.in/reader036/viewer/2022062722/56649f295503460f94c433f3/html5/thumbnails/13.jpg)
The Eight-Point Algorithm (Longuet-Higgins, 1981)
|F | =1.
Minimize:
under the constraint2
Homogenous system, set F33=1
![Page 14: EECS 274 Computer Vision Geometry of Multiple Views](https://reader036.vdocument.in/reader036/viewer/2022062722/56649f295503460f94c433f3/html5/thumbnails/14.jpg)
Least-squares minimization
• Error function:
|F | =1
Minimize:
under the constraint
factor scale :',
),'(')',(')',(
ppdppdppppe TT FFF
![Page 15: EECS 274 Computer Vision Geometry of Multiple Views](https://reader036.vdocument.in/reader036/viewer/2022062722/56649f295503460f94c433f3/html5/thumbnails/15.jpg)
Non-Linear Least-Squares Approach (Luong et al., 1993)
Minimize
with respect to the coefficients of F , using an appropriate rank-2 parameterization (4 parameters instead of 8)
8 point algorithm with least-squares minimizationignores the rank 2 propertyFirst use least squares to find epipoles e and e’ that minimizes |FT e|2 and |Fe’|2
![Page 16: EECS 274 Computer Vision Geometry of Multiple Views](https://reader036.vdocument.in/reader036/viewer/2022062722/56649f295503460f94c433f3/html5/thumbnails/16.jpg)
The Normalized Eight-Point Algorithm (Hartley, 1995)
• Estimation of transformation parameters suffer form poor numerical condition problem
• Center the image data at the origin, and scale it so themean squared distance between the origin and the data points is 2 pixels: q = T p , q’ = T’ p’
• Use the eight-point algorithm to compute F from thepoints q and q’
• Enforce the rank-2 constraint
• Output T F T’
T
i i i i
i i
![Page 17: EECS 274 Computer Vision Geometry of Multiple Views](https://reader036.vdocument.in/reader036/viewer/2022062722/56649f295503460f94c433f3/html5/thumbnails/17.jpg)
Weak calibration experiment
![Page 18: EECS 274 Computer Vision Geometry of Multiple Views](https://reader036.vdocument.in/reader036/viewer/2022062722/56649f295503460f94c433f3/html5/thumbnails/18.jpg)
Trinocular Epipolar Constraints
These constraints are not independent!
Optical centers O1O2O3 defines a trifocal plane
Generally, P does not lie on trifocal plane formed
Trifocal plane intersects retinas along t1, t2, t3
Each line defines two epipoles, e.g., t2 defines e12, e32, wrt O1 and O3
![Page 19: EECS 274 Computer Vision Geometry of Multiple Views](https://reader036.vdocument.in/reader036/viewer/2022062722/56649f295503460f94c433f3/html5/thumbnails/19.jpg)
Trinocular Epipolar Constraints: Transfer
Given p1 and p2 , p3 can be computedas the solution of linear equations.
Geometrically, p1 is found as the intersection of epipolar lines associated with p2 and p3
![Page 20: EECS 274 Computer Vision Geometry of Multiple Views](https://reader036.vdocument.in/reader036/viewer/2022062722/56649f295503460f94c433f3/html5/thumbnails/20.jpg)
Trifocal Constraints
The set of points that project onto an image line l is the plane L that contains the line and pinhole
Point P in L is projected onto p on line l (l=(a,b,c)T)
)(
1
tR
Pp z
KM
M
Recall
P
![Page 21: EECS 274 Computer Vision Geometry of Multiple Views](https://reader036.vdocument.in/reader036/viewer/2022062722/56649f295503460f94c433f3/html5/thumbnails/21.jpg)
Trifocal Constraints
All 3×3 minorsmust be zero!
Calibrated Case
Trifocal Tensorline-line-line correspondence
P
![Page 22: EECS 274 Computer Vision Geometry of Multiple Views](https://reader036.vdocument.in/reader036/viewer/2022062722/56649f295503460f94c433f3/html5/thumbnails/22.jpg)
Trifocal Constraints
Calibrated Case
Given 3 point correspondences, p1, p2, p3 of the same point P, and two lines l2, l3, (passing through p2, and p3), O1p1 must intersect the line l, where the planes L2 and L3 intersect
point-line-line correspondence
011 lpT
![Page 23: EECS 274 Computer Vision Geometry of Multiple Views](https://reader036.vdocument.in/reader036/viewer/2022062722/56649f295503460f94c433f3/html5/thumbnails/23.jpg)
Trifocal Constraints
Uncalibrated Case
P
![Page 24: EECS 274 Computer Vision Geometry of Multiple Views](https://reader036.vdocument.in/reader036/viewer/2022062722/56649f295503460f94c433f3/html5/thumbnails/24.jpg)
Trifocal Constraints
Uncalibrated Case
Trifocal Tensor
![Page 25: EECS 274 Computer Vision Geometry of Multiple Views](https://reader036.vdocument.in/reader036/viewer/2022062722/56649f295503460f94c433f3/html5/thumbnails/25.jpg)
Trifocal Constraints: 3 Points
Pick any two lines l and l through p and p .
Do it again.2 3 2 3
T( p , p , p )=01 2 3
![Page 26: EECS 274 Computer Vision Geometry of Multiple Views](https://reader036.vdocument.in/reader036/viewer/2022062722/56649f295503460f94c433f3/html5/thumbnails/26.jpg)
Properties of the Trifocal Tensor
Estimating the Trifocal Tensor
• Ignore the non-linear constraints and use linear least-squaresa posteriori• Impose the constraints a posteriori
• For any matching epipolar lines, l G l = 0 • The matrices G are singular• Each triple of points 4 independent equations• Each triple of lines 2 independent equations• 4p+2l ≥ 26 need 7 triples of points or 13 triples of lines • The coefficients of tensor satisfy 8 independent constraints in the uncalibrated case (Faugeras and Mourrain, 1995) Reduce the number of independent parameters from 26 to 18
2 1 3T i
1
i
![Page 27: EECS 274 Computer Vision Geometry of Multiple Views](https://reader036.vdocument.in/reader036/viewer/2022062722/56649f295503460f94c433f3/html5/thumbnails/27.jpg)
Multiple Views (Faugeras and Mourrain, 1995)
All 4 × 4 minors have zero determinants
![Page 28: EECS 274 Computer Vision Geometry of Multiple Views](https://reader036.vdocument.in/reader036/viewer/2022062722/56649f295503460f94c433f3/html5/thumbnails/28.jpg)
Two Views
Epipolar Constraint
6 minors
![Page 29: EECS 274 Computer Vision Geometry of Multiple Views](https://reader036.vdocument.in/reader036/viewer/2022062722/56649f295503460f94c433f3/html5/thumbnails/29.jpg)
Three Views
Trifocal Constraint
48 minors
![Page 30: EECS 274 Computer Vision Geometry of Multiple Views](https://reader036.vdocument.in/reader036/viewer/2022062722/56649f295503460f94c433f3/html5/thumbnails/30.jpg)
Four Views
Quadrifocal Constraint(Triggs, 1995)
16 minors
3,...,1,,,,1,Det
l4
k3
j2
i1
lkjiijklijkl
M
M
M
M
![Page 31: EECS 274 Computer Vision Geometry of Multiple Views](https://reader036.vdocument.in/reader036/viewer/2022062722/56649f295503460f94c433f3/html5/thumbnails/31.jpg)
Geometrically, the four rays must intersect in P..
![Page 32: EECS 274 Computer Vision Geometry of Multiple Views](https://reader036.vdocument.in/reader036/viewer/2022062722/56649f295503460f94c433f3/html5/thumbnails/32.jpg)
Quadrifocal Tensorand Lines
Given 4 point correspondences, p1, p2, p3, p4 of the same point P, and 3 lines l2, l3, l4 (passing through p2, and p3, p4), O1p1 must intersect the line l, where the planes L2 , L3, and L4
![Page 33: EECS 274 Computer Vision Geometry of Multiple Views](https://reader036.vdocument.in/reader036/viewer/2022062722/56649f295503460f94c433f3/html5/thumbnails/33.jpg)
Scale-Restraint Condition from Photogrammetry
Trinocular constraints in the presence of calibration or measurement errors