epipolar geometry - silvio savarese · chapter: 9 “epipolar geometry and the fundamental matrix...
TRANSCRIPT
![Page 1: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/1.jpg)
Lecture 5 -Silvio Savarese 5-Feb-14
Professor Silvio Savarese
Computational Vision and Geometry Lab
Lecture 5Epipolar Geometry
![Page 2: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/2.jpg)
Lecture 5 -Silvio Savarese 5-Feb-14
• Why is stereo useful?• Epipolar constraints• Essential and fundamental matrix• Estimating F• Examples
Lecture 5Epipolar Geometry
Reading: [AZ] Chapter: 4 “Estimation – 2D perspective transformationsChapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation”Chapter: 11 “Computation of the Fundamental Matrix F”
[FP] Chapters: 10 “The geometry of multiple views”
![Page 3: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/3.jpg)
Pinhole perspective projectionRecovering structure from a single view
C
Ow
Pp
Calibration rig
Scene
Camera K
From calibration rig
From points and lines at infinity
+ orthogonal lines and planes structure of the scene, K
location/pose of the rig, K
Knowledge about scene (point correspondences, geometry of lines & planes, etc…
![Page 4: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/4.jpg)
Pinhole perspective projectionRecovering structure from a single view
C
Ow
Pp
Calibration rig
Scene
Camera K
Why is it so difficult?
Intrinsic ambiguity of the mapping from 3D to image (2D)
![Page 5: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/5.jpg)
Recovering structure from a single view
Intrinsic ambiguity of the mapping from 3D to image (2D)
Courtesy slide S. Lazebnik
![Page 6: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/6.jpg)
Two eyes help!
![Page 7: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/7.jpg)
O2O1
x2
x1
?
Two eyes help!
This is called triangulation
K =knownK =known
R, T
llX X
l'l
![Page 8: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/8.jpg)
• Find X that minimizes
),(),( 22
2
11
2XMxdXMxd
O1O2
x1
x2
X
Triangulation
![Page 9: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/9.jpg)
Stereo-view geometry
• Correspondence: Given a point in one image, how can I find the corresponding point x’ in another one?
• Camera geometry: Given corresponding points in two images, find camera matrices, position and pose.
• Scene geometry: Find coordinates of 3D point from its projection into 2 or multiple images.
![Page 10: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/10.jpg)
• Epipolar Plane • Epipoles e1, e2
• Epipolar Lines
• Baseline
Epipolar geometry
O1O2
x2
X
x1
e1 e2
= intersections of baseline with image planes
= projections of the other camera center
For details see CS131A
Lecture 9
![Page 11: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/11.jpg)
Example: Converging image planes
e
e’
![Page 12: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/12.jpg)
O1O2
X
e2
x1 x2
e1
Example: Parallel image planes
• Baseline intersects the image plane at infinity
• Epipoles are at infinity
• Epipolar lines are parallel to x axis
![Page 13: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/13.jpg)
Example: Parallel Image Planes
e’ at
infinity
e at
infinity
![Page 14: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/14.jpg)
Example: Forward translation
• The epipoles have same position in both images
• Epipole called FOE (focus of expansion)
O2
e1
e2
O2
![Page 15: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/15.jpg)
- Two views of the same object
- Suppose I know the camera positions and camera matrices
- Given a point on left image, how can I find the corresponding point on right image?
Epipolar Constraint
![Page 16: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/16.jpg)
Epipolar geometry
O1O2
X
![Page 17: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/17.jpg)
- Two views of the same object
- Suppose I know the camera positions and camera matrices
- Given a point on left image, how can I find the corresponding point on right image?
Epipolar Constraint
![Page 18: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/18.jpg)
1
v
u
PMP
0IKM
OO’
pp’
P
R, T
Epipolar Constraint
TRRKMTT
'
1
v
u
PMP
![Page 19: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/19.jpg)
T
O
O’
T = O’ in the camera 1 reference systemR is such that a vector p’ in camera 2 is equal to R p’ in camera 1
The cameras are related by R, T
p’
Camera 1
Camera 2
![Page 20: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/20.jpg)
0IKM
OO’
pp’
P
R, T
Epipolar Constraint
K is known
(canonical cameras)
0IM
100
010
001
K
TRRKMTT
'
TRRMTT
'
![Page 21: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/21.jpg)
OO’
pp’
P
R, T
0)( pRTpT
Epipolar Constraint
is perpendicular to epipolar plane)( pRT
pR p’ in first camera reference system is
![Page 22: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/22.jpg)
baba ][
0
0
0
z
y
x
xy
xz
yz
b
b
b
aa
aa
aa
Cross product as matrix multiplication
![Page 23: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/23.jpg)
OO’
pp’
P
R, T
0)pR(TpT
0pRTpT
E = essential matrix(Longuet-Higgins, 1981)
Epipolar Constraint
![Page 24: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/24.jpg)
• E p2 is the epipolar line associated with p2 (l1 = E p2)
• ET p1 is the epipolar line associated with p1 (l2 = ET p1)
• E e2 = 0 and ET e1 = 0
• E is 3x3 matrix; 5 DOF
• E is singular (rank two)
Epipolar Constraint
O1O2
p2
P
p1
e1e2
l1 l2
021 pEpT
![Page 25: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/25.jpg)
Epipolar Constraint
OO’
pp’
P
0IKM K is unknown
R, T
TRRKMTT
'
![Page 26: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/26.jpg)
OO’
pp’
P
Epipolar Constraint
0pFpT
F = Fundamental Matrix(Faugeras and Luong, 1992)
1
KRTKFT
![Page 27: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/27.jpg)
Epipolar Constraint
O1O2
p2
P
p1
e1e2
• F p2 is the epipolar line associated with p2 (l1 = F p2)
• FT p1 is the epipolar line associated with x1 (l2 = FT p1)
• F e2 = 0 and FT e1 = 0
• F is 3x3 matrix; 7 DOF
• F is singular (rank two)
021 pFpT
![Page 28: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/28.jpg)
Why F is useful?
- Suppose F is known
- No additional information about the scene and camera is given
- Given a point on left image, how can I find the corresponding point on right image?
l’ = FT xx
![Page 29: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/29.jpg)
Why F is useful?
• F captures information about the epipolar geometry of 2 views + camera parameters
• MORE IMPORTANTLY: F gives constraints on how the scene changes under view point transformation (without reconstructing the scene!)
• Powerful tool in:• 3D reconstruction• Multi-view object/scene matching
![Page 30: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/30.jpg)
OO’
pp’
P
Estimating F
The Eight-Point Algorithm
1
v
u
pP
1
v
u
pP 0pFpT
(Hartley, 1995)
(Longuet-Higgins, 1981)
![Page 31: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/31.jpg)
0pFpT
Estimating F
Let’s take 8 corresponding points
11 12 13
21 22 23
31 32 33
'
, ,1 ' 0
1
F F F u
u v F F F v
F F F
11
12
13
21
22
23
31
32
33
', ', , ', ', , ', ',1 0
F
F
F
F
uu uv u vu vv v u v F
F
F
F
F
![Page 32: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/32.jpg)
Estimating F
![Page 33: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/33.jpg)
Estimating F
Lsq. solution by SVD!
• Rank 8 A non-zero solution exists (unique)
• Homogeneous system
• If N>8 F̂
f
1f
W 0f
W
1 1 1 1 1 1 1 1 1 1 1 1
2 2 2 2 2 2 2 2 2 2 2 2
3 3 3 3 3 3 3 3 3 3 3 3
4 4 4 4 4 4 4 4 4 4 4 4
5 5 5 5 5 5 5 5 5 5 5 5
6 6 6 6 6 6 6 6 6 6 6 6
7 7 7 7 7 7
' ' ' ' ' ' 1
' ' ' ' ' ' 1
' ' ' ' ' ' 1
' ' ' ' ' ' 1
' ' ' ' ' ' 1
' ' ' ' ' ' 1
' '
u u u v u v u v v v u v
u u u v u v u v v v u v
u u u v u v u v v v u v
u u u v u v u v v v u v
u u u v u v u v v v u v
u u u v u v u v v v u v
u u u v u v
11
12
13
21
22
23
31
7 7 7 7 7 7
32
8 8 8 8 8 8 8 8 8 8 8 8
33
0
' ' ' ' 1
' ' ' ' ' ' 1
F
F
F
F
F
F
Fu v v v u v
Fu u u v u v u v v v u v
F
![Page 34: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/34.jpg)
0pF̂pT
and estimated F may have full rank (det(F) ≠0)
0F̂F Find F that minimizes
Subject to det(F)=0
But remember: fundamental matrix is Rank2
Frobenius norm (*)
SVD (again!) can be used to solve this problem
^ ^
(*) Sqrt root of the sum of squares of all entries
satisfies:F̂
[HZ] pag 281, chapter 11, “Computation of F”
![Page 35: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/35.jpg)
Data courtesy of R. Mohr and B. Boufama.
![Page 36: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/36.jpg)
Mean errors:
10.0pixel
9.1pixel
![Page 37: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/37.jpg)
Problems with the 8-Point Algorithm
- Recall the structure of W:- do we see any potential
(numerical) issue?
F1f
,0fWLsq solution
by SVD
![Page 38: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/38.jpg)
Problems with the 8-Point Algorithm
• Highly un-balanced (not well conditioned)
• Values of W must have similar magnitude
• This creates problems during the SVD decomposition
f
W
1 1 1 1 1 1 1 1 1 1 1 1
2 2 2 2 2 2 2 2 2 2 2 2
3 3 3 3 3 3 3 3 3 3 3 3
4 4 4 4 4 4 4 4 4 4 4 4
5 5 5 5 5 5 5 5 5 5 5 5
6 6 6 6 6 6 6 6 6 6 6 6
7 7 7 7 7 7
' ' ' ' ' ' 1
' ' ' ' ' ' 1
' ' ' ' ' ' 1
' ' ' ' ' ' 1
' ' ' ' ' ' 1
' ' ' ' ' ' 1
' '
u u u v u v u v v v u v
u u u v u v u v v v u v
u u u v u v u v v v u v
u u u v u v u v v v u v
u u u v u v u v v v u v
u u u v u v u v v v u v
u u u v u v
11
12
13
21
22
23
31
7 7 7 7 7 7
32
8 8 8 8 8 8 8 8 8 8 8 8
33
0
' ' ' ' 1
' ' ' ' ' ' 1
F
F
F
F
F
F
Fu v v v u v
Fu u u v u v u v v v u v
F
0Wf
HZ
pag
10
8
![Page 39: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/39.jpg)
Normalization
IDEA: Transform image coordinate
such that the matrix W become better
conditioned (pre-conditioning)
Apply following transformation T:
(translation and scaling)
• Origin = centroid of image points
• Mean square distance of the data
points from origin is 2 pixels
ii pTq ii pTq (normalization)
![Page 40: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/40.jpg)
1. Normalize coordinates:
2. Use the eight-point algorithm to compute F’q from the
points q and q’ .
1. Enforce the rank-2 constraint.
2. De-normalize Fq:
i i
ii pTq ii pTq
0qFq q
T
The Normalized Eight-Point Algorithm
qF
'TFTF q
T
0)Fdet( q
0. Compute T and T’
![Page 41: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/41.jpg)
With
tra
nsfo
rma
tio
nW
ith
ou
t tr
an
sfo
rmatio
n Mean errors:
10.0pixel
9.1pixel
Mean errors:
1.0pixel
0.9pixel
![Page 42: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/42.jpg)
Same issue for the DLT algorithm
ixix
H
ii xHx [Section 4.4 in AZ]
![Page 43: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/43.jpg)
Lecture 5 -Silvio Savarese 5-Feb-14
• Why is stereo useful?• Epipolar constraints• Essential and fundamental matrix• Estimating F• Examples
Lecture 5Epipolar Geometry
Reading: [AZ] Chapter: 4 “Estimation – 2D perspective transformationsChapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation”Chapter: 11 “Computation of the Fundamental Matrix F”
[FP] Chapters: 10 “The geometry of multiple views”
![Page 44: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/44.jpg)
O1O2
X
e2
x1 x2
e1
Example: Parallel image planes
E=?
K K
K1=K2 = known
Hint :
x
y
z
x parallel to O1O2
R = I t = (T, 0, 0)
![Page 45: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/45.jpg)
00
00
000
0
0
0
T
T
TT
TT
TT
xy
xz
yz
IE
Essential matrix for parallel images
T = [ T 0 0 ]
R = I
RTE
![Page 46: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/46.jpg)
O1
X
e2
x1 x2
e1
Example: Parallel image planes
K K
x
y
z
horizontal!What are the
directions of
epipolar lines?
2
3221
0
00
00
000
E
xT
xT
T
Tl xx
![Page 47: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/47.jpg)
O1
X
e2
x1 x2
e1
Example: Parallel image planes
K K
x
y
z
How are
x1 and x2
related?021 xEx
T
![Page 48: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/48.jpg)
O1
X
e2
x1 x2
e1
Example: Parallel image planes
K K
x
y
z
T
0 0 0 0
p E p 0 1 0 0 0 1 0
0 0 1
u
u v T v u v T Tv Tv
T Tv
How are
x1 and x2
related?
![Page 49: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/49.jpg)
O1O2
X
e2
x1 x2
e1
Example: Parallel image planes
K K
x
y
z
Rectification: making two images “parallel”
Why it is useful? • Epipolar constraint y = y’• New views can be synthesized by linear interpolation
For details on how to rectify
two views see CS131A
Lecture 9
![Page 50: Epipolar Geometry - Silvio Savarese · Chapter: 9 “Epipolar Geometry and the Fundamental Matrix Transformation ... T u ( R pc)@ 0 Epipolar Constraint is perpendicular to epipolar](https://reader033.vdocument.in/reader033/viewer/2022052310/5f0be4b57e708231d432bc81/html5/thumbnails/50.jpg)
Next lecture:
Stereo systems and affine SFM