1 preview at least two views are required to access the depth of a scene point and in turn to...
TRANSCRIPT
![Page 1: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/1.jpg)
1
Preview
• At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure
• Multiple views can be obtained by several cameras or by moving a camera
Chapter 10: The Geometry of Multiple Views
![Page 2: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/2.jpg)
2
Since and
10.1 Two Views
1 1
1
,X x
Z
11 1( ).
xX Z
2
2 2( )x
X Z
Similarly,
2 1X X B 2
1 ( ),x
X B Z
1 2 ,Z Z Z
1 2( ) ( )x x
Z B Z
1 2
Z Bx x
![Page 3: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/3.jpg)
3
10.1.1 Epipolar Geometry
![Page 4: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/4.jpg)
4
◎ The Calibrated Case -- The intrinsic parameters of cameras are known
![Page 5: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/5.jpg)
5
which relates frames O and O’.
![Page 6: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/6.jpg)
6
![Page 7: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/7.jpg)
7
![Page 8: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/8.jpg)
8
![Page 9: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/9.jpg)
9
![Page 10: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/10.jpg)
10
![Page 11: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/11.jpg)
11
![Page 12: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/12.jpg)
12
![Page 13: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/13.jpg)
13
• The matrix associated with the rotation whose
axis is the unit vector a and whose angle is
can be shown to be
[ ]
0
1( [ ])
!i
i
ei
a a (Exercise 10.2)
Therefore, [ ]R Id t ω
![Page 14: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/14.jpg)
14
![Page 15: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/15.jpg)
15
![Page 16: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/16.jpg)
16
◎ The Uncalibrated Case -- Intrinsic parameters of cameras are unknown
![Page 17: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/17.jpg)
17
![Page 18: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/18.jpg)
18
◎ Estimates based on corresponding points between images
![Page 19: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/19.jpg)
19
![Page 20: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/20.jpg)
20
![Page 21: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/21.jpg)
21
○ 8-point algorithm (Longuet-Higgins, 1981)
。 Given 8 point correspondences
This method does not take advantage of rank = 2
( , )i i s p p
x bA
![Page 22: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/22.jpg)
22
![Page 23: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/23.jpg)
23
![Page 24: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/24.jpg)
24
○ Normalized 8-point algorithm (Hartley, 1995)
(1) Translate and scale the image versions of data points so that they are centered at the origin and the average distance to the origin is pixels , 2 : ip piT : ip piT
2
1
( )p pn
Ti i
i
(2) Compute by minimizing (3) Enforce the rank-2 constraint using
the Luong et al. method
![Page 25: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/25.jpg)
25
![Page 26: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/26.jpg)
26
White points: data points White lines: epipolar lines
![Page 27: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/27.jpg)
27
10.2 Three Views ○ Calibrated case:
![Page 28: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/28.jpg)
28
(i) The three constraints are not independent since the three associated planes are intersected at a point P. Any two of them are independent.
(ii) The position of a point (say ) can be predicted from the corresponding two points ( ).
Each pair of cameras define an epipolar constraint
1 12 2 2 23 3 3 31 10, 0, 0p p p p p pT T T
1p
2 3,p p
![Page 29: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/29.jpg)
29
![Page 30: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/30.jpg)
30
![Page 31: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/31.jpg)
31
Image line segment l is the intersection of the plane segment L and the image plane , or is the projection of l onto
Plane segment L is formed by the spatial line segment l and the viewpoint O.
Consider
10.2.1 Trifocal Geometry
![Page 32: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/32.jpg)
32
Let P(x,y,z) be a point on l, and p and l be their i
mage projections. Then, and 0,T M l P
where M: 3 by 4 projection matrix,
( , , ,1)P Tx y z( , , ) ,Ta b cl
0l PT M : the equation of plane L.
where
Rewrite as
0,T L P .TML l
0T l p
![Page 33: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/33.jpg)
33
10.2.2 The Calibrated Case ○ Let the world coordinate system be attached to the first camera. Then, the projection matrices
1 2 2 2 3 3 3( ), ( ), ( )M Id M R M R 0 t t
![Page 34: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/34.jpg)
34
![Page 35: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/35.jpg)
35
Their determinants:
![Page 36: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/36.jpg)
36
![Page 37: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/37.jpg)
37
Combine the above determinants into a vector3 2 1 1
2 3 2 3 2 3 2 31 3 2 2
2 3 2 3 2 3 1 2 32 1 3 3
2 3 2 3 2 3 2 3
T T T T
T T T T
T T T T
b G c G a G G
c G a G b G G
a G b G c G G
l l l l l l l l
l l l l l l l l l
l l l l l l l l
The fourth minor:
The fourth determinant:
![Page 38: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/38.jpg)
38
○ Trilinear Constraints
-- All the above determinants are zero because of degeneration of A
12 3
21 2 3
32 3
0
T
T
T
G
G
G
l l
l l l
l l1 2 3// T iG l l l
1 1 1 1 1 2 30,T T iG p l p l p l l
12 3
21 2 3
32 3
0
T
T T
T
G
G
G
l l
p l l
l l
![Page 39: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/39.jpg)
39
10.2.3 The Uncalibrated Case
![Page 40: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/40.jpg)
40
。 The projection matrices
1 1 2 2 1 2( ), ( ),M K M A K 0 b
。 Trilinear constraints 1
2 32
1 2 33
2 3
0,
T
T T
T
G
G
G
l l
p l l
l l
2 3 2 3i iT i TG b A A b
10.2.4 Estimation of the Trifocal Tensor
○ The three matrix define thetrifocal tensor with 27 coefficients.
3 3 iG 3 3 3
3 3 1 3( ) bM A K
1 2 2 2 3 3 3( ), ( ), ( )M Id M R M R 0 t t
(calibrated)
![Page 41: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/41.jpg)
41
(1) Estimate the trifocal tensor G from point and line correspondences (a) Each triple of matching points provides 4 independent linear equations (b) Each triple of matching lines provides 2 independent linear equations e.g., p points and l lines,
4 2 27p l
(2) Improve the numerical stability of tensor estimation by normalizing image coordinates of points and lines
and the trifocal tensor have 5, 7 and 18 independent coefficients, respectively.
,
![Page 42: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/42.jpg)
42
10.3 More Views
From1
p PMz
![Page 43: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/43.jpg)
43
3 1
3 2
( ) 0
( ) 0
m m P
m m P
T T
T T
u
v
3 1
3 2
0m m
Pm m
T T
T T
u
v
i TiM m
![Page 44: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/44.jpg)
44
![Page 45: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/45.jpg)
45
![Page 46: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/46.jpg)
46
![Page 47: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/47.jpg)
47
![Page 48: 1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained](https://reader035.vdocument.in/reader035/viewer/2022062407/56649e695503460f94b65e54/html5/thumbnails/48.jpg)
48
Each quadrilinearity expresses the four associated planes intersecting at a point