![Page 1: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/1.jpg)
Computer Vision: Calibration and Reconstruction
Raquel Urtasun
TTI Chicago
Feb 7, 2013
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 1 / 56
![Page 2: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/2.jpg)
What did we see in class last week?
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 2 / 56
![Page 3: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/3.jpg)
Panoramas
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 3 / 56
![Page 4: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/4.jpg)
Today’s Readings
Chapter 6 and 11 of Szeliski’s book
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 4 / 56
![Page 5: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/5.jpg)
Let’s look at camera calibration
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 5 / 56
![Page 6: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/6.jpg)
Camera Calibration
Find the quantities internal to the camera that affect the imaging process as wellas the position of the camera with respect to the world
Rotation and translation
Position of image center in the image
Focal length
Different scaling factors for row pixels and column pixels
Skew factor
Lens distortion (pin-cushion effect)
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 6 / 56
![Page 7: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/7.jpg)
Camera Calibration
Find the quantities internal to the camera that affect the imaging process as wellas the position of the camera with respect to the world
Rotation and translation
Position of image center in the image
Focal length
Different scaling factors for row pixels and column pixels
Skew factor
Lens distortion (pin-cushion effect)
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 6 / 56
![Page 8: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/8.jpg)
Camera Calibration
Find the quantities internal to the camera that affect the imaging process as wellas the position of the camera with respect to the world
Rotation and translation
Position of image center in the image
Focal length
Different scaling factors for row pixels and column pixels
Skew factor
Lens distortion (pin-cushion effect)
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 6 / 56
![Page 9: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/9.jpg)
Camera Calibration
Find the quantities internal to the camera that affect the imaging process as wellas the position of the camera with respect to the world
Rotation and translation
Position of image center in the image
Focal length
Different scaling factors for row pixels and column pixels
Skew factor
Lens distortion (pin-cushion effect)
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 6 / 56
![Page 10: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/10.jpg)
Camera Calibration
Find the quantities internal to the camera that affect the imaging process as wellas the position of the camera with respect to the world
Rotation and translation
Position of image center in the image
Focal length
Different scaling factors for row pixels and column pixels
Skew factor
Lens distortion (pin-cushion effect)
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 6 / 56
![Page 11: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/11.jpg)
Camera Calibration
Find the quantities internal to the camera that affect the imaging process as wellas the position of the camera with respect to the world
Rotation and translation
Position of image center in the image
Focal length
Different scaling factors for row pixels and column pixels
Skew factor
Lens distortion (pin-cushion effect)
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 6 / 56
![Page 12: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/12.jpg)
Camera Calibration
Find the quantities internal to the camera that affect the imaging process as wellas the position of the camera with respect to the world
Rotation and translation
Position of image center in the image
Focal length
Different scaling factors for row pixels and column pixels
Skew factor
Lens distortion (pin-cushion effect)
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 6 / 56
![Page 13: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/13.jpg)
Why do we need calibration?
Have good reconstruction
Interact with the 3D world
[Source: Ramani]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 7 / 56
![Page 14: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/14.jpg)
Camera and Calibration Target
Most methods assume that we have a known 3D target in the scene
[Source: Ramani]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 8 / 56
![Page 15: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/15.jpg)
Most common used Procedure
Many algorithms!
Calibration target: 2 planes at right angle with checkerboard patterns (Tsaigrid)
We know positions of pattern corners only with respect to a coordinatesystem of the target
We position camera in front of target and find images of corners
We obtain equations that describe imaging and contain internal parametersof camera
We also find position and orientation of camera with respect to target(camera pose)
[Source: Ramani]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 9 / 56
![Page 16: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/16.jpg)
Most common used Procedure
Many algorithms!
Calibration target: 2 planes at right angle with checkerboard patterns (Tsaigrid)
We know positions of pattern corners only with respect to a coordinatesystem of the target
We position camera in front of target and find images of corners
We obtain equations that describe imaging and contain internal parametersof camera
We also find position and orientation of camera with respect to target(camera pose)
[Source: Ramani]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 9 / 56
![Page 17: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/17.jpg)
Most common used Procedure
Many algorithms!
Calibration target: 2 planes at right angle with checkerboard patterns (Tsaigrid)
We know positions of pattern corners only with respect to a coordinatesystem of the target
We position camera in front of target and find images of corners
We obtain equations that describe imaging and contain internal parametersof camera
We also find position and orientation of camera with respect to target(camera pose)
[Source: Ramani]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 9 / 56
![Page 18: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/18.jpg)
Most common used Procedure
Many algorithms!
Calibration target: 2 planes at right angle with checkerboard patterns (Tsaigrid)
We know positions of pattern corners only with respect to a coordinatesystem of the target
We position camera in front of target and find images of corners
We obtain equations that describe imaging and contain internal parametersof camera
We also find position and orientation of camera with respect to target(camera pose)
[Source: Ramani]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 9 / 56
![Page 19: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/19.jpg)
Most common used Procedure
Many algorithms!
Calibration target: 2 planes at right angle with checkerboard patterns (Tsaigrid)
We know positions of pattern corners only with respect to a coordinatesystem of the target
We position camera in front of target and find images of corners
We obtain equations that describe imaging and contain internal parametersof camera
We also find position and orientation of camera with respect to target(camera pose)
[Source: Ramani]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 9 / 56
![Page 20: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/20.jpg)
Most common used Procedure
Many algorithms!
Calibration target: 2 planes at right angle with checkerboard patterns (Tsaigrid)
We know positions of pattern corners only with respect to a coordinatesystem of the target
We position camera in front of target and find images of corners
We obtain equations that describe imaging and contain internal parametersof camera
We also find position and orientation of camera with respect to target(camera pose)
[Source: Ramani]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 9 / 56
![Page 21: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/21.jpg)
Obtaining 2D-3D correspondences
Canny edge detection
Straight line fitting to detect linked edges. How?
Intersect the lines to obtain the image corners
Matching image corners and 3D target checkerboard corners (sometimesusing manual interaction)
We get pairs (image point)–(world point), (xi , yi )→ (Xi ,Yi ,Zi ).
[Source: Ramani]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 10 / 56
![Page 22: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/22.jpg)
Obtaining 2D-3D correspondences
Canny edge detection
Straight line fitting to detect linked edges. How?
Intersect the lines to obtain the image corners
Matching image corners and 3D target checkerboard corners (sometimesusing manual interaction)
We get pairs (image point)–(world point), (xi , yi )→ (Xi ,Yi ,Zi ).
[Source: Ramani]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 10 / 56
![Page 23: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/23.jpg)
Obtaining 2D-3D correspondences
Canny edge detection
Straight line fitting to detect linked edges. How?
Intersect the lines to obtain the image corners
Matching image corners and 3D target checkerboard corners (sometimesusing manual interaction)
We get pairs (image point)–(world point), (xi , yi )→ (Xi ,Yi ,Zi ).
[Source: Ramani]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 10 / 56
![Page 24: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/24.jpg)
Obtaining 2D-3D correspondences
Canny edge detection
Straight line fitting to detect linked edges. How?
Intersect the lines to obtain the image corners
Matching image corners and 3D target checkerboard corners (sometimesusing manual interaction)
We get pairs (image point)–(world point), (xi , yi )→ (Xi ,Yi ,Zi ).
[Source: Ramani]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 10 / 56
![Page 25: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/25.jpg)
Obtaining 2D-3D correspondences
Canny edge detection
Straight line fitting to detect linked edges. How?
Intersect the lines to obtain the image corners
Matching image corners and 3D target checkerboard corners (sometimesusing manual interaction)
We get pairs (image point)–(world point), (xi , yi )→ (Xi ,Yi ,Zi ).
[Source: Ramani]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 10 / 56
![Page 26: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/26.jpg)
Calibration
Estimate matrix P using 2D-3D correspondences
Estimate K and (R, t) from P
P = K · R · [I3×3 | t]
Left 3x3 submatrix of P is product of upper-triangular matrix andorthogonal matrix
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 11 / 56
![Page 27: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/27.jpg)
Calibration
Estimate matrix P using 2D-3D correspondences
Estimate K and (R, t) from P
P = K · R · [I3×3 | t]
Left 3x3 submatrix of P is product of upper-triangular matrix andorthogonal matrix
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 11 / 56
![Page 28: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/28.jpg)
Calibration
Estimate matrix P using 2D-3D correspondences
Estimate K and (R, t) from P
P = K · R · [I3×3 | t]
Left 3x3 submatrix of P is product of upper-triangular matrix andorthogonal matrix
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 11 / 56
![Page 29: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/29.jpg)
Inherent Constraints
The visual angle between any pair of 2D points must be the same as theangle between their corresponding 3D points.
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 12 / 56
![Page 30: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/30.jpg)
Direct Linear Transform
Simplest to form a set of linear equations (analog to the 2D case)
xi =p00Xi + p01Yi + p02Zi + p03p20Xi + p21Yi + p22Yi + p23
yi =p10Xi + p11Yi + p12Zi + p13p20Xi + p21Yi + p22Yi + p23
with (xi , yi ), the measured 2D points, and (Xi ,Yi ,Zi ) are known 3Dlocations
This can be solve in a linear fashion for P. How?
Similar to the homography case.
This is called the Direct Linear Transform (DLT)
How many unknowns? how many correspondences?
11 or 12 unknowns, 6 correspondences
Are we done?
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 13 / 56
![Page 31: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/31.jpg)
Direct Linear Transform
Simplest to form a set of linear equations (analog to the 2D case)
xi =p00Xi + p01Yi + p02Zi + p03p20Xi + p21Yi + p22Yi + p23
yi =p10Xi + p11Yi + p12Zi + p13p20Xi + p21Yi + p22Yi + p23
with (xi , yi ), the measured 2D points, and (Xi ,Yi ,Zi ) are known 3Dlocations
This can be solve in a linear fashion for P. How?
Similar to the homography case.
This is called the Direct Linear Transform (DLT)
How many unknowns? how many correspondences?
11 or 12 unknowns, 6 correspondences
Are we done?
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 13 / 56
![Page 32: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/32.jpg)
Direct Linear Transform
Simplest to form a set of linear equations (analog to the 2D case)
xi =p00Xi + p01Yi + p02Zi + p03p20Xi + p21Yi + p22Yi + p23
yi =p10Xi + p11Yi + p12Zi + p13p20Xi + p21Yi + p22Yi + p23
with (xi , yi ), the measured 2D points, and (Xi ,Yi ,Zi ) are known 3Dlocations
This can be solve in a linear fashion for P. How?
Similar to the homography case.
This is called the Direct Linear Transform (DLT)
How many unknowns? how many correspondences?
11 or 12 unknowns, 6 correspondences
Are we done?
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 13 / 56
![Page 33: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/33.jpg)
Direct Linear Transform
Simplest to form a set of linear equations (analog to the 2D case)
xi =p00Xi + p01Yi + p02Zi + p03p20Xi + p21Yi + p22Yi + p23
yi =p10Xi + p11Yi + p12Zi + p13p20Xi + p21Yi + p22Yi + p23
with (xi , yi ), the measured 2D points, and (Xi ,Yi ,Zi ) are known 3Dlocations
This can be solve in a linear fashion for P. How?
Similar to the homography case.
This is called the Direct Linear Transform (DLT)
How many unknowns? how many correspondences?
11 or 12 unknowns, 6 correspondences
Are we done?
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 13 / 56
![Page 34: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/34.jpg)
Direct Linear Transform
Simplest to form a set of linear equations (analog to the 2D case)
xi =p00Xi + p01Yi + p02Zi + p03p20Xi + p21Yi + p22Yi + p23
yi =p10Xi + p11Yi + p12Zi + p13p20Xi + p21Yi + p22Yi + p23
with (xi , yi ), the measured 2D points, and (Xi ,Yi ,Zi ) are known 3Dlocations
This can be solve in a linear fashion for P. How?
Similar to the homography case.
This is called the Direct Linear Transform (DLT)
How many unknowns? how many correspondences?
11 or 12 unknowns, 6 correspondences
Are we done?
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 13 / 56
![Page 35: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/35.jpg)
Direct Linear Transform
Simplest to form a set of linear equations (analog to the 2D case)
xi =p00Xi + p01Yi + p02Zi + p03p20Xi + p21Yi + p22Yi + p23
yi =p10Xi + p11Yi + p12Zi + p13p20Xi + p21Yi + p22Yi + p23
with (xi , yi ), the measured 2D points, and (Xi ,Yi ,Zi ) are known 3Dlocations
This can be solve in a linear fashion for P. How?
Similar to the homography case.
This is called the Direct Linear Transform (DLT)
How many unknowns? how many correspondences?
11 or 12 unknowns, 6 correspondences
Are we done?
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 13 / 56
![Page 36: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/36.jpg)
Direct Linear Transform
Simplest to form a set of linear equations (analog to the 2D case)
xi =p00Xi + p01Yi + p02Zi + p03p20Xi + p21Yi + p22Yi + p23
yi =p10Xi + p11Yi + p12Zi + p13p20Xi + p21Yi + p22Yi + p23
with (xi , yi ), the measured 2D points, and (Xi ,Yi ,Zi ) are known 3Dlocations
This can be solve in a linear fashion for P. How?
Similar to the homography case.
This is called the Direct Linear Transform (DLT)
How many unknowns? how many correspondences?
11 or 12 unknowns, 6 correspondences
Are we done?
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 13 / 56
![Page 37: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/37.jpg)
Finding Camera Translation
Once P is recovered, how do I obtain the other matrices?
t is the null vector of matrix P as
Pt = 0
t is the unit singular vector of P corresponding to the smallest singular value
The last column of V, where
P = UDVT
is the SVD of P
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 14 / 56
![Page 38: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/38.jpg)
Finding Camera Translation
Once P is recovered, how do I obtain the other matrices?
t is the null vector of matrix P as
Pt = 0
t is the unit singular vector of P corresponding to the smallest singular value
The last column of V, where
P = UDVT
is the SVD of P
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 14 / 56
![Page 39: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/39.jpg)
Finding Camera Translation
Once P is recovered, how do I obtain the other matrices?
t is the null vector of matrix P as
Pt = 0
t is the unit singular vector of P corresponding to the smallest singular value
The last column of V, where
P = UDVT
is the SVD of P
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 14 / 56
![Page 40: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/40.jpg)
Finding Camera Translation
Once P is recovered, how do I obtain the other matrices?
t is the null vector of matrix P as
Pt = 0
t is the unit singular vector of P corresponding to the smallest singular value
The last column of V, where
P = UDVT
is the SVD of P
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 14 / 56
![Page 41: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/41.jpg)
Finding Camera Orientation and Internal Parameters
Left 3× 3 submatrix M of P is of form
M = KR
where K is an upper triangular matrix, and R is an orthogonal matrix
Any non-singular square matrix M can be decomposed into the product ofan upper-triangular matrix K and an orthogonal matrix R using the RQfactorization
Similar to QR factorization but order of 2 matrices is reversed
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 15 / 56
![Page 42: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/42.jpg)
Finding Camera Orientation and Internal Parameters
Left 3× 3 submatrix M of P is of form
M = KR
where K is an upper triangular matrix, and R is an orthogonal matrix
Any non-singular square matrix M can be decomposed into the product ofan upper-triangular matrix K and an orthogonal matrix R using the RQfactorization
Similar to QR factorization but order of 2 matrices is reversed
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 15 / 56
![Page 43: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/43.jpg)
Finding Camera Orientation and Internal Parameters
Left 3× 3 submatrix M of P is of form
M = KR
where K is an upper triangular matrix, and R is an orthogonal matrix
Any non-singular square matrix M can be decomposed into the product ofan upper-triangular matrix K and an orthogonal matrix R using the RQfactorization
Similar to QR factorization but order of 2 matrices is reversed
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 15 / 56
![Page 44: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/44.jpg)
RQ Factorization
Define the matrices
Rx =
1 0 00 c −s0 s c
,Ry =
c ′ 0 s ′
0 1 0−s ′ 0 c ′
,Rz =
c ′′ −s ′′ 0s ′′ c ′′ 00 0 1
We can compute
c = − M33
(M232 + M2
33)1/2s =
M32
(M232 + M2
33)1/2
Multiply M by Rx . The resulting term at (3, 2) is zero because of the valuesselected for c and s
Multiply the resulting matrix by Ry , after selecting c and s so that theresulting term at position (3, 1) is set to zero
Multiply the resulting matrix by Rz , after selecting c and s so that theresulting term at position (2, 1) is set to zero
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 16 / 56
![Page 45: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/45.jpg)
RQ Factorization
Define the matrices
Rx =
1 0 00 c −s0 s c
,Ry =
c ′ 0 s ′
0 1 0−s ′ 0 c ′
,Rz =
c ′′ −s ′′ 0s ′′ c ′′ 00 0 1
We can compute
c = − M33
(M232 + M2
33)1/2s =
M32
(M232 + M2
33)1/2
Multiply M by Rx . The resulting term at (3, 2) is zero because of the valuesselected for c and s
Multiply the resulting matrix by Ry , after selecting c and s so that theresulting term at position (3, 1) is set to zero
Multiply the resulting matrix by Rz , after selecting c and s so that theresulting term at position (2, 1) is set to zero
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 16 / 56
![Page 46: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/46.jpg)
RQ Factorization
Define the matrices
Rx =
1 0 00 c −s0 s c
,Ry =
c ′ 0 s ′
0 1 0−s ′ 0 c ′
,Rz =
c ′′ −s ′′ 0s ′′ c ′′ 00 0 1
We can compute
c = − M33
(M232 + M2
33)1/2s =
M32
(M232 + M2
33)1/2
Multiply M by Rx . The resulting term at (3, 2) is zero because of the valuesselected for c and s
Multiply the resulting matrix by Ry , after selecting c and s so that theresulting term at position (3, 1) is set to zero
Multiply the resulting matrix by Rz , after selecting c and s so that theresulting term at position (2, 1) is set to zero
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 16 / 56
![Page 47: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/47.jpg)
RQ Factorization
Define the matrices
Rx =
1 0 00 c −s0 s c
,Ry =
c ′ 0 s ′
0 1 0−s ′ 0 c ′
,Rz =
c ′′ −s ′′ 0s ′′ c ′′ 00 0 1
We can compute
c = − M33
(M232 + M2
33)1/2s =
M32
(M232 + M2
33)1/2
Multiply M by Rx . The resulting term at (3, 2) is zero because of the valuesselected for c and s
Multiply the resulting matrix by Ry , after selecting c and s so that theresulting term at position (3, 1) is set to zero
Multiply the resulting matrix by Rz , after selecting c and s so that theresulting term at position (2, 1) is set to zero
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 16 / 56
![Page 48: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/48.jpg)
RQ Factorization
Define the matrices
Rx =
1 0 00 c −s0 s c
,Ry =
c ′ 0 s ′
0 1 0−s ′ 0 c ′
,Rz =
c ′′ −s ′′ 0s ′′ c ′′ 00 0 1
We can compute
c = − M33
(M232 + M2
33)1/2s =
M32
(M232 + M2
33)1/2
Multiply M by Rx . The resulting term at (3, 2) is zero because of the valuesselected for c and s
Multiply the resulting matrix by Ry , after selecting c and s so that theresulting term at position (3, 1) is set to zero
Multiply the resulting matrix by Rz , after selecting c and s so that theresulting term at position (2, 1) is set to zero
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 16 / 56
![Page 49: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/49.jpg)
RQ Factorization
Why does this algorithm work?
MRxRyRz = K
Thus we haveM = KRT
z RTy RT
x = KR
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 17 / 56
![Page 50: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/50.jpg)
RQ Factorization
Why does this algorithm work?
MRxRyRz = K
Thus we haveM = KRT
z RTy RT
x = KR
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 17 / 56
![Page 51: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/51.jpg)
Improved computation of P
We have equations involving homgeneous coordinates
xi = PXi
Thus xi and Xi just have to be proportional
xi × PXi = 0
Let pT1 ,p
T2 ,p
T3 be the row vectors of P
PXi =
pT1 Xi
pT2 Xi
pT3 Xi
Therefore
xi × PXi =
vipT3 Xi − wipT
2 Xi
wipT3 Xi − uipT
2 Xi
uipT3 Xi − vipT
2 Xi
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 18 / 56
![Page 52: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/52.jpg)
Improved computation of P
We have equations involving homgeneous coordinates
xi = PXi
Thus xi and Xi just have to be proportional
xi × PXi = 0
Let pT1 ,p
T2 ,p
T3 be the row vectors of P
PXi =
pT1 Xi
pT2 Xi
pT3 Xi
Therefore
xi × PXi =
vipT3 Xi − wipT
2 Xi
wipT3 Xi − uipT
2 Xi
uipT3 Xi − vipT
2 Xi
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 18 / 56
![Page 53: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/53.jpg)
Improved computation of P
We have equations involving homgeneous coordinates
xi = PXi
Thus xi and Xi just have to be proportional
xi × PXi = 0
Let pT1 ,p
T2 ,p
T3 be the row vectors of P
PXi =
pT1 Xi
pT2 Xi
pT3 Xi
Therefore
xi × PXi =
vipT3 Xi − wipT
2 Xi
wipT3 Xi − uipT
2 Xi
uipT3 Xi − vipT
2 Xi
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 18 / 56
![Page 54: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/54.jpg)
Improved computation of P
We have equations involving homgeneous coordinates
xi = PXi
Thus xi and Xi just have to be proportional
xi × PXi = 0
Let pT1 ,p
T2 ,p
T3 be the row vectors of P
PXi =
pT1 Xi
pT2 Xi
pT3 Xi
Therefore
xi × PXi =
vipT3 Xi − wipT
2 Xi
wipT3 Xi − uipT
2 Xi
uipT3 Xi − vipT
2 Xi
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 18 / 56
![Page 55: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/55.jpg)
Improved computation of P
We have
xi × PXi =
vipT3 Xi − wipT
2 Xi
wipT3 Xi − uipT
2 Xi
uipT3 Xi − vipT
2 Xi
= 0
We can thus write 04 −wiXTi viXT
i
wiXTi 04 −uiXT
i
−viXTi uiXT
i 04
p1
p2
p3
= 0
Third row can be obtained from sum of ui times first row −vi times secondrow
2 independent equations in 11 unknowns (ignoring scale)
With 6 correspondences, we get enough equations to compute matrix P
Ap = 0
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 19 / 56
![Page 56: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/56.jpg)
Improved computation of P
We have
xi × PXi =
vipT3 Xi − wipT
2 Xi
wipT3 Xi − uipT
2 Xi
uipT3 Xi − vipT
2 Xi
= 0
We can thus write 04 −wiXTi viXT
i
wiXTi 04 −uiXT
i
−viXTi uiXT
i 04
p1
p2
p3
= 0
Third row can be obtained from sum of ui times first row −vi times secondrow
2 independent equations in 11 unknowns (ignoring scale)
With 6 correspondences, we get enough equations to compute matrix P
Ap = 0
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 19 / 56
![Page 57: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/57.jpg)
Improved computation of P
We have
xi × PXi =
vipT3 Xi − wipT
2 Xi
wipT3 Xi − uipT
2 Xi
uipT3 Xi − vipT
2 Xi
= 0
We can thus write 04 −wiXTi viXT
i
wiXTi 04 −uiXT
i
−viXTi uiXT
i 04
p1
p2
p3
= 0
Third row can be obtained from sum of ui times first row −vi times secondrow
2 independent equations in 11 unknowns (ignoring scale)
With 6 correspondences, we get enough equations to compute matrix P
Ap = 0
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 19 / 56
![Page 58: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/58.jpg)
Improved computation of P
We have
xi × PXi =
vipT3 Xi − wipT
2 Xi
wipT3 Xi − uipT
2 Xi
uipT3 Xi − vipT
2 Xi
= 0
We can thus write 04 −wiXTi viXT
i
wiXTi 04 −uiXT
i
−viXTi uiXT
i 04
p1
p2
p3
= 0
Third row can be obtained from sum of ui times first row −vi times secondrow
2 independent equations in 11 unknowns (ignoring scale)
With 6 correspondences, we get enough equations to compute matrix P
Ap = 0
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 19 / 56
![Page 59: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/59.jpg)
Improved computation of P
We have
xi × PXi =
vipT3 Xi − wipT
2 Xi
wipT3 Xi − uipT
2 Xi
uipT3 Xi − vipT
2 Xi
= 0
We can thus write 04 −wiXTi viXT
i
wiXTi 04 −uiXT
i
−viXTi uiXT
i 04
p1
p2
p3
= 0
Third row can be obtained from sum of ui times first row −vi times secondrow
2 independent equations in 11 unknowns (ignoring scale)
With 6 correspondences, we get enough equations to compute matrix P
Ap = 0
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 19 / 56
![Page 60: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/60.jpg)
Solving the system
Ap = 0
Minimize ||Ap|| with the constraint ||p|| = 1
P is the unit singular vector of A corresponding to the smallest singular value
The last column of V, where
A = UDVT
is the SVD of A
Called Direct Linear Transformation (DLT)
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 20 / 56
![Page 61: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/61.jpg)
Solving the system
Ap = 0
Minimize ||Ap|| with the constraint ||p|| = 1
P is the unit singular vector of A corresponding to the smallest singular value
The last column of V, where
A = UDVT
is the SVD of A
Called Direct Linear Transformation (DLT)
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 20 / 56
![Page 62: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/62.jpg)
Solving the system
Ap = 0
Minimize ||Ap|| with the constraint ||p|| = 1
P is the unit singular vector of A corresponding to the smallest singular value
The last column of V, where
A = UDVT
is the SVD of A
Called Direct Linear Transformation (DLT)
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 20 / 56
![Page 63: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/63.jpg)
Solving the system
Ap = 0
Minimize ||Ap|| with the constraint ||p|| = 1
P is the unit singular vector of A corresponding to the smallest singular value
The last column of V, where
A = UDVT
is the SVD of A
Called Direct Linear Transformation (DLT)
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 20 / 56
![Page 64: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/64.jpg)
Solving the system
Ap = 0
Minimize ||Ap|| with the constraint ||p|| = 1
P is the unit singular vector of A corresponding to the smallest singular value
The last column of V, where
A = UDVT
is the SVD of A
Called Direct Linear Transformation (DLT)
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 20 / 56
![Page 65: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/65.jpg)
Improving the P estimation
In most applications we have prior knowledge about some of the parametersof K, e.g., pixels are squared, skew is small, optical center near the center ofthe image
Use this constraints and frame the problem as a minimization
Find P using DLTUse as initialization for nonlinear minimization
∑i ρ(xi ,PXi ), with ρ a
robust estimatorUse Levenberg-Marquardt iterative minimization
See 6.2.2. in Szeliski’s book
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 21 / 56
![Page 66: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/66.jpg)
Improving the P estimation
In most applications we have prior knowledge about some of the parametersof K, e.g., pixels are squared, skew is small, optical center near the center ofthe image
Use this constraints and frame the problem as a minimization
Find P using DLT
Use as initialization for nonlinear minimization∑
i ρ(xi ,PXi ), with ρ arobust estimatorUse Levenberg-Marquardt iterative minimization
See 6.2.2. in Szeliski’s book
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 21 / 56
![Page 67: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/67.jpg)
Improving the P estimation
In most applications we have prior knowledge about some of the parametersof K, e.g., pixels are squared, skew is small, optical center near the center ofthe image
Use this constraints and frame the problem as a minimization
Find P using DLTUse as initialization for nonlinear minimization
∑i ρ(xi ,PXi ), with ρ a
robust estimator
Use Levenberg-Marquardt iterative minimization
See 6.2.2. in Szeliski’s book
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 21 / 56
![Page 68: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/68.jpg)
Improving the P estimation
In most applications we have prior knowledge about some of the parametersof K, e.g., pixels are squared, skew is small, optical center near the center ofthe image
Use this constraints and frame the problem as a minimization
Find P using DLTUse as initialization for nonlinear minimization
∑i ρ(xi ,PXi ), with ρ a
robust estimatorUse Levenberg-Marquardt iterative minimization
See 6.2.2. in Szeliski’s book
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 21 / 56
![Page 69: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/69.jpg)
Improving the P estimation
In most applications we have prior knowledge about some of the parametersof K, e.g., pixels are squared, skew is small, optical center near the center ofthe image
Use this constraints and frame the problem as a minimization
Find P using DLTUse as initialization for nonlinear minimization
∑i ρ(xi ,PXi ), with ρ a
robust estimatorUse Levenberg-Marquardt iterative minimization
See 6.2.2. in Szeliski’s book
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 21 / 56
![Page 70: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/70.jpg)
Improving the P estimation
In most applications we have prior knowledge about some of the parametersof K, e.g., pixels are squared, skew is small, optical center near the center ofthe image
Use this constraints and frame the problem as a minimization
Find P using DLTUse as initialization for nonlinear minimization
∑i ρ(xi ,PXi ), with ρ a
robust estimatorUse Levenberg-Marquardt iterative minimization
See 6.2.2. in Szeliski’s book
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 21 / 56
![Page 71: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/71.jpg)
Radial Distorsion
We have assumed that lines are imaged as lines
Significant error for cheap optics and for short focal lengths
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 22 / 56
![Page 72: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/72.jpg)
Radial Distorsion Correction
We can write the correction as
xc − x0 = L(r)(x − x0)
yc − y0 = L(r)(y − y0)
with L(r) = 1 + κ1r2 + κ2r
4 and r2 = (x − x0)2 + (y − y0)2
We thus minimize the following function
f (κ1, κ2) =∑i
(x ′i − xci )2 + (y ′i − yci )
2
using lines known to be straight, with (x ′, y ′) the radial projection of (x , y)on straight line
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 23 / 56
![Page 73: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/73.jpg)
Radial Distorsion Correction
We can write the correction as
xc − x0 = L(r)(x − x0)
yc − y0 = L(r)(y − y0)
with L(r) = 1 + κ1r2 + κ2r
4 and r2 = (x − x0)2 + (y − y0)2
We thus minimize the following function
f (κ1, κ2) =∑i
(x ′i − xci )2 + (y ′i − yci )
2
using lines known to be straight, with (x ′, y ′) the radial projection of (x , y)on straight line
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 23 / 56
![Page 74: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/74.jpg)
Let’s look at 3D alignment
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 24 / 56
![Page 75: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/75.jpg)
Motivation
[Source: W. Burgard]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 25 / 56
![Page 76: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/76.jpg)
3D Alignment
Before we were looking at aligning 2D points, or 2D to 3D points.
Now let’s do it in 3D
Given two corresponding point sets {xi , x′i}, in the case of rigid (Euclidean)motion we want R and t that minimizes
ER3D =∑i
||x′i − Rxi − t||22
This is call the absolute orientation problem
Is this easy to do?
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 26 / 56
![Page 77: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/77.jpg)
3D Alignment
Before we were looking at aligning 2D points, or 2D to 3D points.
Now let’s do it in 3D
Given two corresponding point sets {xi , x′i}, in the case of rigid (Euclidean)motion we want R and t that minimizes
ER3D =∑i
||x′i − Rxi − t||22
This is call the absolute orientation problem
Is this easy to do?
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 26 / 56
![Page 78: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/78.jpg)
3D Alignment
Before we were looking at aligning 2D points, or 2D to 3D points.
Now let’s do it in 3D
Given two corresponding point sets {xi , x′i}, in the case of rigid (Euclidean)motion we want R and t that minimizes
ER3D =∑i
||x′i − Rxi − t||22
This is call the absolute orientation problem
Is this easy to do?
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 26 / 56
![Page 79: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/79.jpg)
3D Alignment
Before we were looking at aligning 2D points, or 2D to 3D points.
Now let’s do it in 3D
Given two corresponding point sets {xi , x′i}, in the case of rigid (Euclidean)motion we want R and t that minimizes
ER3D =∑i
||x′i − Rxi − t||22
This is call the absolute orientation problem
Is this easy to do?
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 26 / 56
![Page 80: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/80.jpg)
3D Alignment
Before we were looking at aligning 2D points, or 2D to 3D points.
Now let’s do it in 3D
Given two corresponding point sets {xi , x′i}, in the case of rigid (Euclidean)motion we want R and t that minimizes
ER3D =∑i
||x′i − Rxi − t||22
This is call the absolute orientation problem
Is this easy to do?
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 26 / 56
![Page 81: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/81.jpg)
Key Idea
If the correct correspondences are known, the correct relativerotation/translation can be calculated in closed form
[Source: W. Burgard]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 27 / 56
![Page 82: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/82.jpg)
Center of Mass
The centroids of the two point clouds c and c ′ can be used to estimate thetranslation
µ =1
N
N∑i=1
xi µ′ =1
N
N∑i=1
x′i
Subtract the corresponding center of mass from every point in the two sets,
x̄i = xi − µx̄′i = x′i − µ′
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 28 / 56
![Page 83: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/83.jpg)
Center of Mass
The centroids of the two point clouds c and c ′ can be used to estimate thetranslation
µ =1
N
N∑i=1
xi µ′ =1
N
N∑i=1
x′i
Subtract the corresponding center of mass from every point in the two sets,
x̄i = xi − µx̄′i = x′i − µ′
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 28 / 56
![Page 84: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/84.jpg)
Solving for the Rotation
Let W =∑N
i=1 x̄′i x̄Ti
We can compute the SVD of W
W = USVT
If the rank is 3 we have a unique solution
R = UVT
t = µ− Rµ′
What if we don’t have correspondences?
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 29 / 56
![Page 85: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/85.jpg)
Solving for the Rotation
Let W =∑N
i=1 x̄′i x̄Ti
We can compute the SVD of W
W = USVT
If the rank is 3 we have a unique solution
R = UVT
t = µ− Rµ′
What if we don’t have correspondences?
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 29 / 56
![Page 86: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/86.jpg)
Solving for the Rotation
Let W =∑N
i=1 x̄′i x̄Ti
We can compute the SVD of W
W = USVT
If the rank is 3 we have a unique solution
R = UVT
t = µ− Rµ′
What if we don’t have correspondences?
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 29 / 56
![Page 87: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/87.jpg)
Solving for the Rotation
Let W =∑N
i=1 x̄′i x̄Ti
We can compute the SVD of W
W = USVT
If the rank is 3 we have a unique solution
R = UVT
t = µ− Rµ′
What if we don’t have correspondences?
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 29 / 56
![Page 88: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/88.jpg)
ICP
If the correspondences are not know, it is not possible to obtain the globalsolution
Iterated Closest Points (ICP) [Besl & McKay 92]
Iterative algorithm that alternates between solving for correspondences andsolving for (R, t)
Does it remained you of any algorithm you have seen before?
Converges to right solution if starting positions are ”close enough”
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 30 / 56
![Page 89: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/89.jpg)
ICP
If the correspondences are not know, it is not possible to obtain the globalsolution
Iterated Closest Points (ICP) [Besl & McKay 92]
Iterative algorithm that alternates between solving for correspondences andsolving for (R, t)
Does it remained you of any algorithm you have seen before?
Converges to right solution if starting positions are ”close enough”
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 30 / 56
![Page 90: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/90.jpg)
ICP
If the correspondences are not know, it is not possible to obtain the globalsolution
Iterated Closest Points (ICP) [Besl & McKay 92]
Iterative algorithm that alternates between solving for correspondences andsolving for (R, t)
Does it remained you of any algorithm you have seen before?
Converges to right solution if starting positions are ”close enough”
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 30 / 56
![Page 91: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/91.jpg)
ICP
If the correspondences are not know, it is not possible to obtain the globalsolution
Iterated Closest Points (ICP) [Besl & McKay 92]
Iterative algorithm that alternates between solving for correspondences andsolving for (R, t)
Does it remained you of any algorithm you have seen before?
Converges to right solution if starting positions are ”close enough”
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 30 / 56
![Page 92: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/92.jpg)
ICP
If the correspondences are not know, it is not possible to obtain the globalsolution
Iterated Closest Points (ICP) [Besl & McKay 92]
Iterative algorithm that alternates between solving for correspondences andsolving for (R, t)
Does it remained you of any algorithm you have seen before?
Converges to right solution if starting positions are ”close enough”
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 30 / 56
![Page 93: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/93.jpg)
ICP Variants
Point subsets (from one or both point sets)
Weighting the correspondences
Data association
Rejecting certain (outlier) point pairs
[Source: W. Burgard]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 31 / 56
![Page 94: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/94.jpg)
ICP Variants
Point subsets (from one or both point sets)
Weighting the correspondences
Data association
Rejecting certain (outlier) point pairs
[Source: W. Burgard]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 31 / 56
![Page 95: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/95.jpg)
ICP Variants
Point subsets (from one or both point sets)
Weighting the correspondences
Data association
Rejecting certain (outlier) point pairs
[Source: W. Burgard]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 31 / 56
![Page 96: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/96.jpg)
ICP Variants
Point subsets (from one or both point sets)
Weighting the correspondences
Data association
Rejecting certain (outlier) point pairs
[Source: W. Burgard]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 31 / 56
![Page 97: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/97.jpg)
Performance of Variants
Speed
Stability (local minima)
Tolerance wrt. noise and/or outliers
Basin of convergence (maximum initial misalignment)
[Source: W. Burgard]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 32 / 56
![Page 98: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/98.jpg)
Performance of Variants
Speed
Stability (local minima)
Tolerance wrt. noise and/or outliers
Basin of convergence (maximum initial misalignment)
[Source: W. Burgard]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 32 / 56
![Page 99: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/99.jpg)
Performance of Variants
Speed
Stability (local minima)
Tolerance wrt. noise and/or outliers
Basin of convergence (maximum initial misalignment)
[Source: W. Burgard]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 32 / 56
![Page 100: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/100.jpg)
Performance of Variants
Speed
Stability (local minima)
Tolerance wrt. noise and/or outliers
Basin of convergence (maximum initial misalignment)
[Source: W. Burgard]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 32 / 56
![Page 101: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/101.jpg)
ICP Variants
Point subsets (from one or both point sets)
Weighting the correspondences
Data association
Rejecting certain (outlier) point pairs
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 33 / 56
![Page 102: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/102.jpg)
ICP Variants
Use all points
Uniform sub-sampling
Random sampling
Normal-space sampling distributed as uniformly as possible
Feature based Sampling
[Source: W. Burgard]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 34 / 56
![Page 103: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/103.jpg)
ICP Variants
Use all points
Uniform sub-sampling
Random sampling
Normal-space sampling distributed as uniformly as possible
Feature based Sampling
[Source: W. Burgard]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 34 / 56
![Page 104: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/104.jpg)
ICP Variants
Use all points
Uniform sub-sampling
Random sampling
Normal-space sampling distributed as uniformly as possible
Feature based Sampling
[Source: W. Burgard]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 34 / 56
![Page 105: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/105.jpg)
ICP Variants
Use all points
Uniform sub-sampling
Random sampling
Normal-space sampling distributed as uniformly as possible
Feature based Sampling
[Source: W. Burgard]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 34 / 56
![Page 106: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/106.jpg)
ICP Variants
Use all points
Uniform sub-sampling
Random sampling
Normal-space sampling distributed as uniformly as possible
Feature based Sampling
[Source: W. Burgard]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 34 / 56
![Page 107: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/107.jpg)
Normal-based sampling
Ensure that samples have normals distributed as uniformly as possible
better for mostly-smooth areas with sparse features
[Source: W. Burgard]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 35 / 56
![Page 108: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/108.jpg)
Feature-based sampling
try to find important points
decrease the number of correspondences
higher efficiency and higher accuracy
requires preprocessing
[Source: W. Burgard]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 36 / 56
![Page 109: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/109.jpg)
ICP Variants
Point subsets (from one or both point sets)
Weighting the correspondences
Data association
Rejecting certain (outlier) point pairs
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 37 / 56
![Page 110: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/110.jpg)
Selection vs. Weighting
Could achieve same effect with weighting
Hard to guarantee that enough samples of important features except at highsampling rates
Weighting strategies turned out to be dependent on the data.
Preprocessing / run-time cost tradeoff (how to find the correct weights?)
[Source: W. Burgard]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 38 / 56
![Page 111: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/111.jpg)
Selection vs. Weighting
Could achieve same effect with weighting
Hard to guarantee that enough samples of important features except at highsampling rates
Weighting strategies turned out to be dependent on the data.
Preprocessing / run-time cost tradeoff (how to find the correct weights?)
[Source: W. Burgard]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 38 / 56
![Page 112: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/112.jpg)
Selection vs. Weighting
Could achieve same effect with weighting
Hard to guarantee that enough samples of important features except at highsampling rates
Weighting strategies turned out to be dependent on the data.
Preprocessing / run-time cost tradeoff (how to find the correct weights?)
[Source: W. Burgard]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 38 / 56
![Page 113: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/113.jpg)
Selection vs. Weighting
Could achieve same effect with weighting
Hard to guarantee that enough samples of important features except at highsampling rates
Weighting strategies turned out to be dependent on the data.
Preprocessing / run-time cost tradeoff (how to find the correct weights?)
[Source: W. Burgard]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 38 / 56
![Page 114: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/114.jpg)
ICP Variants
Point subsets (from one or both point sets)
Weighting the correspondences
Data association
Rejecting certain (outlier) point pairs
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 39 / 56
![Page 115: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/115.jpg)
Data association
Has greatest effect on convergence and speed
Closest point: stable, but slow and requires preprocessing
Normal shooting: slightly better than closest point for smooth structures,worse for noisy or complex structures
Point to plane: lets flat regions slide along each other
Projection: much faster but slightly worst alignments
Closest compatible point: normals, colors
[Source: W. Burgard]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 40 / 56
![Page 116: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/116.jpg)
Data association
Has greatest effect on convergence and speed
Closest point: stable, but slow and requires preprocessing
Normal shooting: slightly better than closest point for smooth structures,worse for noisy or complex structures
Point to plane: lets flat regions slide along each other
Projection: much faster but slightly worst alignments
Closest compatible point: normals, colors
[Source: W. Burgard]Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 40 / 56
![Page 117: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/117.jpg)
Data association
Has greatest effect on convergence and speed
Closest point: stable, but slow and requires preprocessing
Normal shooting: slightly better than closest point for smooth structures,worse for noisy or complex structures
Point to plane: lets flat regions slide along each other
Projection: much faster but slightly worst alignments
Closest compatible point: normals, colors
[Source: W. Burgard]Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 40 / 56
![Page 118: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/118.jpg)
Data association
Has greatest effect on convergence and speed
Closest point: stable, but slow and requires preprocessing
Normal shooting: slightly better than closest point for smooth structures,worse for noisy or complex structures
Point to plane: lets flat regions slide along each other
Projection: much faster but slightly worst alignments
Closest compatible point: normals, colors
[Source: W. Burgard]Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 40 / 56
![Page 119: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/119.jpg)
Data association
Has greatest effect on convergence and speed
Closest point: stable, but slow and requires preprocessing
Normal shooting: slightly better than closest point for smooth structures,worse for noisy or complex structures
Point to plane: lets flat regions slide along each other
Projection: much faster but slightly worst alignments
Closest compatible point: normals, colors
[Source: W. Burgard]Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 40 / 56
![Page 120: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/120.jpg)
ICP Variants
Point subsets (from one or both point sets)
Weighting the correspondences
Data association
Rejecting certain (outlier) point pairs, e.g., Trimmed ICP rejects a %
Where do these 3D points come from?
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 41 / 56
![Page 121: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/121.jpg)
Let’s look into stereo reconstruction
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 42 / 56
![Page 122: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/122.jpg)
Stereo
Public Library, Stereoscopic Looking Room, Chicago, by Phillips, 1923
[Source: N. Snavely]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 43 / 56
![Page 123: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/123.jpg)
Stereo
Stereo matching is the process of taking two or more images andestimating a 3D model of the scene by finding matching pixels in theimages and converting their 2D positions into 3D depths
We perceived depth based on the difference in appearance of the right andleft eye.
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 44 / 56
![Page 124: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/124.jpg)
Stereo
Stereo matching is the process of taking two or more images andestimating a 3D model of the scene by finding matching pixels in theimages and converting their 2D positions into 3D depths
We perceived depth based on the difference in appearance of the right andleft eye.
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 44 / 56
![Page 125: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/125.jpg)
Stereo
Given two images from different viewpoints
The depth is proportional to the inverse of the disparity
How can we compute the depth of each point in the image?
Based on how much each pixel moves between the two images
[Source: N. Snavely]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 45 / 56
![Page 126: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/126.jpg)
Stereo
Given two images from different viewpoints
The depth is proportional to the inverse of the disparity
How can we compute the depth of each point in the image?
Based on how much each pixel moves between the two images
[Source: N. Snavely]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 45 / 56
![Page 127: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/127.jpg)
Stereo
Given two images from different viewpoints
The depth is proportional to the inverse of the disparity
How can we compute the depth of each point in the image?
Based on how much each pixel moves between the two images
[Source: N. Snavely]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 45 / 56
![Page 128: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/128.jpg)
Epipolar Geometry
Pixel in one image x0 projects to an epipolar line segment in the otherimage
The segment is bounded at one end by the projection of the original viewingray at infinity p∞ and at the other end by the projection of the originalcamera center c0 into the second camera, which is known as the epipole e1.
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 46 / 56
![Page 129: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/129.jpg)
Epipolar Geometry
Pixel in one image x0 projects to an epipolar line segment in the otherimage
The segment is bounded at one end by the projection of the original viewingray at infinity p∞ and at the other end by the projection of the originalcamera center c0 into the second camera, which is known as the epipole e1.
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 46 / 56
![Page 130: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/130.jpg)
Epipolar Geometry
If we project the epipolar line in the second image back into the first, we getanother line (segment), this time bounded by the other correspondingepipole e0
Extending both line segments to infinity, we get a pair of correspondingepipolar lines, which are the intersection of the two image planes with theepipolar plane that passes through both camera centers c0 and c1 as wellas the point of interest p
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 47 / 56
![Page 131: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/131.jpg)
Epipolar Geometry
If we project the epipolar line in the second image back into the first, we getanother line (segment), this time bounded by the other correspondingepipole e0
Extending both line segments to infinity, we get a pair of correspondingepipolar lines, which are the intersection of the two image planes with theepipolar plane that passes through both camera centers c0 and c1 as wellas the point of interest p
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 47 / 56
![Page 132: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/132.jpg)
Rectification
The epipolar geometry depends on the relative pose and calibration of thecameras
This can be computed using the fundamental matrix
Once this is computed, we can use the epipolar lines to restrict the searchspace to a 1D search
Rectification, the process of transforming the image so that the search isalong horizontal line
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 48 / 56
![Page 133: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/133.jpg)
Rectification
The epipolar geometry depends on the relative pose and calibration of thecameras
This can be computed using the fundamental matrix
Once this is computed, we can use the epipolar lines to restrict the searchspace to a 1D search
Rectification, the process of transforming the image so that the search isalong horizontal line
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 48 / 56
![Page 134: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/134.jpg)
Rectification
The epipolar geometry depends on the relative pose and calibration of thecameras
This can be computed using the fundamental matrix
Once this is computed, we can use the epipolar lines to restrict the searchspace to a 1D search
Rectification, the process of transforming the image so that the search isalong horizontal line
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 48 / 56
![Page 135: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/135.jpg)
Rectification
The epipolar geometry depends on the relative pose and calibration of thecameras
This can be computed using the fundamental matrix
Once this is computed, we can use the epipolar lines to restrict the searchspace to a 1D search
Rectification, the process of transforming the image so that the search isalong horizontal line
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 48 / 56
![Page 136: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/136.jpg)
Epipolar Geometry
!"#"$%&'(%#)!*( !"#$%&#'% !"($%&#'%
!"#$%&'()*$+',-./)0$12$'$,./)32$4#/%5#6-'3$-/'6*3'76($+'&)/'$8'!+,-!.(*-)/)#$,'%/9$
The disparity for pixel (x1, y1) is (x2 − x1) if the images are rectified
This is a one dimensional search for each pixel
Very challenging to estimate the correspondences
[Source: N. Snavely]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 49 / 56
![Page 137: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/137.jpg)
Epipolar Geometry
!"#"$%&'(%#)!*( !"#$%&#'% !"($%&#'%
!"#$%&'()*$+',-./)0$12$'$,./)32$4#/%5#6-'3$-/'6*3'76($+'&)/'$8'!+,-!.(*-)/)#$,'%/9$
The disparity for pixel (x1, y1) is (x2 − x1) if the images are rectified
This is a one dimensional search for each pixel
Very challenging to estimate the correspondences
[Source: N. Snavely]Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 49 / 56
![Page 138: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/138.jpg)
Epipolar Geometry
!"#"$%&'(%#)!*( !"#$%&#'% !"($%&#'%
!"#$%&'()*$+',-./)0$12$'$,./)32$4#/%5#6-'3$-/'6*3'76($+'&)/'$8'!+,-!.(*-)/)#$,'%/9$
The disparity for pixel (x1, y1) is (x2 − x1) if the images are rectified
This is a one dimensional search for each pixel
Very challenging to estimate the correspondences
[Source: N. Snavely]Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 49 / 56
![Page 139: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/139.jpg)
Fundamental Matrix
Projective geometry depends only on the cameras internal parameters andrelative pose of cameras
Fundamental matrix F encapsulates this geometry
For any pair of points correspoding in both images
xT0 Fx1 = 0
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 50 / 56
![Page 140: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/140.jpg)
Fundamental Matrix
Projective geometry depends only on the cameras internal parameters andrelative pose of cameras
Fundamental matrix F encapsulates this geometry
For any pair of points correspoding in both images
xT0 Fx1 = 0
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 50 / 56
![Page 141: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/141.jpg)
Fundamental Matrix
Projective geometry depends only on the cameras internal parameters andrelative pose of cameras
Fundamental matrix F encapsulates this geometry
For any pair of points correspoding in both images
xT0 Fx1 = 0
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 50 / 56
![Page 142: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/142.jpg)
Epipolar Plane
[Source: Ramani]
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 51 / 56
![Page 143: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/143.jpg)
Pensils of Epipolar Lines
[Source: Ramani]Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 52 / 56
![Page 144: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/144.jpg)
Computation of Fundamental Matrix
F can be computed from correspondences between image points alone
No knowledge of camera internal parameters required
No knowledge of relative pose required
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 53 / 56
![Page 145: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/145.jpg)
Computation of Fundamental Matrix
F can be computed from correspondences between image points alone
No knowledge of camera internal parameters required
No knowledge of relative pose required
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 53 / 56
![Page 146: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/146.jpg)
Computation of Fundamental Matrix
F can be computed from correspondences between image points alone
No knowledge of camera internal parameters required
No knowledge of relative pose required
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 53 / 56
![Page 147: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/147.jpg)
Fundamental Matrix and Projective Geometry
Take x in camera P and find scene point X on ray of x in camera P
Find the image x′ of X in camera P′
Find epipole e′ as image of C in camera P′, e′ = P′C
Find epipolar line l′ from e′ to x′ in P′ as function of x
The fundamental matrix F is defined l′ = Fx
x′ belongs to l′, so x′T l′ = 0, so
x′T
Fx = 0
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 54 / 56
![Page 148: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/148.jpg)
Fundamental Matrix and Projective Geometry
Take x in camera P and find scene point X on ray of x in camera P
Find the image x′ of X in camera P′
Find epipole e′ as image of C in camera P′, e′ = P′C
Find epipolar line l′ from e′ to x′ in P′ as function of x
The fundamental matrix F is defined l′ = Fx
x′ belongs to l′, so x′T l′ = 0, so
x′T
Fx = 0
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 54 / 56
![Page 149: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/149.jpg)
Fundamental Matrix and Projective Geometry
Take x in camera P and find scene point X on ray of x in camera P
Find the image x′ of X in camera P′
Find epipole e′ as image of C in camera P′, e′ = P′C
Find epipolar line l′ from e′ to x′ in P′ as function of x
The fundamental matrix F is defined l′ = Fx
x′ belongs to l′, so x′T l′ = 0, so
x′T
Fx = 0
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 54 / 56
![Page 150: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/150.jpg)
Fundamental Matrix and Projective Geometry
Take x in camera P and find scene point X on ray of x in camera P
Find the image x′ of X in camera P′
Find epipole e′ as image of C in camera P′, e′ = P′C
Find epipolar line l′ from e′ to x′ in P′ as function of x
The fundamental matrix F is defined l′ = Fx
x′ belongs to l′, so x′T l′ = 0, so
x′T
Fx = 0
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 54 / 56
![Page 151: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/151.jpg)
Fundamental Matrix and Projective Geometry
Take x in camera P and find scene point X on ray of x in camera P
Find the image x′ of X in camera P′
Find epipole e′ as image of C in camera P′, e′ = P′C
Find epipolar line l′ from e′ to x′ in P′ as function of x
The fundamental matrix F is defined l′ = Fx
x′ belongs to l′, so x′T l′ = 0, so
x′T
Fx = 0
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 54 / 56
![Page 152: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/152.jpg)
Fundamental Matrix and Projective Geometry
Take x in camera P and find scene point X on ray of x in camera P
Find the image x′ of X in camera P′
Find epipole e′ as image of C in camera P′, e′ = P′C
Find epipolar line l′ from e′ to x′ in P′ as function of x
The fundamental matrix F is defined l′ = Fx
x′ belongs to l′, so x′T l′ = 0, so
x′T
Fx = 0
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 54 / 56
![Page 153: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/153.jpg)
Finding the Fundamental Matrix from known Projections
Take x in camera P and find one scene point on ray from C to x
Point X = P+x satisfies x = PX with P+ = PT (PPT )−1 soPX = PPT (PPT )−1x = x
Image of this point in camera P′ is x′ = P′X = P′P+x
Image of C in camera P′ is epipole e′ = P′C
Epipolar line of x in P′ is
l′ = (e′)× (P′P+x)
l′ = Fx defines the fundamental matrix
F = (P′C)× (P′P+)
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 55 / 56
![Page 154: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/154.jpg)
Finding the Fundamental Matrix from known Projections
Take x in camera P and find one scene point on ray from C to x
Point X = P+x satisfies x = PX with P+ = PT (PPT )−1 soPX = PPT (PPT )−1x = x
Image of this point in camera P′ is x′ = P′X = P′P+x
Image of C in camera P′ is epipole e′ = P′C
Epipolar line of x in P′ is
l′ = (e′)× (P′P+x)
l′ = Fx defines the fundamental matrix
F = (P′C)× (P′P+)
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 55 / 56
![Page 155: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/155.jpg)
Finding the Fundamental Matrix from known Projections
Take x in camera P and find one scene point on ray from C to x
Point X = P+x satisfies x = PX with P+ = PT (PPT )−1 soPX = PPT (PPT )−1x = x
Image of this point in camera P′ is x′ = P′X = P′P+x
Image of C in camera P′ is epipole e′ = P′C
Epipolar line of x in P′ is
l′ = (e′)× (P′P+x)
l′ = Fx defines the fundamental matrix
F = (P′C)× (P′P+)
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 55 / 56
![Page 156: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/156.jpg)
Finding the Fundamental Matrix from known Projections
Take x in camera P and find one scene point on ray from C to x
Point X = P+x satisfies x = PX with P+ = PT (PPT )−1 soPX = PPT (PPT )−1x = x
Image of this point in camera P′ is x′ = P′X = P′P+x
Image of C in camera P′ is epipole e′ = P′C
Epipolar line of x in P′ is
l′ = (e′)× (P′P+x)
l′ = Fx defines the fundamental matrix
F = (P′C)× (P′P+)
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 55 / 56
![Page 157: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/157.jpg)
Finding the Fundamental Matrix from known Projections
Take x in camera P and find one scene point on ray from C to x
Point X = P+x satisfies x = PX with P+ = PT (PPT )−1 soPX = PPT (PPT )−1x = x
Image of this point in camera P′ is x′ = P′X = P′P+x
Image of C in camera P′ is epipole e′ = P′C
Epipolar line of x in P′ is
l′ = (e′)× (P′P+x)
l′ = Fx defines the fundamental matrix
F = (P′C)× (P′P+)Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 55 / 56
![Page 158: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/158.jpg)
Finding the Fundamental Matrix from known Projections
Take x in camera P and find one scene point on ray from C to x
Point X = P+x satisfies x = PX with P+ = PT (PPT )−1 soPX = PPT (PPT )−1x = x
Image of this point in camera P′ is x′ = P′X = P′P+x
Image of C in camera P′ is epipole e′ = P′C
Epipolar line of x in P′ is
l′ = (e′)× (P′P+x)
l′ = Fx defines the fundamental matrix
F = (P′C)× (P′P+)Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 55 / 56
![Page 159: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/159.jpg)
Properties of the fundamental matrix
Matrix 3× 3 since x′TFx = 0
Let F be the fundamental matrix of camera pair (P,P′), the fundamentalmatrix of camera pair (P′,P) is F′ = FT
This is true since xTF′x′ = 0 implies x′TF′Tx = 0, so F′ = FT
Epipolar line of x is l′ = Fx
Epipolar line of x′ is l = FTx′
Epipole e′ is left null space of F, and e is right null space.
All epipolar lines l′ contains epipole e′, so e′T l′ = 0, i.e. e′TFx = 0 for all x,therefore e′TF = 0
F is of rank 2 because F = e′ × (P′P+) and e′× is of rank 2
F has 7 degrees of freedom, there are 9 elements, but scaling is notimportant and det(F) = 0 removes one degree of freedom
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 56 / 56
![Page 160: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/160.jpg)
Properties of the fundamental matrix
Matrix 3× 3 since x′TFx = 0
Let F be the fundamental matrix of camera pair (P,P′), the fundamentalmatrix of camera pair (P′,P) is F′ = FT
This is true since xTF′x′ = 0 implies x′TF′Tx = 0, so F′ = FT
Epipolar line of x is l′ = Fx
Epipolar line of x′ is l = FTx′
Epipole e′ is left null space of F, and e is right null space.
All epipolar lines l′ contains epipole e′, so e′T l′ = 0, i.e. e′TFx = 0 for all x,therefore e′TF = 0
F is of rank 2 because F = e′ × (P′P+) and e′× is of rank 2
F has 7 degrees of freedom, there are 9 elements, but scaling is notimportant and det(F) = 0 removes one degree of freedom
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 56 / 56
![Page 161: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/161.jpg)
Properties of the fundamental matrix
Matrix 3× 3 since x′TFx = 0
Let F be the fundamental matrix of camera pair (P,P′), the fundamentalmatrix of camera pair (P′,P) is F′ = FT
This is true since xTF′x′ = 0 implies x′TF′Tx = 0, so F′ = FT
Epipolar line of x is l′ = Fx
Epipolar line of x′ is l = FTx′
Epipole e′ is left null space of F, and e is right null space.
All epipolar lines l′ contains epipole e′, so e′T l′ = 0, i.e. e′TFx = 0 for all x,therefore e′TF = 0
F is of rank 2 because F = e′ × (P′P+) and e′× is of rank 2
F has 7 degrees of freedom, there are 9 elements, but scaling is notimportant and det(F) = 0 removes one degree of freedom
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 56 / 56
![Page 162: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/162.jpg)
Properties of the fundamental matrix
Matrix 3× 3 since x′TFx = 0
Let F be the fundamental matrix of camera pair (P,P′), the fundamentalmatrix of camera pair (P′,P) is F′ = FT
This is true since xTF′x′ = 0 implies x′TF′Tx = 0, so F′ = FT
Epipolar line of x is l′ = Fx
Epipolar line of x′ is l = FTx′
Epipole e′ is left null space of F, and e is right null space.
All epipolar lines l′ contains epipole e′, so e′T l′ = 0, i.e. e′TFx = 0 for all x,therefore e′TF = 0
F is of rank 2 because F = e′ × (P′P+) and e′× is of rank 2
F has 7 degrees of freedom, there are 9 elements, but scaling is notimportant and det(F) = 0 removes one degree of freedom
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 56 / 56
![Page 163: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/163.jpg)
Properties of the fundamental matrix
Matrix 3× 3 since x′TFx = 0
Let F be the fundamental matrix of camera pair (P,P′), the fundamentalmatrix of camera pair (P′,P) is F′ = FT
This is true since xTF′x′ = 0 implies x′TF′Tx = 0, so F′ = FT
Epipolar line of x is l′ = Fx
Epipolar line of x′ is l = FTx′
Epipole e′ is left null space of F, and e is right null space.
All epipolar lines l′ contains epipole e′, so e′T l′ = 0, i.e. e′TFx = 0 for all x,therefore e′TF = 0
F is of rank 2 because F = e′ × (P′P+) and e′× is of rank 2
F has 7 degrees of freedom, there are 9 elements, but scaling is notimportant and det(F) = 0 removes one degree of freedom
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 56 / 56
![Page 164: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/164.jpg)
Properties of the fundamental matrix
Matrix 3× 3 since x′TFx = 0
Let F be the fundamental matrix of camera pair (P,P′), the fundamentalmatrix of camera pair (P′,P) is F′ = FT
This is true since xTF′x′ = 0 implies x′TF′Tx = 0, so F′ = FT
Epipolar line of x is l′ = Fx
Epipolar line of x′ is l = FTx′
Epipole e′ is left null space of F, and e is right null space.
All epipolar lines l′ contains epipole e′, so e′T l′ = 0, i.e. e′TFx = 0 for all x,therefore e′TF = 0
F is of rank 2 because F = e′ × (P′P+) and e′× is of rank 2
F has 7 degrees of freedom, there are 9 elements, but scaling is notimportant and det(F) = 0 removes one degree of freedom
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 56 / 56
![Page 165: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/165.jpg)
Properties of the fundamental matrix
Matrix 3× 3 since x′TFx = 0
Let F be the fundamental matrix of camera pair (P,P′), the fundamentalmatrix of camera pair (P′,P) is F′ = FT
This is true since xTF′x′ = 0 implies x′TF′Tx = 0, so F′ = FT
Epipolar line of x is l′ = Fx
Epipolar line of x′ is l = FTx′
Epipole e′ is left null space of F, and e is right null space.
All epipolar lines l′ contains epipole e′, so e′T l′ = 0, i.e. e′TFx = 0 for all x,therefore e′TF = 0
F is of rank 2 because F = e′ × (P′P+) and e′× is of rank 2
F has 7 degrees of freedom, there are 9 elements, but scaling is notimportant and det(F) = 0 removes one degree of freedom
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 56 / 56
![Page 166: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/166.jpg)
Properties of the fundamental matrix
Matrix 3× 3 since x′TFx = 0
Let F be the fundamental matrix of camera pair (P,P′), the fundamentalmatrix of camera pair (P′,P) is F′ = FT
This is true since xTF′x′ = 0 implies x′TF′Tx = 0, so F′ = FT
Epipolar line of x is l′ = Fx
Epipolar line of x′ is l = FTx′
Epipole e′ is left null space of F, and e is right null space.
All epipolar lines l′ contains epipole e′, so e′T l′ = 0, i.e. e′TFx = 0 for all x,therefore e′TF = 0
F is of rank 2 because F = e′ × (P′P+) and e′× is of rank 2
F has 7 degrees of freedom, there are 9 elements, but scaling is notimportant and det(F) = 0 removes one degree of freedom
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 56 / 56
![Page 167: Computer Vision: Calibration and Reconstructionurtasun/courses/CV/lecture09.pdf · Computer Vision: Calibration and Reconstruction Raquel Urtasun TTI Chicago Feb 7, 2013 Raquel Urtasun](https://reader031.vdocument.in/reader031/viewer/2022022806/5ccad40888c993fa708c0a6b/html5/thumbnails/167.jpg)
Properties of the fundamental matrix
Matrix 3× 3 since x′TFx = 0
Let F be the fundamental matrix of camera pair (P,P′), the fundamentalmatrix of camera pair (P′,P) is F′ = FT
This is true since xTF′x′ = 0 implies x′TF′Tx = 0, so F′ = FT
Epipolar line of x is l′ = Fx
Epipolar line of x′ is l = FTx′
Epipole e′ is left null space of F, and e is right null space.
All epipolar lines l′ contains epipole e′, so e′T l′ = 0, i.e. e′TFx = 0 for all x,therefore e′TF = 0
F is of rank 2 because F = e′ × (P′P+) and e′× is of rank 2
F has 7 degrees of freedom, there are 9 elements, but scaling is notimportant and det(F) = 0 removes one degree of freedom
Raquel Urtasun (TTI-C) Computer Vision Feb 7, 2013 56 / 56