miniature faking - college of computinghays/compvision/lectures/09.pdfg review: ransac algorithm: n...
TRANSCRIPT
![Page 1: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/1.jpg)
Miniature faking
http://en.wikipedia.org/wiki/File:Jodhpur_tilt_shift.jpg
In close-up photo, the depth of field is limited.
![Page 2: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/2.jpg)
Miniature faking
![Page 3: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/3.jpg)
Miniature faking
http://en.wikipedia.org/wiki/File:Oregon_State_Beavers_Tilt-Shift_Miniature_Greg_Keene.jpg
![Page 4: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/4.jpg)
Review
• Previous section:
– Model fitting and outlier rejection
![Page 5: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/5.jpg)
x
y
b
m
x
y m3 5 3 3 2 2
3 7 11 10 4 3
2 3 1 4 5 2
2 1 0 1 3 3
bSlide from S. Savarese
Review: Hough transform
![Page 6: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/6.jpg)
Review: RANSAC
14INAlgorithm:
1. Sample (randomly) the number of points required to fit the model (#=2)
2. Solve for model parameters using samples
3. Score by the fraction of inliers within a preset threshold of the model
Repeat 1-3 until the best model is found with high confidence
![Page 7: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/7.jpg)
Review: 2D image transformations
Szeliski 2.1
![Page 8: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/8.jpg)
This section – multiple views
• Today – Intro to multiple views and Stereo. Camera Calibration (if we have time).
• Next Lecture – Epipolar Geometry and Fundamental Matrix. Stereo Matching (if there is time).
• Both lectures are the core of what you need for project 3.
![Page 9: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/9.jpg)
Recap: Oriented and Translated Camera
Ow
iw
kw
jw
t
R
![Page 10: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/10.jpg)
Recap: Degrees of freedom
XtRKx
1100
0
1 333231
232221
131211
0
0
z
y
x
trrr
trrr
trrr
v
us
v
u
w
z
y
x
5 6
![Page 11: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/11.jpg)
This Lecture: How to calibrate the camera?
1****
****
****
Z
Y
X
s
sv
su
XtRKx
![Page 12: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/12.jpg)
What can we do with camera calibration?
![Page 13: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/13.jpg)
![Page 14: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/14.jpg)
Computer Vision
James Hays
Stereo:
Intro
Slides by Kristen Grauman
![Page 15: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/15.jpg)
Multiple views
Hartley and Zisserman
Lowe
stereo vision
structure from motion
optical flow
![Page 16: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/16.jpg)
Why multiple views?
• Structure and depth are inherently ambiguous from
single views.
Images from Lana Lazebnik
![Page 17: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/17.jpg)
![Page 18: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/18.jpg)
Why multiple views?
• Structure and depth are inherently ambiguous from
single views.
Optical center
P1
P2
P1’=P2’
![Page 19: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/19.jpg)
• What cues help us to perceive 3d shape and depth?
![Page 20: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/20.jpg)
Shading
[Figure from Prados & Faugeras 2006]
![Page 21: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/21.jpg)
Focus/defocus
[figs from H. Jin and P. Favaro, 2002]
Images from
same point of
view, different
camera
parameters
3d shape / depth
estimates
![Page 22: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/22.jpg)
Texture
[From A.M. Loh. The recovery of 3-D structure using visual texture patterns. PhD thesis]
![Page 23: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/23.jpg)
Perspective effects
Image credit: S. Seitz
![Page 24: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/24.jpg)
Motion
Figures from L. Zhang http://www.brainconnection.com/teasers/?main=illusion/motion-shape
![Page 25: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/25.jpg)
Occlusion
Rene Magritt'e famous
painting Le Blanc-
Seing (literal translation:
"The Blank Signature")
roughly translates as
"free hand“. 1965
![Page 26: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/26.jpg)
If stereo were critical for depth
perception, navigation, recognition,
etc., then this would be a problem
![Page 27: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/27.jpg)
Multi-view geometry problems
• Structure: Given projections of the same 3D point in two
or more images, compute the 3D coordinates of that point
Camera 3
R3,t3 Slide credit:
Noah Snavely
?
Camera 1Camera 2
R1,t1 R2,t2
![Page 28: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/28.jpg)
Multi-view geometry problems
• Stereo correspondence: Given a point in one of the
images, where could its corresponding points be in the
other images?
Camera 3
R3,t3
Camera 1Camera 2
R1,t1 R2,t2Slide credit:
Noah Snavely
![Page 29: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/29.jpg)
Multi-view geometry problems
• Motion: Given a set of corresponding points in two or
more images, compute the camera parameters
Camera 1Camera 2 Camera 3
R1,t1 R2,t2R3,t3
? ? ? Slide credit:
Noah Snavely
![Page 30: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/30.jpg)
Human eye
Fig from Shapiro and Stockman
Pupil/Iris – control
amount of light
passing through lens
Retina - contains
sensor cells, where
image is formed
Fovea – highest
concentration of
cones
Rough analogy with human visual system:
![Page 31: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/31.jpg)
Human stereopsis: disparity
Human eyes fixate on point in space – rotate so that
corresponding images form in centers of fovea.
![Page 32: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/32.jpg)
Stereo photography and stereo viewers
Invented by Sir Charles Wheatstone, 1838Image from fisher-price.com
Take two pictures of the same subject from two slightly
different viewpoints and display so that each eye sees
only one of the images.
![Page 33: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/33.jpg)
http://www.johnsonshawmuseum.org
![Page 34: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/34.jpg)
http://www.johnsonshawmuseum.org
![Page 35: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/35.jpg)
Public Library, Stereoscopic Looking Room, Chicago, by Phillips, 1923
![Page 36: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/36.jpg)
http://www.well.com/~jimg/stereo/stereo_list.html
![Page 37: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/37.jpg)
http://www.well.com/~jimg/stereo/stereo_list.html
![Page 38: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/38.jpg)
Autostereograms
Images from magiceye.com
Exploit disparity as
depth cue using single
image.
(Single image random
dot stereogram, Single
image stereogram)
![Page 39: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/39.jpg)
Images from magiceye.com
Autostereograms
![Page 40: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/40.jpg)
Estimating depth with stereo
• Stereo: shape from “motion” between two views
• We’ll need to consider:
• Info on camera pose (“calibration”)
• Image point correspondences
scene point
optical
center
image plane
![Page 41: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/41.jpg)
Two cameras, simultaneous
views
Single moving camera and
static scene
Stereo vision
![Page 42: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/42.jpg)
Camera parameters
Camera
frame 1
Intrinsic parameters:
Image coordinates relative to
camera Pixel coordinates
Extrinsic parameters:
Camera frame 1 Camera frame 2
Camera
frame 2
• Extrinsic params: rotation matrix and translation vector
• Intrinsic params: focal length, pixel sizes (mm), image center
point, radial distortion parameters
We’ll assume for now that these parameters are
given and fixed.
![Page 43: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/43.jpg)
Geometry for a simple stereo system
• First, assuming parallel optical axes, known camera
parameters (i.e., calibrated cameras):
![Page 44: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/44.jpg)
baseline
optical
center
(left)
optical
center
(right)
Focal
length
World
point
image point
(left)
image point
(right)
Depth of p
![Page 45: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/45.jpg)
• Assume parallel optical axes, known camera parameters
(i.e., calibrated cameras). What is expression for Z?
Similar triangles (pl, P, pr) and
(Ol, P, Or):
Geometry for a simple stereo system
Z
T
fZ
xxT rl
disparitylr xx
TfZ
lr xx
TfZ
lr xx
TfZ
lr xx
TfZ
-
![Page 46: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/46.jpg)
Depth from disparity
image I(x,y) image I´(x´,y´)Disparity map D(x,y)
(x´,y´)=(x+D(x,y), y)
So if we could find the corresponding points in two images,
we could estimate relative depth…
![Page 47: Miniature faking - College of Computinghays/compvision/lectures/09.pdfG Review: RANSAC Algorithm: N I 14 1. Sample (randomly) the number of points required to fit the model (#=2) 2](https://reader033.vdocument.in/reader033/viewer/2022052805/60585caed468521c8313b0e2/html5/thumbnails/47.jpg)
Where do we need to search?
Intermission…