freeman ran sac 2010
Post on 07-Apr-2018
221 Views
Preview:
TRANSCRIPT
-
8/6/2019 Freeman Ran Sac 2010
1/165
Homographies and RANSAC
Computer vision 6.869
Bill Freeman and Antonio Torralba
March 30, 2011
Wednesday, March 30, 2011
-
8/6/2019 Freeman Ran Sac 2010
2/165
Homographies and RANSAC
Homographies
RANSAC
Building panoramas
Phototourism
2
Wednesday, March 30, 2011
-
8/6/2019 Freeman Ran Sac 2010
3/165
Depth-based ambiguity of position
Camera A Camera B
Wednesday, March 30, 2011
-
8/6/2019 Freeman Ran Sac 2010
4/165
Under what conditions can you know where
to translate each point of image A to where
it would appear in camera B (with calibrated
cameras), knowing nothing about image
depths?
4
Camera A Camera B
Wednesday, March 30, 2011
-
8/6/2019 Freeman Ran Sac 2010
5/165
(a) camera rotation
5
Wednesday, March 30, 2011
-
8/6/2019 Freeman Ran Sac 2010
6/165
and (b) imaging a planar surface
6
Wednesday, March 30, 2011
-
8/6/2019 Freeman Ran Sac 2010
7/165
Geometry of perspective projection
7
pinhole
sensor plane
Wednesday, March 30, 2011
-
8/6/2019 Freeman Ran Sac 2010
8/165
Geometry of perspective projection
7
pinhole
inverted copy
of sensor plane
Wednesday, March 30, 2011
-
8/6/2019 Freeman Ran Sac 2010
9/165
Geometry of perspective projection
7
pinhole
inverted copy
of sensor plane
Lets look at this scene from above...
Wednesday, March 30, 2011
-
8/6/2019 Freeman Ran Sac 2010
10/165
Two cameras with same center of projection
common pinhole
position of the
cameras
Wednesday, March 30, 2011
-
8/6/2019 Freeman Ran Sac 2010
11/165
Two cameras with same center of projection
camera A
common pinhole
position of the
cameras
Wednesday, March 30, 2011
-
8/6/2019 Freeman Ran Sac 2010
12/165
Two cameras with same center of projection
camera A camera B
common pinhole
position of the
cameras
Wednesday, March 30, 2011
-
8/6/2019 Freeman Ran Sac 2010
13/165
Two cameras with same center of projection
camera A camera B
Can generate any synthetic camera view
as long as it has the same center of projection!
common pinhole
position of the
cameras
Wednesday, March 30, 2011
-
8/6/2019 Freeman Ran Sac 2010
14/165
camera A camera B
camera A center
Two cameras with offset centers of projection
Wednesday, March 30, 2011
-
8/6/2019 Freeman Ran Sac 2010
15/165
camera A camera B
camera A center
camera B center
Two cameras with offset centers of projection
Wednesday, March 30, 2011
-
8/6/2019 Freeman Ran Sac 2010
16/165
camera A camera B
camera A center
camera B center
Two cameras with offset centers of projection
Wednesday, March 30, 2011
-
8/6/2019 Freeman Ran Sac 2010
17/165
camera A camera B
camera A center
camera B center
Two cameras with offset centers of projection
Wednesday, March 30, 2011
-
8/6/2019 Freeman Ran Sac 2010
18/165
Recap
When we only rotate the camera (around nodal
point) depth does not matter It only performs a 2D warp
one-to-one mapping of the 2D plane
plus of course reveals stuff that was outside the fieldof view
Now we just need to figure out this mapping
A B
Wednesday, March 30, 2011
-
8/6/2019 Freeman Ran Sac 2010
19/165
Aligning images: translation?
Wednesday, March 30, 2011
Ali i i l i ?
-
8/6/2019 Freeman Ran Sac 2010
20/165
Aligning images: translation?
left on top right on top
Wednesday, March 30, 2011
Ali i i t l ti ?
-
8/6/2019 Freeman Ran Sac 2010
21/165
Aligning images: translation?
Translations are not enough to align the images
left on top right on top
Wednesday, March 30, 2011
h h
-
8/6/2019 Freeman Ran Sac 2010
22/165
homography
Wednesday, March 30, 2011
homograph
-
8/6/2019 Freeman Ran Sac 2010
23/165
homography
Wednesday, March 30, 2011
homography
-
8/6/2019 Freeman Ran Sac 2010
24/165
homography
Wednesday, March 30, 2011
homography
-
8/6/2019 Freeman Ran Sac 2010
25/165
homography
How many pairs of points does it take to specify M_10?
Wednesday, March 30, 2011
Homography
-
8/6/2019 Freeman Ran Sac 2010
26/165
Homography
PP2
PP1
S
eeSzeliskiSect2
.1.5,
Mappingfro
mone
c
ameratoanother.
Wednesday, March 30, 2011
Homography
-
8/6/2019 Freeman Ran Sac 2010
27/165
Homography Projective mapping between any two projection
planes with the same center of projection called Homography
represented as 3x3 matrix in homogenous
coordinates
PP2
PP1H pp
S
eeSzeliskiSect2
.1.5,
Mappingfromone
c
ameratoanother.
Wednesday, March 30, 2011
Homography
-
8/6/2019 Freeman Ran Sac 2010
28/165
Homography Projective mapping between any two projection
planes with the same center of projection called Homography
represented as 3x3 matrix in homogenous
coordinates
PP2
PP1H pp
To apply a homography H Compute p = Hp (regular matrix multiply)
Convert p from homogeneous to image
coordinates (divide by w)S
eeSzeliskiSect2.1.5,
Mappingfromone
c
ameratoanother.
Wednesday, March 30, 2011
-
8/6/2019 Freeman Ran Sac 2010
29/165
Images of planar
objects, taken bygenerically offsetcameras, are alsorelated by ahomography.
FromS
zeliskibo
ok
camera A
camera B
Wednesday, March 30, 2011
Measurements on planes
-
8/6/2019 Freeman Ran Sac 2010
30/165
CSE 576, Spring 2008 Projective Geometry 6
Measurements on planes
Wednesday, March 30, 2011
Measurements on planes
-
8/6/2019 Freeman Ran Sac 2010
31/165
CSE 576, Spring 2008 Projective Geometry 6
Measurements on planes
Wednesday, March 30, 2011
Measurements on planes
-
8/6/2019 Freeman Ran Sac 2010
32/165
CSE 576, Spring 2008 Projective Geometry 6
Measurements on planes
Approach: unwarp then measure
Wednesday, March 30, 2011
Measurements on planes
-
8/6/2019 Freeman Ran Sac 2010
33/165
CSE 576, Spring 2008 Projective Geometry 6
Measurements on planes
Approach: unwarp then measure
Wednesday, March 30, 2011
Measurements on planes
-
8/6/2019 Freeman Ran Sac 2010
34/165
CSE 576, Spring 2008 Projective Geometry 6
1 2 3 4
1
2
3
4
Measurements on planes
Approach: unwarp then measure
Wednesday, March 30, 2011
Measurements on planes
-
8/6/2019 Freeman Ran Sac 2010
35/165
CSE 576, Spring 2008 Projective Geometry 6
1 2 3 4
1
2
3
4
Measurements on planes
Approach: unwarp then measure
Wednesday, March 30, 2011
Measurements on planes
-
8/6/2019 Freeman Ran Sac 2010
36/165
CSE 576, Spring 2008 Projective Geometry 6
1 2 3 4
1
2
3
4
Measurements on planes
Approach: unwarp then measure
Wednesday, March 30, 2011
Measurements on planes
-
8/6/2019 Freeman Ran Sac 2010
37/165
CSE 576, Spring 2008 Projective Geometry 6
1 2 3 4
1
2
3
4
Measurements on planes
Approach: unwarp then measure
How to unwarp?
Wednesday, March 30, 2011
Image rectification
-
8/6/2019 Freeman Ran Sac 2010
38/165
CSE 576, Spring 2008 Projective Geometry 7
Image rectification
To unwarp (rectify) an image solve for homography H given p and p
solve equations of the form: wp = Hp
linear in unknowns: w and coefficients ofH
H is defined up to an arbitrary scale factor
how many points are necessary to solve forH?
p p
Wednesday, March 30, 2011
Solving for homographies
-
8/6/2019 Freeman Ran Sac 2010
39/165
CSE 576, Spring 2008 Projective Geometry 8
g g p
Wednesday, March 30, 2011
Solving for homographies
-
8/6/2019 Freeman Ran Sac 2010
40/165
CSE 576, Spring 2008 Projective Geometry 8
g g p
Wednesday, March 30, 2011
Solving for homographies
-
8/6/2019 Freeman Ran Sac 2010
41/165
CSE 576, Spring 2008 Projective Geometry 8
g g p
Wednesday, March 30, 2011
Solving for homographies
-
8/6/2019 Freeman Ran Sac 2010
42/165
CSE 576, Spring 2008 Projective Geometry 8
g g p
Wednesday, March 30, 2011
Solving for homographies
-
8/6/2019 Freeman Ran Sac 2010
43/165
CSE 576, Spring 2008 Projective Geometry 9
g g
A h 0
Wednesday, March 30, 2011
Solving for homographies
-
8/6/2019 Freeman Ran Sac 2010
44/165
CSE 576, Spring 2008 Projective Geometry 9
A h 02n 9 9 2n
Wednesday, March 30, 2011
Solving for homographies
-
8/6/2019 Freeman Ran Sac 2010
45/165
CSE 576, Spring 2008 Projective Geometry 9
A h 0
Defines a least squares problem:
2n 9 9 2n
Wednesday, March 30, 2011
Solving for homographies
-
8/6/2019 Freeman Ran Sac 2010
46/165
CSE 576, Spring 2008 Projective Geometry 9
A h 0
Defines a least squares problem:
2n 9 9 2n
Since h is only defined up to scale, solve for unit vector
Wednesday, March 30, 2011
Solving for homographies
-
8/6/2019 Freeman Ran Sac 2010
47/165
CSE 576, Spring 2008 Projective Geometry 9
A h 0
Defines a least squares problem:
2n 9 9 2n
Since h is only defined up to scale, solve for unit vector
Solution: = eigenvector ofATA with smallest eigenvalue Works with 4 or more points
Wednesday, March 30, 2011
Image warping with homographies
-
8/6/2019 Freeman Ran Sac 2010
48/165
image plane below
homography so
that image isparallel to floor
homography sothat image is
parallel to rightwall
Wednesday, March 30, 2011
Image warping with homographies
-
8/6/2019 Freeman Ran Sac 2010
49/165
image plane below
black areawhere no pixelmaps to
homography so
that image isparallel to floor
homography sothat image is
parallel to rightwall
Wednesday, March 30, 2011
Image warping with homographies
-
8/6/2019 Freeman Ran Sac 2010
50/165
image plane below
black areawhere no pixelmaps to
homography so
that image isparallel to floor
homography sothat image is
parallel to rightwall
Wednesday, March 30, 2011
automatic image mosaicing
-
8/6/2019 Freeman Ran Sac 2010
51/165
Wednesday, March 30, 2011
automatic image mosaicing
-
8/6/2019 Freeman Ran Sac 2010
52/165
Basic Procedure
Take a sequence of images from the same position. Rotate the camera about its optical center (entrance pupil).
Robustly compute the homography transformation
between second image and first.
Transform (warp) the second image to overlap with first.
Blend the two together to create a mosaic.
If there are more images, repeat.
Wednesday, March 30, 2011
Robust feature matching through
-
8/6/2019 Freeman Ran Sac 2010
53/165
RANSAC
15-463: Computational Photography
Alexei Efros, CMU, Fall 2005
with a lot of slides stolen from
Steve Seitz and Rick Szeliski
Krister ParmstrandNikon D70. Stitched Panorama. The sky has been retouched. No other image manipulation.
Wednesday, March 30, 2011
Feature matching
-
8/6/2019 Freeman Ran Sac 2010
54/165
?
descriptors for left image feature points descriptors for right image feature points
Wednesday, March 30, 2011
Strategies to match images robustly
-
8/6/2019 Freeman Ran Sac 2010
55/165
(a) Working with individual features: For each feature
point, find most similar point in other image (SIFTdistance)
Reject ambiguous matches where there are too many similar points
(b) Working with all the features: Given some good featurematches, look for possible homographies relating the twoimages
Reject homographies that dont have many feature matches.
25
Wednesday, March 30, 2011
(a) Feature-space outlier rejection
-
8/6/2019 Freeman Ran Sac 2010
56/165
Lets not match all features, but only these that
have similar enough matches?
How can we do it?
SSD(patch1,patch2) < threshold
How to set threshold?Not so easy.
Wednesday, March 30, 2011
Feature-space outlier rejection
-
8/6/2019 Freeman Ran Sac 2010
57/165
A better way [Lowe, 1999]:
1-NN: SSD of the closest match
2-NN: SSD of the second-closest match
Look at how much better 1-NN is than 2-NN, e.g. 1-NN/2-NN
That is, is our best match so much better than the rest?
Wednesday, March 30, 2011
Feature-space outlier rejection
-
8/6/2019 Freeman Ran Sac 2010
58/165
A better way [Lowe, 1999]:
1-NN: SSD of the closest match
2-NN: SSD of the second-closest match
Look at how much better 1-NN is than 2-NN, e.g. 1-NN/2-NN
That is, is our best match so much better than the rest?
Wednesday, March 30, 2011
Feature-space outlier rejection
-
8/6/2019 Freeman Ran Sac 2010
59/165
Wednesday, March 30, 2011
Feature-space outlier rejection
-
8/6/2019 Freeman Ran Sac 2010
60/165
Can we now compute H from the bluepoints?
Wednesday, March 30, 2011
Feature-space outlier rejection
-
8/6/2019 Freeman Ran Sac 2010
61/165
Can we now compute H from the bluepoints?
No! Still too many outliers
What can we do?
Wednesday, March 30, 2011
(b) Matching many features--looking fora good homography
-
8/6/2019 Freeman Ran Sac 2010
62/165
What do we do about the bad matches?
Note: at this point we dont know which ones are good/bad
Simplified illustration with translation instead of homography
Wednesday, March 30, 2011
RA d SA l C
-
8/6/2019 Freeman Ran Sac 2010
63/165
RAndom SAmple Consensus
Select one match, count inliers
Wednesday, March 30, 2011
RAndom SAmple Consensus
-
8/6/2019 Freeman Ran Sac 2010
64/165
RAndom SAmple Consensus
Select one match, count inliers
0 inliers
Wednesday, March 30, 2011
RAndom SAmple Consensus
-
8/6/2019 Freeman Ran Sac 2010
65/165
RAndom SAmple Consensus
Select one match, count inliers
4 inliers
Wednesday, March 30, 2011
RAndom SAmple Consensus
-
8/6/2019 Freeman Ran Sac 2010
66/165
RAndom SAmple Consensus
Select one match, count inliersSelect one match, count inliers
Keep match with largest set of inliers
Wednesday, March 30, 2011
At the end: Least squares fit
-
8/6/2019 Freeman Ran Sac 2010
67/165
At the end: Least squares fit
Find average translation vector,but with only inliers
Wednesday, March 30, 2011
At the end: Least squares fit
-
8/6/2019 Freeman Ran Sac 2010
68/165
At the end: Least squares fit
Find average translation vector,but with only inliers
Wednesday, March 30, 2011
Reference
-
8/6/2019 Freeman Ran Sac 2010
69/165
M. A. Fischler, R. C.Bolles. Random SampleConsensus: A Paradigm forModel Fitting with
Applications to ImageAnalysis and AutomatedCartography. Comm. of theACM, Vol 24, pp 381-395,1981.
http://portal.acm.org/citation.cfm?id=358692
35
Wednesday, March 30, 2011
RANSAC for estimating homography
http://portal.acm.org/citation.cfm?id=358692http://portal.acm.org/citation.cfm?id=358692http://portal.acm.org/citation.cfm?id=358692http://portal.acm.org/citation.cfm?id=358692http://portal.acm.org/citation.cfm?id=358692http://portal.acm.org/citation.cfm?id=358692 -
8/6/2019 Freeman Ran Sac 2010
70/165
RANSAC for estimating homography
Wednesday, March 30, 2011
RANSAC for estimating homography
-
8/6/2019 Freeman Ran Sac 2010
71/165
RANSAC for estimating homography
RANSAC loop:
Wednesday, March 30, 2011
RANSAC for estimating homography
-
8/6/2019 Freeman Ran Sac 2010
72/165
RANSAC for estimating homography
RANSAC loop:
Select four feature pairs (at random)
Wednesday, March 30, 2011
RANSAC for estimating homography
-
8/6/2019 Freeman Ran Sac 2010
73/165
RANSAC for estimating homography
RANSAC loop:
Select four feature pairs (at random)
Compute homography H (exact)
Wednesday, March 30, 2011
RANSAC for estimating homography
-
8/6/2019 Freeman Ran Sac 2010
74/165
RANSAC for estimating homography
RANSAC loop:
Select four feature pairs (at random)
Compute homography H (exact)
Compute inliers where ||pi, Hpi|| <
Wednesday, March 30, 2011
RANSAC for estimating homography
-
8/6/2019 Freeman Ran Sac 2010
75/165
RANSAC for estimating homography
RANSAC loop:
Select four feature pairs (at random)
Compute homography H (exact)
Compute inliers where ||pi, Hpi|| <
Wednesday, March 30, 2011
RANSAC for estimating homography
-
8/6/2019 Freeman Ran Sac 2010
76/165
RANSAC for estimating homography
RANSAC loop:
Select four feature pairs (at random)
Compute homography H (exact)
Compute inliers where ||pi, Hpi|| <
Keep largest set of inliers
Wednesday, March 30, 2011
RANSAC for estimating homography
-
8/6/2019 Freeman Ran Sac 2010
77/165
RANSAC for estimating homography
RANSAC loop:
Select four feature pairs (at random)
Compute homography H (exact)
Compute inliers where ||pi, Hpi|| <
Keep largest set of inliers
Re-compute least-squares H estimate using all ofthe inliers
Wednesday, March 30, 2011
Simple example: fit a line
-
8/6/2019 Freeman Ran Sac 2010
78/165
p p
Rather than homography H (8 numbers)
fit y=ax+b (2 numbers a, b) to 2D pairs
37
Wednesday, March 30, 2011
Simple example: fit a line
-
8/6/2019 Freeman Ran Sac 2010
79/165
p p
38
Wednesday, March 30, 2011
Simple example: fit a line
-
8/6/2019 Freeman Ran Sac 2010
80/165
p p
Pick 2 points
38
Wednesday, March 30, 2011
Simple example: fit a line
-
8/6/2019 Freeman Ran Sac 2010
81/165
p p
Pick 2 points
Fit line
38
Wednesday, March 30, 2011
Simple example: fit a line
-
8/6/2019 Freeman Ran Sac 2010
82/165
p p
Pick 2 points
Fit line
Count inliers
38
3 inlier
Wednesday, March 30, 2011
Simple example: fit a line
-
8/6/2019 Freeman Ran Sac 2010
83/165
p p
39
Wednesday, March 30, 2011
Simple example: fit a line
-
8/6/2019 Freeman Ran Sac 2010
84/165
Pick 2 points
39
Wednesday, March 30, 2011
-
8/6/2019 Freeman Ran Sac 2010
85/165
Simple example: fit a line
-
8/6/2019 Freeman Ran Sac 2010
86/165
Pick 2 points
Fit line
Count inliers
39
4 inlier
Wednesday, March 30, 2011
Simple example: fit a line
-
8/6/2019 Freeman Ran Sac 2010
87/165
40
Wednesday, March 30, 2011
Simple example: fit a line
-
8/6/2019 Freeman Ran Sac 2010
88/165
Pick 2 points
40
Wednesday, March 30, 2011
Simple example: fit a line
-
8/6/2019 Freeman Ran Sac 2010
89/165
Pick 2 points
Fit line
40
Wednesday, March 30, 2011
Simple example: fit a line
-
8/6/2019 Freeman Ran Sac 2010
90/165
Pick 2 points
Fit line
Count inliers
40
9 inlier
Wednesday, March 30, 2011
Simple example: fit a line
-
8/6/2019 Freeman Ran Sac 2010
91/165
41
Wednesday, March 30, 2011
Simple example: fit a line
-
8/6/2019 Freeman Ran Sac 2010
92/165
Pick 2 points
41
Wednesday, March 30, 2011
Simple example: fit a line
-
8/6/2019 Freeman Ran Sac 2010
93/165
Pick 2 points
Fit line
41
Wednesday, March 30, 2011
Simple example: fit a line
-
8/6/2019 Freeman Ran Sac 2010
94/165
Pick 2 points
Fit line
Count inliers
41
8 inlier
Wednesday, March 30, 2011
Simple example: fit a line
-
8/6/2019 Freeman Ran Sac 2010
95/165
42
Wednesday, March 30, 2011
Simple example: fit a line
-
8/6/2019 Freeman Ran Sac 2010
96/165
Use biggest set of inliers
42
Wednesday, March 30, 2011
Simple example: fit a line
-
8/6/2019 Freeman Ran Sac 2010
97/165
Use biggest set of inliers
Do least-square fit
42
Wednesday, March 30, 2011
RANSAC
-
8/6/2019 Freeman Ran Sac 2010
98/165
red:rejected by 2nd nearestneighbor criterion
blue:Ransac outliers
yellow:inliers
Wednesday, March 30, 2011
Robustness
-
8/6/2019 Freeman Ran Sac 2010
99/165
44
Wednesday, March 30, 2011
Robustness
-
8/6/2019 Freeman Ran Sac 2010
100/165
Proportion of inliers in our pairs is G (for
good)
Our model needs P pairs
P=4 for homography
44
Wednesday, March 30, 2011
Robustness
-
8/6/2019 Freeman Ran Sac 2010
101/165
Proportion of inliers in our pairs is G (for
good)
Our model needs P pairs
P=4 for homography
Probability that we pick P inliers?
44
Wednesday, March 30, 2011
Robustness
-
8/6/2019 Freeman Ran Sac 2010
102/165
Proportion of inliers in our pairs is G (for
good)
Our model needs P pairs
P=4 for homography
Probability that we pick P inliers?
GP
44
Wednesday, March 30, 2011
Robustness
-
8/6/2019 Freeman Ran Sac 2010
103/165
Proportion of inliers in our pairs is G (for
good)
Our model needs P pairs
P=4 for homography
Probability that we pick P inliers?
GP
Probability that after N RANSAC iterations
we have not picked a set of inliers?44
Wednesday, March 30, 2011
-
8/6/2019 Freeman Ran Sac 2010
104/165
Robustness: example Matlab: p=4; x=0.5; n=1000; (1-x^p)^n
-
8/6/2019 Freeman Ran Sac 2010
105/165
Proportion of inliers G=0.5
Probability that we pick P=4 inliers?
0.54=0.0625 (6% chance)
Probability that we have not picked a set of
inliers?
N=100 iterations:
(1-0.54)100=0.00157 (1 chance in 600)
N=1000 iterations:
1 chance in 1e28 45
p ; ; ; ( p)
Wednesday, March 30, 2011
Robustness: example
-
8/6/2019 Freeman Ran Sac 2010
106/165
Proportion of inliers G=0.3
Probability that we pick P=4 inliers?
0.34=0.0081 (0.8% chance)
Probability that we have not picked a set of
inliers?
N=100 iterations:
(1-0.34)100=0.44 (1 chance in 2)
N=1000 iterations:
1 chance in 3400 46
Wednesday, March 30, 2011
Robustness: example
-
8/6/2019 Freeman Ran Sac 2010
107/165
Proportion of inliers G=0.1
Probability that we pick P=4 inliers?
0.14=0.0001 (0.01% chances, 1 in 10,000)
Probability that we have not picked a set of
inliers?
N=100 iterations: (1-0.14)100=0.99
N=1000 iterations: 90%
N=10,000: 36%
N=100,000: 1 in 22,00047
Wednesday, March 30, 2011
Robustness: conclusions
-
8/6/2019 Freeman Ran Sac 2010
108/165
48
Wednesday, March 30, 2011
Robustness: conclusions
-
8/6/2019 Freeman Ran Sac 2010
109/165
Effect of number of parameters of model/
number of necessary pairs
48
Wednesday, March 30, 2011
Robustness: conclusions
-
8/6/2019 Freeman Ran Sac 2010
110/165
Effect of number of parameters of model/
number of necessary pairs
Bad exponential
48
Wednesday, March 30, 2011
Robustness: conclusions
-
8/6/2019 Freeman Ran Sac 2010
111/165
Effect of number of parameters of model/
number of necessary pairs
Bad exponential
Effect of percentage of inliers
48
Wednesday, March 30, 2011
Robustness: conclusions
-
8/6/2019 Freeman Ran Sac 2010
112/165
Effect of number of parameters of model/
number of necessary pairs
Bad exponential
Effect of percentage of inliers
Base of the exponential
48
Wednesday, March 30, 2011
Robustness: conclusions
-
8/6/2019 Freeman Ran Sac 2010
113/165
Effect of number of parameters of model/
number of necessary pairs
Bad exponential
Effect of percentage of inliers
Base of the exponential
Effect of number of iterations
48
Wednesday, March 30, 2011
Robustness: conclusions
-
8/6/2019 Freeman Ran Sac 2010
114/165
Effect of number of parameters of model/
number of necessary pairs
Bad exponential
Effect of percentage of inliers
Base of the exponential
Effect of number of iterations
Good exponential
48
Wednesday, March 30, 2011
RANSAC recap
-
8/6/2019 Freeman Ran Sac 2010
115/165
For fitting a model with low number P of
parameters (8 for homographies)
Loop
Select P random data points
Fit model
Count inliers
(other data points well fit by this model) Keep model with largest number of inliers
49
Wednesday, March 30, 2011
Example: Recognising
-
8/6/2019 Freeman Ran Sac 2010
116/165
Panoramas
M. Brown and D. Lowe,University of British Columbia
* M. Brown and D. Lowe. Automatic Panoramic Image Stitching using Invariant
Features. International Journal of Computer Vision, 74(1), pages 59-73, 2007 (pdf
3.5Mb | bib)
* M. Brown and D. G. Lowe. Recognising Panoramas. In Proceedings of the 9th
International Conference on Computer Vision (ICCV2003), pages 1218-1225, Nice,
France, 2003 (pdf 820kb | ppt | bib)
Wednesday, March 30, 2011
Recognising Panoramas?
-
8/6/2019 Freeman Ran Sac 2010
117/165
Wednesday, March 30, 2011
RANSAC for Homography
-
8/6/2019 Freeman Ran Sac 2010
118/165
Wednesday, March 30, 2011
RANSAC for Homography
-
8/6/2019 Freeman Ran Sac 2010
119/165
Wednesday, March 30, 2011
-
8/6/2019 Freeman Ran Sac 2010
120/165
Probabilistic model for verification
-
8/6/2019 Freeman Ran Sac 2010
121/165
Wednesday, March 30, 2011
-
8/6/2019 Freeman Ran Sac 2010
122/165
Finding the panoramas
-
8/6/2019 Freeman Ran Sac 2010
123/165
Wednesday, March 30, 2011
Finding the panoramas
-
8/6/2019 Freeman Ran Sac 2010
124/165
Wednesday, March 30, 2011
Finding the panoramas
-
8/6/2019 Freeman Ran Sac 2010
125/165
Wednesday, March 30, 2011
Results
-
8/6/2019 Freeman Ran Sac 2010
126/165
Wednesday, March 30, 2011
-
8/6/2019 Freeman Ran Sac 2010
127/165
61
Wednesday, March 30, 2011
Benefits of Laplacian image compositing
-
8/6/2019 Freeman Ran Sac 2010
128/165
62M. Brown and D. Lowe. Automatic Panoramic Image Stitching using Invariant
Features. International Journal of Computer Vision, 74(1), pages 59-73, 2007
Wednesday, March 30, 2011
Photo Tourism:
-
8/6/2019 Freeman Ran Sac 2010
129/165
2006 Noah Snavely
Photo Tourism:Exploring Photo Collections in 3D
Noah Snavely
Steven M. SeitzUniversity of Washington
Richard SzeliskiMicrosoft Research
2006 Noah Snavely
Wednesday, March 30, 2011
-
8/6/2019 Freeman Ran Sac 2010
130/165
2006 Noah Snavely
Wednesday, March 30, 2011
-
8/6/2019 Freeman Ran Sac 2010
131/165
2006 Noah Snavely
Wednesday, March 30, 2011
-
8/6/2019 Freeman Ran Sac 2010
132/165
2006 Noah Snavely
15,464
Wednesday, March 30, 2011
-
8/6/2019 Freeman Ran Sac 2010
133/165
2006 Noah Snavely
15,464
37,383
Wednesday, March 30, 2011
-
8/6/2019 Freeman Ran Sac 2010
134/165
2006 Noah Snavely
15,464
76,389
37,383
Wednesday, March 30, 2011
Movie
-
8/6/2019 Freeman Ran Sac 2010
135/165
2006 Noah Snavely
Wednesday, March 30, 2011
Photo Tourism overview
-
8/6/2019 Freeman Ran Sac 2010
136/165
2006 Noah Snavely
Wednesday, March 30, 2011
Photo Tourism overview
-
8/6/2019 Freeman Ran Sac 2010
137/165
2006 Noah Snavely
Input photographs
Wednesday, March 30, 2011
Photo Tourism overview
-
8/6/2019 Freeman Ran Sac 2010
138/165
2006 Noah Snavely
Scene
reconstruction
Input photographs
Wednesday, March 30, 2011
Photo Tourism overview
-
8/6/2019 Freeman Ran Sac 2010
139/165
2006 Noah Snavely
Scene
reconstruction
Input photographsRelative camerapositions and orientations
Point cloud
Sparse correspondence
Wednesday, March 30, 2011
Photo Tourism overview
-
8/6/2019 Freeman Ran Sac 2010
140/165
2006 Noah Snavely
Scene
reconstruction PhotoExplorer
Input photographsRelative camerapositions and orientations
Point cloud
Sparse correspondence
Wednesday, March 30, 2011
Photo Tourism overview
-
8/6/2019 Freeman Ran Sac 2010
141/165
2006 Noah Snavely
Scene
reconstructionPhoto
ExplorerInput photographs
Wednesday, March 30, 2011
Photo Tourism overview
-
8/6/2019 Freeman Ran Sac 2010
142/165
2006 Noah Snavely
Scene
reconstructionPhoto
ExplorerInput photographs
Wednesday, March 30, 2011
Scene reconstruction
-
8/6/2019 Freeman Ran Sac 2010
143/165
2006 Noah Snavely
Wednesday, March 30, 2011
Scene reconstruction
Automatically estimate
position, orientation, and focal length of cameras
3D positions of feature points
-
8/6/2019 Freeman Ran Sac 2010
144/165
2006 Noah Snavely
Wednesday, March 30, 2011
Scene reconstruction
Automatically estimate
position, orientation, and focal length of cameras
3D positions of feature points
-
8/6/2019 Freeman Ran Sac 2010
145/165
2006 Noah Snavely
Feature detection
Wednesday, March 30, 2011
Scene reconstruction
Automatically estimate
position, orientation, and focal length of cameras
3D positions of feature points
-
8/6/2019 Freeman Ran Sac 2010
146/165
2006 Noah Snavely
Feature detection
Pairwisefeature matching
Wednesday, March 30, 2011
Scene reconstruction
Automatically estimate
position, orientation, and focal length of cameras
3D positions of feature points
-
8/6/2019 Freeman Ran Sac 2010
147/165
2006 Noah Snavely
Feature detection
Pairwisefeature matching
Correspondenceestimation
Wednesday, March 30, 2011
Scene reconstruction
Automatically estimate
position, orientation, and focal length of cameras
3D positions of feature points
-
8/6/2019 Freeman Ran Sac 2010
148/165
2006 Noah Snavely
Feature detection
Pairwisefeature matching
Incrementalstructure
from motion
Correspondenceestimation
Wednesday, March 30, 2011
Feature detection
Detect features using SIFT [Lowe, IJCV 2004]
-
8/6/2019 Freeman Ran Sac 2010
149/165
2006 Noah Snavely
Wednesday, March 30, 2011
Feature detection
Detect features using SIFT [Lowe, IJCV 2004]
-
8/6/2019 Freeman Ran Sac 2010
150/165
2006 Noah Snavely
Wednesday March 30 2011
Feature detection
Detect features using SIFT [Lowe, IJCV 2004]
-
8/6/2019 Freeman Ran Sac 2010
151/165
2006 Noah Snavely
Wednesday March 30 2011
Feature detection
Detect features using SIFT [Lowe, IJCV 2004]
-
8/6/2019 Freeman Ran Sac 2010
152/165
2006 Noah Snavely
Wednesday March 30 2011
Feature detection
Detect features using SIFT [Lowe, IJCV 2004]
-
8/6/2019 Freeman Ran Sac 2010
153/165
2006 Noah Snavely
Wednesday March 30 2011
Feature matching
Match features between each pair of images
-
8/6/2019 Freeman Ran Sac 2010
154/165
2006 Noah Snavely
Wednesday March 30 2011
Feature matching
Match features between each pair of images
-
8/6/2019 Freeman Ran Sac 2010
155/165
2006 Noah Snavely
Wednesday March 30 2011
Feature matchingRefine matching using RANSAC [Fischler & Bolles 1987]
to estimate fundamental matrices between pairs(See 6.801/6.866 for fundamental matrix, or Hartley and Zisserman, Multi-View
Geometry.
See also the fundamental matrix song: http://danielwedge.com/fmatrix/ )
http://danielwedge.com/fmatrix/http://danielwedge.com/fmatrix/ -
8/6/2019 Freeman Ran Sac 2010
156/165
2006 Noah Snavely
Wednesday March 30 2011
Feature matchingRefine matching using RANSAC [Fischler & Bolles 1987]
to estimate fundamental matrices between pairs(See 6.801/6.866 for fundamental matrix, or Hartley and Zisserman, Multi-View
Geometry.
See also the fundamental matrix song: http://danielwedge.com/fmatrix/ )
http://danielwedge.com/fmatrix/http://danielwedge.com/fmatrix/http://danielwedge.com/fmatrix/ -
8/6/2019 Freeman Ran Sac 2010
157/165
2006 Noah Snavely
Wednesday March 30 2011
http://danielwedge.com/fmatrix/ -
8/6/2019 Freeman Ran Sac 2010
158/165
Structure from motion
-
8/6/2019 Freeman Ran Sac 2010
159/165
2006 Noah Snavely
Camera 1
Camera 2
Camera 3
Wednesday, March 30, 2011
Structure from motion
-
8/6/2019 Freeman Ran Sac 2010
160/165
2006 Noah Snavely
Camera 1
Camera 2
Camera 3
Wednesday, March 30, 2011
Structure from motion
p1
p4
p3p2
-
8/6/2019 Freeman Ran Sac 2010
161/165
2006 Noah Snavely
Camera 1
Camera 2
Camera 3
p5
p6
p7
Wednesday, March 30, 2011
Structure from motion
p1
p4
p3p2
-
8/6/2019 Freeman Ran Sac 2010
162/165
2006 Noah Snavely
Camera 1
Camera 2
Camera 3
R1,t1R2,t2
R3,t3
p5
p6
p7
Wednesday, March 30, 2011
Structure from motion
p1
p4
p3p2
-
8/6/2019 Freeman Ran Sac 2010
163/165
2006 Noah Snavely
Camera 1
Camera 2
Camera 3
R1,t1R2,t2
R3,t3
p5
p6
p7
Wednesday, March 30, 2011
Structure from motion
p1
p4
p3p2
minimizef(R,T,P)
-
8/6/2019 Freeman Ran Sac 2010
164/165
2006 Noah Snavely
Camera 1
Camera 2
Camera 3
R1,t1R2,t2
R3,t3
p5
p6
p7
( , , )
Wednesday, March 30, 2011
Links
Code available: http://phototour.cs.washington.edu/bundler/
http://phototour.cs.washington.edu/ http://livelabs.com/photosynth/
http://livelabs.com/photosynth/http://livelabs.com/photosynth/http://phototour.cs.washington.edu/http://phototour.cs.washington.edu/http://phototour.cs.washington.edu/bundler/http://phototour.cs.washington.edu/bundler/ -
8/6/2019 Freeman Ran Sac 2010
165/165
2006 Noah Snavely
http://www.cs.cornell.edu/~snavely/
Wednesday, March 30, 2011
http://www.cs.cornell.edu/~snavely/http://www.cs.cornell.edu/~snavely/http://livelabs.com/photosynth/
top related