twenty five years of ransac: mlesac a new cost function for ransac philip torr oxford brookes code...
Post on 22-Dec-2015
222 views
TRANSCRIPT
![Page 1: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/1.jpg)
Twenty Five Years of RANSAC:MLESAC a new cost function for
RANSAC
Philip TorrPhilip Torr
Oxford BrookesOxford Brookes
Code AvailableCode Available
http://cms.brookes.ac.uk/staff/PhilipTorr/http://cms.brookes.ac.uk/staff/PhilipTorr/
[email protected]@brookes.ac.uk
![Page 2: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/2.jpg)
Robust estimation
What if set of matches contains gross outliers?What if set of matches contains gross outliers?
![Page 3: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/3.jpg)
Previous work
Suprisingly, at the time, algorithmic work in Suprisingly, at the time, algorithmic work in statistics quite weak.statistics quite weak.
– L1 norms.L1 norms.
– M-estimators.M-estimators.
![Page 4: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/4.jpg)
Quiz, L1 norm
Is the L1 norm robust, in the sense that it is Is the L1 norm robust, in the sense that it is resistant to outliers?resistant to outliers?
– A) for one dimension?A) for one dimension?
– B) for greater than one dimension?B) for greater than one dimension?
![Page 5: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/5.jpg)
Quiz Solutions
Contrary to widely held belief L1 norm is not robust Contrary to widely held belief L1 norm is not robust for dimension greater than 1for dimension greater than 1
Dimension 1 it leads to the median so has some Dimension 1 it leads to the median so has some robustness.robustness.
Dimension 2+ L1 norm is as vulnerable to outliers as Dimension 2+ L1 norm is as vulnerable to outliers as L2 (this observation although it might appear obvious L2 (this observation although it might appear obvious has eluded many eminent vision researchers).has eluded many eminent vision researchers).
![Page 6: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/6.jpg)
L1 Problem
![Page 7: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/7.jpg)
M-Estimators
No good algorithm for obtaining themNo good algorithm for obtaining them
Not justified from a Bayesian perspectiveNot justified from a Bayesian perspective
![Page 8: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/8.jpg)
What we would like to do
If we are rational we might be interested to know If we are rational we might be interested to know the maximum a posteriori solutionthe maximum a posteriori solution
This will lead to an energy minimization problem This will lead to an energy minimization problem where the energy might be arbitrarywhere the energy might be arbitrary
![Page 9: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/9.jpg)
Example: Gaussian Error in Variable Model (EVM)
Noise on points is (possibly a robust mixture):Noise on points is (possibly a robust mixture):
Where points lie on a manifold defined by q Where points lie on a manifold defined by q implicit relationsimplicit relations
![Page 10: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/10.jpg)
Parameters:
Non-Latent Variables αα parameters of manifold (i.e. 7 for parameters of manifold (i.e. 7 for FF))
Latent Variables ββparameters of location of data on manifoldparameters of location of data on manifold
γγindicator variables, one per datum, whether indicator variables, one per datum, whether inlier or outlierinlier or outlier
![Page 11: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/11.jpg)
Total least squares
![Page 12: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/12.jpg)
Bayesian Form of the Non-robust & Robust likelihood L
![Page 13: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/13.jpg)
Problem
How to minimize this complex (or any other How to minimize this complex (or any other complex) posterior likelihood.complex) posterior likelihood.
Answer of course by data driven search of Answer of course by data driven search of the space (e.g. RANSAC)the space (e.g. RANSAC)
![Page 14: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/14.jpg)
RANSAC
Repeat M times:Repeat M times:– Sample minimal number of matches to estimate Sample minimal number of matches to estimate
two view relation.two view relation.
– Calculate number of inliers or posterior likelihood Calculate number of inliers or posterior likelihood for relation.for relation.
– Choose relation to maximize number of inliers.Choose relation to maximize number of inliers.
![Page 15: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/15.jpg)
RANSAC line fitting example
Task:Task:
Estimate best lineEstimate best line
![Page 16: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/16.jpg)
RANSAC line fitting example
Sample two pointsSample two points
![Page 17: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/17.jpg)
RANSAC line fitting example
Fit LineFit Line
![Page 18: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/18.jpg)
RANSAC line fitting example
Total number of points Total number of points within a threshold of line.within a threshold of line.
![Page 19: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/19.jpg)
RANSAC line fitting example
Repeat, until get a Repeat, until get a good resultgood result
![Page 20: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/20.jpg)
RANSAC line fitting example
Repeat, until get a Repeat, until get a good resultgood result
![Page 21: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/21.jpg)
RANSAC line fitting example
Repeat, until get a Repeat, until get a good resultgood result
![Page 22: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/22.jpg)
How do we minimize a posterior energy?
So before we mentioned Bayesian estimation.So before we mentioned Bayesian estimation.
We explain first why just the RANSAC error We explain first why just the RANSAC error might not be so good.might not be so good.
Then how we can maximize an arbitrary Then how we can maximize an arbitrary posterior likelihood.posterior likelihood.
![Page 23: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/23.jpg)
Cost function
RANSAC can be vulnerable to the correct RANSAC can be vulnerable to the correct choice of the threshold:choice of the threshold:– Too large all hypotheses are ranked equally.Too large all hypotheses are ranked equally.– Too small leads to an unstable fit.Too small leads to an unstable fit.
The interesting thing is that the same strategy The interesting thing is that the same strategy can be followed with any modification of the can be followed with any modification of the cost function.cost function.
![Page 24: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/24.jpg)
Problem with RANSAC;threshold too high
![Page 25: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/25.jpg)
Problem with RANSAC;threshold too high
This solution…This solution…
![Page 26: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/26.jpg)
Problem with RANSAC;threshold too high
Is as good as this Is as good as this solutionsolution
![Page 27: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/27.jpg)
Problem with RANSAC;threshold too low-no support
![Page 28: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/28.jpg)
Problem 1; cost function
Examples of other cost functionsExamples of other cost functions– Least Median Squares; i.e. take the sample that Least Median Squares; i.e. take the sample that
minimized the median of the residuals.minimized the median of the residuals.– MLESAC/MLESAC use the posterior or likelihood MLESAC/MLESAC use the posterior or likelihood
of the data.of the data.– MINPRAN (Stewart), makes assumptions about MINPRAN (Stewart), makes assumptions about
randomness of datarandomness of data
![Page 29: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/29.jpg)
LMS
Repeat M times:Repeat M times:– Sample minimal number of matches to estimate Sample minimal number of matches to estimate
two view relation.two view relation.
– Calculate error of all data.Calculate error of all data.
– Choose relation to minimize median of errors.Choose relation to minimize median of errors.
![Page 30: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/30.jpg)
Pros and Cons LMS
PROPRO– Do not need any threshold for inliers.Do not need any threshold for inliers.
CONCON– Cannot work for more than 50% outliers.Cannot work for more than 50% outliers.– Problems if a lot of data belongs to a sub manifold Problems if a lot of data belongs to a sub manifold
(e.g. dominate plane in the image)(e.g. dominate plane in the image)
![Page 31: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/31.jpg)
Con: LMS, subspace problem
Median error is same Median error is same for two solutions.for two solutions.
![Page 32: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/32.jpg)
Con: LMS, subspace problem
No good solution if the No good solution if the number of outliers >50%number of outliers >50%
![Page 33: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/33.jpg)
Pros LMS
One major advantage of LMS is that it can One major advantage of LMS is that it can yield a robust estimate of the variance of the yield a robust estimate of the variance of the errors.errors.
But care should be taken to use the right But care should be taken to use the right formula, as this depends on the distribution of formula, as this depends on the distribution of the errors, and degrees of freedom in the the errors, and degrees of freedom in the errors (codimension).errors (codimension).
![Page 34: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/34.jpg)
Choosing the right model
We shall return to this laterWe shall return to this later
![Page 35: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/35.jpg)
Robust Maximum Likelihood Estimation
Random Sampling can optimize any function:Random Sampling can optimize any function:
outlier
inlier ρ with ρ
222
222
i tet
teeed iR
Better, robust cost function, MLESACBetter, robust cost function, MLESAC
![Page 36: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/36.jpg)
Mixture (Maxture) of Gaussian/Uniform?
Red-mixture, green-uniform, blue-Gaussian.Red-mixture, green-uniform, blue-Gaussian.
![Page 37: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/37.jpg)
MAPSAC/MLESAC
This solution…This solution…
![Page 38: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/38.jpg)
MLESAC/MLESAC
Is better than this Is better than this solutionsolution
![Page 39: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/39.jpg)
MLESAC
Add in prior to get to MAP solutionAdd in prior to get to MAP solution
Interesting thing is that with MLESAC one could Interesting thing is that with MLESAC one could sample less than the minimal number of points to sample less than the minimal number of points to make an estimate (using prior as extra information).make an estimate (using prior as extra information).
Any posterior can be optimized; random sampling Any posterior can be optimized; random sampling good for matching AND FUNCTION OPTIMIZATION! good for matching AND FUNCTION OPTIMIZATION! e.g. MLESAC is a cheap way to optimize objective e.g. MLESAC is a cheap way to optimize objective functions regardless of outliers or not.functions regardless of outliers or not.
![Page 40: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/40.jpg)
MLESAC
Once the benefits of MLESAC are seen there Once the benefits of MLESAC are seen there is no reason to continue to use RANSAC; is no reason to continue to use RANSAC; – in many situations the improvement in the solution in many situations the improvement in the solution
can be markedcan be marked– Especially if want to use prior information (e.g. the Especially if want to use prior information (e.g. the
F matrix changing smoothly over time).F matrix changing smoothly over time).– Gives an optimized solution Gives an optimized solution
AT NO EXTRA COST!AT NO EXTRA COST!
![Page 41: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/41.jpg)
A RANSAC system for SAM
![Page 42: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/42.jpg)
Structure and Motion Recovery
1. Finding Features 1. Finding Features 2. Matching Features2. Matching Features 3. Extracting Epipolar3. Extracting Epipolar GeometryGeometry
7. VRML Models7. VRML Models4. Extract edges4. Extract edges5. Match edges5. Match edges
6. Estimate Depth Map 6. Estimate Depth Map (dynamic programming)(dynamic programming)
![Page 43: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/43.jpg)
Guide matches with Geometry
Plane
C
C’
m
m’
11
'
'
987
654
321
y
x
y
x
hhhhhhhhh
Hxx '0'Fxxt
0
1
'
'
1
987
654
321
y
x
yx
fffffffff
![Page 44: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/44.jpg)
Concatenated Image space
2 Views- consider 4D space of image 2 Views- consider 4D space of image coordinates (x,y,x’,y’).coordinates (x,y,x’,y’).
Fundamental matrix is a 3D manifold in Fundamental matrix is a 3D manifold in this space.this space.
Homography is a 2D manifold in this Homography is a 2D manifold in this space.space.
![Page 45: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/45.jpg)
Estimation of Motion model like fitting a manifold to space of 4D image
points in two images:
•Problem compounded in higher dimensionsProblem compounded in higher dimensions
![Page 46: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/46.jpg)
Images of the same scene from different viewpoints
Feature Detectors need to consistently locate the position within the image of a landmark on the
3D object.
Stage 1 Corner Detection
![Page 47: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/47.jpg)
Typical Features Detected
![Page 48: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/48.jpg)
Stage 2 Feature Matching
Images of the same scene from different viewpoints
Initial Feature correspondence via Cross Correlation.
![Page 49: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/49.jpg)
Stage 2 Feature Matching
Initial Feature correspondence via Cross Correlation
Many outliers.
![Page 50: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/50.jpg)
Stage 3 Estimation of Epipolar Geometry
Images of the same scene from different viewpoints
Corresponding features must lie on correspondingepipolar lines.
All epipolar lines intersect at a common point.
![Page 51: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/51.jpg)
Example: robust computationInterest points(500/image)
Putative correspondences (268)
Outliers (117)
Inliers (151)
Final inliers (262)
![Page 52: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/52.jpg)
Automatic camera recovery:
![Page 53: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/53.jpg)
Video
![Page 54: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/54.jpg)
Video
![Page 55: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/55.jpg)
How many samples?
Choose Choose NN so that, with probability so that, with probability pp, at least one random , at least one random sample is free from outliers. e.g. sample is free from outliers. e.g. pp=0.99=0.99
sepN 11log/1log
peNs 111
proportion of outliersproportion of outliers ee
ss 5%5% 10%10% 20%20% 25%25% 30%30% 40%40% 50%50%
22 22 33 55 66 77 1111 171733 33 44 77 99 1111 1919 3535
44 33 55 99 1313 1717 3434 7272
55 44 66 1212 1717 2626 5757 146146
66 44 77 1616 2424 3737 9797 29329377 44 88 2020 3333 5454 163163 588588
88 55 99 2626 4444 7878 272272 11771177
![Page 56: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/56.jpg)
Adaptively determining the number of samples
ee is often unknown a priori, so pick worst case, e.g. 50%, and is often unknown a priori, so pick worst case, e.g. 50%, and
adapt if more inliers are found, e.g. 80% would yield adapt if more inliers are found, e.g. 80% would yield ee=0.2 =0.2
– NN=∞, =∞, sample_count sample_count =0=0
– While While N N >>sample_countsample_count repeat repeat Choose a sample and count the number of inliersChoose a sample and count the number of inliers Set e=1-(number of inliers)/(total number of points)Set e=1-(number of inliers)/(total number of points) Recompute Recompute NN from from ee Increment the Increment the sample_countsample_count by 1 by 1
– TerminateTerminate sepN 11log/1log
![Page 57: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/57.jpg)
Number of Samples II
Make take many Make take many more samples than more samples than one would think due one would think due to degenerate point to degenerate point sets.sets.
![Page 58: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/58.jpg)
Number of Samples II
These two These two points are points are inliers.inliers.
![Page 59: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/59.jpg)
Number of Samples II
And yet the And yet the estimate estimate yielded is poor.yielded is poor.
![Page 60: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/60.jpg)
Work of Tordof and Murray
Try and sample more frequently from strong Try and sample more frequently from strong matches.matches.
![Page 61: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/61.jpg)
Model Selection!!
![Page 62: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/62.jpg)
Problem 2, what model to fit?
There are many cases when we do not know There are many cases when we do not know the relation between the images, there may a the relation between the images, there may a choice of many.choice of many.
In this case a Bayesian solution might be to In this case a Bayesian solution might be to evaluate the likelihood of each.evaluate the likelihood of each.
![Page 63: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/63.jpg)
Problem Degeneracy F or H?:Torr, Fitzgibbon & Zisserman 97, 99
AA
Two problem cases Two problem cases for SAM (structure for SAM (structure and motion) and motion) recovery.recovery.
A Camera RotationA Camera Rotation
B Planar object.B Planar object.
BB
Problem:Problem: Model Selection to determine whether Model Selection to determine whether FF or or HH
![Page 64: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/64.jpg)
Problem Degeneracy F or H?:Torr, Fitzgibbon & Zisserman 97, 99
AA
Two problem cases Two problem cases for SAM (structure for SAM (structure and motion) and motion) recovery.recovery.
A Camera RotationA Camera Rotation
B Planar object.B Planar object.
BB
Solution:Solution: was to use was to use GRIC GRIC criteria, to be explained later.criteria, to be explained later.
![Page 65: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/65.jpg)
When Homography describes scene:
A: Camera rotates, no new structure A: Camera rotates, no new structure information.information.
B: 2 views have a plane in common; can B: 2 views have a plane in common; can not put structure into the same projective not put structure into the same projective frame (3 degrees of freedom).frame (3 degrees of freedom).
![Page 66: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/66.jpg)
This is an example of fitting manifolds of varying dimension:
2 Views----Consider Image coordinates 4D 2 Views----Consider Image coordinates 4D spacespace
Dimension 3 Dimension 3 Dimension 2Dimension 2
BilinearBilinear FF Matrix Matrix Homography Homography HH
LinearLinear Affine Affine FF Matrix Matrix Affinity Affinity AA
(non generic: quadratic transformations, (non generic: quadratic transformations, dimension 2.)dimension 2.)
Three views: same dimension for manifolds.Three views: same dimension for manifolds.
![Page 67: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/67.jpg)
Concatenated or Joint Image Space
xxyy
X’X’
Y’Y’
Image coordinates in higher dimensional spaceImage coordinates in higher dimensional space
(x,y,x’,y’,…)(x,y,x’,y’,…)
![Page 68: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/68.jpg)
Generic ProblemDetermine the degree and dimension ofDetermine the degree and dimension ofa manifold in a manifold in dd dimensional space. dimensional space.
![Page 69: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/69.jpg)
Examples
•Problem compounded in higher dimensionsProblem compounded in higher dimensions
![Page 70: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/70.jpg)
Robust Model Selection
•Curve Dim 2, degree 2Curve Dim 2, degree 2
•Line Dim 1, degree 1Line Dim 1, degree 1
•Point Dim 0, degree 1Point Dim 0, degree 1
•Outliers make a hard problem very hard!Outliers make a hard problem very hard!
![Page 71: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/71.jpg)
Bayesian Model Comparison
![Page 72: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/72.jpg)
A Horrible integral:
Can we simplify it?Can we simplify it?
Integrate out the Integrate out the ββ and and γγ..
![Page 73: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/73.jpg)
Solve Via Sampling
Can use RANSAC to generate an importance Can use RANSAC to generate an importance sampling function and use sampling to sampling function and use sampling to evaluate the integral evaluate the integral – (Torr & Davidson ECCV 2002/PAMI 2002)(Torr & Davidson ECCV 2002/PAMI 2002)
![Page 74: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/74.jpg)
GRIC approximation, Torr IJCV 2002
Quick hack: some combination of likelihood Quick hack: some combination of likelihood of model and a penalty term for complexity…of model and a penalty term for complexity…
Helps to penalize latent and model Helps to penalize latent and model parameters separately.parameters separately.
![Page 75: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/75.jpg)
Summary of Idea
Can base approximation around MAP Can base approximation around MAP solution,solution,
Thus need to find MAP solution for each Thus need to find MAP solution for each model.model.
![Page 76: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/76.jpg)
Sampling Problem
If lots of points lie on low D space data driven If lots of points lie on low D space data driven estimates can be quite poor.estimates can be quite poor.
E.g. many points lie near a single point, then hard to E.g. many points lie near a single point, then hard to get a line that passes through all the inliers.get a line that passes through all the inliers.
This gets worse in higher dimensionsThis gets worse in higher dimensions
Example for motion estimation, many points lie on a Example for motion estimation, many points lie on a plane.plane.
![Page 77: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/77.jpg)
Sampling Problem
![Page 78: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/78.jpg)
Sampling Problem
![Page 79: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/79.jpg)
Sampling Problem
![Page 80: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/80.jpg)
Solution
Derive a sampling strategy that takes into Derive a sampling strategy that takes into account the multiple models.account the multiple models.
Estimate lower dimensional ones first, e.g. Estimate lower dimensional ones first, e.g. planesplanes
Sample points on and off the planes to Sample points on and off the planes to estimate the higher dimensional manifolds estimate the higher dimensional manifolds e.g. F etc.e.g. F etc.
![Page 81: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/81.jpg)
Model Selection and Multiple Motions
![Page 82: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/82.jpg)
Lorry Sequence
Camera zooms out, lorry translates leftCamera zooms out, lorry translates left
![Page 83: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/83.jpg)
Lorry Match Segmentation
![Page 84: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/84.jpg)
Conclusion
Minimize the correct error from the outset.Minimize the correct error from the outset.
Be wary of the model selection problem.Be wary of the model selection problem.
![Page 85: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/85.jpg)
END
Matlab Code and paper available:Matlab Code and paper available: Matching, est F, sfm, segmentation.Matching, est F, sfm, segmentation.
http://cms.brookes.ac.uk/staff/PhilipTorr/http://cms.brookes.ac.uk/staff/PhilipTorr/
philtorr@[email protected]@[email protected]
![Page 86: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/86.jpg)
Bib
P.H.S. Torr, and D.W. Murray. P.H.S. Torr, and D.W. Murray. The Development and Comparison of Robust Methods for Estimating the FundaThe Development and Comparison of Robust Methods for Estimating the Fundamental Matrixmental Matrix. In . In International Journal of Computer Vision,International Journal of Computer Vision, pages 271—300, v 24, n 3, 1997. pages 271—300, v 24, n 3, 1997.
P.H.S. Torr and A. Zisserman. P.H.S. Torr and A. Zisserman. MLESAC: A New Robust Estimator with Application to Estimating Image GeomeMLESAC: A New Robust Estimator with Application to Estimating Image Geometrytry. In . In Journal of Computer Vision and Image UnderstandingJournal of Computer Vision and Image Understanding, pages 138—156, , pages 138—156, 78(1), 2000.78(1), 2000.
P.H.S. Torr. P.H.S. Torr. Geometric Motion Segmentation and Model SelectionGeometric Motion Segmentation and Model Selection. In . In Philosophical Transactions of the Royal Society APhilosophical Transactions of the Royal Society A , pages 1321—1340, 1998., pages 1321—1340, 1998.
P.H.S. Torr, A. Fitzgibbon and A. Zisserman. P.H.S. Torr, A. Fitzgibbon and A. Zisserman. The Problem of Degeneracy in Structure and Motion Recovery from UncalibrateThe Problem of Degeneracy in Structure and Motion Recovery from Uncalibrated Image Sequencesd Image Sequences. In . In International Journal of Computer VisionInternational Journal of Computer Vision, 32(1), pages 27—45, 1999., 32(1), pages 27—45, 1999.
P.H.S. Torr and C. Davidson. P.H.S. Torr and C. Davidson. IMPSAC: A synthesis of importance sampling and random sample consensusIMPSAC: A synthesis of importance sampling and random sample consensus. . In In IEEE Trans Pattern Analysis and Machine Intelligence, IEEE Trans Pattern Analysis and Machine Intelligence, 25(3), pages 354-365, 25(3), pages 354-365, 2003.2003.
P.H.S. Torr. P.H.S. Torr. Bayesian Model Estimation and Selection for Epipolar Geometry and Generic MBayesian Model Estimation and Selection for Epipolar Geometry and Generic Manifold Fittinganifold Fitting. In . In International Journal of Computer VisionInternational Journal of Computer Vision, 50(1), pages 27—45, 2002., 50(1), pages 27—45, 2002.
![Page 87: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/87.jpg)
Bib
D. Myatt, P.H.S. Torr, S. Nasuto, and R. Craddock. D. Myatt, P.H.S. Torr, S. Nasuto, and R. Craddock. NAPSAC: High Noise, High Dimensional Robust EstimationNAPSAC: High Noise, High Dimensional Robust Estimation, In , In Proceedings British Proceedings British Machine Vision ConferenceMachine Vision Conference, pages 458-467,, pages 458-467, 2002. 2002. (oral)(oral). .
P.H.S. Torr and D.W. Murray. P.H.S. Torr and D.W. Murray. Outlier detection and motion segmentationOutlier detection and motion segmentation. In . In SPIE sensor SPIE sensor fusion conference VIfusion conference VI, pages 432—443, Sept. 1993. , pages 432—443, Sept. 1993. (oral)(oral). .
P.H.S. Torr, A. Zisserman and S. Maybank. P.H.S. Torr, A. Zisserman and S. Maybank. Robust Detection of Degeneracy.Robust Detection of Degeneracy. In In The Fifth The Fifth International Conference on Computer VisionInternational Conference on Computer Vision, pages 1037—1044, 1995. , pages 1037—1044, 1995. (oral)(oral)..
P.H.S. Torr, A. Zisserman, and D.W. Murray. P.H.S. Torr, A. Zisserman, and D.W. Murray. Motion Clustering using the Trilinear ConstraintMotion Clustering using the Trilinear Constraint. In . In Europe-China workshop on Geometric Europe-China workshop on Geometric Modelling and Invariants for Computer VisionModelling and Invariants for Computer Vision, pages 118—125, 1995. , pages 118—125, 1995. (oral)(oral)..
P.A. Beardsley, P.H.S. Torr and A. Zisserman. P.A. Beardsley, P.H.S. Torr and A. Zisserman. 3D Model Acquisition from Extended Image Sequences3D Model Acquisition from Extended Image Sequences. In . In The Fourth European The Fourth European Conference on Computer VisionConference on Computer Vision, pages 683—695, 1996. , pages 683—695, 1996. (oral)(oral). .
P.H.S. Torr and A. Zisserman. P.H.S. Torr and A. Zisserman. Feature Based Methods for Structure and Motion EstimationFeature Based Methods for Structure and Motion Estimation. In . In International Workshop on Vision Algorithms, International Workshop on Vision Algorithms, pages 278-295, 1999.pages 278-295, 1999.(oral-panel (oral-panel discussion)discussion). .
F. Schaffalitzky, A. Zisserman, R. Hartley and P.H.S. Torr. F. Schaffalitzky, A. Zisserman, R. Hartley and P.H.S. Torr. A Six Point Solution for Structure and MotionA Six Point Solution for Structure and Motion. In . In The Sixth European Conference on The Sixth European Conference on Computer VisionComputer Vision, pages 632—648, 2000. , pages 632—648, 2000. (poster)(poster). .
![Page 88: Twenty Five Years of RANSAC: MLESAC a new cost function for RANSAC Philip Torr Oxford Brookes Code Available](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d7e5503460f94a6096e/html5/thumbnails/88.jpg)
Related Bib
P.H.S. Torr and A. Fitzgibbon. P.H.S. Torr and A. Fitzgibbon. Invariant Fitting of Two View Geometry or “In Defiance of Invariant Fitting of Two View Geometry or “In Defiance of the eight point algorithm”the eight point algorithm”, In , In IEEE Trans Pattern Analysis and Machine IEEE Trans Pattern Analysis and Machine Intelligence, Intelligence, 26(5), pages 648-651, 2004. 26(5), pages 648-651, 2004.
P.H.S. Torr, A. Zisserman. P.H.S. Torr, A. Zisserman. Robust Parameterization and Computation of the Trifocal Robust Parameterization and Computation of the Trifocal TensorTensor. . Image and Vision Computing,Image and Vision Computing, pages 591—607, v 15, pages 591—607, v 15, 1997. 1997.
P.H.S. Torr, A. Zisserman. P.H.S. Torr, A. Zisserman. Performance Performance CharacterisationCharacterisation of Fundamental Matrix Estimation Under Image Degradat of Fundamental Matrix Estimation Under Image Degradationion. In . In Machine Vision and Applications, Machine Vision and Applications, pages 321—333, v pages 321—333, v 9, 1997.9, 1997.