model fitting - cvg · 3. radial quadrifocal tensor •linearly compute radial quadrifocal tensor...
TRANSCRIPT
![Page 1: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/1.jpg)
Model fitting
Hough transformRANSACEM
![Page 2: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/2.jpg)
2
Last week: multi-view geometry
perspective camera(2 constraints / feature)
radial camera (uncalibrated)(1 constraints / feature)
3 constraints allow to reconstruct 3D point
more constraints also tell something about cameras
multilinear constraints known as epipolar, trifocal and quadrifocal constraints
(0,0)
l=(y,-x)
(x,y)
![Page 3: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/3.jpg)
3
Radial quadrifocal tensor
• Linearly compute radial quadrifocal tensor Qijkl
from 15 pts in 4 views
• Reconstruct 3D scene and use it for calibration(2x2x2x2 tensor)
(2x2x2 tensor)
Not easy for real data, hard to avoid degenerate cases (e.g. 3 optical axes intersect in single point). However, degenerate case leads to simpler 3 view algorithm for pure rotation
• Radial trifocal tensor Tijk from 7 points in 3 views
• Reconstruct 2D panorama and use it for calibration
(x,y)
![Page 4: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/4.jpg)
4
Dealing with Wide FOV Camera
• Two-step linear approach to compute radial distortion
• Estimates distortion polynomial of arbitrary degree
(Thirthala and Pollefeys CVPR05)
undistorted image
estimated distortion(4-8 coefficients)
![Page 5: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/5.jpg)
5
Dealing with Wide FOV Camera
• Two-step linear approach to compute radial distortion
• Estimates distortion polynomial of arbitrary degree
(Thirthala and Pollefeys CVPR05)
unfolded cubemapestimated distortion
(4-8 coefficients)
![Page 6: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/6.jpg)
6
Non-parametric distortion calibration
• Models fish-eye lenses, cata-dioptricsystems, etc.
(Thirthala and Pollefeys, ICCV’05)
normalized radius
angl
e
![Page 7: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/7.jpg)
7
Non-parametric distortion calibration
• Models fish-eye lenses, cata-dioptricsystems, etc.
• results
(Thirthala and Pollefeys, ICCV’05)
normalized radius
angl
e
90o
![Page 8: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/8.jpg)
Schedule (tentative)
8
# date topic1 Sep.22 Introduction and geometry2 Sep.29 Invariant features 3 Oct.6 Camera models and calibration4 Oct.13 Multiple-view geometry5 Oct.20 Model fitting (RANSAC, EM, …)6 Oct.27 Stereo Matching7 Nov.3 Segmentation8 Nov.10 2D Shape Matching9 Nov.17 Shape from Silhouettes10 Nov.24 Optical Flow11 Dec.1 Structure from motion12 Dec.8 Tracking (Kalman, particle filter)13 Dec.15 Object recognition14 Dec.22 Object category recognition
![Page 9: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/9.jpg)
9
Fitting
• Choose a parametric object/some objects to represent a set of tokens
• Most interesting case is when criterion is not local– can’t tell whether a
set of points lies on a line by looking only at each point and the next.
• Three main questions:– what object
represents this set of tokens best?
– which of several objects gets which token?
– how many objects are there?
(you could read line for object here, or circle, or ellipse or...)
![Page 10: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/10.jpg)
10
Hough transform : straight lines
implementation :
1. the parameter space is discretised2. a counter is incremented at each cell
where the lines pass 3. peaks are detected
yaxb +−= )(
![Page 11: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/11.jpg)
11
Hough transform : straight linesproblem : unbounded parameter domain
vertical lines require infinite a
ρθθ = + sincos yx
Each point will add a cosine function in the (θ,ρ) parameter space
alternative representation:
![Page 12: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/12.jpg)
12
tokensvotes
![Page 13: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/13.jpg)
13
Hough transform : straight lines
Square : Circle :
![Page 14: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/14.jpg)
14
Hough transform : straight lines
![Page 15: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/15.jpg)
15
Mechanics of the Hough transform
• Construct an array representing θ, d
• For each point, render the curve (θ, d) into this array, adding one at each cell
• Difficulties– how big should the
cells be? (too big, and we cannot distinguish between quite different lines; too small, and noise causes lines to be missed)
• How many lines?– count the peaks in the
Hough array• Who belongs to which
line?– tag the votes
• Hardly ever satisfactory in practice, because problems with noise and cell size defeat it
![Page 16: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/16.jpg)
16
tokens votes
![Page 17: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/17.jpg)
17
![Page 18: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/18.jpg)
18
![Page 19: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/19.jpg)
19
![Page 20: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/20.jpg)
20
Cascaded hough transformTuytelaars and Van Gool ICCV‘98
![Page 21: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/21.jpg)
21
Line fitting can be max.likelihood - but choice ofmodel is important
standard least-squares
total least-squares
![Page 22: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/22.jpg)
22
Who came from which line?
• Assume we know how many lines there are - but which lines are they?– easy, if we know who came from which
line• Three strategies
– Incremental line fitting– K-means– Probabilistic (later!)
![Page 23: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/23.jpg)
23
![Page 24: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/24.jpg)
24
![Page 25: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/25.jpg)
25
![Page 26: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/26.jpg)
26
![Page 27: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/27.jpg)
27
![Page 28: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/28.jpg)
28
![Page 29: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/29.jpg)
29
![Page 30: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/30.jpg)
30
![Page 31: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/31.jpg)
31
![Page 32: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/32.jpg)
32
![Page 33: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/33.jpg)
33
![Page 34: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/34.jpg)
34
![Page 35: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/35.jpg)
35
![Page 36: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/36.jpg)
36
![Page 37: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/37.jpg)
37
Robustness
• As we have seen, squared error can be a source of bias in the presence of noise points– One fix is EM - we’ll do this shortly– Another is an M-estimator
• Square nearby, threshold far away
– A third is RANSAC• Search for good points
![Page 38: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/38.jpg)
38
![Page 39: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/39.jpg)
39
![Page 40: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/40.jpg)
40
![Page 41: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/41.jpg)
41
![Page 42: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/42.jpg)
42
M-estimators
• Generally, minimize
where is the residual
( )( )σθρ ;,iii
xr∑
( )θ,ii xr
![Page 43: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/43.jpg)
43
![Page 44: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/44.jpg)
44
![Page 45: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/45.jpg)
45
![Page 46: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/46.jpg)
46
Too small
![Page 47: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/47.jpg)
47
Too large
![Page 48: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/48.jpg)
48
![Page 49: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/49.jpg)
49
RANSAC
• Choose a small subset uniformly at random
• Fit to that• Anything that is close
to result is signal; all others are noise
• Refit• Do this many times
and choose the best
• Issues– How many times?
• Often enough that we are likely to have a good line
– How big a subset?• Smallest possible
– What does close mean?• Depends on the
problem– What is a good line?
• One where the number of nearby points is so big it is unlikely to be all outliers
![Page 50: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/50.jpg)
50
![Page 51: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/51.jpg)
51
Distance threshold
Choose t so probability for inlier is α (e.g. 0.95) • Often empirically• Zero-mean Gaussian noise σ then follows
distribution with m=codimension of model
2⊥d
2mχ
(dimension+codimension=dimension space)
Codimension Model t 2
1 line,F 3.84σ2
2 H,P 5.99σ2
3 T 7.81σ2
![Page 52: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/52.jpg)
52
How many samples?
Choose N so that, with probability p, at least one random sample is free from outliers. e.g. p=0.99
( ) ( )( )sepN −−−= 11log/1log
( )( ) peNs −=−− 111
proportion of outliers es 5% 10% 20% 25% 30% 40% 50%2 2 3 5 6 7 11 173 3 4 7 9 11 19 354 3 5 9 13 17 34 725 4 6 12 17 26 57 1466 4 7 16 24 37 97 2937 4 8 20 33 54 163 5888 5 9 26 44 78 272 1177
alternatively, use Poisson:
with k=0 (no occurrence)
λ=3: f≈0.05λ=4.5: f≈0.01
![Page 53: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/53.jpg)
53
Acceptable consensus set?
• Typically, terminate when inlier ratio reaches expected ratio of inliers
( )neT −= 1
![Page 54: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/54.jpg)
54
Adaptively determining the number of samples
e is often unknown a priori, so pick worst case, e.g. 50%, and adapt if more inliers are found, e.g. 80% would yield e=0.2
– N=∞, sample_count =0– While N >sample_count repeat
• Choose a sample and count the number of inliers• Set e=1-(number of inliers)/(total number of points)• Recompute N from e• Increment the sample_count by 1
– Terminate
( ) ( )( )( )sepN −−−= 11log/1log
![Page 55: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/55.jpg)
55
Step 1. Extract featuresStep 2. Compute a set of potential matchesStep 3. do
Step 3.1 select minimal sample (i.e. 7 matches)Step 3.2 compute solution(s) for FStep 3.3 determine inliers
until Γ(#inliers,#samples)<95%
( ) samples#7 )1(1matches#
inliers#−−=Γ
#inliers 90% 80% 70% 60% 50%
#samples 5 13 35 106 382
Step 4. Compute F based on all inliersStep 5. Look for additional matchesStep 6. Refine F based on all correct matches
(generate hypothesis)
(verify hypothesis)
}
RANSAC for Fundamental Matrix
![Page 56: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/56.jpg)
56
Step 1. Extract featuresStep 2. Compute a set of potential matchesStep 3. do
Step 3.1 select minimal sample (i.e. 7 matches)Step 3.2 compute solution(s) for FStep 3.3 Randomize verification
3.3.1 verify if inlier while hypothesis is still promising
while Γ(#inliers,#samples)<95% Step 4. Compute F based on all inliersStep 5. Look for additional matchesStep 6. Refine F based on all correct matches
(generate hypothesis)
(verify hypothesis)
}
Randomized RANSAC for Fundamental Matrix
}
![Page 57: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/57.jpg)
57
Example: robust computation
Interest points(500/image)(640x480)
Putative correspondences (268)(Best match,SSD<20,±320)Outliers (117)(t=1.25 pixel; 43 iterations)
Inliers (151)
Final inliers (262)(2 MLE-inlier cycles; d⊥=0.23→d⊥=0.19;IterLev-Mar=10)
#in 1-e adapt. N
6 2% 20M
10 3% 2.5M
44 16% 6,922
58 21% 2,291
73 26% 911
151 56% 43
from H&Z
![Page 58: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/58.jpg)
58
More on robust estimation
• LMedS, an alternative to RANSAC(minimize Median residual in stead of maximizing inlier count)
• Enhancements to RANSAC– Randomized RANSAC– Sample ‘good’ matches more frequently– …
• RANSAC is also somewhat robust to bugs, sometimes it just takes a bit longer…
![Page 59: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/59.jpg)
59
RANSAC for quasi-degenerate data
• Often data can be almost degeneratee.g. 337 matches on plane, 11 off plane
• RANSAC gets confused by quasi-degenerate data
Probability of valid non-degenerate sampleProbability of success for RANSAC (aiming for 99%)
planar points only provide 6 in stead of 8 linearly independent equations for F
![Page 60: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/60.jpg)
60
RANSAC for quasi-degenerate data(QDEGSAC) (Frahm and Pollefeys, CVPR06)
QDEGSAC estimates robust rank of datamatrix (i.e. large % of rows approx. fit in a lower dimensional subspace)
![Page 61: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/61.jpg)
61
Experiments for computing F matrix
Comparison to Chum et al. (CVPR’05)
Data courtesy of Chum et al.
QDEGSAC (and DEGENSAC) are successful at dealing with quasi-degenerate data
![Page 62: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/62.jpg)
62
Experiments for computing P matrix
QDEGSAC is general and can deal with other robust estimation problemsQDEGSAC does not require to know which degeneracies can occur
Similar results for H, Q, …
![Page 63: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/63.jpg)
63
Fitting curves other than lines
• In principle, an easy generalisation– The probability of
obtaining a point, given a curve, is given by a negative exponential of distance squared
• In practice, rather hard– It is generally
difficult to compute the distance between a point and a curve
![Page 64: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/64.jpg)
64
Missing variable problems
• In many vision problems, if some variables were known the maximum likelihood inference problem would be easy– fitting; if we knew which line each token came
from, it would be easy to determine line parameters
– segmentation; if we knew the segment each pixel came from, it would be easy to determine the segment parameters
– fundamental matrix estimation; if we knew which feature corresponded to which, it would be easy to determine the fundamental matrix
– etc.• This sort of thing happens in statistics, too
![Page 65: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/65.jpg)
65
Missing variable problems
• Strategy– estimate appropriate values for the missing
variables– plug these in, now estimate parameters– re-estimate appropriate values for missing
variables, continue• e.g.
– guess which line gets which point– now fit the lines– now reallocate points to lines, using our knowledge
of the lines– now refit, etc.
• We’ve seen this line of thought before (k-means)
![Page 66: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/66.jpg)
66
Missing variables - strategy
• We have a problem with parameters, missing variables
• This suggests:• Iterate until
convergence– replace missing
variable with expected values, given fixedvalues of parameters
– fix missing variables, choose parameters to maximise likelihood given fixed values of missing variables
• e.g., iterate till convergence– allocate each point to a
line with a weight, which is the probability of the point given the line
– refit lines to the weighted set of points
• Converges to local extremum
• Somewhat more general form is available
![Page 67: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/67.jpg)
67
Lines and robustness
• We have one line, and n points
• Some come from the line, some from “noise”
• This is a mixture model:
• We wish to determine– line parameters– p(comes from line)
P p o i n t| l i n e a n d n o ( )
e p a r a m s= P p o i n t| l i n e( )P c o m e s f r o m l( )+
P p o i n t| n o i s e( )P c o m e s f r o m n( )= P p o i n t| l i n e( )λ + P p o i n t| n o i s e( )(1− λ)
![Page 68: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/68.jpg)
68
Estimating the mixture model
• Introduce a set of hidden variables, δ, one for each point. They are one when the point is on the line, and zero when off.
• If these are known, the negative log-likelihood becomes (the line’s parameters are φ, c):
• Here K is a normalising constant, kn is the noise intensity (we’ll choose this later).
Qc x;θ( )=δ i
xi cosφ + yi sinφ + c( )2
2σ 2
+
1− δi( )kn
i
∑ + K
![Page 69: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/69.jpg)
69
Substituting for delta
• We shall substitute the expected value of δ, for a given θ
• recall θ=(φ, c, λ)• E(δ_i)=1.
P(δ_i=1|θ)+0....
• Notice that if kn is small and positive, then if distance is small, this value is close to 1 and if it is large, close to zero
P δi = 1|θ,xi( )=P xi | δi = 1,θ( )P δi = 1( )
P xi |δ i = 1,θ( )P δi = 1( )+ P xi | δi = 0,θ( )P δ i = 0( )
=exp −1
2σ 2 xi cosφ + yi sinϕ + c[ ]2( )λexp −1
2σ 2 xi cosφ + yi sinϕ + c[ ]2( )λ + exp −kn( )1− λ( )
![Page 70: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/70.jpg)
70
Algorithm for line fitting
• Obtain some start point
• Now compute δ’s using formula above
• Now compute maximum likelihood estimate of
– φ, c come from fitting to weighted points
– λ comes by counting
• Iterate to convergence
θ 0( ) = φ 0( ),c 0( ),λ 0( )( )
θ 1( )
![Page 71: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/71.jpg)
71
![Page 72: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/72.jpg)
72
The expected values of the deltas at the maximum(notice the one value close to zero).
![Page 73: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/73.jpg)
73
Closeup of the fit
![Page 74: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/74.jpg)
74
Choosing parameters
• What about the noise parameter, and the sigma for the line?– several methods
• from first principles knowledge of the problem (seldom really possible)
• play around with a few examples and choose (usually quite effective, as precise choice doesn’t matter much)
– notice that if kn is large, this says that points very seldom come from noise, however far from the line they lie• usually biases the fit, by pushing outliers into
the line• rule of thumb; its better to fit to the better
fitting points, within reason; if this is hard to do, then the model could be a problem
![Page 75: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/75.jpg)
75
Other examples
• Segmentation– a segment is a
gaussian that emits feature vectors (which could contain colour; or colour and position; or colour, texture and position).
– segment parameters are mean and (perhaps) covariance
– if we knew which segment each point belonged to, estimating these parameters would be easy
– rest is on same lines as fitting line
• Fitting multiple lines– rather like fitting one
line, except there are more hidden variables
– easiest is to encode as an array of hidden variables, which represent a table with a one where the i’th point comes from the j’th line, zeros otherwise
– rest is on same lines as above
![Page 76: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/76.jpg)
76
Issues with EM
• Local maxima– can be a serious nuisance in some problems– no guarantee that we have reached the “right”
maximum• Starting
– k means to cluster the points is often a good idea
![Page 77: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/77.jpg)
77
Local maximum
![Page 78: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/78.jpg)
78
which is an excellent fit to some points
![Page 79: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/79.jpg)
79
and the deltas for this maximum
![Page 80: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/80.jpg)
80
A dataset that is well fitted by four lines
![Page 81: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/81.jpg)
81
Result of EM fitting, with one line (or at least, one available local maximum).
![Page 82: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/82.jpg)
82
Result of EM fitting, with two lines (or at least, one available local maximum).
![Page 83: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/83.jpg)
83
Seven lines can produce a rather logical answer
![Page 84: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/84.jpg)
84
Figure from “Color and Texture Based Image Segmentation Using EM and Its Application to Content Based Image Retrieval”,S.J. Belongie et al., Proc. Int. Conf. Computer Vision, 1998, c1998, IEEE
Segmentation with EM
![Page 85: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/85.jpg)
85
Motion segmentation with EM
• Model image pair (or video sequence) as consisting of regions of parametric motion– affine motion is
popular
• Now we need to– determine which
pixels belong to which region
– estimate parameters
• Likelihood– assume
• Straightforward missing variable problem, rest is calculation
vx
vy
=a bc d
xy
+tx
ty
I x, y,t( )= I x + vx, y + vy,t +1( )+noise
![Page 86: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/86.jpg)
86
Three frames from the MPEG “flower garden” sequence
Figure from “Representing Images with layers,”, by J. Wang and E.H. Adelson, IEEE Transactions on Image Processing, 1994, c 1994, IEEE
![Page 87: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/87.jpg)
87
Grey level shows region no. with highest probability
Segments and motion fields associated with themFigure from “Representing Images with layers,”, by J. Wang and E.H. Adelson, IEEE Transactions on Image Processing, 1994, c 1994, IEEE
![Page 88: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/88.jpg)
88
If we use multiple frames to estimate the appearanceof a segment, we can fill in occlusions; so we canre-render the sequence with some segments removed.
Figure from “Representing Images with layers,”, by J. Wang and E.H. Adelson, IEEE Transactions on Image Processing, 1994, c 1994, IEEE
![Page 89: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/89.jpg)
89
Structure from motion without correspondences
Dellaert et al.00
EMMCMCChain Flipping
![Page 90: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/90.jpg)
90
Some generalities
• Many, but not all problems that can be attacked with EM can also be attacked with RANSAC– need to be able to get
a parameter estimate with a manageably small number of random choices.
– RANSAC is usually better
• Didn’t present in the most general form– in the general form,
the likelihood may not be a linear function of the missing variables
– in this case, one takes an expectation of the likelihood, rather than substituting expected values of missing variables
– Issue doesn’t seem to arise in vision applications.
![Page 91: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/91.jpg)
91
Model Selection
• We wish to choose a model to fit to data– e.g. is it a line or a
circle?– e.g is this a
perspective or orthographic camera?
– e.g. is there an aeroplane there or is it noise?
• Issue– In general, models
with more parameters will fit a dataset better, but are poorer at prediction
– This means we can’t simply look at the negative log-likelihood (or fitting error)
![Page 92: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/92.jpg)
92
Top is not necessarily a betterfit than bottom(actually, almost always worse)
![Page 93: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/93.jpg)
93
![Page 94: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/94.jpg)
94
We can discount the fitting error with some term in the number of parameters in the model.
![Page 95: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/95.jpg)
95
Discounts
• AIC (an information criterion)– choose model with
smallest value of
– p is the number of parameters
• BIC (Bayes information criterion)– choose model with
smallest value of
– N is the number of data points
• Minimum description length– same criterion as
BIC, but derived in a completely different way
−2L D;θ*( )+ p log N
−2L D;θ*( )+ 2 p
![Page 96: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/96.jpg)
96
Example: Dominant planes in USfM(Pollefeys et al., ECCV’2002)
Two or three views only sharing points in a plane cause problem for uncalibrated structure from motion
Solution: – Detect problem through model selection– Delay computation of ambiguous parameters until more information is
available
![Page 97: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/97.jpg)
97
GRIC: Geometric Robust Information Criterion
(Torr et al., ICCV’98)
robust! (otherwise outliers will make you get the most expensive model every time,i.e. the model is never good enough for them)
(2D or 3D)
(6 for points in 3 views)
(11x3-15 (PPP) or 8x2(HH))
(inspired from Kanatani’s GIC)
structure complexity
model complexity
co-dimension
![Page 98: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/98.jpg)
98
F vs. H PPP vs. HH
![Page 99: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/99.jpg)
99
HHgric
PPPgric
![Page 100: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/100.jpg)
100
Farmhouse 3D models
Strategy:– Detect problem using model selection– Reconstruct planes and 3D separately– Perform self-calibration– Assemble reconstruction – compute poses (for planar parts)– Refine complete reconstruction
(bundle adjustment)
![Page 101: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/101.jpg)
102
Cross-validation
• Split data set into two pieces, fit to one, and compute negative log-likelihood on the other
• Average over multiple different splits
• Choose the model with the smallest value of this average
• The difference in averages for two different models is an estimate of the difference in KL divergence of the models from the source of the data
![Page 102: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/102.jpg)
103
Model averaging
• Very often, it is smarter to use multiple models for prediction than just one
• e.g. motion capture data– there are a small
number of schemes that are used to put markers on the body
– given we know the scheme S and the measurements D, we can estimate the configuration of the body X
• We want
• If it is obvious what the scheme is from the data, then averaging makes little difference
• If it isn’t, then not averaging underestimates the variance of X --- we think we have a more precise estimate than we do.
P X | D( )= P X | S1, D( )P S1 | D( )+
P X | S2, D( )P S2 | D( )+
P X | S3, D( )P S3 | D( )
![Page 103: Model fitting - CVG · 3. Radial quadrifocal tensor •Linearly compute radial quadrifocal tensor Q. ijkl. from 15 pts in 4 views •Reconstruct 3D scene and use it for calibration](https://reader033.vdocument.in/reader033/viewer/2022050221/5f6737a193f10f15167b7ad6/html5/thumbnails/103.jpg)
104
Next week:Stereo Matching