Download - 3D Vision
![Page 1: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/1.jpg)
3D VISION
Interest Points
![Page 2: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/2.jpg)
correspondence and alignment Correspondence: matching points,
patches, edges, or regions across images
≈
![Page 3: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/3.jpg)
correspondence and alignment Alignment: solving the transformation
that makes two things match better
![Page 4: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/4.jpg)
Example: estimating “fundamental matrix” that corresponds two views
![Page 5: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/5.jpg)
Example: tracking points
frame 0 frame 22 frame 49
x xx
Your problem 1 for HW 2!
![Page 6: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/6.jpg)
Human eye movements
![Page 7: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/7.jpg)
Human eye movements
![Page 8: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/8.jpg)
Interest points
Suppose you have to click on some point, go away and come back after I deform the image, and click on the same points again. Which points would
you choose?
original
deformed
![Page 9: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/9.jpg)
Overview of Keypoint Matching
Af Bf
B1
B2
B3A1
A2 A3
Tffd BA ),(
1. Find a set of distinctive key-
points
3. Extract and normalize the region content
2. Define a region around each
keypoint
4. Compute a local descriptor from the normalized region
5. Match local descriptors
![Page 10: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/10.jpg)
Goals for Keypoints
Detect points that are repeatable and distinctive
![Page 11: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/11.jpg)
Choosing interest points
Where would you tell your friend to meet you?
![Page 12: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/12.jpg)
Moravec corner detector (1980)
We should easily recognize the point by looking through a small window
Shifting a window in any direction should give a large change in intensity
![Page 13: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/13.jpg)
Moravec corner detector
flat
![Page 14: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/14.jpg)
Moravec corner detector
flat
![Page 15: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/15.jpg)
Moravec corner detector
flat edge
![Page 16: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/16.jpg)
Moravec corner detector
flat edgecorner
isolated point
![Page 17: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/17.jpg)
Moravec corner detectorChange of intensity for the shift [u,v]:
2
,
( , ) ( , ) ( , ) ( , )x y
E u v w x y I x u y v I x y
IntensityShifted intensity
Window function
Four shifts: (u,v) = (1,0), (1,1), (0,1), (-1, 1)Look for local maxima in min{E}
![Page 18: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/18.jpg)
Problems of Moravec detector
Noisy response due to a binary window function Only a set of shifts at every 45 degree is considered Responds too strong for edges because only
minimum of E is taken into account
Harris corner detector (1988) solves these problems.
![Page 19: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/19.jpg)
Harris corner detector
Noisy response due to a binary window function Use a Gaussian function
![Page 20: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/20.jpg)
Harris corner detectorOnly a set of shifts at every 45 degree is considered Consider all small shifts by Taylor’s expansion
yxyx
yxy
yxx
yxIyxIyxwC
yxIyxwB
yxIyxwA
BvCuvAuvuE
,
,
2
,
2
22
),(),(),(
),(),(
),(),(
2),(
![Page 21: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/21.jpg)
Harris corner detector
( , ) ,u
E u v u v Mv
Equivalently, for small shifts [u,v] we have a bilinear approximation:
2
2,
( , ) x x y
x y x y y
I I IM w x y
I I I
, where M is a 22 matrix computed from image derivatives:
![Page 22: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/22.jpg)
Harris corner detector
Responds too strong for edges because only minimum of E is taken into accountA new corner measurement
![Page 23: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/23.jpg)
Harris corner detector
( , ) ,u
E u v u v Mv
Intensity change in shifting window: eigenvalue analysis
1, 2 – eigenvalues of M
direction of the slowest
change
direction of the fastest
change
(max)-1/2
(min)-1/2
Ellipse E(u,v) = const
![Page 24: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/24.jpg)
Harris corner detector
1
2
Corner1 and 2 are large,
1 ~ 2;
E increases in all directions
1 and 2 are small;
E is almost constant in all directions
edge 1 >> 2
edge 2 >> 1
flat
Classification of image points using eigenvalues of M:
![Page 25: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/25.jpg)
Harris corner detector
Measure of corner response:
2det traceR M k M
1 2
1 2
det
trace
M
M
(k – empirical constant, k = 0.04-0.06)
![Page 26: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/26.jpg)
Another view
![Page 27: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/27.jpg)
Another view
![Page 28: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/28.jpg)
Another view
![Page 29: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/29.jpg)
Harris corner detector (input)
![Page 30: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/30.jpg)
Corner response R
![Page 31: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/31.jpg)
Threshold on R
![Page 32: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/32.jpg)
Local maximum of R
![Page 33: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/33.jpg)
Harris corner detector
![Page 34: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/34.jpg)
Harris Detector: Summary Average intensity change in direction [u,v] can
be expressed as a bilinear form:
Describe a point in terms of eigenvalues of M:measure of corner response
A good (corner) point should have a large intensity change in all directions, i.e. R should be large positive
( , ) ,u
E u v u v Mv
2
1 2 1 2R k
![Page 35: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/35.jpg)
Harris Detector: Some Properties Partial invariance to affine intensity change
Only derivatives are used => invariance to intensity shift I I + b Intensity scale: I a I
R
x (image coordinate)
threshold
R
x (image coordinate)
![Page 36: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/36.jpg)
Harris Detector: Some Properties Rotation invariance
Ellipse rotates but its shape (i.e. eigenvalues) remains the same
Corner response R is invariant to image rotation
![Page 37: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/37.jpg)
Harris Detector is rotation invariant
Repeatability rate:# correspondences
# possible correspondences
![Page 38: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/38.jpg)
Harris Detector: Some Properties
But: non-invariant to image scale!
All points will be classified as edges
Corner !
![Page 39: 3D Vision](https://reader036.vdocument.in/reader036/viewer/2022062517/56812dd0550346895d9311b6/html5/thumbnails/39.jpg)
Harris Detector: Some Properties Quality of Harris detector for different
scale changes
Repeatability rate:# correspondences
# possible correspondences