the blue and green colors are actually the samehays/compvision/lectures/05.pdf · harris corner...
TRANSCRIPT
![Page 1: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/1.jpg)
![Page 2: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/2.jpg)
![Page 3: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/3.jpg)
Recap: Frequency Domain
![Page 4: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/4.jpg)
Fourier Bases
This change of basis is the Fourier Transform
Teases away fast vs. slow changes in the image.
![Page 5: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/5.jpg)
Fourier Bases
in Matlab, check out: imagesc(log(abs(fftshift(fft2(im)))));
![Page 6: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/6.jpg)
Review
1. Match the spatial domain image to the Fourier magnitude image
1 54
A
32
C
B
D
E
![Page 7: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/7.jpg)
Why does a lower resolution image still make sense to us? What do we lose?
Image: http://www.flickr.com/photos/igorms/136916757/
Sampling
![Page 8: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/8.jpg)
Throw away every other row and
column to create a 1/2 size image
Subsampling by a factor of 2
![Page 9: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/9.jpg)
• 1D example (sinewave):
Source: S. Marschner
Aliasing problem
![Page 10: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/10.jpg)
Source: S. Marschner
• 1D example (sinewave):
Aliasing problem
![Page 11: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/11.jpg)
• Sub-sampling may be dangerous….
• Characteristic errors may appear:
– “car wheels rolling the wrong way in movies”
– “Checkerboards disintegrate in ray tracing”
– “Striped shirts look funny on color television”
Source: D. Forsyth
Aliasing problem
![Page 12: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/12.jpg)
Aliasing in video
Slide by Steve Seitz
![Page 13: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/13.jpg)
Source: A. Efros
Aliasing in graphics
![Page 14: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/14.jpg)
Sampling and aliasing
![Page 15: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/15.jpg)
• When sampling a signal at discrete intervals, the sampling frequency must be 2 fmax
• fmax = max frequency of the input signal
• This will allows to reconstruct the original perfectly from the sampled version
good
bad
v v v
Nyquist-Shannon Sampling Theorem
![Page 16: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/16.jpg)
Anti-aliasing
Solutions:
• Sample more often
• Get rid of all frequencies that are greater than half the new sampling frequency
– Will lose information
– But it’s better than aliasing
– Apply a smoothing filter
![Page 17: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/17.jpg)
Algorithm for downsampling by factor of 2
1. Start with image(h, w)
2. Apply low-pass filter
im_blur = imfilter(image, fspecial(‘gaussian’, 7, 1))
3. Sample every other pixel
im_small = im_blur(1:2:end, 1:2:end);
![Page 18: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/18.jpg)
Anti-aliasing
Forsyth and Ponce 2002
![Page 19: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/19.jpg)
Subsampling without pre-filtering
1/4 (2x zoom) 1/8 (4x zoom)1/2
Slide by Steve Seitz
![Page 20: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/20.jpg)
Subsampling with Gaussian pre-filtering
G 1/4 G 1/8Gaussian 1/2
Slide by Steve Seitz
![Page 21: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/21.jpg)
Interest Points and Corners
Computer Vision
James Hays
Slides from Rick Szeliski, Svetlana Lazebnik, Derek Hoiem and Grauman&Leibe 2008 AAAI Tutorial
Read Szeliski 7.1.1 and 7.1.2
![Page 22: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/22.jpg)
Correspondence across views
• Correspondence: matching points, patches, edges, or regions across images
≈
![Page 23: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/23.jpg)
Example: estimating “fundamental matrix” that corresponds two views
Slide from Silvio Savarese
![Page 24: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/24.jpg)
Example: structure from motion
![Page 25: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/25.jpg)
Applications
• Feature points are used for:– Image alignment
– 3D reconstruction
– Motion tracking
– Robot navigation
– Indexing and database retrieval
– Object recognition
![Page 26: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/26.jpg)
Project 2: interest points and local features
• Note: “interest points” = “keypoints”, also sometimes called “features”
![Page 27: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/27.jpg)
This class: interest points
• Suppose you have to click on some point, go away and come back after I deform the image, and click on the same points again.
– Which points would you choose?
original
deformed
![Page 28: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/28.jpg)
Overview of Keypoint Matching
K. Grauman, B. Leibe
AfBf
A1
A2 A3
Tffd BA ),(
1. Find a set of
distinctive key-
points
2. Define a region
around each
keypoint
3. Compute a local
descriptor from the
normalized region
4. Match local
descriptors
![Page 29: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/29.jpg)
Goals for Keypoints
Detect points that are repeatable and distinctive
![Page 30: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/30.jpg)
Invariant Local Features
Image content is transformed into local feature coordinates that are
invariant to translation, rotation, scale, and other imaging parameters
Features Descriptors
![Page 31: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/31.jpg)
Why extract features?
• Motivation: panorama stitching• We have two images – how do we combine them?
![Page 32: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/32.jpg)
Local features: main components
1) Detection: Identify the
interest points
2) Description: Extract vector
feature descriptor
surrounding each interest
point.
3) Matching: Determine
correspondence between
descriptors in two views
],,[ )1()1(
11 dxx x
],,[ )2()2(
12 dxx x
Kristen Grauman
![Page 33: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/33.jpg)
Characteristics of good features
• Repeatability• The same feature can be found in several images despite geometric
and photometric transformations
• Saliency• Each feature is distinctive
• Compactness and efficiency• Many fewer features than image pixels
• Locality• A feature occupies a relatively small area of the image; robust to
clutter and occlusion
![Page 34: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/34.jpg)
Goal: interest operator repeatability
• We want to detect (at least some of) the
same points in both images.
• Yet we have to be able to run the detection
procedure independently per image.
No chance to find true matches!
Kristen Grauman
![Page 35: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/35.jpg)
Goal: descriptor distinctiveness
• We want to be able to reliably determine
which point goes with which.
• Must provide some invariance to geometric
and photometric differences between the two
views.
?
Kristen Grauman
![Page 36: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/36.jpg)
Local features: main components
1) Detection: Identify the
interest points
2) Description:Extract vector
feature descriptor
surrounding each interest
point.
3) Matching: Determine
correspondence between
descriptors in two views
![Page 37: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/37.jpg)
Many Existing Detectors Available
K. Grauman, B. Leibe
Hessian & Harris [Beaudet ‘78], [Harris ‘88]Laplacian, DoG [Lindeberg ‘98], [Lowe 1999]Harris-/Hessian-Laplace [Mikolajczyk & Schmid ‘01]Harris-/Hessian-Affine [Mikolajczyk & Schmid ‘04]EBR and IBR [Tuytelaars & Van Gool ‘04]MSER [Matas ‘02]Salient Regions [Kadir & Brady ‘01] Others…
![Page 38: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/38.jpg)
Corner Detection: Basic Idea
• We should easily recognize the point by looking through a small window
• Shifting a window in any direction should give a large change in intensity
“edge”:
no change along
the edge
direction
“corner”:
significant
change in all
directions
“flat” region:
no change in
all directions
Source: A. Efros
![Page 39: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/39.jpg)
Corner Detection: Mathematics
2
,
( , ) ( , ) ( , ) ( , )x y
E u v w x y I x u y v I x y
Change in appearance of window w(x,y)
for the shift [u,v]:
I(x, y)E(u, v)
E(3,2)
w(x, y)
![Page 40: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/40.jpg)
Corner Detection: Mathematics
2
,
( , ) ( , ) ( , ) ( , )x y
E u v w x y I x u y v I x y
I(x, y)E(u, v)
E(0,0)
w(x, y)
Change in appearance of window w(x,y)
for the shift [u,v]:
![Page 41: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/41.jpg)
Corner Detection: Mathematics
2
,
( , ) ( , ) ( , ) ( , )x y
E u v w x y I x u y v I x y
IntensityShifted intensity
Window function
orWindow function w(x,y) =
Gaussian1 in window, 0 outside
Source: R. Szeliski
Change in appearance of window w(x,y)
for the shift [u,v]:
![Page 42: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/42.jpg)
Corner Detection: Mathematics
2
,
( , ) ( , ) ( , ) ( , )x y
E u v w x y I x u y v I x y
We want to find out how this function behaves for
small shifts
Change in appearance of window w(x,y)
for the shift [u,v]:
E(u, v)
![Page 43: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/43.jpg)
Corner Detection: Mathematics
2
,
( , ) ( , ) ( , ) ( , )x y
E u v w x y I x u y v I x y
We want to find out how this function behaves for
small shifts
Change in appearance of window w(x,y)
for the shift [u,v]:
But this is very slow to compute naively.
O(window_width2 * shift_range2 * image_width2)
O( 112 * 112 * 6002 ) = 5.2 billion of these
14.6 thousand per pixel in your image
![Page 44: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/44.jpg)
Corner Detection: Mathematics
2
,
( , ) ( , ) ( , ) ( , )x y
E u v w x y I x u y v I x y
We want to find out how this function behaves for
small shifts
Change in appearance of window w(x,y)
for the shift [u,v]:
Recall Taylor series expansion. A function f can be
approximated around point a as
![Page 45: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/45.jpg)
Recall: Taylor series expansion
A function f can be approximated as
Approximation of
f(x) = ex
centered at f(0)
![Page 46: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/46.jpg)
Different
derivations
exist.
This is the
textbook
version.
![Page 47: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/47.jpg)
Corner Detection: Mathematics
The quadratic approximation simplifies to
2
2,
( , )x x y
x y x y y
I I IM w x y
I I I
where M is a second moment matrix computed from image
derivatives:
v
uMvuvuE ][),(
M
![Page 48: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/48.jpg)
yyyx
yxxx
IIII
IIIIyxwM ),(
x
II x
y
II y
y
I
x
III yx
Corners as distinctive interest points
2 x 2 matrix of image derivatives (averaged in
neighborhood of a point).
Notation:
![Page 49: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/49.jpg)
The surface E(u,v) is locally approximated by a
quadratic form. Let’s try to understand its shape.
Interpreting the second moment matrix
v
uMvuvuE ][),(
yx yyx
yxx
III
IIIyxwM
,2
2
),(
![Page 50: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/50.jpg)
Consider a horizontal “slice” of E(u, v):
Interpreting the second moment matrix
This is the equation of an ellipse.
const][
v
uMvu
![Page 51: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/51.jpg)
Consider a horizontal “slice” of E(u, v):
Interpreting the second moment matrix
This is the equation of an ellipse.
RRM
2
11
0
0
The axis lengths of the ellipse are determined by the
eigenvalues and the orientation is determined by R
direction of the
slowest change
direction of the
fastest change
(max)-1/2
(min)-1/2
const][
v
uMvu
Diagonalization of M:
![Page 52: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/52.jpg)
Interpreting the eigenvalues
1
2
“Corner”
1 and 2 are large,
1 ~ 2;
E increases in all
directions
1 and 2 are small;
E is almost constant
in all directions
“Edge”
1 >> 2
“Edge”
2 >> 1
“Flat”
region
Classification of image points using eigenvalues
of M:
![Page 53: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/53.jpg)
Corner response function
“Corner”
R > 0
“Edge”
R < 0
“Edge”
R < 0
“Flat”
region
|R| small
2
2121
2 )()(trace)det( MMR
α: constant (0.04 to 0.06)
![Page 54: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/54.jpg)
Harris corner detector
1) Compute M matrix for each image window to
get their cornerness scores.
2) Find points whose surrounding window gave
large corner response (f> threshold)
3) Take the points of local maxima, i.e., perform
non-maximum suppression
C.Harris and M.Stephens. “A Combined Corner and Edge Detector.” Proceedings of the 4th Alvey Vision Conference: pages 147—151, 1988.
![Page 55: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/55.jpg)
Harris Detector [Harris88]
• Second moment matrix
)()(
)()()(),(
2
2
DyDyx
DyxDx
IDIIII
IIIg
64
1. Image
derivatives
2. Square of
derivatives
3. Gaussian
filter g(I)
Ix Iy
Ix2 Iy
2 IxIy
g(Ix2) g(Iy
2) g(IxIy)
222222 )]()([)]([)()( yxyxyx IgIgIIgIgIg
])),([trace()],(det[ 2
DIDIhar
4. Cornerness function – both eigenvalues are strong
har5. Non-maxima suppression
1 2
1 2
det
trace
M
M
(optionally, blur first)
![Page 56: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/56.jpg)
Harris Corners – Why so complicated?
• Can’t we just check for regions with lots of gradients in the x and y directions?
– No! A diagonal line would satisfy that criteria
Current
Window
![Page 57: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/57.jpg)
Harris Detector [Harris88]
• Second moment matrix
)()(
)()()(),(
2
2
DyDyx
DyxDx
IDIIII
IIIg
66
1. Image
derivatives
2. Square of
derivatives
3. Gaussian
filter g(I)
Ix Iy
Ix2 Iy
2 IxIy
g(Ix2) g(Iy
2) g(IxIy)
222222 )]()([)]([)()( yxyxyx IgIgIIgIgIg
])),([trace()],(det[ 2
DIDIhar
4. Cornerness function – both eigenvalues are strong
har5. Non-maxima suppression
1 2
1 2
det
trace
M
M
(optionally, blur first)
![Page 58: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/58.jpg)
Harris Corners – Why so complicated?
• What does the structure matrix look here?
CC
CC
Current
Window
![Page 59: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/59.jpg)
Harris Corners – Why so complicated?
• What does the structure matrix look here?
00
0C
Current
Window
![Page 60: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/60.jpg)
Harris Corners – Why so complicated?
• What does the structure matrix look here?
C
C
0
0
Current
Window
![Page 61: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/61.jpg)
Harris Detector: Steps
![Page 62: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/62.jpg)
Harris Detector: Steps
Compute corner response R
![Page 63: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/63.jpg)
Harris Detector: Steps
Find points with large corner response: R>threshold
![Page 64: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/64.jpg)
Harris Detector: Steps
Take only the points of local maxima of R
![Page 65: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/65.jpg)
Harris Detector: Steps
![Page 66: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/66.jpg)
Invariance and covariance
• We want corner locations to be invariant to photometric
transformations and covariant to geometric transformations
• Invariance: image is transformed and corner locations do not change
• Covariance: if we have two transformed versions of the same image,
features should be detected in corresponding locations
![Page 67: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/67.jpg)
Affine intensity change
• Only derivatives are used =>
invariance to intensity shift I I + b
• Intensity scaling: I a I
R
x (image coordinate)
threshold
R
x (image coordinate)
Partially invariant to affine intensity change
I a I + b
![Page 68: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/68.jpg)
Image translation
• Derivatives and window function are shift-invariant
Corner location is covariant w.r.t. translation
![Page 69: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/69.jpg)
Image rotation
Second moment ellipse rotates but its shape
(i.e. eigenvalues) remains the same
Corner location is covariant w.r.t. rotation
![Page 70: The blue and green colors are actually the samehays/compvision/lectures/05.pdf · Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores](https://reader033.vdocument.in/reader033/viewer/2022053101/605ddb150d8dc5695e02eb98/html5/thumbnails/70.jpg)
Scaling
All points will
be classified
as edges
Corner
Corner location is not covariant to scaling!