lecture’6:’’...
TRANSCRIPT
![Page 1: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/1.jpg)
Lecture 6 - !!!
Fei-Fei Li!
Lecture 6: Finding Features (part 1/2)
Professor Fei-‐Fei Li Stanford Vision Lab
2-‐Oct-‐14 1
![Page 2: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/2.jpg)
Lecture 6 - !!!
Fei-Fei Li!
What we will learn today?
• Local invariant features – MoHvaHon – Requirements, invariances
• Keypoint localizaHon – Harris corner detector
• Scale invariant region selecHon – AutomaHc scale selecHon – Difference-‐of-‐Gaussian (DoG) detector
• SIFT: an image region descriptor
2-‐Oct-‐14 2
Next lecture (#7)
![Page 3: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/3.jpg)
Lecture 6 - !!!
Fei-Fei Li!
What we will learn today?
• Local invariant features – MoHvaHon – Requirements, invariances
• Keypoint localizaHon – Harris corner detector
• Scale invariant region selecHon – AutomaHc scale selecHon – Difference-‐of-‐Gaussian (DoG) detector
• SIFT: an image region descriptor
2-‐Oct-‐14 3
Some background reading: Rick Szeliski, Chapter 14.1.1; David Lowe, IJCV 2004
![Page 4: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/4.jpg)
Lecture 6 - !!!
Fei-Fei Li!
Image matching: a challenging problem
2-‐Oct-‐14 4
![Page 5: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/5.jpg)
Lecture 6 - !!!
Fei-Fei Li!
by Diva Sian
by swashford
Slid
e cr
edit
: St
eve
Seit
z
2-‐Oct-‐14 5
Image matching: a challenging problem
![Page 6: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/6.jpg)
Lecture 6 - !!!
Fei-Fei Li!
Harder Case
by Diva Sian by scgbt
Slid
e cr
edit
: St
eve
Seit
z
2-‐Oct-‐14 6
![Page 7: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/7.jpg)
Lecture 6 - !!!
Fei-Fei Li!
Harder SHll?
NASA Mars Rover images
Slid
e cr
edit
: St
eve
Seit
z
2-‐Oct-‐14 7
![Page 8: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/8.jpg)
Lecture 6 - !!!
Fei-Fei Li!
Answer Below (Look for Hny colored squares)
NASA Mars Rover images with SIFT feature matches (Figure by Noah Snavely) Sl
ide
cred
it:
Stev
e Se
itz
2-‐Oct-‐14 8
![Page 9: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/9.jpg)
Lecture 6 - !!!
Fei-Fei Li!
MoHvaHon for using local features • Global representaHons have major limitaHons • Instead, describe and match only local regions • Increased robustness to
– Occlusions
– ArHculaHon
– Intra-‐category variaHons
θqφ
dq
φ
θ
d
2-‐Oct-‐14 9
![Page 10: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/10.jpg)
Lecture 6 - !!!
Fei-Fei Li!
General Approach N
pix
els
N pixels
Similarity measure Af
e.g. color
Bf
e.g. color
B1 B2
B3 A1
A2 A3
Tffd BA <),(
1. Find a set of distinctive key- points
3. Extract and normalize the region content
2. Define a region around each keypoint
4. Compute a local descriptor from the normalized region
5. Match local descriptors
Slid
e cr
edit
: Ba
stia
n Le
ibe
2-‐Oct-‐14 10
![Page 11: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/11.jpg)
Lecture 6 - !!!
Fei-Fei Li!
Common Requirements • Problem 1:
– Detect the same point independently in both images
No chance to match!
We need a repeatable detector! Slid
e cr
edit
: D
arya
Fro
lova
, D
enis
Sim
akov
2-‐Oct-‐14 11
![Page 12: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/12.jpg)
Lecture 6 - !!!
Fei-Fei Li!
Common Requirements • Problem 1:
– Detect the same point independently in both images
• Problem 2: – For each point correctly recognize the corresponding one
We need a reliable and distinctive descriptor!
?
Slid
e cr
edit
: D
arya
Fro
lova
, D
enis
Sim
akov
2-‐Oct-‐14 12
![Page 13: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/13.jpg)
Lecture 6 - !!!
Fei-Fei Li!
Invariance: Geometric TransformaHons
Slid
e cr
edit
: St
eve
Seit
z
2-‐Oct-‐14 13
![Page 14: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/14.jpg)
Lecture 6 - !!!
Fei-Fei Li!
Levels of Geometric Invariance
Slid
e cr
edit
: Ba
stia
n Le
ibe
2-‐Oct-‐14 14
CS131 CS231a
![Page 15: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/15.jpg)
Lecture 6 - !!!
Fei-Fei Li!
Invariance: Photometric TransformaHons
• Ofen modeled as a linear transformaHon: – Scaling + Offset
Slid
e cr
edit
: Ti
nne
Tuyt
elaa
rs
2-‐Oct-‐14 15
![Page 16: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/16.jpg)
Lecture 6 - !!!
Fei-Fei Li!
Requirements • Region extracHon needs to be repeatable and accurate
– Invariant to translaHon, rotaHon, scale changes – Robust or covariant to out-‐of-‐plane (≈affine) transformaHons – Robust to lighHng variaHons, noise, blur, quanHzaHon
• Locality: Features are local, therefore robust to occlusion and cluier.
• QuanHty: We need a sufficient number of regions to cover the object.
• DisHncHveness: The regions should contain “interesHng” structure.
• Efficiency: Close to real-‐Hme performance.
Slid
e cr
edit
: Ba
stia
n Le
ibe
2-‐Oct-‐14 16
![Page 17: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/17.jpg)
Lecture 6 - !!!
Fei-Fei Li!
Many ExisHng Detectors Available
• Hessian & Harris [Beaudet ‘78], [Harris ‘88] • Laplacian, DoG [Lindeberg ‘98], [Lowe ‘99] • Harris-‐/Hessian-‐Laplace [Mikolajczyk & Schmid ‘01] • Harris-‐/Hessian-‐Affine [Mikolajczyk & Schmid ‘04] • EBR and IBR [Tuytelaars & Van Gool ‘04] • MSER [Matas ‘02] • Salient Regions [Kadir & Brady ‘01] • Others…
• Those detectors have become a basic building block for many recent applica8ons in Computer Vision.
Slid
e cr
edit
: Ba
stia
n Le
ibe
2-‐Oct-‐14 17
![Page 18: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/18.jpg)
Lecture 6 - !!!
Fei-Fei Li!
Keypoint LocalizaHon
• Goals:
– Repeatable detecHon – Precise localizaHon – InteresHng content
⇒ Look for two-‐dimensional signal changes
Slid
e cr
edit
: Ba
stia
n Le
ibe
2-‐Oct-‐14 18
![Page 19: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/19.jpg)
Lecture 6 - !!!
Fei-Fei Li!
Finding Corners
• Key property: – In the region around a corner, image gradient has two or more dominant direcHons
• Corners are repeatable and dis8nc8ve
C.Harris and M.Stephens. "A Combined Corner and Edge Detector.“ Proceedings of the 4th Alvey Vision Conference, 1988.
Slid
e cr
edit
: Sv
etla
na L
azeb
nik
2-‐Oct-‐14 19
![Page 20: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/20.jpg)
Lecture 6 - !!!
Fei-Fei Li!
Corners as DisHncHve Interest Points • Design criteria
– We should easily recognize the point by looking through a small window (locality)
– Shifing the window in any direc8on should give a large change in intensity (good localiza8on)
“edge”: no change along the edge direction
“corner”: significant change in all directions
“flat” region: no change in all directions Sl
ide
cred
it:
Alyo
sha
Efro
s
2-‐Oct-‐14 20
![Page 21: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/21.jpg)
Lecture 6 - !!!
Fei-Fei Li!
Harris Detector FormulaHon
• Change of intensity for the shif [u,v]: [ ]2
,( , ) ( , ) ( , ) ( , )
x yE u v w x y I x u y v I x y= + + −∑
Intensity Shifted intensity
Window function
or Window function w(x,y) =
Gaussian 1 in window, 0 outside
Slid
e cr
edit
: Ri
ck S
zelis
ki
2-‐Oct-‐14 21
![Page 22: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/22.jpg)
Lecture 6 - !!!
Fei-Fei Li!
Harris Detector FormulaHon • This measure of change can be approximated by:
where M is a 2×2 matrix computed from image derivaHves:
⎥⎦
⎤⎢⎣
⎡≈
vu
MvuvuE ][),(
M
Sum over image region – the area we are checking for corner
Gradient with respect to x, times gradient with respect to y
2
2,( , ) x x y
x y x y y
I I IM w x y
I I I⎡ ⎤
= ⎢ ⎥⎢ ⎥⎣ ⎦
∑
Slid
e cr
edit
: Ri
ck S
zelis
ki
2-‐Oct-‐14 22
![Page 23: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/23.jpg)
Lecture 6 - !!!
Fei-Fei Li!
Harris Detector FormulaHon
Ix Image I IxIy Iy
where M is a 2×2 matrix computed from image derivaHves:
M
Sum over image region – the area we are checking for corner
Gradient with respect to x, times gradient with respect to y
2
2,( , ) x x y
x y x y y
I I IM w x y
I I I⎡ ⎤
= ⎢ ⎥⎢ ⎥⎣ ⎦
∑
Slid
e cr
edit
: Ri
ck S
zelis
ki
2-‐Oct-‐14 23
![Page 24: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/24.jpg)
Lecture 6 - !!!
Fei-Fei Li!
What Does This Matrix Reveal? • First, let’s consider an axis-‐aligned corner:
• This means: – Dominant gradient direcHons align with x or y axis – If either λ is close to 0, then this is not a corner, so look for locaHons where both are large.
• What if we have a corner that is not aligned with the image axes?
⎥⎦
⎤⎢⎣
⎡=
⎥⎥⎦
⎤
⎢⎢⎣
⎡=
∑∑∑∑
2
12
2
00λ
λ
yyx
yxx
IIIIII
M
Slid
e cr
edit
: D
avid
Jac
obs
2-‐Oct-‐14 24
![Page 25: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/25.jpg)
Lecture 6 - !!!
Fei-Fei Li!
What Does This Matrix Reveal? • First, let’s consider an axis-‐aligned corner:
• This means: – Dominant gradient direcHons align with x or y axis – If either λ is close to 0, then this is not a corner, so look for locaHons where both are large.
• What if we have a corner that is not aligned with the image axes?
⎥⎦
⎤⎢⎣
⎡=
⎥⎥⎦
⎤
⎢⎢⎣
⎡=
∑∑∑∑
2
12
2
00λ
λ
yyx
yxx
IIIIII
M
Slid
e cr
edit
: D
avid
Jac
obs
2-‐Oct-‐14 25
![Page 26: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/26.jpg)
Lecture 6 - !!!
Fei-Fei Li!
General Case
• Since M is symmetric, we have
• We can visualize M as an ellipse with axis lengths determined by the eigenvalues and orientaHon determined by R
RRM ⎥⎦
⎤⎢⎣
⎡= −
2
11
00λ
λ
Direction of the slowest change
Direction of the fastest change
(λmax)-1/2
(λmin)-1/2
adap
ted
from
Dar
ya F
rolo
va,
Den
is S
imak
ov
(Eigenvalue decomposition)
2-‐Oct-‐14 26
![Page 27: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/27.jpg)
Lecture 6 - !!!
Fei-Fei Li!
InterpreHng the Eigenvalues • ClassificaHon of image points using eigenvalues of M:
λ1
“Corner” λ1 and λ2 are large, λ1 ~ λ2; E increases in all direcHons
λ1 and λ2 are small; E is almost constant in all direcHons “Edge”
λ1 >> λ2
“Edge” λ2 >> λ1
“Flat” region
Slid
e cr
edit
: Kr
iste
n G
raum
an
λ2
2-‐Oct-‐14 27
![Page 28: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/28.jpg)
Lecture 6 - !!!
Fei-Fei Li!
Corner Response FuncHon
• Fast approximaHon – Avoid compuHng the eigenvalues
– α: constant (0.04 to 0.06)
λ2
“Corner” θ > 0
“Edge” θ < 0
“Edge” θ < 0
“Flat” region
θ = det(M )−α trace(M )2 = λ1λ2 −α(λ1 +λ2 )2
λ1 Slid
e cr
edit
: Kr
iste
n G
raum
an
2-‐Oct-‐14 28
![Page 29: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/29.jpg)
Lecture 6 - !!!
Fei-Fei Li!
Window FuncHon w(x,y)
• OpHon 1: uniform window – Sum over square window
– Problem: not rotaHon invariant
• OpHon 2: Smooth with Gaussian – Gaussian already performs weighted sum
– Result is rotaHon invariant
1 in window, 0 outside
2
2,( , ) x x y
x y x y y
I I IM w x y
I I I⎡ ⎤
= ⎢ ⎥⎢ ⎥⎣ ⎦
∑
Gaussian
2
2,
x x y
x y x y y
I I IM
I I I⎡ ⎤
= ⎢ ⎥⎢ ⎥⎣ ⎦
∑
2
2( ) x x y
x y y
I I IM g
I I Iσ
⎡ ⎤= ∗⎢ ⎥
⎢ ⎥⎣ ⎦
Slid
e cr
edit
: Ba
stia
n Le
ibe
2-‐Oct-‐14 29
![Page 30: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/30.jpg)
Lecture 6 - !!!
Fei-Fei Li!
Summary: Harris Detector [Harris88] • Compute second moment matrix
(autocorrelaHon matrix) 1. Image derivatives
Ix Iy
2. Square of derivatives
Ix2 Iy2 IxIy
3. Gaussian filter g(σI) g(Ix2) g(Iy2) g(IxIy)
R
2
2
( ) ( )( , ) ( )
( ) ( )x D x y D
I D Ix y D y D
I I IM g
I I Iσ σ
σ σ σσ σ
⎡ ⎤= ∗⎢ ⎥
⎢ ⎥⎣ ⎦
2 2 2 2 2 2( ) ( ) [ ( )] [ ( ) ( )]x y x y x yg I g I g I I g I g Iα= − − +
θ = det[M (σ I ,σ D )]−α[trace(M (σ I ,σ D ))]2
4. Cornerness function – two strong eigenvalues
5. Perform non-maximum suppression
Slid
e cr
edit
: Kr
ysti
an M
ikol
ajcz
yk
2-‐Oct-‐14 30
![Page 31: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/31.jpg)
Lecture 6 - !!!
Fei-Fei Li!
Harris Detector: Workflow
Slid
e ad
apte
d fr
om D
arya
Fro
lova
, D
enis
Sim
akov
2-‐Oct-‐14 31
![Page 32: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/32.jpg)
Lecture 6 - !!!
Fei-Fei Li!
Harris Detector: Workflow -‐ computer corner responses θ
Slid
e ad
apte
d fr
om D
arya
Fro
lova
, D
enis
Sim
akov
2-‐Oct-‐14 32
![Page 33: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/33.jpg)
Lecture 6 - !!!
Fei-Fei Li!
Slid
e ad
apte
d fr
om D
arya
Fro
lova
, D
enis
Sim
akov
Harris Detector: Workflow -‐ Take only the local maxima of θ, where θ>threshold
2-‐Oct-‐14 33
![Page 34: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/34.jpg)
Lecture 6 - !!!
Fei-Fei Li!
Harris Detector: Workflow -‐ ResulHng Harris points
Slid
e ad
apte
d fr
om D
arya
Fro
lova
, D
enis
Sim
akov
2-‐Oct-‐14 34
![Page 35: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/35.jpg)
Lecture 6 - !!!
Fei-Fei Li!
Harris Detector – Responses [Harris88]
Effect: A very precise corner detector.
Slid
e cr
edit
: Kr
ysti
an M
ikol
ajcz
yk
2-‐Oct-‐14 35
![Page 36: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/36.jpg)
Lecture 6 - !!!
Fei-Fei Li!
Harris Detector – Responses [Harris88]
Slid
e cr
edit
: Kr
ysti
an M
ikol
ajcz
yk
2-‐Oct-‐14 36
![Page 37: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/37.jpg)
Lecture 6 - !!!
Fei-Fei Li!
Harris Detector – Responses [Harris88]
Slid
e cr
edit
: Kr
iste
n G
raum
an
• Results are well suited for finding stereo correspondences
2-‐Oct-‐14 37
![Page 38: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/38.jpg)
Lecture 6 - !!!
Fei-Fei Li!
Harris Detector: ProperHes
• TranslaHon invariance?
Slid
e cr
edit
: Kr
iste
n G
raum
an
2-‐Oct-‐14 38
![Page 39: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/39.jpg)
Lecture 6 - !!!
Fei-Fei Li!
Harris Detector: ProperHes
• TranslaHon invariance • RotaHon invariance?
Ellipse rotates but its shape (i.e. eigenvalues) remains the same
Corner response θ is invariant to image rotation
Slid
e cr
edit
: Kr
iste
n G
raum
an
2-‐Oct-‐14 39
![Page 40: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/40.jpg)
Lecture 6 - !!!
Fei-Fei Li!
Harris Detector: ProperHes
• TranslaHon invariance • RotaHon invariance • Scale invariance?
Not invariant to image scale!
All points will be classified as edges!
Corner
Slid
e cr
edit
: Kr
iste
n G
raum
an
2-‐Oct-‐14 40
![Page 41: Lecture’6:’’ Finding’Features’(part1/2)vision.stanford.edu/teaching/cs131_fall1415/lectures/Lecture6... · Lecture 6 - !!! Fei-Fei Li! Requirements’ • Region’extracHon’needs’to’be’repeatable’and’accurate’](https://reader034.vdocument.in/reader034/viewer/2022042300/5ecad5cf600730616c1815a4/html5/thumbnails/41.jpg)
Lecture 6 - !!!
Fei-Fei Li!
What we have learned today?
• Local invariant features – MoHvaHon – Requirements, invariances
• Keypoint localizaHon – Harris corner detector
• Scale invariant region selecHon – AutomaHc scale selecHon – Difference-‐of-‐Gaussian (DoG) detector
• SIFT: an image region descriptor
2-‐Oct-‐14 41
Next lecture (#7)
Some background reading: Rick Szeliski, Chapter 14.1.1; David Lowe, IJCV 2004