![Page 1: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/1.jpg)
Local invariant features
Cordelia Schmid INRIA, Grenoble
![Page 2: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/2.jpg)
Local features
( ) local descriptor
Several / many local descriptors per image Robust to occlusion/clutter + no object segmentation required
Photometric : distinctive Invariant : to image transformations + illumination changes
![Page 3: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/3.jpg)
Local features: interest points
![Page 4: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/4.jpg)
Local features: Contours/segments
![Page 5: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/5.jpg)
Local features: segmentation
![Page 6: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/6.jpg)
Application: Matching
Find corresponding locations in the image
![Page 7: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/7.jpg)
Illustration – Matching
Interest points extracted with Harris detector (~ 500 points)
![Page 8: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/8.jpg)
Matching
Interest points matched based on cross-correlation (188 pairs)
Illustration – Matching
![Page 9: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/9.jpg)
Global constraints
Global constraint - Robust estimation of the fundamental matrix
99 inliers 89 outliers
Illustration – Matching
![Page 10: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/10.jpg)
Application: Panorama stitching
Images courtesy of A. Zisserman.
![Page 11: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/11.jpg)
Application: Instance-level recognition
Search for particular objects and scenes in large databases
…
![Page 12: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/12.jpg)
Finding the object despite possibly large changes in scale, viewpoint, lighting and partial occlusion
requires invariant description
Viewpoint Scale
Lighting Occlusion
Difficulties
![Page 13: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/13.jpg)
Difficulties
• Very large images collection need for efficient indexing
– Flickr has 2 billion photographs, more than 1 million added daily
– Facebook has 15 billion images (~27 million added daily)
– Large personal collections
– Video collections, i.e., YouTube
![Page 14: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/14.jpg)
Search photos on the web for particular places
Find these landmarks ...in these images and 1M more
Applications
![Page 15: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/15.jpg)
Applications
• Take a picture of a product or advertisement find relevant information on the web
[Pixee – Milpix]
![Page 16: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/16.jpg)
Applications
• Finding stolen/missing objects in a large collection …
![Page 17: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/17.jpg)
Applications
• Copy detection for images and videos
Search in 200h of video Query video
![Page 18: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/18.jpg)
Local features - history
• Line segments [Lowe’87, Ayache’90]
• Interest points & cross correlation [Z. Zhang et al. 95]
• Rotation invariance with differential invariants [Schmid&Mohr’96]
• Scale & affine invariant detectors [Lindeberg’98, Lowe’99, Tuytelaars&VanGool’00, Mikolajczyk&Schmid’02, Matas et al.’02]
• Dense detectors and descriptors [Leung&Malik’99, Fei-Fei& Perona’05, Lazebnik et al.’06]
• Contour and region (segmentation) descriptors [Shotton et al.’05, Opelt et al.’06, Ferrari et al.’06, Leordeanu et al.’07]
![Page 19: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/19.jpg)
Example for line segments
images 600 x 600
![Page 20: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/20.jpg)
Example for line segments
248 / 212 line segments extracted
![Page 21: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/21.jpg)
Matched line segments
89 matched line segments - 100% correct
![Page 22: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/22.jpg)
3D reconstruction of line segments
![Page 23: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/23.jpg)
Problems of line segments
• Often only partial extraction – Line segments broken into parts – Missing parts
• Information not very discriminative – 1D information – Similar for many segments
• Potential solutions – Pairs and triplets of segments – Interest points
![Page 24: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/24.jpg)
Overview
• Harris interest points
• Comparing interest points
• Scale & affine invariant interest points
• Evaluation and comparison of different detectors
• Region descriptors and their performance
![Page 25: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/25.jpg)
Harris detector [Harris & Stephens’88]
Based on the idea of auto-correlation
Important difference in all directions => interest point
![Page 26: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/26.jpg)
Images
• We can think of an image as a function, f, from R2 to R: – f( x, y ) gives the intensity at position ( x, y ) – the image is defined over a rectangle with a finite range
• A color image is just three functions pasted together. We can write this as a “vector-valued” function:
![Page 27: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/27.jpg)
Digital images
• In computer vision we operate on digital images: – Sample the 2D space on a regular grid – Quantize each sample (round to nearest integer)
• The image can now be represented as a matrix of integer values (pixels)
6279231191201054010109621278340105819746460048176135518819168049211292637077089144147187102622082552520166123620311666312717109930
![Page 28: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/28.jpg)
Harris detector
Auto-correlation function for a point and a shift
![Page 29: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/29.jpg)
Harris detector
Auto-correlation function for a point and a shift
small in all directions
large in one directions
large in all directions {
→ uniform region → contour → interest point
![Page 30: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/30.jpg)
Harris detector
![Page 31: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/31.jpg)
Harris detector
Discret shifts are avoided based on the auto-correlation matrix
with first order approximation
![Page 32: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/32.jpg)
Harris detector
Auto-correlation matrix
the sum can be smoothed with a Gaussian
![Page 33: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/33.jpg)
Harris detector
• Auto-correlation matrix
– captures the structure of the local neighborhood – measure based on eigenvalues of this matrix
• 2 strong eigenvalues • 1 strong eigenvalue • 0 eigenvalue
=> interest point => contour => uniform region
![Page 34: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/34.jpg)
Interpreting the eigenvalues
λ1
λ2
“Corner” λ1 and λ2 are large,
λ1 ~ λ2; E increases in all
directions
λ1 and λ2 are small; E is almost constant
in all directions “Edge” λ1 >> λ2
“Edge” λ2 >> λ1
“Flat” region
Classification of image points using eigenvalues of M:
![Page 35: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/35.jpg)
Corner response function
“Corner” R > 0
“Edge” R < 0
“Edge” R < 0
“Flat” region
|R| small
α: constant (0.04 to 0.06)
![Page 36: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/36.jpg)
Harris detector
Reduces the effect of a strong contour
• Cornerness function
• Interest point detection – Treshold (absolut, relatif, number of corners) – Local maxima
![Page 37: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/37.jpg)
Harris Detector: Steps
![Page 38: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/38.jpg)
Harris Detector: Steps Compute corner response R
![Page 39: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/39.jpg)
Harris Detector: Steps Find points with large corner response: R>threshold
![Page 40: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/40.jpg)
Harris Detector: Steps Take only the points of local maxima of R
![Page 41: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/41.jpg)
Harris Detector: Steps
![Page 42: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/42.jpg)
Harris detector: Summary of steps
1. Compute Gaussian derivatives at each pixel 2. Compute second moment matrix M in a Gaussian
window around each pixel 3. Compute corner response function R 4. Threshold R 5. Find local maxima of response function (non-maximum
suppression)
![Page 43: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/43.jpg)
Harris - invariance to transformations
• Geometric transformations – translation
– rotation
– similitude (rotation + scale change)
– affine (valide for local planar objects)
• Photometric transformations – Affine intensity changes (I → a I + b)
![Page 44: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/44.jpg)
Harris Detector: Invariance Properties • Rotation
Ellipse rotates but its shape (i.e. eigenvalues) remains the same
Corner response R is invariant to image rotation
![Page 45: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/45.jpg)
Harris Detector: Invariance Properties
• Affine intensity change
Only derivatives are used => invariance to intensity shift I → I + b Intensity scale: I → a I
R
x (image coordinate)
threshold
R
x (image coordinate)
Partially invariant to affine intensity change, dependent on type of threshold
![Page 46: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/46.jpg)
Harris Detector: Invariance Properties
• Scaling
All points will be classified as
edges
Corner
Not invariant to scaling
![Page 47: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/47.jpg)
Overview
• Harris interest points
• Comparing interest points (SSD, ZNCC, Derivatives, SIFT)
• Scale & affine invariant interest points
• Evaluation and comparison of different detectors
• Region descriptors and their performance
![Page 48: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/48.jpg)
Comparison of patches - SSD
image 1 image 2
SSD : sum of square difference
Comparison of the intensities in the neighborhood of two interest points
Small difference values similar patches
![Page 49: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/49.jpg)
Comparison of patches
SSD :
Invariance to photometric transformations?
Intensity changes (I → I + b)
Intensity changes (I → aI + b)
=> Normalizing with the mean of each patch
=> Normalizing with the mean and standard deviation of each patch
![Page 50: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/50.jpg)
Cross-correlation ZNCC
ZNCC: zero normalized cross correlation
zero normalized SSD
ZNCC values between -1 and 1, 1 when identical patches in practice threshold around 0.5
![Page 51: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/51.jpg)
Local descriptors
• Greyvalue derivatives
• Differential invariants [Koenderink’87] – combinations of derivatives
• SIFT descriptor [Lowe’99]
![Page 52: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/52.jpg)
• The gradient points in the direction of most rapid increase in intensity
Greyvalue derivatives: Image gradient
• The gradient of an image:
•
• The gradient direction is given by – how does this relate to the direction of the edge?
• The edge strength is given by the gradient magnitude
Source: Steve Seitz
![Page 53: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/53.jpg)
Differentiation and convolution
• Recall, for 2D function, f(x,y):
• We could approximate this as
-1 1
Source: D. Forsyth, D. Lowe
• Convolution with the filter
![Page 54: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/54.jpg)
Finite difference filters
• Other approximations of derivative filters exist:
Source: K. Grauman
![Page 55: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/55.jpg)
Effects of noise • Consider a single row or column of the image
– Plotting intensity as a function of position gives a signal The image cannot be displayed. Your computer may not have enough memory
• Where is the edge? Source: S. Seitz
![Page 56: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/56.jpg)
Solution: smooth first
• To find edges, look for peaks in
f
g
f * g
Source: S. Seitz
![Page 57: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/57.jpg)
• Differentiation is convolution, and convolution is associative:
• This saves us one operation:
Derivative theorem of convolution
f
Source: S. Seitz
![Page 58: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/58.jpg)
Local descriptors • Greyvalue derivatives
– Convolution with Gaussian derivatives
![Page 59: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/59.jpg)
Gaussian Kernel
• Gaussian filters have infinite support, but discrete filters use finite kernels
• Rule of thumb: set filter half-width to about 3 σ
![Page 60: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/60.jpg)
Local descriptors – rotation invariance
Invariance to image rotation : differential invariants [Koen87]
gradient magnitude
Laplacian
![Page 61: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/61.jpg)
Laplacian of Gaussian (LOG)
![Page 62: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/62.jpg)
SIFT descriptor [Lowe’99]
• Approach – 8 orientations of the gradient – 4x4 spatial grid – soft-assignment to spatial bins, dimension 128 – normalization of the descriptor to norm one – comparison with Euclidean distance
gradient 3D histogram
→ →
image patch
y
x
![Page 63: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/63.jpg)
Local descriptors - rotation invariance
• Estimation of the dominant orientation – extract gradient orientation – histogram over gradient orientation – peak in this histogram
• Rotate patch in dominant direction
0 2 π
![Page 64: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/64.jpg)
Local descriptors – illumination change
in case of an affine transformation
• Normalization of the image patch with mean and variance
• Robustness to illumination changes
![Page 65: Local invariant features · • Evaluation and comparison of different detectors • Region descriptors and their performance . Harris detector [Harris & Stephens’88] Based on the](https://reader033.vdocument.in/reader033/viewer/2022060605/605a46ce5af1c36dd26b5a0b/html5/thumbnails/65.jpg)
Invariance to scale changes
• Scale change between two images
• Scale factor s can be eliminated
• Support region for calculation!! – In case of a convolution with Gaussian derivatives defined by