feature detection and descriptors charles hatt nisha kiran lulu zhang
TRANSCRIPT
![Page 1: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/1.jpg)
Feature Detection and Descriptors
Charles HattNisha KiranLulu Zhang
![Page 2: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/2.jpg)
Overview
• Background– Motivation– Timeline and related work
• SIFT / SIFT Extensions– PCA – SIFT– GLOH
• DAISY• Performance Evaluation
![Page 3: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/3.jpg)
Scope
• We cover local descriptors• Basic Procedure: – Find patches or key points– Compute a descriptor– Match to other points
• Local vs Global:– Robust to occlusion and clutter– Stable under image transforms
![Page 4: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/4.jpg)
Color Histogram: A Global Descriptor
![Page 5: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/5.jpg)
Motivation
![Page 6: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/6.jpg)
Object Recognition
![Page 7: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/7.jpg)
Robot Self Localization
• DARPA urban challenge, cars can recognize four way stops
![Page 8: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/8.jpg)
Image Retrieval
![Page 9: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/9.jpg)
Image Retrieval
![Page 10: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/10.jpg)
Tracking
![Page 11: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/11.jpg)
Things We Did in Class
• Image stitching• Image alignment
![Page 12: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/12.jpg)
Good Descriptors are Invariant to
![Page 13: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/13.jpg)
Timeline
• Cross correlation• Canny Edge Detector 1986• Harris Corner Detector 1988• Moment Invariants 1991• SIFT 1999• Shape Context 2002• PCA-SIFT 2004• Spin Images 2005• GLOH 2005• Daisy 2008
![Page 14: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/14.jpg)
Cross Correlation
![Page 15: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/15.jpg)
Cross Correlation
![Page 16: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/16.jpg)
Moment Invariants
• a = degreep + q = orderId(x, y) = image gradient in direction d
d = horizontal or vertical• Invariant to convolution, blurring, affine transforms;
can compute any order or degree• Higher order sensitive to small photometric distortions
![Page 17: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/17.jpg)
Spin Images (Johnson 97)
![Page 18: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/18.jpg)
Spin Images (Johnson 97)
![Page 19: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/19.jpg)
Spin Images (Lazebnik 05)
• Normalized patch implies invariant to intensity changes; invariant to rotation.
• Usually there are 10 bins for intensity, 5 bins for distance from center in the histogram.
• Descriptor is 50 elements.
![Page 20: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/20.jpg)
Shape Context
![Page 21: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/21.jpg)
Scale Invariant Features
![Page 22: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/22.jpg)
Characteristic of good features• Repeatability– The same feature can be found in several images
despite the geometric and photometric transformation
• Saliency– Each feature has a distinctive description
• Compactness and efficiency– Many fewer features than image pixels
• Locality– Features occupy very small area of the image, robust
to clutter and occlusion
![Page 23: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/23.jpg)
Good features - Corners in image
• Harris corner detector • Key idea: in the region around the corner, image
gradient has two or more dominant directions• Invariant to – Rotation– Partially to affine intensity change
• I = I + b (Invariant) – Only derivatives are used• I = a*I (Not in this case)
– Not invariant to scale
![Page 24: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/24.jpg)
Not invariant to scale
All points will be classified as edges
Corner !
![Page 25: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/25.jpg)
Scale Invariant Detection
• Consider regions (e.g. circles) of different sizes around a point
• Regions of corresponding sizes will look the same in both images
![Page 26: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/26.jpg)
Scale invariant feature detection
• Goal: independently detect corresponding regions in scaled versions of the same image
• Need scale selection mechanism for finding characteristic region size that is covariant with the image transformation
![Page 27: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/27.jpg)
Recall: Edge detection
• Convolution with derivative of Gaussian => Edge at maximum of derivative
• Convolution with second derivative of Gaussian => Edge at zero crossing
![Page 28: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/28.jpg)
f
dg/dx
f*dg/dx
Edge
Derivative ofGaussian
Edge = maximumof derivative
![Page 29: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/29.jpg)
f Edge
Second derivative of Gaussian(Laplacian)
Edge = Zero crossingof second derivative
![Page 30: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/30.jpg)
Scale selection
• Define the characteristic scale as the scale that produces peak of Laplacian response
![Page 31: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/31.jpg)
![Page 32: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/32.jpg)
SIFT stages
• Scale space extrema detection• Keypoint localization• Orientation assignment• Keypoint descriptor
![Page 33: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/33.jpg)
Scale space extrema detection• Approximate Laplacian of Gaussian with Difference of
Gaussian – Computationally less intensive– Invariant to scale
• Images of the same size(vertical) form an octave. Each octave have certain level of blurred images.
![Page 34: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/34.jpg)
![Page 35: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/35.jpg)
Maxima/Minima selection in DoG
SIFT: Find the local maxima of difference of Gaussian in space and scale
![Page 36: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/36.jpg)
Keypoint localization
• Lot of keypoints detected• Sub pixel localization: Accurate location of
keypoints • Eliminating points with low contrast• Eliminating edge responses
![Page 37: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/37.jpg)
Sub pixel localization
![Page 38: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/38.jpg)
Eliminating extra keypoints
• If the magnitude of intensity at the current pixel in the DoG image (that is being checked for maxima/minima) is less than a certain value, it is rejected.
• Removing edges – Idea similar to Harris corner detector
![Page 39: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/39.jpg)
• Until now, we have seen scale invariance• Now, let’s make the keypoint rotation
invariant
![Page 40: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/40.jpg)
Orientation assignment
• Key idea: Collect gradient directions and magnitudes around each keypoint. Then figure out the most prominent orientations in that region. Assign these orientations to the keypoint
• Size of the orientation collection region depends on the scale. Bigger the scale, bigger the collection region.
![Page 41: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/41.jpg)
• Compute gradient magnitude and orientations for each pixel and then construct a histogram
• Peak of the histogram taken as the keypoint orientation
![Page 42: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/42.jpg)
Keypoint descriptor
![Page 43: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/43.jpg)
• Based on 16*16 patches• 4*4 subregions• 8 bins in each subregion• 4*4*8=128 dimensions in total
![Page 44: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/44.jpg)
PCA-SIFT
• PCA-SIFT is a modification of SIFT, which changes how the keypoint descriptors are constructed
• Basic Idea: Use PCA(Principal Component Analysis) to represent the gradient patch around the keypoint
• PCA stages– Computing projection matrix– Constructing PCA-SIFT descriptor
![Page 45: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/45.jpg)
Computing projection matrix
• Select a representative set of pictures and detect all keypoints in these pictures
• For each keypoint– Extract an image patch around it with size 41*41 pixels– Calculate horizontal and vertical gradients, resulting in
a vector of size 39*39*2 = 3042• Put all these vectors into a k*3042 matrix A where
k is the number of keypoints detected• Calculate the covariance matrix of A
![Page 46: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/46.jpg)
Contd..
• Compute the eigenvectors and the eigenvalues of cov A.
• Select the first n eigenvectors; the projection matrix is a n*3042 matrix composed of these eigenvectors
• The projection matrix is only computed once and saved.
![Page 47: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/47.jpg)
Dimension reduction through PCA
The image patches do not span the entire space of pixel values, and also not the Smaller space of patches from natural images. They consist of highly restricted set of patches that passed the first three stages of SIFT.
![Page 48: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/48.jpg)
Constructing PCA-SIFT descriptor
• Input: location of keypoint, scale, orientation.• Extract 41*41 patch around the keypoint at
the given scale, rotated to its orientation• Calculate 39*39 horizontal and vertical
gradients, resulting in a vector of size 3042• Multiply this vector using the precomputed
n*3042 projection matrix• This results in a PCA-SIFT descriptor of size n
![Page 49: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/49.jpg)
![Page 50: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/50.jpg)
Eigenspace construction
![Page 51: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/51.jpg)
Effect of PCA dimension
Hypothesis:First several components of the PCA subspace are sufficient for encoding variations caused by keypoint identity, while the later components represent details that are not useful, of potentially detrimental, such as distortion from projective warp
![Page 52: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/52.jpg)
Gradient Location Orientation Histogram
• Another SIFT – Extension• Gradients quantized into 16 bins• Log Polar location grid – 3 bins for radius: 6, 11, 15– 8 bins for direction: 0, π/4, π/2, … 7π/4
![Page 53: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/53.jpg)
GLOH
17 location bins, 16 gradient bins per location bin272 elements -> down to 128 with PCA
![Page 54: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/54.jpg)
GLOH Results
192 correct, 208 false positive… Not as bad as it sounds
![Page 55: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/55.jpg)
DAISY
• An efficient dense local descriptor• Similar to SIFT and GLOH – Descriptor is fundamentally based on pixel
gradient histograms– Has key differences that make it much faster for
dense matching.• Original application: Wide-baseline stereo• Other applications: Face Recognition
![Page 56: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/56.jpg)
SIFT GLOH*
+ Good Performance+ Better Localization- Not suitable for dense computation
+ Good Performance- Not suitable for dense computation
* K. Mikolajczyk and C. Schmid. A Performance Evaluation of Local Descriptors. PAMI’04.
![Page 57: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/57.jpg)
DAISY
+ Suitable for dense computation + Improved performance:*
+ Precise localization+ Rotational Robustness
![Page 58: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/58.jpg)
DAISY
• Parameters:
![Page 59: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/59.jpg)
![Page 60: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/60.jpg)
Computation Steps
• Compute H (number of histogram bins) orientation maps, G_i (0 < i < H), one for each gradient orientation
• G_o(u,v) = Gradient norm in direction o, at pixel u,v. If gradient norm is < 0, G_o(u,v) = 0
![Page 61: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/61.jpg)
Computation Steps
• Each orientation map is then repeatedly convolved with a Gaussian kernel to obtain “convolved” orientation maps.
![Page 62: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/62.jpg)
Computation Steps
• There are a total of Q (the number of ‘rings’ in the DAISY) levels of convolution.
![Page 63: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/63.jpg)
Computation Steps
• Each pixel now has Q vectors, each H long, of the form:
• Each of these vectors in normalized to unit norm, which helps preserve viewpoint invariance.
![Page 64: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/64.jpg)
• Full Descriptor• Total Size– Q*T*H + H– In this case, 200
![Page 65: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/65.jpg)
Computational Complexity
Configuration: H=8, T=8, Q=3S=25
122 Multiplications/pixel119 Summations/pixel25 Sampling/pixel
Configuration: 16 4x4 arrays, 8 bins.
1280 Multiplications/pixel512 Summations/pixel256 Sampling/pixel
DAISY SIFT
![Page 66: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/66.jpg)
DAISY vs SIFT• Computation Time:
• Reasons:– DAISY descriptors share histograms.– Computation pipeline enables efficient memory
access pattern and histogram layers are separated early • Easily parallelized
![Page 67: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/67.jpg)
Performance with parallel cores
• Computation time falls almost linearly.
![Page 68: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/68.jpg)
Choosing the Best DAISY
• Winder et al.• Tested a wide variety of gradient and steerable
filter based configurations for calculating image gradients at each pixel and found the best parameters for each.
• Found best configurations for different applications.
![Page 69: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/69.jpg)
Choosing the Best DAISY
• Real-time applications: – DAISY configuration:• 1 or 2 rings• 4 bins
– Rectification of image gradients to length one, no use of PCA and quantization of histogram values to a bit depth of 2-3.
![Page 70: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/70.jpg)
Choosing the Best DAISY
• Applications requiring good discrimination: – 2nd order steerable filters at two spatial scales– Application of PCA
• Large-database applications (low storage requirements and computational burden)– Steerable filters with H=4 histogram bins, Q=2
rings, T= 8 segments– Rectified gradients with 4 histogram bins, Q=1 ring
and T= 8 segments
![Page 71: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/71.jpg)
Reported Applications of DAISY
• Wide-baseline stereo• Face recognition
![Page 72: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/72.jpg)
Depth Map Estimation
• DAISY descriptors used to measure similarities across images
• Graph cut based reconstruction algorithm used to generate maps.
• Occlusion masks are used to properly deal with occlusions
![Page 73: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/73.jpg)
Occlusion maps
![Page 74: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/74.jpg)
Depth Map Accuracy
• Ground truth – Laser scan
![Page 75: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/75.jpg)
Depth Map Results
![Page 76: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/76.jpg)
Face Recognition
• Dense descriptor computation is necessary for recognizing faces due to wide baseline nature of facial images.– DAISY descriptors calculated and matched using
recursive grid search– Matches distances are vectorized and input to a
Support Vector Machine (SVM)
![Page 77: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/77.jpg)
![Page 78: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/78.jpg)
Recognition Rate compared to previous, similar methods
Olivetti Research Lab Database
FERET Database
FERET Fafb – Varying facial expressions
FERET Fafb – Varying illumination
![Page 79: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/79.jpg)
Local Descriptor Matching
• Methods for matching descriptor vectors– Exhaustive Search– Recursive Grid Search– KD trees
![Page 80: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/80.jpg)
Recursive Grid Search• Finds the local descriptor for each section of the
template image in a grid (DT).• Find the local descriptor for the corresponding
section in the query image (DQ).• Distance is computed between DT and DQ, as well as
the descriptors of DQ’s neighbors at a distance d.• Point showing minimum distance (DT2) is consider
for further analysis.• Descriptors Neighbors for DT2, at a distance, d/2, are
calculated…
![Page 81: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/81.jpg)
Recursive Grid Search
![Page 82: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/82.jpg)
KD Trees
• Search for nearest neighbor of an n-dimensional point.
• Guaranteed to be log2 (n) depth• Has been shown to run in O(log n) average
time.• Pre-processing time is O(n log n)
![Page 83: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/83.jpg)
KD Trees
![Page 84: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/84.jpg)
KD Trees
![Page 85: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/85.jpg)
KD Trees
![Page 86: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/86.jpg)
KD Trees
![Page 87: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/87.jpg)
KD Trees
![Page 88: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/88.jpg)
KD Trees
![Page 89: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/89.jpg)
KD Trees
![Page 90: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/90.jpg)
KD Trees
![Page 91: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/91.jpg)
KD Trees
![Page 92: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/92.jpg)
KD Trees
![Page 93: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/93.jpg)
KD Trees
![Page 94: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/94.jpg)
KD Trees
![Page 95: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/95.jpg)
Performance Evaluation
• Mikolajczyk Schmid 2005• Detecting • Normalizing • Describing• Matching• Graphing
![Page 96: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/96.jpg)
Detecting
• 10 descriptors will be tested, but first what will they be tested on
• Harris points• Harris Laplace• Hessian Laplace• Harris Affine• Hessian Affine
![Page 97: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/97.jpg)
Normalizing
• With respect to size: 41 pixels• Orientation: Dominant gradient• Illumination: normalize standard deviation and
mean of pixel intensities
![Page 98: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/98.jpg)
![Page 99: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/99.jpg)
Matching
• For histogram based methods, Euclidean distance
• For non-histogram based methods, Mahalanobis distance (S = covariance matrix)
• After distance is calculated, two regions match if D < threshold
• Nearest neighbor threshold
![Page 100: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/100.jpg)
Data Set
• Original image is subjected to …• Rotations: 30-45 deg around optical axis• Scale: camera zoom 2 – 2.5x• Blur: defocusing• Viewpoint: frontal to foreshortened• Light: aperture varied• Compression: JPEG at 5% quality
![Page 101: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/101.jpg)
Evaluation Criteria
• For each patch, compute distance; does d<t?• Compare to ground truth• Count number of correct and false matches• Recall vs 1-Precision graphs
• To build curves, change t and repeat. Now you can use recall and 1-precision to build graphs.
![Page 102: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/102.jpg)
Notes
• If recall = 1 for any precision, we have a perfect descriptor
• Slowly increasing curve => descriptor is affected by the type of noise or transformation we applied to it
• Generally, if the curve for one type of descriptor is higher than the other, it is more robust to that type of transformation
![Page 103: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/103.jpg)
Hessian-Affine detector on Structured Scene
![Page 104: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/104.jpg)
![Page 105: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/105.jpg)
![Page 106: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/106.jpg)
![Page 107: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/107.jpg)
Hessian Laplace Regions
![Page 108: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/108.jpg)
![Page 109: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/109.jpg)
![Page 110: Feature Detection and Descriptors Charles Hatt Nisha Kiran Lulu Zhang](https://reader035.vdocument.in/reader035/viewer/2022062309/56649eb15503460f94bb7614/html5/thumbnails/110.jpg)
Conclusion
• Feature detection and descriptors through the ages