similarity measures spring 2009 ben-gurion university of the negev
Post on 17-Jan-2018
221 Views
Preview:
DESCRIPTION
TRANSCRIPT
Similarity Measures
Spring 2009
Ben-Gurion University of the Negev
Sensor Fusion Spring 2009
Instructor
• Dr. H. B Mitchell email: harveymitchell@walla.co.il
Sensor Fusion Spring 2009
Similarity Measures
Fundamental importance in all fusion Conceptually consists of two parts: Transformation T. This extracts characteristics from input image and
represents it as a feature vector Distance D. This quantifies similarity of feature vectors. Symbolically:
Sensor Fusion Spring 2009
Similarity Measures. Example
Content Based Information Retrieval (CBIR) Input: Database containing many images Requirement: Retrieve images which are similar to test image Transformation T: Represent each image in database as multi-
dimensional feature vector extracted using low-level descriptors: co-occurrence matrix, color histogram, wavelets, moments etc
Distance D: Euclidean, Manhattan, Minkowski, Hausdorff distances
Sensor Fusion Spring 2009
Similarity Measure. Shape Context
In matching shapes introduced the shape context for each point on the contour C:
which is a two-dimensional histogram We measure the similarity between two shapes with shapes contexts and using the test.
Sensor Fusion Spring 2009
Similarity Measures Metric
A similarity measure is a metric if it obeys following:
Experiments show that valid similarity measures generally obey the first three requirements but not the triangle inequality.
Similarity measures therefore not metric
Sensor Fusion Spring 2009
Similarity Measures: Global vs Local
Can classify similarity measures as global or local: Global measures act on the entire images and return a single scalar
value Local measures act on small patches. They return a local similarity
map. We can convert global measures into local measures and vice
versa.
Sensor Fusion Spring 2009
Similarity Measures: Global vs Local
Example. Global similarity measure can be used as a local measure:
(m,n)
Local Window W(m,n)
Sensor Fusion Spring 2009
Similarity Measures: Global vs Local
Example. Local similarity measure can be used as a global measure:
Then we aggregate the local similarity map to obtain a global measure e.g.
(m,n)
Local Window W(m,n)
Sensor Fusion Spring 2009
Global Measures Without Spatial Alignment
These similarity measures are very robust but tend to have low discriminative power.
Examples are the probabilistic measures: Chernoff Bhattacharyya Jeffrey’s-Matusita Kullback-Leibler Symmetric Kullback-Leibler
The measures may be applied locally. In this case typical window size is 20x20. This is needed to ensure we have sufficient pixels to calculate the probability densities p(x) and q(x).
Sensor Fusion Spring 2009
Distance.
If we represent the distributions p(a) and q(b) as discrete histograms we may use the test. This test assumes common bins.
If two images A and B have histograms
Then test is
Problems. (1) Optimal value for K. Rule of thumb says no bin should contain less than 5 samples. (2) No cross-bin correlations allowed
Sensor Fusion Spring 2009
Distance. Example
The test is simple and widely used similarity measure. In shape matching we have two closed contours c and C. Each point on c is characterized by a 2D histogram
and each point on C is characterized by If is associated with the point , then the match between the
two histograms is
We may define the overall similarity as
We may also impose an order constraint on : If then we require
Sensor Fusion Spring 2009
Optimal Equal-Sized Histogram Bins.
A recent suggestion is the following empirical formula for the optimum number of equal-sized bins in interval [0,1]:
where H(l) is number of samples which fall in kth bin N is the total number of samples k is the number of equal spaced bins
Sensor Fusion Spring 2009
Earth Mover’s Distance.
Earth Movers Distance allows cross-bin associations Given two histograms and Treat P as “supplies” where = amount of kth supply Treat Q as “demands” where = amount of lth demand EMD is defined as minimum normalized work required to transform
P into Q: where is the amount transferred from to . is work involved in transferring one unit from to
Project: Circular Earth Mover’s Distance Project: Pele-Werman EMD variant (A linear time histogram materic
for improved SIFT matching. Pele and Werman ECCV 2008)
Sensor Fusion Spring 2009
Earth Mover’s Distance
If the histograms P and Q are normalized then EMD is Mallows distance. If the histograms each have N bins then
where the min is taken over all permutations of In one dimension this reduces to
Example.
Not normalized. Each box is a unit. EMD=0
Normalized. Each box is probability=0.25. For p=1, EMD=0.5
Sensor Fusion Spring 2009
Earth Mover’s Distance
Recent development in EMD is A linear time histogram metric for improved SIFT matching by Pele and Werman ECCV 2008 Project
Sensor Fusion Spring 2009
Color-based Image Retrieval
Jeffrey divergence
Quadratic form distance
Earth Mover Distance
χ2 statistics
L1 distance
Sensor Fusion Spring 2009
Global Measure With Spatial Alignment
Tend to have higher discriminative power but require accurate registration.
The measures tend to give lower similarity values as mis-registration increases. They are therefore used in spatial alignment algorithms.
Simple measures are mse mae Correlation coefficient
Mutual Information
Sensor Fusion Spring 2009
Ordinal Measures
The mse, mae, correlation coefficient are sensitive to illumination changes. Mutual information and other ordinal measures are designed to overcome this sensitivity by using ordered gray-levels.
Ordinal measures are insensitive to illumination changes if order of the gray-levels is maintained.
Two classical measures are Spearman Kendall
Experiments on stereo matching showed Kendall measure gave best results from all ordinal measures.
Sensor Fusion Spring 2009
Ordinal Measures
Kemeny-Snell distance comapres the realtive ranking of each ordered pair of locations in one image with its relative ranking in the second image. Smaller values of d indicate more agreement.
Luo et al. suggests the measure is very powerful. However computational complexity is very high.
Sensor Fusion Spring 2009
Local Ordinal Measures. Bhat-Nayar
We may apply the Spearman, Kendall and Kemeny-Snell distances locally. However recommendation is the local windows measure at least 20x20.
Bhat-Nayar is a powerful specially designed local ordinal operator. The window size in Bhat-Nayar is from 3x3 to 13x13.
Bhat-Nayar uses a local window containing K gray-levels in A and B with rank vectors
Bhat-Nayar creates a composite rank vector by ranking wrt where and
Sensor Fusion Spring 2009
Local Ordinal Measures. Bhat-Nayar Bhat-Nayar creates a composite rank vector by ranking wrt where and Example. A: [10 20 30 50 40 70 60 90 80] B: [90 60 70 50 40 80 10 30 20] rA: [1 2 3 5 4 7 6 9 8] rB: [9 6 7 5 4 8 1 3 2] s : [9 6 7 4 5 1 8 2 3] Consider k=6. . Therefore Thus . Therefore
Sensor Fusion Spring 2009
Local Ordinal Measures. Bhat-Nayar
Bhat-Nayar assumes all ranks are unique. How can we handle ties? Project: Use fuzzy ranks in the Bhat-Nayar scheme.
Sensor Fusion Spring 2009
Ordinal Measures. Mittal-Ramesh
The ordinal similarity measures (spearman, kendall, Kemeny-Snell and Bhat-Nayar) are robust to changes in illumination but not to Gaussian noise.
Small amounts of Gaussian noise can completely change the rankings between pixels that are not far from each other in gray-level.
Mittal-Ramesh measure is an ordinal measure which also takes into accunt the pixel gray-levels. In general this should give best results. However the Mittal-Ramesh operator has very high computational complexity.
Sensor Fusion Spring 2009
Example
Comparative results from four algorithms: Normalized Cross-Correlation
Census Algorithm Bhat-Nayar Algorithm
Mittal-Ramesh Algorithm
Sensor Fusion Spring 2009
Binary Image Measures
Special measures are used for binary images Following is a local binary measure. Given two binary images A and B the corresponding distance transforms are:
where
Local distance map is
0 0 1 0 0 1 1 0 0
Distance transform
Sensor Fusion Spring 2009
Binary Image Measures
May convert the local similarity map L(m,n) into a global measure. This is the Hausdorff distance:
Hausdorff distance is very sensitive to noise. Robust alternatives are:
top related