multi-attribute spaces: calibration for attribute fusion and similarity search university of oxford...
TRANSCRIPT
![Page 1: Multi-Attribute Spaces: Calibration for Attribute Fusion and Similarity Search University of Oxford 5 th December 2012 Walter Scheirer, Neeraj Kumar, Peter](https://reader035.vdocument.in/reader035/viewer/2022081420/551ac021550346b2288b530f/html5/thumbnails/1.jpg)
Multi-Attribute Spaces: Calibration for Attribute Fusion and Similarity Search
University of Oxford 5th December 2012
Walter Scheirer, Neeraj Kumar, Peter N. Belhumeur, Terrance E. Boult,CVPR 2012
![Page 2: Multi-Attribute Spaces: Calibration for Attribute Fusion and Similarity Search University of Oxford 5 th December 2012 Walter Scheirer, Neeraj Kumar, Peter](https://reader035.vdocument.in/reader035/viewer/2022081420/551ac021550346b2288b530f/html5/thumbnails/2.jpg)
Attributes based image description
4-Legged
Orange
Striped
Furry
White
Symmetric
Ionic columns
Classical
Male
Asian
Beard
Smiling
Slide Courtesy: Neeraj Kumar
![Page 3: Multi-Attribute Spaces: Calibration for Attribute Fusion and Similarity Search University of Oxford 5 th December 2012 Walter Scheirer, Neeraj Kumar, Peter](https://reader035.vdocument.in/reader035/viewer/2022081420/551ac021550346b2288b530f/html5/thumbnails/3.jpg)
Attribute Classifiers
Attribute and Simile Classifiers for Face VerificationN. Kumar, A. C. Berg, P. N. Belhumeur, and S. K. Nayar ICCV 2009
FaceTracer: A Search Engine for Large Collections of Images with Faces N. Kumar, P. N. Belhumeur, and S. K. Nayar ICCV 2009
![Page 4: Multi-Attribute Spaces: Calibration for Attribute Fusion and Similarity Search University of Oxford 5 th December 2012 Walter Scheirer, Neeraj Kumar, Peter](https://reader035.vdocument.in/reader035/viewer/2022081420/551ac021550346b2288b530f/html5/thumbnails/4.jpg)
Attributes Fusion
FaceTracer: “smiling asian men with glasses”Slide Courtesy: Neeraj Kumar
![Page 5: Multi-Attribute Spaces: Calibration for Attribute Fusion and Similarity Search University of Oxford 5 th December 2012 Walter Scheirer, Neeraj Kumar, Peter](https://reader035.vdocument.in/reader035/viewer/2022081420/551ac021550346b2288b530f/html5/thumbnails/5.jpg)
Score Normalization: Problem
• Necessary to prevent high confidence for one attribute from dominating the results.
• Ideal normalization technique should,1) Normalize scores to a uniform range say, [0,1]2) Assign perceptual quality to the scores.
• Positive and negative distributions of different classifiers do not necessarily follow same distribution.
• Fitting a Gaussian or any other distribution to scores satisfies condition 1 but doesn’t satisfy condition 2.
Negative Scores Distributions Positive Scores Distributions
![Page 6: Multi-Attribute Spaces: Calibration for Attribute Fusion and Similarity Search University of Oxford 5 th December 2012 Walter Scheirer, Neeraj Kumar, Peter](https://reader035.vdocument.in/reader035/viewer/2022081420/551ac021550346b2288b530f/html5/thumbnails/6.jpg)
Score Normalization: Solution
• Model distance between positive scores and the negative scores .
• If we knew distribution of negative scores, we could do a hypothesis test for each positive score using that distribution.
• Unfortunately, we don’t know anything about overall negative distribution.
But, we know something about tail of the negative score distribution.
![Page 7: Multi-Attribute Spaces: Calibration for Attribute Fusion and Similarity Search University of Oxford 5 th December 2012 Walter Scheirer, Neeraj Kumar, Peter](https://reader035.vdocument.in/reader035/viewer/2022081420/551ac021550346b2288b530f/html5/thumbnails/7.jpg)
Extreme Value Theory
• Central Limit Theorem:
• The “mean” of a sufficiently large iid random variables will be distributed according to Normal distribution
• Extreme Value Theory:
• The maximum of a sufficiently large iid random variable will be distributed according to Gumbell, Frechet or Weibull distribution.
• If the values are bounded from above and below, the the values are distributed according to “Weibull” distribution.
![Page 8: Multi-Attribute Spaces: Calibration for Attribute Fusion and Similarity Search University of Oxford 5 th December 2012 Walter Scheirer, Neeraj Kumar, Peter](https://reader035.vdocument.in/reader035/viewer/2022081420/551ac021550346b2288b530f/html5/thumbnails/8.jpg)
Weibull Distribution
• Weibull Distribution
CDF
k and λ are shape and location parameters respectively.
PDF CDF
![Page 9: Multi-Attribute Spaces: Calibration for Attribute Fusion and Similarity Search University of Oxford 5 th December 2012 Walter Scheirer, Neeraj Kumar, Peter](https://reader035.vdocument.in/reader035/viewer/2022081420/551ac021550346b2288b530f/html5/thumbnails/9.jpg)
Extreme Value Theory: Application
Tail
Overall Negative Score Distribution
Maximum values of random variables
• Tail of negative scores can be seen as a collection of maxima of some random variables. • Hence it follows Weibull distribution according to Extreme Value Theory.
![Page 10: Multi-Attribute Spaces: Calibration for Attribute Fusion and Similarity Search University of Oxford 5 th December 2012 Walter Scheirer, Neeraj Kumar, Peter](https://reader035.vdocument.in/reader035/viewer/2022081420/551ac021550346b2288b530f/html5/thumbnails/10.jpg)
W-score normalization: Procedure
For any classifier,
• Fix the decision boundary on the scores(Ideally this should be at score = 0 )
• Select maximum N (tail size) samples from negative side of the boundary.
• Fit a Weibull Distribution to these tail scores.
• Renormalize scores using Cumulative Density Function (CDF) of this Weibull distribution.
![Page 11: Multi-Attribute Spaces: Calibration for Attribute Fusion and Similarity Search University of Oxford 5 th December 2012 Walter Scheirer, Neeraj Kumar, Peter](https://reader035.vdocument.in/reader035/viewer/2022081420/551ac021550346b2288b530f/html5/thumbnails/11.jpg)
Results: Dataset
• “Labeled Faces In The Wild” dataset.
• About 13,000 images of 5000 celebrities.
• 75 different attribute classification scores available from “Attribute and Simile Classifiers for Face Verification”. Kumar et al. ICCV 09.
Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments.
![Page 12: Multi-Attribute Spaces: Calibration for Attribute Fusion and Similarity Search University of Oxford 5 th December 2012 Walter Scheirer, Neeraj Kumar, Peter](https://reader035.vdocument.in/reader035/viewer/2022081420/551ac021550346b2288b530f/html5/thumbnails/12.jpg)
Results
![Page 13: Multi-Attribute Spaces: Calibration for Attribute Fusion and Similarity Search University of Oxford 5 th December 2012 Walter Scheirer, Neeraj Kumar, Peter](https://reader035.vdocument.in/reader035/viewer/2022081420/551ac021550346b2288b530f/html5/thumbnails/13.jpg)
Multi Attribute Fusion:
• Joint score can be computed as multiplication of individual attribute probabilities.
• Attributes may not be independent.
• Low probability due to:• bad classifier • absence of images belonging to an attribute.
• Instead of product, authors propose use l1 norm of probabilities as a fusion score.
![Page 14: Multi-Attribute Spaces: Calibration for Attribute Fusion and Similarity Search University of Oxford 5 th December 2012 Walter Scheirer, Neeraj Kumar, Peter](https://reader035.vdocument.in/reader035/viewer/2022081420/551ac021550346b2288b530f/html5/thumbnails/14.jpg)
Results
![Page 15: Multi-Attribute Spaces: Calibration for Attribute Fusion and Similarity Search University of Oxford 5 th December 2012 Walter Scheirer, Neeraj Kumar, Peter](https://reader035.vdocument.in/reader035/viewer/2022081420/551ac021550346b2288b530f/html5/thumbnails/15.jpg)
Similarity Search:
• Given an image and a set of attributes, find nearest images.
• Perceived difference between images in different ranges might be similar.
• Distances between query attribute and its nearest neighbor needs to be normalized.
• Normalize query attribute scores on query image.• Get nearest neighbor distances.• Fit Weibull distribution to distances.
![Page 16: Multi-Attribute Spaces: Calibration for Attribute Fusion and Similarity Search University of Oxford 5 th December 2012 Walter Scheirer, Neeraj Kumar, Peter](https://reader035.vdocument.in/reader035/viewer/2022081420/551ac021550346b2288b530f/html5/thumbnails/16.jpg)
Summary
• Provides way of normalizing scores intuitively.
• Provides way for combining attributes.
• Relies on finding the right threshold and tail size. Requires fair bit of tuning.
![Page 17: Multi-Attribute Spaces: Calibration for Attribute Fusion and Similarity Search University of Oxford 5 th December 2012 Walter Scheirer, Neeraj Kumar, Peter](https://reader035.vdocument.in/reader035/viewer/2022081420/551ac021550346b2288b530f/html5/thumbnails/17.jpg)
Questions?