Multi-Attribute Spaces: Calibration for Attribute Fusion and Similarity Search Walter Scheirer, Neeraj Kumar, Peter N. Belhumeur, Terrance E. Boult, CVPR 2012 University of Oxford 5th December 2012 Attributes based image description 4-Legged White Male Orange Symmetric Asian Striped Ionic columns Beard Furry Classical Smiling Slide Courtesy: Neeraj Kumar Attribute Classifiers Attribute and Simile Classifiers for Face Verification N. Kumar, A. C. Berg, P. N. Belhumeur, and S. K. Nayar ICCV 2009 FaceTracer: A Search Engine for Large Collections of Images with Faces N. Kumar, P. N. Belhumeur, and S. K. Nayar ICCV 2009 Attributes Fusion FaceTracer: “smiling asian men with glasses” Slide Courtesy: Neeraj Kumar Score Normalization: Problem • Necessary to prevent high confidence for one attribute from dominating the results. • Ideal normalization technique should, 1) Normalize scores to a uniform range say, [0,1] 2) Assign perceptual quality to the scores. • Positive and negative distributions of different classifiers do not necessarily follow same distribution. • Fitting a Gaussian or any other distribution to scores satisfies condition 1 but doesn’t satisfy condition 2. Negative Scores Distributions Positive Scores Distributions Score Normalization: Solution • Model distance between positive scores and the negative scores . • If we knew distribution of negative scores, we could do a hypothesis test for each positive score using that distribution. • Unfortunately, we don’t know anything about overall negative distribution. But, we know something about tail of the negative score distribution. Extreme Value Theory • Central Limit Theorem: • The “mean” of a sufficiently large iid random variables will be distributed according to Normal distribution • Extreme Value Theory: • The maximum of a sufficiently large iid random variable will be distributed according to Gumbell, Frechet or Weibull distribution. • If the values are bounded from above and below, the the values are distributed according to “Weibull” distribution. Weibull Distribution • Weibull Distribution PDF CDF k and λ are shape and location parameters respectively. PDF CDF Extreme Value Theory: Application Overall Negative Score Distribution Tail Maximum values of random variables • Tail of negative scores can be seen as a collection of maxima of some random variables. • Hence it follows Weibull distribution according to Extreme Value Theory. W-score normalization: Procedure For any classifier, • Fix the decision boundary on the scores (Ideally this should be at score = 0 ) • Select maximum N (tail size) samples from negative side of the boundary. • Fit a Weibull Distribution to these tail scores. • Renormalize scores using Cumulative Density Function (CDF) of this Weibull distribution. Results: Dataset • “Labeled Faces In The Wild” dataset. • About 13,000 images of 5000 celebrities. • 75 different attribute classification scores available from “Attribute and Simile Classifiers for Face Verification”. Kumar et al. ICCV 09. Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments. Results Multi Attribute Fusion: • Joint score can be computed as multiplication of individual attribute probabilities. • Attributes may not be independent. • Low probability due to: • bad classifier • absence of images belonging to an attribute. • Instead of product, authors propose use l1 norm of probabilities as a fusion score. Results Similarity Search: • Given an image and a set of attributes, find nearest images. • Perceived difference between images in different ranges might be similar. • Distances between query attribute and its nearest neighbor needs to be normalized. • Normalize query attribute scores on query image. • Get nearest neighbor distances. • Fit Weibull distribution to distances. Summary • Provides way of normalizing scores intuitively. • Provides way for combining attributes. • Relies on finding the right threshold and tail size. Requires fair bit of tuning. Questions?