Minimal Loss Hashing for Compact Binary Codes

Mohammad Norouzi

David Fleet

University of Toronto

Near Neighbor Search

Near Neighbor Search

Near Neighbor Search

Similarity-Preserving Binary Hashing

Why binary codes?

Sub-linear search using hash indexing

(even exhaustive linear search is fast)

Binary codes are storage-efficient

input vector

parametermatrix

binaryquantization

Random projections used by locality-sensitive hashing

(LSH) and related techniques [Indyk & Motwani ‘98;

Charikar ’02; Raginsky & Lazebnik ’09]

Similarity-Preserving Binary Hashing

Hash function

kth row of W

Learning Binary Hash Functions

Reasons to learn hash functions:

to find more compact binary codes

to preserve general similarity measures

Previous work

boosting [Shakhnarovich et al ’03]

neural nets [Salakhutdinov & Hinton 07; Torralba et al 07]

spectral methods [Weiss et al ’08]

loss-based methods [Kulis & Darrel ‘09]

…

Formulation

Input data:

Similarity labels:

Hash function:

Binary codes:

Loss Function

Hash code quality measured by a loss function:

similarity label

binarycodes : code for item 1

: code for item 2

: similarity label

cost

measures consistency

Similar items should map to nearby hash codes

Dissimilar items should map to very different codes

Hinge Loss

Similar items should map to codes within a radius of bits

Dissimilar items should map to codes no closer than bits

Empirical Loss

Good:

incorporates quantization and Hamming distance

Not so good:

discontinuous, non-convex objective function

Given training pairs with similarity labels

We minimize an upper bound on empirical loss,

inspired by structural SVM formulations

[Taskar et al ‘03; Tsochantaridis et al ‘04; Yu &

Joachims ‘09]

Bound on loss

LHS = RHS

Bound on loss

Remarks: piecewise linear in W convex-concave in W relates to structural SVM with latent variables

[Yu & Joachims ‘09]

Bound on Empirical Loss

Loss-adjusted inference

Exact

Efficient

Perceptron-like Learning

Initialize with LSH

Iterate over pairs

• Compute , the codes given by

• Solve loss-adjusted inference

• Update

[McAllester et al.., 2010]

Experiment: Euclidean ANN

Similarity based on Euclidean distance

Datasets LabelMe (GIST) MNIST (pixels) PhotoTourism (SIFT) Peekaboom (GIST) Nursery (8D attributes) 10D Uniform

Experiment: Euclidean ANN

22K LabelMe

512 GIST

20K training

2K testing

~1% of pairs are similar

Evaluation

Precision: #hits / number of items retrieved

Recall: #hits / number of similar items

Techniques of interest

MLHMLH – minimal loss hashing (This work)

LSHLSH – locality-sensitive hashing (Charikar ‘02)

SHSH – spectral hashing (Weiss, Torralba & Fergus ‘09)

SIKHSIKH – shift-Invariant kernel hashing (Raginsky & Lazebnik ‘09)

BRE BRE – Binary reconstructive embedding (Kulis & Darrel ‘09)

Euclidean Labelme – 32 bits

Euclidean Labelme – 32 bits

Euclidean Labelme – 32 bits

Euclidean Labelme – 64 bits

Euclidean Labelme – 64 bits

Euclidean Labelme – 128 bits

Euclidean Labelme – 256 bits

Experiment: Semantic ANN

Semantic similarity measure based on annotations(object labels) from LabelMe database:

512D GIST, 20K training, 2K testing

Techniques of interest

MLHMLH – minimal loss hashing

NNNN – nearest neighbor in GIST space

NNCA NNCA – multilayer network with RBM pre-training and nonlinear NCA fine tuning [Torralba, et al. ’09; Salakhutdinov & Hinton ’07]

Semantic LabelMe

Semantic LabelMe

Summary

A formulation for learning binary hash functions

based on

structured prediction with latent variables

hinge-like loss function for similarity search

Experiments show that with minimal loss hashing

binary codes can be made more compact

semantic similarity based on human labels can be preserved

Thank you!

Questions?