selecting distinctive 3d shape descriptors for similarity retrieval

Selecting Distinctive 3D Shape Descriptors for Similarity

Retrieval

Philip Shilane and Thomas Funkhouser

Large Databases of 3D Shapes

Mechanical CAD(National Design Repository)

Molecular Biology(Protein Databank)

Computer Graphics(Princeton Shape Benchmark)

Shape Retrieval

3D Model Model

Database

BestMatche

s

Local Matches for Retrieval

3D Model Model

Database

BestMatche

s

Local Matches for Retrieval

3D Model Model

Database

BestMatche

s

i

i YXC ),(

Cost Function

Local Matches for Retrieval

3D Model Model

Database

BestMatche

s

i

i YXC ),(

Cost Function

Using many local descriptors is slow.

Local Matches for Retrieval

3D Model Model

Database

BestMatche

s

i

i YXC ),(

Cost Function

Using many local descriptors is slow.Many descriptors do

not represent distinguishing parts.

Local Matches for Retrieval

3D Model Model

Database

BestMatche

s

i

i YXC ),(

Cost Function

Focusing on the distinctive regions improves retrieval time and accuracy.

Related Work

Selecting Local Descriptors• Random

Mori 2001Frome 2004

Related Work

Selecting Local Descriptors• Random• Salient

Gal 2005Lee 2005Frintrop 2004

Related Work

Selecting Local Descriptors• Random• Salient• Likelihood

Johnson 2000Shan 2004

Distinction = Retrieval Performance

QueryDescriptors

The distinction of each local descriptor is based on how well it retrieves shapes of the correct class.

Retrieval Results

Distinction = Retrieval Performance

QueryDescriptors

The distinct descriptors that distinguish between classes are classification dependent.

Retrieval Results

Approach

Descriptors

Distinction

We want a predicted distinction score for each descriptor on the model.

ApproachWe map descriptors into a 1D space where we learn distinction from a training set.

Dis

tinc

tion

1D Parameterization

Descriptors

Distinction

Approach

Descriptors

Distinction

Likelihood of shape descriptors is a 1D function that groups descriptors with similar distinction.

Likelihood Parameterization

System Overview

Likelihood

RetrievalEvaluation

Training

Query

ShapeDB

LocalDescriptors

DescriptorDB

Likelihood EvaluateDistinction

LocalDescriptors

Classification

Shape

DistinctionFunction

Match

RetrievalList

SelectDescriptors

System Overview

Likelihood

RetrievalEvaluation

Training

Query

ShapeDB

LocalDescriptors

DescriptorDB

Likelihood EvaluateDistinction

LocalDescriptors

Classification

Shape

DistinctionFunction

Match

RetrievalList

SelectDescriptors

System Overview

Likelihood

RetrievalEvaluation

Training

Query

ShapeDB

LocalDescriptors

DescriptorDB

Likelihood EvaluateDistinction

LocalDescriptors

Classification

Shape

DistinctionFunction

Match

RetrievalList

SelectDescriptors

System Overview

Likelihood

RetrievalEvaluation

Training

Query

ShapeDB

LocalDescriptors

DescriptorDB

Likelihood EvaluateDistinction

LocalDescriptors

Classification

Shape

DistinctionFunction

Match

RetrievalList

SelectDescriptors

)(21exp

2)( 1

2

21

xxxdensityt

d

Multi-dimensional normal density [Johnson 2000]

matrix covariance d x d vectorfeaturemean

vectorfeature ldimensiona d

x

Likelihood of Descriptors

Likelihood of Descriptors

)(21)(

))(densityln()(

1

xxxp

xxpt

The likelihood function is proportional to the descriptor density.

matrix covariance d x d vectorfeaturemean

vectorfeature ldimensiona d

x

Map from Descriptors to LikelihoodFlat regions are the most common while wing tips

and the cockpit area are rarer.

Less Likely

More Likely

System Overview

Likelihood

RetrievalEvaluation

Training

Query

ShapeDB

LocalDescriptors

DescriptorDB

Likelihood EvaluateDistinction

LocalDescriptors

Classification

Shape

DistinctionFunction

Match

RetrievalList

SelectDescriptors

Measuring Distinction

0.33

QueryDescriptors

Evaluation Metric

Evaluate the retrieval performance of every query descriptor.

Retrieval Results

Measuring Distinction

0.33

1.0

QueryDescriptors

Evaluation Metric

Some descriptors are better for retrieval than others.

Retrieval Results

System Overview

Likelihood

RetrievalEvaluation

Training

Query

ShapeDB

LocalDescriptors

DescriptorDB

Likelihood EvaluateDistinction

LocalDescriptors

Classification

Shape

DistinctionFunction

Match

RetrievalList

SelectDescriptors

Build Distinction FunctionMeasure likelihood and retrieval performance of each descriptor.

Build Distinction FunctionMeasure likelihood and retrieval performance of each descriptor.

Build Distinction FunctionMeasure likelihood and retrieval performance of each descriptor.

Build Distinction FunctionRetrieval performance is averaged within each likelihood bin.

Descriptor DistinctionA likelihood mapping separates descriptors with different retrieval performance.

Less Likely

More Likely

Less Likely

More Likely

Descriptor DistinctionThe most common features are the worst for retrieval.

Predicting Distinction

Distinction Function

Descriptors

Distinction

The likelihood mapping predicts descriptor distinction.

System Overview

Likelihood

RetrievalEvaluation

Training

Query

ShapeDB

LocalDescriptors

DescriptorDB

Likelihood EvaluateDistinction

LocalDescriptors

Classification

Shape

DistinctionFunction

Match

RetrievalList

SelectDescriptors

Selecting Distinctive DescriptorsThe k most distinctive descriptors with a minimum distance constraint are selected.

Mesh Descriptors DistinctionScores

3 SelectedDescriptors

Matching with Selected Descriptors

k

i

ki

k YXCYX ),(

3D Model Model

Database

BestMatche

s

Results

• Examples of Distinctive Descriptors• Evaluation for Retrieval

Distinctive Descriptor ExamplesDescriptors on the head and neck represent

consistent regions of the models.

Distinctive Descriptor ExamplesDescriptors on the front of the jet are consistent as

opposed to on the wings.

ChallengeThe wheels are consistent features for cars.

Shape Database

• 100 Models in 10 Classes from the Princeton Shape Benchmark

• Models come from different branchesof the hierarchical classification

Shape Descriptors• Mass per Shell Shape Histogram

(SHELLS)Ankerst 1999

• Spherical Harmonics of the Gaussian Euclidean Distance Transform (SHD)

Kazhdan 2003

0.25 0.5 1.0 2.0

Radius of Descriptors Considered

Local vs. Global DescriptorsUsing local descriptors improves retrieval relative to global descriptors.

Global vs Local

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Recall

Prec

isio

n

GlobalAll Local

Focus on Distinctive DescriptorsUsing a small number of distinct descriptors maintains retrieval performance while improving retrieval time.

Global vs Local

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Recall

Prec

isio

n

GlobalAll Local10 Distinct3 Distinct

Alternative Selection Techniques

Selection Techniques

-5%

0%

5%

10%

15%

20%

10% 30% 50% 70% 90%

Recall

% Im

prov

emen

t Pre

cisi

on

Johnson 2000 (DB)

Random

Alternative Selection Techniques

Selection Techniques

-5%

0%

5%

10%

15%

20%

10% 30% 50% 70% 90%

Recall

% Im

prov

emen

t Pre

cisi

on

Johnson 2000(Model)Johnson 2000 (DB)

Random

Alternative Selection Techniques

Selection Techniques

-5%

0%

5%

10%

15%

20%

10% 30% 50% 70% 90%

Recall

% Im

prov

emen

t Pre

cisi

on

Distinctive

Johnson 2000(Model)Johnson 2000 (DB)

Random

Distinction improves retrieval more than other techniques.

Conclusion

• Method to select distinctive descriptors

• Distinctive descriptors can improve retrieval

• Mapping descriptors through likelihood and learned retrieval performance to distinction is better than other alternatives

• Distinction is independent of type of descriptor

Future Work

• Explore other definitions of likelihood including mixture models

Future Work

• Explore other definitions of likelihood including mixture models

• Consider non-likelihood parameterizations

Future Work

• Explore other definitions of likelihood including mixture models

• Consider non-likelihood parameterizations

• Combine descriptors while accounting for deformation [Funkhouser and Shilane, SGP]

Acknowledgements

Szymon RusinkiewiczJoshua PodolakPrinceton Graphics Group

Funding Sources:National Science Foundation Grant CCR-0093343

and Grant 11S-0121446Air Force Research Laboratory Grant FA8650-04-1-

1718

The End