a sparse texture representation using affine-invariant regions svetlana lazebnik, jean ponce...

A Sparse Texture A Sparse Texture Representation Using Representation Using Affine-Invariant RegionsAffine-Invariant Regions

Svetlana Lazebnik, Jean PonceSvetlana Lazebnik, Jean PonceBeckman InstituteUniversity of Illinois, Urbana, USA

Cordelia SchmidCordelia SchmidINRIA Rhône-AlpesGrenoble, France

Supported in part by the UIUC Campus Research Board, the UIUC/CNRS Collaborative Research Agreement, the National Science Foundation under grant IRI-990709, and by the European project LAVA (IST-2001-34405).

GoalGoalDevelop a texture representation invariant to:

– viewpoint changes

– non-rigid deformations

Our ApproachOur Approach• Affine-invariant regions: robustness against geometric

transformations• A sparse representation: saliency, compactness

Without spatial selection With spatial selection

OutlineOutline

Affine Region DetectorsAffine Region DetectorsHarris detector (H) Laplacian detector (L)

[Lindeberg & Gårding 1997, Mikolajczyk & Schmid 2002]

Affine Rectification ProcessAffine Rectification Process

Patch 2Patch 1

Rectified patches (rotational ambiguity)

Spin Images as Intensity DescriptorsSpin Images as Intensity Descriptors

• Range spin images: Johnson & Hebert (1998)• Two-dimensional histogram:

distance from center × intensity value

Signatures and EMDSignatures and EMD

• Signatures

S = {(m1 , w1) , … , (mk , wk)} mi — representative of ith cluster wi — weight (relative size) of ith cluster

• Earth Mover’s Distance [Rubner, Tomasi & Guibas 1998] – Computed from ground distances d(mi , m'j)

– Can compare signatures of different sizes – Insensitive to the number of clusters

EvaluationEvaluation

• Retrieval and classification• Two experiments:

– Viewpoint-invariant texture recognition– Brodatz database

Viewpoint-Invariant Texture RecognitionViewpoint-Invariant Texture RecognitionData set: 10 textures, 20 samples each

ResultsResults

• Retrieval evaluation strategy: Picard et al. 1993, Liu & Picard 1996, Xu et al. 2000

• Gabor-like filters: Schmid 2000

Classification ResultsClassification Results

??

A Closer LookA Closer LookProblem: viewpoint- and lighting-dependent appearance changes

A Closer LookA Closer LookProblem: viewpoint- and lighting-dependent appearance changes

Brodatz Database EvaluationBrodatz Database Evaluation

• 111 classes, 9 samples each• No affine invariance required• Shape channel:

Retrieval ResultsRetrieval Results

Better results: Xu, Georgescu, Comaniciu & Meer (2000)

Classification ResultsClassification Results

SummarySummary• Sparse representation• Flexible approach to invariance• Spin images as intensity descriptors

Future WorkFuture Work

• Evaluate more detector types [Kadir & Brady 2001, Tuytelaars & Van Gool 2001]

• Compare spin images with descriptors of similar dimensionality (e.g. SIFT)

• Enhance representation with spatial relations • Learning from multi-texture images• Texture segmentation

ICCV 2003ICCV 2003• Neighborhood statistics • Learning from multi-texture images• Texture segmentation