semi-local affine parts for object recognition
DESCRIPTION
Semi-Local Affine Parts for Object Recognition. Svetlana Lazebnik, Jean Ponce University of Illinois at Urbana-Champaign Cordelia Schmid INRIA Rh ô ne-Alpes BMVC 2004. Overview. Goal: Learning models for recognition of 3D object classes Challenges: Geometric invariance - PowerPoint PPT PresentationTRANSCRIPT
Semi-Local Affine PartsSemi-Local Affine Partsfor Object Recognitionfor Object Recognition
Svetlana Lazebnik, Jean PonceSvetlana Lazebnik, Jean PonceUniversity of Illinois at Urbana-ChampaignUniversity of Illinois at Urbana-Champaign
Cordelia Schmid Cordelia Schmid INRIA RhINRIA Rhôône-Alpesne-Alpes
BMVC 2004BMVC 2004
OverviewOverview• Goal:
– Learning models for recognition of 3D object classes• Challenges:
– Geometric invariance– Robustness to clutter, occlusion– Weakly supervised learning
• Proposed approach: – An object representation using semi-local affine parts
Low-Level Features: Local Affine RegionsLow-Level Features: Local Affine Regions
• This work: Laplacian detector (Gårding & Lindeberg, 1996)• Other detectors: Kadir et al. (2004), Matas et al. (2002),
Mikolajczyk & Schmid (2002), Tuytelaars & Van Gool (2004), etc.
• In practice: two-image matching followed by validation
Learning PartsLearning Parts• Ideal approach: simultaneous correspondence search
across entire training set
validation setinitial pair
candidate part
Two-Image MatchingTwo-Image Matching• Goal: to find collections of local affine regions that can be
mapped onto each other using a single affine transformation
• Implementation: greedy search based on geometric and photometric consistency constraints– Returns multiple correspondence hypotheses
– Automatically determines number of regions in correspondence
– Works on unsegmented, cluttered images (weakly supervised learning)
A
Matching: DetailsMatching: Details• Initialization:
– Identify triples of neighboring regions (i, j, k) in first image– Find all triples (i', j', k') in the second image such that i'
(resp. j', k') is a potential match of i (resp. j, k), and j', k' are neighbors of i'
i
j
k
i'
j'
k'
Matching (cont.)Matching (cont.)• Beginning with each seed triple, iterate:
– Estimate the affine transformation between centers of corresponding regions in current group of matches
A
Matching (cont.)Matching (cont.)• Beginning with each seed triple, iterate:
– Estimate the affine transformation between centers of corresponding regions in current group of matches
– Determine geometric consistency of current group of matches
• Geometric consistency criteria:– Distance between ellipse centers
(residual)– Difference of major and minor axis
lengths– Difference of ellipse orientations
Matching (cont.)Matching (cont.)• Beginning with each seed triple, iterate:
– Estimate the affine transformation between centers of corresponding regions in current group of matches
– Determine geometric consistency of current group of matches
– Search for additional matches in the neighborhood of the current group
Matching: 3D ObjectsMatching: 3D Objects
Matching: 3D ObjectsMatching: 3D Objects
closeup closeup
Matching: FacesMatching: Faces
spurious match ???
Finding Repeated Patterns and Finding Repeated Patterns and SymmetriesSymmetries
Learning Object Models for RecognitionLearning Object Models for Recognition• Match multiple pairs of training images to produce a
set of candidate parts• Use additional validation images to evaluate
repeatability of parts and individual regions • Retain a fixed number of parts having the best
repeatability score
Recognition Experiment: ButterfliesRecognition Experiment: Butterflies
• 26 training images per class– 8 initial pairs– 10 validation images
• 437 test images• 619 images total
Admiral Swallowtail Machaon Monarch 1 Monarch 2 Peacock Zebra
Butterfly PartsButterfly Parts
RecognitionRecognition
• Top 10 parts per class used for recognition• Relative repeatability score:• Classification results:
total number of regions detectedtotal part size
Total part size (smallest/largest)
Classification Rate vs. Classification Rate vs. Number of PartsNumber of Parts
Detection Results (ROC Curves)Detection Results (ROC Curves)
Circles: reference relative repeatability rates. Red square: ROC equal error rate (in parentheses)
Successful Detection ExamplesSuccessful Detection ExamplesTraining images
Test images (blue: occluded regions)
All regions found in the test images
Unsuccessful Detection ExamplesUnsuccessful Detection ExamplesTraining images
Test images (blue: occluded regions)
All regions found in the test images
Future WorkFuture Work• Goal:
– Recognize highly variable, non-rigid object categories
• Proposed approach: – Treat semi-local affine parts as “black boxes”– Model spatial relations between parts– Learn these relations from training data in a weakly
supervised fashion