![Page 1: Evaluation of features detectors and descriptors based on 3D objects P. Moreels - P. Perona California Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/56649d605503460f94a41be4/html5/thumbnails/1.jpg)
Evaluation of features detectors and descriptors based on 3D objects
P. Moreels - P. Perona
California Institute of Technology
![Page 2: Evaluation of features detectors and descriptors based on 3D objects P. Moreels - P. Perona California Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/56649d605503460f94a41be4/html5/thumbnails/2.jpg)
Large baseline stereo
Features – what for ?
Stitching
Object recognition
[Dorko & Schmid’05]
[Lowe ’04][Brown & Lowe ’03]
[Tuytelaars & Van Gool ’00]
![Page 3: Evaluation of features detectors and descriptors based on 3D objects P. Moreels - P. Perona California Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/56649d605503460f94a41be4/html5/thumbnails/3.jpg)
Moving the viewpoint
![Page 4: Evaluation of features detectors and descriptors based on 3D objects P. Moreels - P. Perona California Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/56649d605503460f94a41be4/html5/thumbnails/4.jpg)
232 keypoints extracted
Features stabilityFeatures stability is not perfect…
240 keypoints extracted
![Page 5: Evaluation of features detectors and descriptors based on 3D objects P. Moreels - P. Perona California Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/56649d605503460f94a41be4/html5/thumbnails/5.jpg)
First stage – feature detector
difference ofgaussians
[Crowley’84]
Kadir & Brady[Kadir’02]
Harris[Harris’88]
Affineinvariant
Harris[Mikolajczyk’02]
![Page 6: Evaluation of features detectors and descriptors based on 3D objects P. Moreels - P. Perona California Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/56649d605503460f94a41be4/html5/thumbnails/6.jpg)
Second stage – feature descriptorSIFT Steerable filters
Differential invariants Shape context
[Lowe ’04] [Freeman’91 ]
[Belongie’02 ][Schmid’97 ]
![Page 7: Evaluation of features detectors and descriptors based on 3D objects P. Moreels - P. Perona California Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/56649d605503460f94a41be4/html5/thumbnails/7.jpg)
Evaluations – Mikolajczyk ’03-’05
• Large viewpoint change• Computation of ground truth positions via a homography
![Page 8: Evaluation of features detectors and descriptors based on 3D objects P. Moreels - P. Perona California Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/56649d605503460f94a41be4/html5/thumbnails/8.jpg)
Evaluations – Mikolajczyk ’03-’05
[CVPR’03] [PAMI’04]
[submitted]
• SIFT-based descriptors rule !
• All affine-invariant detectors are good, they should all be used together.
![Page 9: Evaluation of features detectors and descriptors based on 3D objects P. Moreels - P. Perona California Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/56649d605503460f94a41be4/html5/thumbnails/9.jpg)
2D vs. 3D
Ranking of detectors/descriptorscombinations are modified whenswitching from 2D to 3D objects
![Page 10: Evaluation of features detectors and descriptors based on 3D objects P. Moreels - P. Perona California Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/56649d605503460f94a41be4/html5/thumbnails/10.jpg)
Dataset – 100 3D objects
![Page 11: Evaluation of features detectors and descriptors based on 3D objects P. Moreels - P. Perona California Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/56649d605503460f94a41be4/html5/thumbnails/11.jpg)
Viewpoints 45° apart
![Page 12: Evaluation of features detectors and descriptors based on 3D objects P. Moreels - P. Perona California Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/56649d605503460f94a41be4/html5/thumbnails/12.jpg)
Viewpoints 45° apart
![Page 13: Evaluation of features detectors and descriptors based on 3D objects P. Moreels - P. Perona California Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/56649d605503460f94a41be4/html5/thumbnails/13.jpg)
Viewpoints 45° apart
![Page 14: Evaluation of features detectors and descriptors based on 3D objects P. Moreels - P. Perona California Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/56649d605503460f94a41be4/html5/thumbnails/14.jpg)
Viewpoints 45° apart
![Page 15: Evaluation of features detectors and descriptors based on 3D objects P. Moreels - P. Perona California Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/56649d605503460f94a41be4/html5/thumbnails/15.jpg)
Ground truth - Epipolar constraints
![Page 16: Evaluation of features detectors and descriptors based on 3D objects P. Moreels - P. Perona California Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/56649d605503460f94a41be4/html5/thumbnails/16.jpg)
Testing setup
Unrelated images used to load the database of features.
![Page 17: Evaluation of features detectors and descriptors based on 3D objects P. Moreels - P. Perona California Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/56649d605503460f94a41be4/html5/thumbnails/17.jpg)
Distance ratio
• Correct matches are highly distinctive lower ratio
• Incorrect correspondences are ‘random correspondences’ low distinctiveness and ratio close to 1
[Lowe’04]
![Page 18: Evaluation of features detectors and descriptors based on 3D objects P. Moreels - P. Perona California Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/56649d605503460f94a41be4/html5/thumbnails/18.jpg)
Are we accepting wrong matches ?
• Manual user classification into correct and incorrect triplets• Comparison with a simpler system: 2 views, only one epipolar constraint.
Pietro said maybe don’t need this slide – I think it is important to justify our 3-cameras setup
![Page 19: Evaluation of features detectors and descriptors based on 3D objects P. Moreels - P. Perona California Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/56649d605503460f94a41be4/html5/thumbnails/19.jpg)
Detectors / descriptors tested
Detectors Descriptors
• Harris• Hessian• Harris-affine• Hessian-affine• Difference-of-gaussians• MSER• Kadir-Brady
• SIFT• steerable filters• differential invariants• shape context• PCA-SIFT
![Page 20: Evaluation of features detectors and descriptors based on 3D objects P. Moreels - P. Perona California Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/56649d605503460f94a41be4/html5/thumbnails/20.jpg)
Results – viewpoint change M
ahal
anob
is d
ista
nce
No
‘bac
kgro
und’
imag
es
![Page 21: Evaluation of features detectors and descriptors based on 3D objects P. Moreels - P. Perona California Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/56649d605503460f94a41be4/html5/thumbnails/21.jpg)
Results – lighting / scale changes
Change in light – result averaged over 3 lighting conditions.
Change in scale - 7.0mm to 14.6mm
![Page 22: Evaluation of features detectors and descriptors based on 3D objects P. Moreels - P. Perona California Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/56649d605503460f94a41be4/html5/thumbnails/22.jpg)
Conclusions
• Automated ground truth for 3D objects/scenes
• Ranking changes from 2D to 3D
• Stability is much lower for 3D
• Detectors – affine-rectified detectors are indeed best
• Descriptors – SIFT and shape context performed best.
• Application: use ground truth in order to learn probability densities: ‘how does a correct match look like ?’