pyramids of features for categorization greg griffin and will coulter (see lazebnik et al., cvpr...
Post on 21-Dec-2015
215 views
TRANSCRIPT
![Page 1: Pyramids of Features For Categorization Greg Griffin and Will Coulter (see Lazebnik et al., CVPR 2006, too)](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d6b5503460f94a4a786/html5/thumbnails/1.jpg)
Pyramids of FeaturesFor Categorization
Greg Griffin and Will Coulter(see Lazebnik et al., CVPR 2006, too)
![Page 2: Pyramids of Features For Categorization Greg Griffin and Will Coulter (see Lazebnik et al., CVPR 2006, too)](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d6b5503460f94a4a786/html5/thumbnails/2.jpg)
Intuition: Approximates optimal partial matching
U
Adapted from http://people.csail.mit.edu/kgrauman/slides/pyr_match_iccv2005.ppt
![Page 3: Pyramids of Features For Categorization Greg Griffin and Will Coulter (see Lazebnik et al., CVPR 2006, too)](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d6b5503460f94a4a786/html5/thumbnails/3.jpg)
Intuition [cont’d]: Combine bags of features with spatial information
0 5 10 15 20 25 30 35 40 45 500
1
2
3
4
5
6
7
8
9
10
0 5 10 15 20 25 30 35 40 45 500
1
2
3
4
5
6
7
8
9
10
category
![Page 4: Pyramids of Features For Categorization Greg Griffin and Will Coulter (see Lazebnik et al., CVPR 2006, too)](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d6b5503460f94a4a786/html5/thumbnails/4.jpg)
Disadvantages
• Same objects in different quadrants
• Objects sliced by bins
![Page 5: Pyramids of Features For Categorization Greg Griffin and Will Coulter (see Lazebnik et al., CVPR 2006, too)](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d6b5503460f94a4a786/html5/thumbnails/5.jpg)
Possible Solutions
• Flipping / rotating image
• Sliding / shuffling histogram bins
![Page 6: Pyramids of Features For Categorization Greg Griffin and Will Coulter (see Lazebnik et al., CVPR 2006, too)](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d6b5503460f94a4a786/html5/thumbnails/6.jpg)
Possible Solutions [cont’d]
• Split histogram in powers of three
![Page 7: Pyramids of Features For Categorization Greg Griffin and Will Coulter (see Lazebnik et al., CVPR 2006, too)](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d6b5503460f94a4a786/html5/thumbnails/7.jpg)
Implementation Overview
Image and Feature Extraction (SIFT on regular grid)
↓
Vocabulary Translation (200 words (k-means))
↓
Histogram Generation (flips, slides, arbitrary mixing)
↓
Matching (full or partial pyramid)
↓
Decision (best match, voting, SVM)
![Page 8: Pyramids of Features For Categorization Greg Griffin and Will Coulter (see Lazebnik et al., CVPR 2006, too)](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d6b5503460f94a4a786/html5/thumbnails/8.jpg)
Sanity Check 1Graphical mini-confusion matrix (and rot. invariance)
![Page 9: Pyramids of Features For Categorization Greg Griffin and Will Coulter (see Lazebnik et al., CVPR 2006, too)](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d6b5503460f94a4a786/html5/thumbnails/9.jpg)
Sanity Check 2
![Page 10: Pyramids of Features For Categorization Greg Griffin and Will Coulter (see Lazebnik et al., CVPR 2006, too)](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d6b5503460f94a4a786/html5/thumbnails/10.jpg)
Scene Database
![Page 11: Pyramids of Features For Categorization Greg Griffin and Will Coulter (see Lazebnik et al., CVPR 2006, too)](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d6b5503460f94a4a786/html5/thumbnails/11.jpg)
Scene Database
81.1% vs 67.4%
(100)
![Page 12: Pyramids of Features For Categorization Greg Griffin and Will Coulter (see Lazebnik et al., CVPR 2006, too)](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d6b5503460f94a4a786/html5/thumbnails/12.jpg)
Caltech 101 64.6% vs. 33.1% (30 , 16)
![Page 13: Pyramids of Features For Categorization Greg Griffin and Will Coulter (see Lazebnik et al., CVPR 2006, too)](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d6b5503460f94a4a786/html5/thumbnails/13.jpg)
Caltech 101
minaret windsor chair joshua tree okapi
cougar body beaver crocodile ant
97.6% , 86.7% 94.6% , 80.0% 87.9% , 40.0% 87.8% , 33.3%
27.6% , 6.7% 27.5% , 13.3% 25.0% , 13.3% 25.0% , 0.0%
(30 , 16)
![Page 14: Pyramids of Features For Categorization Greg Griffin and Will Coulter (see Lazebnik et al., CVPR 2006, too)](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d6b5503460f94a4a786/html5/thumbnails/14.jpg)
Caltech 101
![Page 15: Pyramids of Features For Categorization Greg Griffin and Will Coulter (see Lazebnik et al., CVPR 2006, too)](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d6b5503460f94a4a786/html5/thumbnails/15.jpg)
Work in Progress
• 256 Performance– 64 times more work scene database– 6.4 times more work than 101
• SVM– one-vs-all weighting issues– speed it up?– improve performance
• Improvements– Flip, Slide, Arbitrary
• Powers-of-3 histogram bins
![Page 16: Pyramids of Features For Categorization Greg Griffin and Will Coulter (see Lazebnik et al., CVPR 2006, too)](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d6b5503460f94a4a786/html5/thumbnails/16.jpg)
Open Questions
• Performance of arbitrary match bins– Try random sampling?– Allow multiple best matches?
• Chess/pattern example
• Grid example
• Optimal kernel level weights
![Page 17: Pyramids of Features For Categorization Greg Griffin and Will Coulter (see Lazebnik et al., CVPR 2006, too)](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d6b5503460f94a4a786/html5/thumbnails/17.jpg)
![Page 18: Pyramids of Features For Categorization Greg Griffin and Will Coulter (see Lazebnik et al., CVPR 2006, too)](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d6b5503460f94a4a786/html5/thumbnails/18.jpg)
Implementation Details
• [block diagram]• Images (288^2,b&w,squished) and feature
extraction (-weak,-pca,+sift)• Vocab generation (200 words, 20,000-
small)• Histogram(fliplr,flipud,slide,arbitrary,bag of
features)• match (full&partial pyramids)• Decision (best match,voting,SVM)