recognition using regions (demo) sudheendra v. outline generating multiple segmentations...
Post on 30-Dec-2015
215 Views
Preview:
TRANSCRIPT
Recognition using Regions (Demo)
Sudheendra V
Outline
• Generating multiple segmentations– Normalized cuts [Ren & Malik (2003)]
• Uniform regions– Watershed transform [Arbel´aez1et al. (2009)]
• Non-uniform• Multiple scales
• Discovering object regions using LDA– ground truth segments– multiple segmentation– single segmentation– hierarchical segmentation
Multiple Segmentations
• Normalized cuts– segmentation as graph partitioning
• nodes -> pixels, edge between neighboring pixels
• edge weight -> affinity between pixels• partition graph into K components
– parameters• number of partitions K
– properties• similar sized partitions (normalized)• preserves region boundaries for large enough
K– multiple segmentations
• vary number of partitions K• resize image to different resolutions
http://www.cs.sfu.ca/~mori/research/superpixels/
affinity matrix
Normalized cuts (examples)
K = 4
K = 6
K = 7
Normalized cuts (examples)
K = 3
K = 5
K = 7
Extra edge •“normalized” regions
Normalized cuts (examples)
K = 3
K = 5
K = 7
Multiple Segmentations• Watershed transform
– contours are detected using texture, edge cues and oriented watershed transform used to determine contour scale
– parameters• thresholding scale for contours, k
– properties• variable sized regions• preserves region boundaries
– multiple segmentations• vary thresholding scale k• resize image to different resolutions
http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/
Contours at multiple scales Threshold at a scale
Watershed (examples)
K = 200
K = 180
K = 160
Threshold at different contour scales K to generate multiple segmentations
Watershed (examples)
K = 195
K = 183
K = 169
Threshold at different contour scales K to generate multiple segmentations
Watershed (Hierarchical segmentation)
Thresholding scales in an increasing sequence produces a hierarchical segmentation
K = 175
K = 155
K = 190
K = 140
Watershed (Hierarchical segmentation)
Thresholding scales in an increasing sequence produces a hierarchical segmentation
K = 175
K = 155
K = 200
K = 145
Ncuts vs Watershed Ncuts
Watershed
Comparison of multiple segmentations generated using Ncuts vs Watershed
Ncuts vs Watershed Ncuts
Watershed
Comparison of multiple segmentations generated using Ncuts vs Watershed
Outline
• Generating multiple segmentations– Normalized cuts [Ren & Malik (2003)]
• Uniform regions– Watershed transform [Arbel´aez1et al. (2009)]
• Non-uniform• Multiple scales
• Discovering object regions using LDA– ground truth segments– multiple segmentation– single segmentation– hierarchical segmentation
Multiple Segmentations
Discovering object regions using LDA
• Approach
• Parameters – number of topics to discover
Generate multiple segmentations
Extract local features (SIFT)
Bag of words rep for each
segment
Use LDA to discover topics
based on word co-occurrence
Rank segments based on
similarity to topic
Dataset
Note that the paper uses a larger set containing ~ 4000 images (MSRC_v0)
MSRC_v2 dataset
• 23 categories
• 591 images
• 1648 objects
Distribution of categories in MSRC_v2
Implementation details
– Dense sift on edge points and 3 different scales
– 2000 visual words– 8 segmentations using different parameters– ~ 40k segments in total– LDA takes ~ 10 mins
Ground Truth Segments
• ground truth segments are directly used• number of topics set to 25 (~ num categories)
Top 20 segments in terms of similarity of word distribution to a topic
Ground Truth SegmentsNumber of topics = 25
Ground Truth SegmentsNumber of topics = 50
Ground Truth SegmentsNumber of topics = 75
• Quantitative results
Overlap score on top 20 segments
Average precision (area under precision-recall curve)
Ground Truth Segments
Multiple segmentations• Normalized cuts
– k = {3, 5, 7, 9}– 2 resolutions
Multiple segmentations
Multiple segmentations
Multiple segmentations vs. Ground truth
• Quantitative results
Overlap score on top 20 segments
Average precision (area under precision-recall curve)
Multiple segmentations
Multiple segmentations• Effect of number of images
Overlap score on top 20 segments
• topics are easier to discover with more object instances
Single segmentation
Multiple segmentations vs. Single segmentation
Single segmentation returns partial objects for some classes
Single segmentation
Multiple segmentations vs. Single segmentation
Single segmentation returns partial objects for some classes
Single segmentation
Multiple segmentations vs. Single segmentation
• Quantitative results
Overlap score on top 20 segments
Average precision (area under precision-recall curve)
Single/Multiple segmentations
Hierarchical segmentationHierarchical segmentation vs. Multiple segmentation (Ncuts)
Watershed threshold set such there are 12 leaf nodes and entire hierarchical tree is used by LDA
Hierarchical segmentation
Hierarchical segmentation vs. Multiple segmentation (Ncuts)
Hierarchical segmentationHierarchical segmentation vs. Multiple segmentation (Ncuts)
• Quantitative results
Overlap score on top 20 segments
Hierarchical segmentation
Contour-based watershed method
• does better for objects with few internal contours (grass, sky)
• is worse for objects with large number of contours (flower, airplane)
Conclusion• Generating multiple segmentations
– ncuts and watershed provide different tradeoffs– bottom-up segmentation needs different parameters
for different objects
• Discovering objects using LDA– number of topics matters quite a bit– topics are easier to discover with more examples– multiple segmentation does better than because
different objects require different parameters – contour-based watershed method does better for
objects with few internal contours
top related