lecture’12:’clustering’and’ segmentaon’ - artificial...
TRANSCRIPT
Lecture 12 - !!!
Fei-Fei Li!
Lecture 12: Clustering and Segmenta5on
Professor Fei-‐Fei Li Stanford Vision Lab
28-‐Oct-‐14 1
Lecture 12 - !!!
Fei-Fei Li!
What we will learn today
• Introduc5on to segmenta5on and clustering • Gestalt theory for perceptual grouping • Agglomera5ve clustering
28-‐Oct-‐14 2
Reading: [FP] Chapters: 14.2, 14.4
Lecture 12 - !!!
Fei-Fei Li! 28-‐Oct-‐14 3
Lecture 12 - !!!
Fei-Fei Li!
Image Segmenta5on
• Goal: iden5fy groups of pixels that go together
Slide credit: Steve Seitz, Kristen Grauman
28-‐Oct-‐14 4
Lecture 12 - !!!
Fei-Fei Li!
The Goals of Segmenta5on
• Separate image into coherent “objects” Image Human segmenta5on
Slide credit: Svetlana Lazebnik
28-‐Oct-‐14 5
Lecture 12 - !!!
Fei-Fei Li!
The Goals of Segmenta5on
• Separate image into coherent “objects” • Group together similar-‐looking pixels for efficiency of further processing
X. Ren and J. Malik. Learning a classifica5on model for segmenta5on. ICCV 2003.
“superpixels”
Slid
e cr
edit
: Sv
etla
na L
azeb
nik
28-‐Oct-‐14 6
Lecture 12 - !!!
Fei-Fei Li!
Segmenta5on for feature support
50x50 Patch 50x50 Patch
Slide: Derek Hoiem
28-‐Oct-‐14 7
Lecture 12 - !!!
Fei-Fei Li!
Segmenta5on for efficiency
[Felzenszwalb and Huttenlocher 2004]
[Hoiem et al. 2005, Mori 2005] [Shi and Malik 2001] Slide: Derek Hoiem
28-‐Oct-‐14 8
Lecture 12 - !!!
Fei-Fei Li!
Segmenta5on as a result
Rother et al. 2004
28-‐Oct-‐14 9
Lecture 12 - !!!
Fei-Fei Li!
Types of segmenta5ons
Oversegmentation Undersegmentation
Multiple Segmentations
28-‐Oct-‐14 10
Lecture 12 - !!!
Fei-Fei Li!
One way to think about “segmenta5on” is Clustering
Clustering: group together similar points and represent them with a single token
Key Challenges: 1) What makes two points/images/patches similar? 2) How do we compute an overall grouping from pairwise similari5es?
Slide: Derek Hoiem
28-‐Oct-‐14 11
Lecture 12 - !!!
Fei-Fei Li!
Why do we cluster? • Summarizing data
– Look at large amounts of data – Patch-‐based compression or denoising – Represent a large con5nuous vector with the cluster number
• Coun2ng
– Histograms of texture, color, SIFT vectors
• Segmenta2on – Separate the image into different regions
• Predic2on – Images in the same cluster may have the same labels
Slide: Derek Hoiem
28-‐Oct-‐14 12
Lecture 12 - !!!
Fei-Fei Li!
How do we cluster?
• Agglomera5ve clustering – Start with each point as its own cluster and itera5vely merge the closest clusters
• K-‐means (next lecture) – Itera5vely re-‐assign points to the nearest cluster center
• Mean-‐shig clustering (next lecture) – Es5mate modes of pdf
• Spectral clustering (CS231a, winter quarter) – Split the nodes in a graph based on assigned links with similarity weights
28-‐Oct-‐14 13
Lecture 12 - !!!
Fei-Fei Li!
General ideas
• Tokens – whatever we need to group (pixels, points, surface elements, etc., etc.)
• Bokom up clustering
– tokens belong together because they are locally coherent
• Top down clustering – tokens belong together because they lie on the same visual en5ty (object, scene…)
> These two are not mutually exclusive
28-‐Oct-‐14 14
Lecture 12 - !!!
Fei-Fei Li!
Examples of Grouping in Vision
Determining image regions
Grouping video frames into shots
Object-‐level grouping
Figure-‐ground
Slide credit: Kristen Grauman
What things should be grouped?
What cues indicate groups?
28-‐Oct-‐14 15
Lecture 12 - !!!
Fei-Fei Li!
Similarity
Slide credit: Kristen Grauman
28-‐Oct-‐14 16
Lecture 12 - !!!
Fei-Fei Li!
Symmetry
Slide credit: Kristen Grauman
28-‐Oct-‐14 17
Lecture 12 - !!!
Fei-Fei Li!
Common Fate
Image credit: Arthus-‐Bertrand (via F. Durand)
Slide credit: Kristen Grauman
28-‐Oct-‐14 18
Lecture 12 - !!!
Fei-Fei Li!
Proximity
Slide credit: Kristen Grauman
28-‐Oct-‐14 19
Lecture 12 - !!!
Fei-Fei Li!
Muller-‐Lyer Illusion
• What makes the bokom line look longer than the top line?
28-‐Oct-‐14 20
Lecture 12 - !!!
Fei-Fei Li!
What we will learn today
• Introduc5on to segmenta5on and clustering • Gestalt theory for perceptual grouping • Agglomera5ve clustering
28-‐Oct-‐14 21
Lecture 12 - !!!
Fei-Fei Li!
The Gestalt School • Grouping is key to visual percep5on • Elements in a collec5on can have proper5es that result from rela5onships – “The whole is greater than the sum of its parts”
Illusory/subjec5ve contours
Occlusion
Familiar configura5on
hkp://en.wikipedia.org/wiki/Gestalt_psychology Slid
e cr
edit
: Sv
etla
na L
azeb
nik
28-‐Oct-‐14 22
Lecture 12 - !!!
Fei-Fei Li!
Gestalt Theory • Gestalt: whole or group
– Whole is greater than sum of its parts – Rela5onships among parts can yield new proper5es/features
• Psychologists iden5fied series of factors that predispose set of elements to be grouped (by human visual system)
Untersuchungen zur Lehre von der Gestalt, Psychologische Forschung, Vol. 4, pp. 301-350, 1923 http://psy.ed.asu.edu/~classics/Wertheimer/Forms/forms.htm
“I stand at the window and see a house, trees, sky. Theoretically I might say there were 327 brightnesses and nuances of colour. Do I have "327"? No. I have sky, house, and trees.”
Max Wertheimer (1880-1943)
28-‐Oct-‐14 23
Lecture 12 - !!!
Fei-Fei Li!
Gestalt Factors
• These factors make intui5ve sense, but are very difficult to translate into algorithms.
Imag
e so
urce
: Fo
rsyt
h &
Pon
ce
28-‐Oct-‐14 24
Lecture 12 - !!!
Fei-Fei Li!
Con5nuity through Occlusion Cues
28-‐Oct-‐14 25
Lecture 12 - !!!
Fei-Fei Li!
Con5nuity through Occlusion Cues
Con5nuity, explana5on by occlusion
28-‐Oct-‐14 26
Lecture 12 - !!!
Fei-Fei Li!
Con5nuity through Occlusion Cues
Imag
e so
urce
: Fo
rsyt
h &
Pon
ce
28-‐Oct-‐14 27
Lecture 12 - !!!
Fei-Fei Li!
Con5nuity through Occlusion Cues
Imag
e so
urce
: Fo
rsyt
h &
Pon
ce
28-‐Oct-‐14 28
Lecture 12 - !!!
Fei-Fei Li!
Figure-‐Ground Discrimina5on
28-‐Oct-‐14 29
Lecture 12 - !!!
Fei-Fei Li!
The Ul5mate Gestalt?
28-‐Oct-‐14 30
Lecture 12 - !!!
Fei-Fei Li!
What we will learn today
• Introduc5on to segmenta5on and clustering • Gestalt theory for perceptual grouping • Agglomera5ve clustering
28-‐Oct-‐14 31
Lecture 12 - !!!
Fei-Fei Li!
Clustering: distance measure
Clustering is an unsupervised learning method. Given items , the goal is to group them into clusters. We need a pairwise distance/similarity func5on between items, and some5mes the desired number of clusters. When data (e.g. images, objects, documents) are represented by feature vectors, a commonly used similarity measure is the cosine similarity. Let be two data vectors. There is angle between the two vectors . The cosine similarity is defined as In contrast, Euclidean distance measure would be
28-‐Oct-‐14 32
technical
note
Lecture 12 - !!!
Fei-Fei Li!
Agglomera5ve clustering
28-‐Oct-‐14 33
Slide credit: Andrew Moore
Lecture 12 - !!!
Fei-Fei Li!
Agglomera5ve clustering
28-‐Oct-‐14 34
Slide credit: Andrew Moore
Lecture 12 - !!!
Fei-Fei Li!
Agglomera5ve clustering
28-‐Oct-‐14 35
Slide credit: Andrew Moore
Lecture 12 - !!!
Fei-Fei Li!
Agglomera5ve clustering
28-‐Oct-‐14 36
Slide credit: Andrew Moore
Lecture 12 - !!!
Fei-Fei Li!
Agglomera5ve clustering
28-‐Oct-‐14 37
Slide credit: Andrew Moore
Lecture 12 - !!!
Fei-Fei Li!
Agglomera5ve clustering How to define cluster similarity? -‐ Average distance between points, -‐ maximum distance -‐ minimum distance -‐ Distance between means or medoids
How many clusters? -‐ Clustering creates a dendrogram (a tree) -‐ Threshold based on max number of clusters
or based on distance between merges di
stan
ce
28-‐Oct-‐14 38
Lecture 12 - !!!
Fei-Fei Li!
Agglomera5ve Hierarchical Clustering
28-‐Oct-‐14 39
technical
note
Different ways to define the “nearest clusters”:
Lecture 12 - !!!
Fei-Fei Li!
Conclusions: Agglomera5ve Clustering
Good • Simple to implement, widespread applica5on • Clusters have adap5ve shapes • Provides a hierarchy of clusters Bad • May have imbalanced clusters • S5ll have to choose number of clusters or threshold
• Need to use an “ultrametric” to get a meaningful hierarchy
28-‐Oct-‐14 40
Lecture 12 - !!!
Fei-Fei Li!
What we have learned today?
• Introduc5on to segmenta5on and clustering • Gestalt theory for perceptual grouping • Agglomera5ve clustering
28-‐Oct-‐14 41