Download - Context-Aware Clustering
1
Context-Aware Clustering
Junsong Yuan and Ying WuEECS Dept., Northwestern University
2
Contextual pattern and co-occurrences
?
Spatial contexts provide useful cues for clustering
3
K-means revisit
Assumption: data samples are independent
Binary label indicator
Limitation: contextual information of spatial dependency is not considered in clustering data samples
EM Update
4
Clustering higher-level patterns
• Regularized k-means
Distortion in original feature space
Distortion in hamming spacecharactering contextual patterns
Same as traditionalK-means clustering
Regularization term due tocontextual patterns
Not a smooth term!
5
Chicken and Egg Problem• Hamming distance in clustering contextual patterns
• Matrix form
• Cannot minimize J1 and J2 separately !
J1 is coupled with J2
6
Decoupling
Fix Update
Fix Update
7
Nested-EM solution
NestedE-step
M-stepUpdate and separately
the nested-EM algorithm can converge in finite steps.
Theorem of convergence
8
Simulation results (feature space)
9
Simulation results (spatial space)
10
11
K-m
eansInitialization
12
1st roundFinal Phrases
13
14
15
16
Multiple-feature clustering• Dataset: handwritten numerical (‘0’-‘9’) from UCI data set
– Each digit has three different types of features– Contextual pattern corresponds to compositional feature
• Different types of features serve as contexts of each other– Clustering each type of features into 10 “words”– Clustering 10 “phrases” based on a word-lexicon of size 3x10
17
Conclusion• A context-aware clustering formulation proposed
– Targets on higher-level compositional patterns in terms of co-occurrences
– Discovered contextual patterns can feed back to improve the primitive feature clustering
• An efficient nested-EM solution which is guaranteed to converge in finite steps
• Successful applications in image pattern discovery and multiple-feature clustering– Can be applied to other general clustering problems