Download - Context-Aware Clustering
![Page 1: Context-Aware Clustering](https://reader036.vdocument.in/reader036/viewer/2022062810/56815e76550346895dccf929/html5/thumbnails/1.jpg)
1
Context-Aware Clustering
Junsong Yuan and Ying WuEECS Dept., Northwestern University
![Page 2: Context-Aware Clustering](https://reader036.vdocument.in/reader036/viewer/2022062810/56815e76550346895dccf929/html5/thumbnails/2.jpg)
2
Contextual pattern and co-occurrences
?
Spatial contexts provide useful cues for clustering
![Page 3: Context-Aware Clustering](https://reader036.vdocument.in/reader036/viewer/2022062810/56815e76550346895dccf929/html5/thumbnails/3.jpg)
3
K-means revisit
Assumption: data samples are independent
Binary label indicator
Limitation: contextual information of spatial dependency is not considered in clustering data samples
EM Update
![Page 4: Context-Aware Clustering](https://reader036.vdocument.in/reader036/viewer/2022062810/56815e76550346895dccf929/html5/thumbnails/4.jpg)
4
Clustering higher-level patterns
• Regularized k-means
Distortion in original feature space
Distortion in hamming spacecharactering contextual patterns
Same as traditionalK-means clustering
Regularization term due tocontextual patterns
Not a smooth term!
![Page 5: Context-Aware Clustering](https://reader036.vdocument.in/reader036/viewer/2022062810/56815e76550346895dccf929/html5/thumbnails/5.jpg)
5
Chicken and Egg Problem• Hamming distance in clustering contextual patterns
• Matrix form
• Cannot minimize J1 and J2 separately !
J1 is coupled with J2
![Page 6: Context-Aware Clustering](https://reader036.vdocument.in/reader036/viewer/2022062810/56815e76550346895dccf929/html5/thumbnails/6.jpg)
6
Decoupling
Fix Update
Fix Update
![Page 7: Context-Aware Clustering](https://reader036.vdocument.in/reader036/viewer/2022062810/56815e76550346895dccf929/html5/thumbnails/7.jpg)
7
Nested-EM solution
NestedE-step
M-stepUpdate and separately
the nested-EM algorithm can converge in finite steps.
Theorem of convergence
![Page 8: Context-Aware Clustering](https://reader036.vdocument.in/reader036/viewer/2022062810/56815e76550346895dccf929/html5/thumbnails/8.jpg)
8
Simulation results (feature space)
![Page 9: Context-Aware Clustering](https://reader036.vdocument.in/reader036/viewer/2022062810/56815e76550346895dccf929/html5/thumbnails/9.jpg)
9
Simulation results (spatial space)
![Page 10: Context-Aware Clustering](https://reader036.vdocument.in/reader036/viewer/2022062810/56815e76550346895dccf929/html5/thumbnails/10.jpg)
10
![Page 11: Context-Aware Clustering](https://reader036.vdocument.in/reader036/viewer/2022062810/56815e76550346895dccf929/html5/thumbnails/11.jpg)
11
K-m
eansInitialization
![Page 12: Context-Aware Clustering](https://reader036.vdocument.in/reader036/viewer/2022062810/56815e76550346895dccf929/html5/thumbnails/12.jpg)
12
1st roundFinal Phrases
![Page 13: Context-Aware Clustering](https://reader036.vdocument.in/reader036/viewer/2022062810/56815e76550346895dccf929/html5/thumbnails/13.jpg)
13
![Page 14: Context-Aware Clustering](https://reader036.vdocument.in/reader036/viewer/2022062810/56815e76550346895dccf929/html5/thumbnails/14.jpg)
14
![Page 15: Context-Aware Clustering](https://reader036.vdocument.in/reader036/viewer/2022062810/56815e76550346895dccf929/html5/thumbnails/15.jpg)
15
![Page 16: Context-Aware Clustering](https://reader036.vdocument.in/reader036/viewer/2022062810/56815e76550346895dccf929/html5/thumbnails/16.jpg)
16
Multiple-feature clustering• Dataset: handwritten numerical (‘0’-‘9’) from UCI data set
– Each digit has three different types of features– Contextual pattern corresponds to compositional feature
• Different types of features serve as contexts of each other– Clustering each type of features into 10 “words”– Clustering 10 “phrases” based on a word-lexicon of size 3x10
![Page 17: Context-Aware Clustering](https://reader036.vdocument.in/reader036/viewer/2022062810/56815e76550346895dccf929/html5/thumbnails/17.jpg)
17
Conclusion• A context-aware clustering formulation proposed
– Targets on higher-level compositional patterns in terms of co-occurrences
– Discovered contextual patterns can feed back to improve the primitive feature clustering
• An efficient nested-EM solution which is guaranteed to converge in finite steps
• Successful applications in image pattern discovery and multiple-feature clustering– Can be applied to other general clustering problems