random swap em algorithm for gmm and image segmentation qinpei zhao, ville hautamäki, ismo...
TRANSCRIPT
Random Swap EM algorithm for GMM and Image Segmentation
Qinpei Zhao, Ville Hautamäki, Ismo Kärkkäinen, Pasi Fränti
Speech & Image Processing UnitDepartment of Computer Science, University of Joensuu
Box 111, Fin-80101 JoensuuFINLAND
Outline
Background & StatusRS-EMApplication
Background: Mixture Model
Background: EM algorithm
EM algorithm -> {α, Θ} E-step (Expectation):
M-step (Maximization):
Iterate E,M step until convergence α- mixing coefficient
Θ- model parameters, eg. {μ,∑}
( 1) ( 1)( , ) [log ( , | ) | , ]i iQ E p X Y X
( ) ( 1)argmax ( , )i iQ
Local MaximaLet’s describe it as mountain climbing……
600km
2160m 3099m
Initialization Effect
Initialization and Result(1) Initialization and Result(2)
Sub-optimal Example
The situation of local maxima trap
Status
Standard EM for Mixture Models(1977) Deterministic Annealing EM (DAEM) (1998) Split-Merge EM (SMEM) (2000) Greedy EM (2002) RS-EM coming…
Outline
Background & StatusRS-EM (Random Swap)Application
RSEM: Motivations Random manner Prevent from staying near the unstable or
hyperbolic fixed points of EM. Prevent from its stable fixed points corresponding
to insignificant local maxima of the likelihood function
Avoid the slow convergence of EM algorithm Less sensitive to its initialization
Formulas SMEM
Greedy EM
RSEM
Random Swap EMAfter EM
Afte
r Sw
ap
After EM
Comparisons(1)
Comparisons(2)
Q1 Q2 S1 S4
Outline
Background & StatusRS-EMApplication
Application Image Segmentation Color Quantization Image Retrieval ……
Conclusion Introduce Randomization into algorithm Performs better Without heavy time complexity Wider applications
Thanks!☺