Dependent Hierarchical Normalized Random Measures for Dynamic Topic Modeling

Changyou Chen, Nan Ding and Wray Buntine

ICML 2012

Presented by: Mingyuan ZhouDuke University, ECE

October 24, 2012

Introduction

• NRM: normalized random measures with independent increments

• Superposition, subsampling and point transition of NRM• Dependent hierarchical NRM• Dynamic topic modeling

• Poisson process

• Completely random measures (CRM)

Normalized Random Measures

• Completely random measures (CRM)

Normalized Random Measures

• Slice sampling NRMsRef: Griffin, J.E. and Walker, S.G. Posterior simulation of normalized random measure mixtures. J. Comput. Graph. Stat., 2011.

Normalized Random Measures

• Normalized generalized gamma process

Normalized Random Measures

• Ideas:– Inherit topics from the previous time frame through three

dependency operators: • Superposition• Subsampling• Point transition

– Generate new topics

Dynamic topic modeling with dependent hierarchical NRMs

Dynamic topic modeling with dependent hierarchical NRMs

Dynamic topic modeling with dependent hierarchical NRMs

Dynamic topic modeling with dependent hierarchical NRMs

• Properties of the dependence operators

Dynamic topic modeling with dependent hierarchical NRMs

• Reformulated model

Dynamic topic modeling with dependent hierarchical NRMs

• Original and reformulated model

Dynamic topic modeling with dependent hierarchical NRMs

• Sampling under the Chinese restaurant metaphor

Sampling

• Sampling under the Chinese restaurant metaphor

Sampling

Sampling

Sampling

Sampling

• Power-law in the NGG

Experiments

Experiments

Experiments

Experiments

Experiments

Experiments

Conclusions