type-based mcmcpliang/papers/type-naacl2010-talk.pdf · type-based mcmc naacl 2010 { los angeles...

Type-Based MCMC

NAACL 2010 – Los Angeles

Percy Liang Michael I. Jordan Dan Klein

Type-Based MCMC

NAACL 2010 – Los Angeles

Percy Liang Michael I. Jordan Dan Klein

Learning latent-variable models

· · · DT NNS VBD · · ·

· · · some stocks rose · · ·

· · · NN NNS VBD · · ·

· · · tech stocks fell · · ·

· · · DT NNS VBD · · ·

· · · the stocks soared · · ·

· · · WRB NNS VBD · · ·

· · · how stocks dipped · · ·

· · · DT NNS VBD · · ·

· · · many stocks created · · ·

· · · DT NNS VBD · · ·

· · · NN NNS VBD · · ·

· · · DT NNS VBD · · ·

· · · WRB NNS VBD · · ·

· · · DT NNS VBD · · ·

· · · NN NNS VBD · · ·

· · · DT NNS VBD · · ·

· · · WRB NNS VBD · · ·

· · · DT NNS VBD · · ·

· · · NN NNS VBD · · ·

· · · DT NNS VBD · · ·

· · · WRB NNS VBD · · ·

· · · DT NNS VBD · · ·

· · · NN NNS VBD · · ·

· · · DT NNS VBD · · ·

· · · WRB NNS VBD · · ·

· · · DT NNS VBD · · ·

Variables are highly dependent

· · · DT NNS VBD · · ·

· · · NN NNS VBD · · ·

· · · DT NNS VBD · · ·

· · · WRB NNS VBD · · ·

· · · DT NNS VBD · · ·

Simple: token-based (e.g., Gibbs sampler)

· · · DT NNS VBD · · ·

· · · NN NNS VBD · · ·

· · · DT NNS VBD · · ·

· · · WRB NNS VBD · · ·

· · · DT NNS VBD · · ·

Simple: token-based (e.g., Gibbs sampler)⇒ local optima, slow mixing

· · · DT NNS VBD · · ·

· · · NN NNS VBD · · ·

· · · DT NNS VBD · · ·

· · · WRB NNS VBD · · ·

· · · DT NNS VBD · · ·

Classic: sentence-based (e.g., EM)

· · · DT NNS VBD · · ·

· · · NN NNS VBD · · ·

· · · DT NNS VBD · · ·

· · · WRB NNS VBD · · ·

· · · DT NNS VBD · · ·

Structural dependencies

· · · DT NNS VBD · · ·

· · · NN NNS VBD · · ·

· · · DT NNS VBD · · ·

· · · WRB NNS VBD · · ·

· · · DT NNS VBD · · ·

dynamic programming

· · · DT NNS VBD · · ·

· · · NN NNS VBD · · ·

· · · DT NNS VBD · · ·

· · · WRB NNS VBD · · ·

· · · DT NNS VBD · · ·

dynamic programming

New: type-based

· · · DT NNS VBD · · ·

· · · NN NNS VBD · · ·

· · · DT NNS VBD · · ·

· · · WRB NNS VBD · · ·

· · · DT NNS VBD · · ·

dynamic programming

New: type-based

Parameter dependencies

· · · DT NNS VBD · · ·

· · · NN NNS VBD · · ·

· · · DT NNS VBD · · ·

· · · WRB NNS VBD · · ·

· · · DT NNS VBD · · ·

dynamic programming

New: type-based

exchangeability

Dependencies between adjacent variables

NN VBZ NN NNworker demands meeting resistance

NN NNS VBG NNworker demands meeting resistance

Token-based: update only one variable at a time

NN VBZ VBG NNworker demands meeting resistance

Token-based: update only one variable at a timeProblem: need to go downhill before uphill

Sentence-based: update all variables in a sentence

Token-based: update only one variable at a timeProblem: need to go downhill before uphill

Dependencies between variables with shared parameters

VBZ VBD ...stocks rose ...VBZ VBD ...stocks fell ...VBZ VBD ...stocks took ...VBZ VBD ...stocks shot ...

NNS VBD ...stocks rose ...NNS VBD ...stocks fell ...NNS VBD ...stocks took ...NNS VBD ...stocks shot ...

NNS VBD ...stocks rose ...VBZ VBD ...stocks fell ...VBZ VBD ...stocks took ...VBZ VBD ...stocks shot ...

Token-based:

NNS VBD ...stocks rose ...NNS VBD ...stocks fell ...VBZ VBD ...stocks took ...VBZ VBD ...stocks shot ...

Token-based:

NNS VBD ...stocks rose ...NNS VBD ...stocks fell ...NNS VBD ...stocks took ...VBZ VBD ...stocks shot ...

Token-based:

Token-based: need to go downhill a lot before going uphill

Type-based: update all variables of one type

1. Parameter dependencies create deeper valleys

2. Sentence-based cannot handle these dependencies

What exactly is a type?

How can we update all variables of a type efficiently?

Formal setup

Parameters θ: vector of conditional probabilities

Formal setup

θT:V,N: from state V, probability of transitioning to state N

Formal setup

p(θ) is product of Dirichlets [prior]

Formal setup

Choices z: specifies values of latent and observed variables

Formal setup

Choices z: specifies values of latent and observed variables8 9

· · · V N · · ·stocks

Formal setup

z ∈ z[T :V,N at 8]

· · · V N · · ·stocks

Formal setup

z ∈ z j(z) [parameter used][T :V,N at 8] T:V,N

· · · V N · · ·stocks

Formal setup

· · · V N · · ·stocks

p(z | θ) =∏

z∈z θj(z) [likelihood]

Formal setup

· · · V N · · ·stocks

p(z | θ) =∏

p(z) =∫p(z | θ)p(θ)dθ [marginal likelihood]

Formal setup

· · · V N · · ·stocks

p(z | θ) =∏

Observations x: observed part of z (e.g., the words)

Formal setup

· · · V N · · ·stocks

p(z | θ) =∏

Observations x: observed part of z (e.g., the words)

Goal: sample from p(z | x) [not p(z | x,θ)]6

Exchangeability

Sufficient statistics n: # times parameters were used in z

Exchangeability

Sufficient statistics n: # times parameters were used in znT:V,N: # times that state V transitioned to state N

Exchangeability

Rewrite likelihood:

p(z | θ) =∏

j θnj

Exchangeability

Rewrite likelihood:

p(z | θ) =∏

j θnj

j = simple function of n and θ

Exchangeability

Rewrite likelihood:

p(z | θ) =∏

j θnj

p(z) = simple function of n

Exchangeability

Rewrite likelihood:

p(z | θ) =∏

j θnj

p(z) = simple function of n (key: exchangeability)

Exchangeability

Rewrite likelihood:

p(z | θ) =∏

j θnj

· · · D V V · · ·many stocks rose

· · · D N V · · ·tech stocks were

Exchangeability

Rewrite likelihood:

p(z | θ) =∏

j θnj

· · · D N V · · ·many stocks rose

· · · D V V · · ·tech stocks were

Exchangeability

Rewrite likelihood:

p(z | θ) =∏

j θnj

Exchangeability

Rewrite likelihood:

p(z | θ) =∏

j θnj

z1 and z2 have same sufficient statistics

Exchangeability

Rewrite likelihood:

p(z | θ) =∏

j θnj

z1 and z2 have same sufficient statistics ⇒ p(z1) = p(z2)

Type of a variable: its dependent parameter components

For HMM, type = assignment to Markov blanket

· · · D V · · ·many stocks rose

type( ) = (D, stocks, V)

· · · D V · · ·tech stocks were

· · · D V · · ·the stocks have

· · · D V · · ·how stocks from

· · · D N V · · ·the stocks have

· · · D N V · · ·how stocks from

Assignments[VVNN]

· · · D V V · · ·the stocks have

Assignments[VVNN][VNVN]

· · · D V V · · ·how stocks from

Assignments[VVNN][VNVN][VNNV]

Assignments[VVNN][VNVN][VNNV][NVVN]

Assignments[VVNN][VNVN][VNNV][NVVN][NVNV]

Assignments[VVNN][VNVN][VNNV][NVVN][NVNV][NNVV]

p(assignment) only depends on number of Vs and Ns8

Sampling same-type variables

Goal: sample an assignment of a set of same-type variables

m = 0 m = 1 m = 2 m = 3 m = 4p0 [VVVV] p1 [VVVN]

p1 [VVNV]p1 [VNVV]p1 [NVVV]

p2 [VVNN]p2 [VNVN]p2 [VNNV]p2 [NVVN]p2 [NVNV]p2 [NNVV]

p3 [VNNN]p3 [NVNN]p3 [NNVN]p3 [NNNV]

p4 [NNNN]

Algorithm:

1. Choose m ∈ {0, . . . , B} with prob. ∝(

p4 [NNNN]

Algorithm:

1. Choose m ∈ {0, . . . , B} with prob. ∝(

2. Choose assignment uniformly from column m

p4 [NNNN]

Full algorithm

Iterate: · · · D V V · · ·many stocks rose

Full algorithm

Iterate:1. Choose a position

e.g., 2

Full algorithm

e.g., 2

2. Add variables with same typee.g., (D, stocks, V)

Full algorithm

e.g., 2

3. Sample me.g., 1 ⇒ {V, V, V, N}

· · · D V · · ·tech stocks were

· · · D V · · ·the stocks have

· · · D V · · ·how stocks from

Full algorithm

e.g., 2

3. Sample me.g., 1 ⇒ {V, V, V, N}

4. Sample assignmente.g., [VNVV]

Experimental setup: sampling algorithms

stocks

trading

worker

demands

meeting

resistance

stocks

investors

stocks

trading

worker

demands

meeting

resistance

stocks

investors

stocks

Token-based sampler (Token)1. Choose token

stocks

trading

worker

demands

meeting

resistance

stocks

investors

stocks

2. Update conditioned on rest

stocks

trading

worker

demands

meeting

resistance

stocks

investors

stocks

Sentence-based sampler (Sentence)1. Choose sentence

stocks

trading

worker

demands

meeting

resistance

stocks

investors

stocks

trading

worker

demands

meeting

resistance

stocks

investors

stocks

Type-based sampler (Type)1. Choose type

stocks

trading

worker

demands

meeting

resistance

stocks

investors

stocks

Type-based sampler (Type)1. Choose type

Experimental setup: models/tasks/datasets

Hidden Markov model (HMM)

Part-of-speech induction

Dataset: WSJ(49K sentences, 45 tags)

Unigram segmentation model (USM) [Goldwater et al., 2006]

l o o k a t t h e b o o k

Word segmentation

Dataset: CHILDES(9.7K sentences)

Word segmentation

Probabilistic tree-substitution grammar (PTSG) [Cohn et al., 2009]

Word segmentation

Token versus Type

HMM USM PTSG

Token versus Type

HMM USM PTSG

3 6 9 12

time (hr.)

-7.3e6

-6.9e6

-6.5e6

Token versus Type

HMM USM PTSG

3 6 9 12

time (hr.)

-7.3e6

-6.9e6

-6.5e6

Token versus Type

HMM USM PTSG

3 6 9 12

time (hr.)

-7.3e6

-6.9e6

-6.5e6

2 4 6 8

time (min.)

-2.1e5

-1.9e5

-1.7e5

Token versus Type

HMM USM PTSG

3 6 9 12

time (hr.)

-7.3e6

-6.9e6

-6.5e6

2 4 6 8

time (min.)

-2.1e5

-1.9e5

-1.7e5

3 6 9 12

time (hr.)

-6.1e6

-5.9e6

-5.7e6

-5.5e6

-5.3e6

Token versus Type

HMM USM PTSG

3 6 9 12

time (hr.)

-7.3e6

-6.9e6

-6.5e6

2 4 6 8

time (min.)

-2.1e5

-1.9e5

-1.7e5

3 6 9 12

time (hr.)

-6.1e6

-5.9e6

-5.7e6

-5.5e6

-5.3e6

3 6 9 12

time (hr.)

2 4 6 8

time (min.)

Sensitivity to initialization

(use few params.)η = 0

V V V Vworker demands meeting resistance

(use many params.)η = 1

P N D Vworker demands meeting resistance

l o o k a t t h e b o o k l o o k a t t h e b o o k

HMM USM PTSG

0.2 0.4 0.6 0.8 1.0

-7.2e6

-7.0e6

-6.8e6

-6.6e6

0.2 0.4 0.6 0.8 1.0

HMM USM PTSG

0.2 0.4 0.6 0.8 1.0

-7.2e6

-7.0e6

-6.8e6

-6.6e6

0.2 0.4 0.6 0.8 1.0

HMM USM PTSG

0.2 0.4 0.6 0.8 1.0

-7.2e6

-7.0e6

-6.8e6

-6.6e6

0.2 0.4 0.6 0.8 1.0

-3.4e5

-3.1e5

-2.7e5

-2.3e5

-1.9e5

0.2 0.4 0.6 0.8 1.0

HMM USM PTSG

0.2 0.4 0.6 0.8 1.0

-7.2e6

-7.0e6

-6.8e6

-6.6e6

0.2 0.4 0.6 0.8 1.0

-3.4e5

-3.1e5

-2.7e5

-2.3e5

-1.9e5

0.2 0.4 0.6 0.8 1.0

-5.7e6

-5.6e6

-5.5e6

-5.4e6

0.2 0.4 0.6 0.8 1.0

HMM USM PTSG

0.2 0.4 0.6 0.8 1.0

-7.2e6

-7.0e6

-6.8e6

-6.6e6

0.2 0.4 0.6 0.8 1.0

-3.4e5

-3.1e5

-2.7e5

-2.3e5

-1.9e5

0.2 0.4 0.6 0.8 1.0

-5.7e6

-5.6e6

-5.5e6

-5.4e6

0.2 0.4 0.6 0.8 1.0

Type less sensitive than Token

Alternatives

Can we get the gains of Type via simpler means?

Alternatives

• Annealing (Tokenanneal)

– Use p(z | x)1/T , temperature T from 5 to 1

Annealing

HMM USM PTSG

Annealing

HMM USM PTSG

3 6 9 12

time (hr.)

-7.3e6

-6.9e6

-6.5e6

3 6 9 12

time (hr.)

Annealing

HMM USM PTSG

3 6 9 12

time (hr.)

-7.3e6

-6.9e6

-6.5e6

3 6 9 12

time (hr.)

Tokenanneal

[anneal. hurts]

Annealing

HMM USM PTSG

3 6 9 12

time (hr.)

-7.3e6

-6.9e6

-6.5e6

2 4 6 8

time (min.)

-2.1e5

-1.9e5

-1.7e5

3 6 9 12

time (hr.)

2 4 6 8

time (min.)

Tokenanneal

[anneal. hurts] [anneal. helps]

Annealing

HMM USM PTSG

3 6 9 12

time (hr.)

-7.3e6

-6.9e6

-6.5e6

2 4 6 8

time (min.)

-2.1e5

-1.9e5

-1.7e5

3 6 9 12

time (hr.)

-6.1e6

-5.9e6

-5.7e6

-5.5e6

-5.3e6

3 6 9 12

time (hr.)

2 4 6 8

time (min.)

Tokenanneal

[anneal. hurts] [anneal. helps] [no effect]

Alternatives

– More (random) mobility through space, but insufficient

Alternatives

• Sentence-based (Sentence)

Sentence

2 4 6 8

time (min.)

-2.1e5

-1.9e5

-1.7e5

2 4 6 8

time (min.)

Sentence

2 4 6 8

time (min.)

-2.1e5

-1.9e5

-1.7e5

2 4 6 8

time (min.)

Sentence

2 4 6 8

time (min.)

-2.1e5

-1.9e5

-1.7e5

2 4 6 8

time (min.)

Sentence

Sentence performs comparably to Token,

but worse than Type

Sentence

2 4 6 8

time (min.)

-2.1e5

-1.9e5

-1.7e5

2 4 6 8

time (min.)

Sentence

Sentence performs comparably to Token,

but worse than Type

Sentence requires dynamic programming,

computationally more expensive than Token and Type

Alternatives

– Sentence handles structural dependencies

– Type handles parameter dependencies

Alternatives

– Sentence handles structural dependencies

– Type handles parameter dependencies

– Intuition: parameter dependencies more important inunsupervised learning

Summary and outlook

General strategy: update many dependent variables tractably

Summary and outlook

Variables sharing parameters very dependent

Type-based sampler updates exactly these variables

Summary and outlook

Techniques for tractability:

Old: exploit dynamic programming structure

Summary and outlook

Old: exploit dynamic programming structureNew: think about exchangeability

Summary and outlook

Tokens versus types:

Summary and outlook

• Older work operate on types (e.g., model merging)

Summary and outlook

• Older work operate on types (e.g., model merging)Larger updates, but greedy and brittle

Summary and outlook

• Recent methods operate on sentences or tokensSmaller updates, but softer and more robust

Summary and outlook

• Recent methods operate on sentences or tokensSmaller updates, but softer and more robust

Type-based sampling combines advantages of both

Thank you!

type-based mcmcpliang/papers/type-naacl2010-talk.pdf · type-based mcmc naacl 2010 { los angeles...

Documents

original irp- mcmc

mcmc annual report_2008

an introduction to mcmc for machine learning - vision...

metaphor for nlp (at naacl 2015)

dnssec mcmc amir

replica exchange mcmc

mcmc gains accrediation

distributional semantic models - tutorial at naacl-hlt

partnership working mcmc

schmidt mcmc 2010

trans-dimensional mcmc peter...

mcmc gibbs intro

the nltk framenet lexicon api - naacl: north american...

1 mcmc estimation mcmc = markov chain monte carlo an...

semantic role labeling tutorial - naacl hlt...

naacl hlt 2019

mcmc theory

mcmc sheldon ross

mcmc summary & conclusion

introduction to bayesian mcmc models - home (en) ·...