6601: NATURAL LANGUAGE PROCESSINGDaniel Kohlsdorf, Thad Starner

Wednesday, November 27, 13

LANGUAGE MODELS

Wednesday, November 27, 13

NGRAM MODELS

That's great, it starts with an earthquake, birds and snakes, an aeroplane -Lenny Bruce is not afraid. Eye of a hurricane, to yourself churn -world serves its own needs, regardless of your own needs. Feed it up a knock,speed, grunt no, strength no. Ladder structure clatter with fear of height,down height. Wire in a fire, represent the seven games in a government forhire and a combat site. Left her, wasn't coming in a hurry with the furiesbreathing down your neck. Team by team reporters baffled, trump, tetheredcrop. Look at that low plane! Fine then. Uh oh, overflow, population,common group, but it'll do. Save yourself, serve yourself. World serves itsown needs, listen to your heart bleed. Tell me with the rapture and thereverent in the right - right. You vitriolic, patriotic, slam, fight, brightlight, feeling pretty psyched.

It's the end of the world as we know it.It's the end of the world as we know it.It's the end of the world as we know it and I feel fine.

n = 1n = 2n = 3n = 4

Wednesday, November 27, 13

COUNTING NGRAMS

Corpus: latin Body

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Region 1

P(ngram4)

Wednesday, November 27, 13

CLASSIFICATION1234567891011121315 1234567891011121315

1 2

P(Spam)

P(EMAIL|SPAM) P(EMAIL|SPAM)

Wednesday, November 27, 13

MIXTURES OF MULTINOMIALS1234567891011121315 1234567891011121315

1 2

Unobserved

Observed

Wednesday, November 27, 13

DISCRETE HIDDEN MARKOV MOEDL1234567891011121315 1234567891011121315

Transitions

Observations

Wednesday, November 27, 13

GRAMMARS

Wednesday, November 27, 13

CONTEXT FREE GRAMMARWednesday, November 27, 13

CYK - ALGORITHM / PARSINGWednesday, November 27, 13

IMAGE COMPOSITIONWednesday, November 27, 13

EXPECTATION GRAMMARSWednesday, November 27, 13