![Page 1: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/1.jpg)
Statistical Parsing and CKY Algorithm
Many slides from Ray Mooney and Michael Collins
Instructor: Wei Xu Ohio State University
![Page 2: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/2.jpg)
TA Office Hours for HW#2• Dreese 390: - 03/28 Tue 10:00AM-12:00 noon - 03/30 Thu 10:00AM-12:00 noon - 04/04 Tue 10:00AM-12:00 noon
• Readings: - textbook http://ciml.info/dl/v0_99/ciml-v0_99-ch17.pdf - slide #28,29.30: https://cocoxu.github.io/courses/
5525_slides_spring17/15_more_memm.pdf
Wuwei Lan
![Page 3: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/3.jpg)
Syntactic Parsing
Syntax
![Page 4: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/4.jpg)
Parsing• Given a string of terminals and a CFG, determine if the string
can be generated by the CFG: - also return a parse tree for the string - also return all possible parse trees for the string
• Must search space of derivations for one that derives the given string. - Top-Down Parsing - Bottom-Up Parsing
![Page 5: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/5.jpg)
Simple CFG for ATIS English
S → NP VP S → Aux NP VP S → VP NP → Pronoun NP → Proper-Noun NP → Det Nominal Nominal → Noun Nominal → Nominal Noun Nominal → Nominal PP VP → Verb VP → Verb NP VP → VP PP PP → Prep NP
Det → the | a | that | this Noun → book | flight | meal | money Verb → book | include | prefer Pronoun → I | he | she | me Proper-Noun → Houston | NWA Aux → does Prep → from | to | on | near | through
Grammar Lexicon
![Page 6: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/6.jpg)
S
VP
Verb NP
book Det Nominal
that Noun
flight
book that flight
Parsing Example
![Page 7: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/7.jpg)
Top Down ParsingS
NP VP
Pronoun
• Start searching space of derivations for the start symbol.
![Page 8: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/8.jpg)
S
NP VP
Pronoun
bookX
Top Down Parsing
![Page 9: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/9.jpg)
S
NP VP
ProperNoun
Top Down Parsing
![Page 10: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/10.jpg)
S
NP VP
ProperNoun
bookX
Top Down Parsing
![Page 11: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/11.jpg)
S
NP VP
Det Nominal
Top Down Parsing
![Page 12: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/12.jpg)
S
NP VP
Det Nominal
bookX
Top Down Parsing
![Page 13: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/13.jpg)
S
Aux NP VP
Top Down Parsing
![Page 14: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/14.jpg)
S
Aux NP VP
bookX
Top Down Parsing
![Page 15: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/15.jpg)
S
VP
Top Down Parsing
![Page 16: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/16.jpg)
S
VP
Verb
Top Down Parsing
![Page 17: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/17.jpg)
S
VP
Verb
book
Top Down Parsing
![Page 18: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/18.jpg)
S
VP
Verb
bookX
that
Top Down Parsing
![Page 19: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/19.jpg)
S
VP
Verb NP
Top Down Parsing
![Page 20: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/20.jpg)
S
VP
Verb NP
book
Top Down Parsing
![Page 21: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/21.jpg)
S
VP
Verb NP
book Pronoun
Top Down Parsing
![Page 22: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/22.jpg)
S
VP
Verb NP
book Pronoun
Xthat
Top Down Parsing
![Page 23: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/23.jpg)
S
VP
Verb NP
book ProperNoun
Top Down Parsing
![Page 24: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/24.jpg)
S
VP
Verb NP
book ProperNoun
Xthat
Top Down Parsing
![Page 25: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/25.jpg)
S
VP
Verb NP
book Det Nominal
Top Down Parsing
![Page 26: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/26.jpg)
S
VP
Verb NP
book Det Nominal
that
Top Down Parsing
![Page 27: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/27.jpg)
S
VP
Verb NP
book Det Nominal
that Noun
Top Down Parsing
![Page 28: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/28.jpg)
S
VP
Verb NP
book Det Nominal
that Noun
flight
Top Down Parsing
![Page 29: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/29.jpg)
book that flight
• Start searching space of reverse derivations from the terminal symbols in the string.
Bottom Up Parsing
![Page 30: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/30.jpg)
book that flight
Noun
Bottom Up Parsing
![Page 31: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/31.jpg)
book that flight
Noun
Nominal
Bottom Up Parsing
![Page 32: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/32.jpg)
book that flight
Noun
Nominal Noun
Nominal
Bottom Up Parsing
![Page 33: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/33.jpg)
book that flight
Noun
Nominal Noun
Nominal
X
Bottom Up Parsing
![Page 34: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/34.jpg)
book that flight
Noun
Nominal PP
Nominal
Bottom Up Parsing
![Page 35: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/35.jpg)
book that flight
Noun Det
Nominal PP
Nominal
Bottom Up Parsing
![Page 36: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/36.jpg)
book that flight
Noun Det
NP
Nominal
Nominal PP
Nominal
Bottom Up Parsing
![Page 37: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/37.jpg)
book that
Noun Det
NP
Nominal
flight
Noun
Nominal PP
Nominal
Bottom Up Parsing
![Page 38: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/38.jpg)
book that
Noun Det
NP
Nominal
flight
Noun
Nominal PP
Nominal
Bottom Up Parsing
![Page 39: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/39.jpg)
book that
Noun Det
NP
Nominal
flight
Noun
S
VP
Nominal PP
Nominal
Bottom Up Parsing
![Page 40: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/40.jpg)
book that
Noun Det
NP
Nominal
flight
Noun
S
VP
X
Nominal PP
Nominal
Bottom Up Parsing
![Page 41: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/41.jpg)
book that
Noun Det
NP
Nominal
flight
Noun
Nominal PP
Nominal
X
Bottom Up Parsing
![Page 42: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/42.jpg)
book that
Verb Det
NP
Nominal
flight
Noun
Bottom Up Parsing
![Page 43: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/43.jpg)
book that
Verb
VP
Det
NP
Nominal
flight
Noun
Bottom Up Parsing
![Page 44: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/44.jpg)
Det
book that
Verb
VP
S
NP
Nominal
flight
Noun
Bottom Up Parsing
![Page 45: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/45.jpg)
Det
book that
Verb
VP
S
XNP
Nominal
flight
Noun
Bottom Up Parsing
![Page 46: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/46.jpg)
book that
Verb
VP
VP
PP
Det
NP
Nominal
flight
Noun
Bottom Up Parsing
![Page 47: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/47.jpg)
book that
Verb
VP
VP
PP
Det
NP
Nominal
flight
Noun
X
Bottom Up Parsing
![Page 48: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/48.jpg)
book that
Verb
VP
Det
NP
Nominal
flight
Noun
NP
Bottom Up Parsing
![Page 49: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/49.jpg)
book that
Verb
VP
Det
NP
Nominal
flight
Noun
Bottom Up Parsing
![Page 50: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/50.jpg)
book that
Verb
VP
Det
NP
Nominal
flight
Noun
S
Bottom Up Parsing
![Page 51: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/51.jpg)
Top Down vs. Bottom Up• Top down never explores options that will not lead to a full
parse, but can explore many options that never connect to the actual sentence. • Bottom up never explores options that do not connect to the
actual sentence but can explore options that can never lead to a full parse. • Relative amounts of wasted search depend on how much the
grammar branches in each direction.
![Page 52: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/52.jpg)
CYK Algorithm
Syntax
![Page 53: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/53.jpg)
Dynamic Programming Parsing• CKY (Cocke-Kasami-Younger) algorithm based on bottom-up
parsing and requires first normalizing the grammar. • First grammar must be converted to Chomsky normal form
(CNF) in which productions must have either exactly 2 non-terminal symbols on the RHS or 1 terminal symbol (lexicon rules). • Parse bottom-up storing phrases formed from all substrings
in a triangular table (chart).
![Page 54: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/54.jpg)
Dynamic Programming• a general algorithm design technique for solving problems
defined by recurrences with overlapping subproblems • first invented by Richard Bellman in 1950s • “programming” here means “planning” or finding an optimal
program, as also seen in the term “linear programming” • Main idea: • setup a recurrence of smaller subproblems • solve subproblems once and record solutions in a table
(avoid any recalculation)
![Page 55: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/55.jpg)
![Page 56: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/56.jpg)
ATIS English Grammar Conversion
S → NP VP S → Aux NP VP
S → VP
NP → Pronoun NP → Proper-Noun NP → Det Nominal Nominal → Noun Nominal → Nominal Noun Nominal → Nominal PP VP → Verb VP → Verb NP VP → VP PP PP → Prep NP
Original Grammar Chomsky Normal FormS → NP VP S → X1 VP X1 → Aux NP S → book | include | prefer S → Verb NP S → VP PP NP → I | he | she | me NP → Houston | NWA NP → Det Nominal Nominal → book | flight | meal | money Nominal → Nominal Noun Nominal → Nominal PP VP → book | include | prefer VP → Verb NP VP → VP PP PP → Prep NP
Note that, although not shown here, original grammar contain all the lexical entires.
![Page 57: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/57.jpg)
Exercise
![Page 58: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/58.jpg)
CKY Parser Book the flight through Houston
i= 0
1
2
3
4
j= 1 2 3 4 5
Cell[i,j] contains all constituents (non-terminals) covering words i +1 through j
![Page 59: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/59.jpg)
CKY Parser
i= 0
1
2
3
4
Cell[i,j] contains all constituents (non-terminals) covering words i +1 through j
Book the flight through Houstonj= 1 2 3 4 5
![Page 60: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/60.jpg)
CKY Parser
S, VP, Verb, Nominal, Noun
Det
Nominal, Noun
None
NP
Book the flight through Houston
![Page 61: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/61.jpg)
CKY Parser
S, VP, Verb, Nominal, Noun
Det
Nominal, Noun
None
NP
VP
Book the flight through Houston
![Page 62: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/62.jpg)
CKY Parser
S, VP, Verb, Nominal, Noun
Det
Nominal, Noun
None
NP
VPS
Book the flight through Houston
![Page 63: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/63.jpg)
CKY Parser
S, VP, Verb, Nominal, Noun
Det
Nominal, Noun
None
NP
VPS
Book the flight through Houston
![Page 64: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/64.jpg)
CKY Parser
S, VP, Verb, Nominal, Noun
Det
Nominal, Noun
None
NP
VPS
Prep
None
None
None
Book the flight through Houston
![Page 65: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/65.jpg)
CKY Parser
S, VP, Verb, Nominal, Noun
Det
Nominal, Noun
None
NP
VPS
Prep
None
None
None
NP ProperNoun
PP
Book the flight through Houston
![Page 66: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/66.jpg)
CKY Parser
S, VP, Verb, Nominal, Noun
Det
Nominal, Noun
None
NP
VPS
Prep
None
None
None
NP ProperNoun
PP
Nominal
Book the flight through Houston
![Page 67: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/67.jpg)
CKY Parser
S, VP, Verb, Nominal, Noun
Det
Nominal, Noun
None
NP
VPS
Prep
None
None
None
NP ProperNoun
PP
Nominal
NP
Book the flight through Houston
![Page 68: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/68.jpg)
CKY Parser
S, VP, Verb, Nominal, Noun
Det
Nominal, Noun
None
NP
VPS
Prep
None
None
None
NP ProperNoun
PP
Nominal
NP
VP
Book the flight through Houston
![Page 69: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/69.jpg)
CKY Parser
S, VP, Verb, Nominal, Noun
Det
Nominal, Noun
None
NP
VPS
Prep
None
None
None
NP ProperNoun
PP
Nominal
NP
SVP
Book the flight through Houston
![Page 70: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/70.jpg)
CKY Parser
S, VP, Verb, Nominal, Noun
Det
Nominal, Noun
None
NP
VPS
Prep
None
None
None
NP ProperNoun
PP
Nominal
NP
VPSVP
Book the flight through Houston
![Page 71: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/71.jpg)
CKY Parser
S, VP, Verb, Nominal, Noun
Det
Nominal, Noun
None
NP
VPS
Prep
None
None
None
NP ProperNoun
PP
Nominal
NP
VPSVP
S
Book the flight through Houston
![Page 72: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/72.jpg)
CKY Parser
S, VP, Verb, Nominal, Noun
Det
Nominal, Noun
None
NP
VPS
Prep
None
None
None
NP ProperNoun
PP
Nominal
NP
VPSVP
S Parse Tree #1
Book the flight through Houston
![Page 73: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/73.jpg)
CKY Parser
S, VP, Verb, Nominal, Noun
Det
Nominal, Noun
None
NP
VPS
Prep
None
None
None
NP ProperNoun
PP
Nominal
NP
VPSVP
S Parse Tree #2
Book the flight through Houston
![Page 74: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/74.jpg)
The Problem with Parsing: Ambiguity
INPUT:She announced a program to promote safety in trucks and vans
+POSSIBLE OUTPUTS:
S
NP
She
VP
announced NP
NP
a program
VP
to promote NP
safety PP
in NP
trucks and vans
S
NP
She
VP
announced NP
NP
NP
a program
VP
to promote NP
safety PP
in NP
trucks
and NP
vans
S
NP
She
VP
announced NP
NP
a program
VP
to promote NP
NP
sa fety PP
in NP
trucks
and NP
va ns
S
NP
Sh e
VP
announced NP
NP
a program
VP
to promote NP
safety
PP
in NP
trucks and vans
S
NP
She
VP
announced NP
NP
NP
a program
VP
to promote NP
safe ty
PP
in NP
trucks
and NP
va ns
S
NP
She
VP
announced NP
NP
NP
a program
VP
to promote NP
safe ty
PP
in NP
trucks and va ns
And there are more...
![Page 75: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/75.jpg)
Probabilistic Context Free Grammars (PCFG)
Syntax
![Page 76: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/76.jpg)
![Page 77: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/77.jpg)
![Page 78: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/78.jpg)
![Page 79: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/79.jpg)
![Page 80: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/80.jpg)
![Page 81: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/81.jpg)
![Page 82: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/82.jpg)
![Page 83: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/83.jpg)
![Page 84: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/84.jpg)
![Page 85: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/85.jpg)
![Page 86: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/86.jpg)
![Page 87: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/87.jpg)
![Page 88: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/88.jpg)
![Page 89: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/89.jpg)
![Page 90: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/90.jpg)
![Page 91: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/91.jpg)
![Page 92: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/92.jpg)
![Page 93: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/93.jpg)
split point
![Page 94: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/94.jpg)
![Page 95: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/95.jpg)
O(n3|N|3)
O(n2) for l, i choices
![Page 96: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/96.jpg)
![Page 97: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/97.jpg)
S(saw)
NP(man)
DT(the)
the
NN(man)
man
VP(saw)
VP(saw)
Vt(saw)
saw
NP(dog)
DT(the)
the
NN(dog)
dog
PP(with)
IN(with)
with
NP(telescope)
DT(the)
the
NN(telescope)
telescope
p(t) = q(S(saw) !2 NP(man) VP(saw))⇥q(NP(man) !2 DT(the) NN(man))⇥q(VP(saw) !1 VP(saw) PP(with))⇥q(VP(saw) !1 Vt(saw) NP(dog))⇥q(PP(with) !1 IN(with) NP(telescope))⇥ . . .
Parsing with Lexicalized CFGs
I The new form of grammar looks just like a Chomsky normalform CFG, but with potentially O(|⌃|2 ⇥ |N |3) possible rules.
I Naively, parsing an n word sentence using the dynamicprogramming algorithm will take O(n3|⌃|2|N |3) time. But|⌃| can be huge!!
I Crucial observation: at most O(n2 ⇥ |N |3) rules can beapplicable to a given sentence w1, w2, . . . wn of length n.This is because any rules which contain a lexical item that isnot one of w1 . . . wn, can be safely discarded.
I The result: we can parse in O(n5|N |3) time.
![Page 98: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/98.jpg)
Dependency Parsing
Syntax
![Page 99: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/99.jpg)
![Page 100: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/100.jpg)
![Page 101: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/101.jpg)
![Page 102: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/102.jpg)
![Page 103: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/103.jpg)
![Page 104: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/104.jpg)
![Page 105: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/105.jpg)
![Page 106: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/106.jpg)
![Page 107: Statistical Parsing and CKY Algorithm · Dynamic Programming Parsing • CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar](https://reader033.vdocument.in/reader033/viewer/2022060520/604e49db925356260d6a0460/html5/thumbnails/107.jpg)