Download - Context Free Grammar (CFG)
![Page 1: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/1.jpg)
Context Free Grammar (CFG)
(These slides are modified from Dan Jurafsky’s slides.)
![Page 2: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/2.jpg)
2
Syntax By grammar, or syntax, we have in mind the kind of
implicit knowledge of your native language that you had mastered by the time you were 3 years old without explicit instruction
Not the kind of stuff you were later taught in “grammar” school
![Page 3: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/3.jpg)
3
Syntax Why should you care?
Grammars (and parsing) are key components in many applications
Grammar checkers
Dialogue management
Question answering
Information extraction
Machine translation
![Page 4: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/4.jpg)
4
Constituency A sequence of words that acts as a single unit
Noun phrases
Verb phrases
These units form coherent classes that behave in similar ways
For example, we can say that noun phrases can come before verbs
![Page 5: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/5.jpg)
5
Constituency For example, following are all noun phrases in
English...
Why? One piece of evidence is that they can all
![Page 6: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/6.jpg)
6
Context-Free Grammars Context-free grammars (CFGs)
Also known as Phrase structure grammars
Backus-Naur form
Consist of
Rules
Terminals
Non-terminals
![Page 7: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/7.jpg)
7
Context-Free Grammars Terminals
words
Non-Terminals
The constituents in a language Such as noun phrases, verb phrases and sentences
Rules
Rules are equations that consist of a single non-terminal on the left and any number of terminals and non-terminals on the right.
![Page 8: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/8.jpg)
8
Some NP Rules
Here are some rules for our noun phrases
Together, these describe two kinds of NPs. One that consists of a determiner followed by a nominal
And another that says that proper names are NPs.
The third rule illustrates two things: An explicit disjunction
A recursive definition
![Page 9: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/9.jpg)
9
L0 Grammar
![Page 10: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/10.jpg)
10
Derivations
A “derivation” is a sequence of rules applied to a string that accounts for that string.
![Page 11: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/11.jpg)
11
Definition
More formally, a CFG consists of
![Page 12: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/12.jpg)
12
Parsing
Parsing is the process of taking a string and a grammar and returning a (or multiple) parse tree(s) for that string
It is completely analogous to running a finite-state transducer with a tape
It’s just more powerful there are languages we can capture with CFGs that we can’t capture with finite-state machines.
![Page 13: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/13.jpg)
13
All the morning flights from Denver to Tampa leaving before 10
![Page 14: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/14.jpg)
14
All the morning flights from Denver to Tampa leaving before 10
Which word is central (most important)?
![Page 15: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/15.jpg)
15
All the morning flights from Denver to Tampa leaving before 10
Which word is central (most important)?
![Page 16: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/16.jpg)
16
NP Structure All the morning flights from Denver to Tampa leaving before 10
Clearly this NP is really about flights. That’s the central critical noun in this NP. Such word is called as the head.
We can dissect this kind of NP into the stuff that can come before the head, and the stuff that can come after it.
![Page 17: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/17.jpg)
17
Determiners Noun phrases can start with determiners...
Determiners can be
Simple lexical items: the, this, a, an, etc. A car
Or simple possessives John’s car
Or complex recursive versions of that John’s sister’s husband’s son’s car
![Page 18: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/18.jpg)
18
Nominals Contains the head and any pre- and post- modifiers of
the head.
Pre- Quantifiers, cardinals, ordinals...
Three cars
Adjectives and Aps
large cars
Ordering constraints
Three large cars
?large three cars
![Page 19: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/19.jpg)
19
Postmodifiers Three kinds
Prepositional phrases Flights from Seattle
Non-finite clauses Flights arriving before noon
Relative clauses Flights that serve breakfast
Same general (recursive) rule to handle these Nominal Nominal PP
Nominal Nominal GerundVP Nominal Nominal RelClause
![Page 20: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/20.jpg)
20
Agreement
Constraints that hold among various constituents.
For example, in English, determiners and the head nouns in NPs have to agree in their number.
Which of the following cannot be parsed by the rule
NP Det Nominal ?
(O) This flight
(O) Those flights
(X) This flights
(X) Those flight
![Page 21: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/21.jpg)
21
Agreement
Constraints that hold among various constituents.
For example, in English, determiners and the head nouns in NPs have to agree in their number.
Which of the following cannot be parsed by the rule
NP Det Nominal ?
This rule does not handle agreement! (The rule does not detect whether the agreement is correct or not.)
(O) This flight
(O) Those flights
(X) This flights
(X) Those flight
![Page 22: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/22.jpg)
22
Problem Our earlier NP rules are clearly deficient since they
don’t capture the agreement constraint
NP Det Nominal Accepts, and assigns correct structures, to grammatical examples
(this flight)
But its also happy with incorrect examples (*these flight)
Such a rule is said to overgenerate.
We’ll come back to this in a bit
![Page 23: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/23.jpg)
23
Verb Phrases
English VPs consist of a head verb along with 0 or more following constituents which we’ll call arguments.
![Page 24: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/24.jpg)
24
Subcategorization But, even though there are many valid VP rules in
English, not all verbs are allowed to participate in all those VP rules.
We can subcategorize the verbs in a language according to the sets of VP rules that they participate in.
This is a modern take on the traditional notion of transitive/intransitive.
Modern grammars may have 100s or such classes.
![Page 25: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/25.jpg)
25
Subcategorization Sneeze: John sneezed
Find: Please find [a flight to NY]NP
Give: Give [me]NP[a cheaper fare]NP
Help: Can you help [me]NP[with a flight]PP
Prefer: I prefer [to leave earlier]TO-VP
Told: I was told [United has a flight]S
…
![Page 26: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/26.jpg)
26
Subcategorization *John sneezed the book
*I prefer United has a flight
*Give with a flight
As with agreement phenomena, we need a way to formally express the constraints!
![Page 27: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/27.jpg)
27
Why?
Right now, the various rules for VPs overgenerate.
They permit the presence of strings containing verbs and arguments that don’t go together
For example
VP -> V NP therefore
Sneezed the book is a VP since “sneeze” is a verb and “the book” is a valid NP
![Page 28: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/28.jpg)
28
Possible CFG Solution
Possible solution for agreement.
Can use the same trick for all the verb/VP classes.
SgS -> SgNP SgVP
PlS -> PlNp PlVP
SgNP -> SgDet SgNom
PlNP -> PlDet PlNom
PlVP -> PlV NP
SgVP ->SgV Np
…
![Page 29: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/29.jpg)
29
CFG Solution for Agreement It works and stays within the power of CFGs
But its ugly
And it doesn’t scale all that well because of the interaction among the various constraints explodes the number of rules in our grammar.
![Page 30: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/30.jpg)
30
To conclude CFGs are simple and capture a lot of basic
syntactic structure in English.
But there are problems Don’t handle “agreement” and “subcategorization” Overgenerate!
Advanced grammars LFG HPSG
Construction grammar XTAG
![Page 31: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/31.jpg)
31
Treebanks Treebanks are corpora in which each sentence has
been paired with a parse tree (presumably the right one).
![Page 32: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/32.jpg)
32
Penn Treebank Penn TreeBank is a widely used treebank.
Most well known is the Wall Street Journal section of the Penn TreeBank.
1 M words from the 1987-1989 Wall Street Journal.
![Page 33: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/33.jpg)
33
Heads in Trees Finding heads in treebank trees is a task that arises
frequently in many applications.
Particularly important in statistical parsing
We can visualize this task by annotating the nodes of a parse tree with the heads of each corresponding node.
![Page 34: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/34.jpg)
34
Lexically Decorated Tree
![Page 35: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/35.jpg)
35
Head Finding The standard way to do head finding is to use a simple
set of tree traversal rules specific to each non-terminal in the grammar.
![Page 36: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/36.jpg)
4/25/2011 Speech and Language Processing - Jurafsky and Martin 36
Noun Phrases
![Page 37: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/37.jpg)
37
Treebank Uses Treebanks (and headfinding) are particularly critical to
the development of statistical parsers
Chapter 14
Also valuable to Corpus Linguistics
Investigating the empirical details of various constructions in a given language
![Page 38: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/38.jpg)
38
Dependency Grammars In CFG-style phrase-structure grammars the main
focus is on constituents.
But it turns out you can get a lot done with just binary relations among the words in an utterance.
In a dependency grammar framework, a parse is a tree where the nodes stand for the words in an utterance
The links between the words represent dependency relations between pairs of words. Relations may be typed (labeled), or not.
![Page 39: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/39.jpg)
4/25/2011 Speech and Language Processing - Jurafsky and Martin 39
Dependency Relations
![Page 40: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/40.jpg)
40
Dependency Parse
They hid the letter on the shelf
![Page 41: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/41.jpg)
41
Dependency Parsing The dependency approach has a number of
advantages over full phrase-structure parsing. Deals well with free word order languages where the
constituent structure is quite fluid
Parsing is much faster than CFG-bases parsers Dependency structure often captures the syntactic
relations needed by later applications CFG-based approaches often extract this same information from
trees anyway.
![Page 42: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/42.jpg)
42
Dependency Parsing There are two modern approaches to dependency
parsing
Optimization-based approaches that search a space of trees for the tree that best matches some criteria
Shift-reduce approaches that greedily take actions based on the current word and state.
![Page 43: Context Free Grammar (CFG)](https://reader031.vdocument.in/reader031/viewer/2022020911/62019a63db00f37881115f6b/html5/thumbnails/43.jpg)
43
Summary Context-free grammars can be used to model
various facts about the syntax of a language.
When paired with parsers, such grammars constitute a critical component in many applications.
Constituency is a key phenomena easily captured with CFG rules. But agreement and subcategorization do pose significant
problems
Treebanks pair sentences in corpus with their corresponding trees.