cs460/626 : natural language processing/speech nlp and the
TRANSCRIPT
CS460/626 : Natural Language Processing/Speech NLP and the WebProcessing/Speech, NLP and the Web
(Lecture 28– Grammar; Constituency, Dependency)Dependency)
Pushpak BhattacharyyaPushpak BhattacharyyaCSE Dept., IIT Bombay
21 t M h 201121st March, 2011
Grammar
A finite set of rules that generates only and all sentences of athat generates only and all sentences of a language.that assigns an appropriate structural g pp pdescription to each one.
Grammatical Analysis Techniques
Two main devices
Breaking up a StringLabeling the Constituents
– MorphologicalC i l
– SequentialHi hi l – Categorial
– Functional – Hierarchical– Transformational
Breaking up and LabelingSequential Breaking up
Seq enti l B e king p nd Mo phologi lSequential Breaking up and Morphological LabelingSequential Breaking up and Categorial Labeling Seque a ea g up a d Ca ego a abe gSequential Breaking up and Functional Labeling
Hierarchical Breaking upHierarchical Breaking upHierarchical Breaking up and Categorial LabelingHierarchical Breaking up and Functional Labelingg p g
Sequential Breaking up
• That student solved the problems.
that student solve ed the problem s+ + + + + +
Sequential Breaking up and Morphological LabelingMorphological Labeling
That student solved the problems.
th t t d t l d h blthat student solve ed the problem s
word word stem affix word stem affix
Sequential Breaking up and Categorial LabelingCategorial Labeling
This boy can solve the problem.this boy can solve the problem
Det N Aux V Det N
• They called her a taxi.They call ed taxi
Pron V Affi N
her
P on
a
DetPron V Affix NPron Det
Sequential Breaking up and Functional LabelingFunctional Labeling
They called taxiher a
Subject Verbal IndirectDirectSubject Verbal IndirectObject
Direct Object
They called taxiher a
Subject Verbal DirectObject
Indirect Object
Hierarchical Breaking up
Old men and women
Old Old men
andmen and women
andwomen
Old men and women Old men and women
womenandmenmenOld menOld
Hierarchical Breaking up and Categorial LabelingCategorial Labeling
Poor John ran away.
S
VPNP
V AdvA N
ran awayPoor John
Hierarchical Breaking up and F ti l L b liFunctional Labeling
• Immediate Constituent (IC) AnalysisImmediate Constituent (IC) Analysis• Construction types in terms of the function of the
constituents:– Predication (subject + predicate)– Modification (modifier + head)– Complementation (verbal + complement)– Subordination (subordinator + dependent unit)– Coordination (independent unit + coordinator)
Predication
[Birds]subject [fly]predicate
SS
PredicateSubject
Birds flyy
Modification
[A]modifier [flower]head
John [slept]head [in the room]modifier
S
PredicateSubject PredicateSubject
John H d M difiJohn Head Modifier
slept In the roomslept
ComplementationComplementationHe [saw]verbal [a lake]complement
S
PredicateSubject
He Verbal Complement
saw a lake
19/12/2004 ICON 2004 Tutorial
SubordinationSubordinationJohn slept [in]subordinator [the room]dependent unit
S
PredicateSubject
John Head Modifier
slept Subordinator Dependent Unit
the roomin
Coordination
[John came in time] independent unit [but]coordinatorp[Mary was not ready] independent unit
SS
C di tI d d t U it Independent UnitCoordinatorIndependent Unit Independent Unit
John came in time but Mary was not ready
An Example
SIn the morning, the sky looked much brighter.An Example
S
HeadModifier
Subordinator DU PredicateSubject
HeadHead Verbal ComplementModifierModifier
HeadModifier
In the morning, the sky looked much brighter
Hierarchical Breaking up and Categorial / Functional LabelingCategorial / Functional Labeling
Hierarchical Breaking up coupled with C t i l /F ti l L b li iCategorial /Functional Labeling is a very powerful device.B t th bi iti hi hBut there are ambiguities which demand something more powerful.E L f G dE.g., Love of God
Someone loves GodG d lGod loves someone
Hierarchical Breaking up
Categorial Labeling Functional Labeling
Love of God Love of God
Noun Ph
Prepositional Phrase
Head ModifierPhrase Phrase
DUSub
Godoflove love of God
Types of Generative GrammarypFinite State Model
( i l)(sequential)Phrase Structure Model
(sequential + hierarchical) + (categorial)
Transformational ModelTransformational Model (sequential + hierarchical + transformational) +
(categorial + functional)(categorial + functional)
Phrase Structure Grammar (PSG)Phrase Structure Grammar (PSG)
A phrase-structure grammar G consists of a f t l (V T S P) hfour tuple (V, T, S, P), where V is a finite set of alphabets (or vocabulary)
E g N V A Adv P NP VP AP AdvP PPE.g., N, V, A, Adv, P, NP, VP, AP, AdvP, PP, student, sing, etc.
T is a finite set of terminal symbols: T ⊂ VT is a finite set of terminal symbols: T ⊂ VE.g., student, sing, etc.
S is a distinguished non-terminal symbol, also g y ,called start symbol: S ∈ VP is a set of productions.
Noun Phrases
• John • the student • the intelligent student
NP NP NPNP
N
NP
NDet
NP
NDet AdjP
John studentthe studentthe intelligent
j
Noun Phrase
• his first five PhD students
NP
QuantDet Ord NN
five
Quant
his
Det
first
Ord
students
N
PhD
N
fivehis first studentsPhD
Noun PhraseNoun Phrase• The five best students of my class
NP
fi
Quant
th
Det NAP PP
fivethe studentsbest of my class
Verb Phrases• can sing • can hit the ball
VP VPVP
VAux
VP
NPAux V
singcan the ball
NP
can
Aux
hit
V
g
Verb PhraseVerb Phrase• Can give a flower to Mary
VP
NPAux V PP
a flower
NP
can
Aux
give
V
to Mary
PP
g o a y
Verb Phrase• may make John the chairman
VP
NPAux V NP
John
NP
may
Aux
make
V
the chairman
NP
y e c a a
Verb Phrase
• may find the book very interesting
VP
NPAux V AP
the bookmay find very interesting
Prepositional Phrases
• in the classroom • near the river
PPPP
NPPNPP
the rivernearthe classroomin
Adjective PhrasesAdjective Phrases• intelligent • very honest • fond of sweets
AP AP AP
A ADegree PPA
intelligent honestvery of sweetsfond
Adjective Phrase
• very worried that she might have done badly in the i tassignment
AP
S’Degree A S
very
Degree
worried
A
that she might have done badly in the
very worried
assignment
Phrase Structure Rules
Rewrite Rules:
• The boy hit the ball.
Rewrite Rules:(i) S NP VP(ii) NP D t N(ii) NP Det N(iii) VP V NP
h(iv) Det the(v) N man, ball(v) V hit
We interpret each rule X Y as the instruction rewrite X as Y.
DerivationDerivation• The boy hit the ball.
SentenceNP + VP (i)Det + N + VP (ii)Det + N + V + NP (iii)The + N + V + NP (iv)The + N + V + NP (iv)The + boy + V + NP (v)The + boy + hit + NP (vi)The + boy + hit + Det + N (ii)The + boy + hit + the + N (iv)The + boy + hit + the + ball (v)The + boy + hit + the + ball (v)
PSG Parse Tree
The boy hit the ball.S
VPNP
VND t NPVNDet
the
NP
Nb Dethitthe Nboy Dethit
the ballthe ball
PSG Parse Tree
John wrote those words in the Book of ProverbsSProverbs.S
VPNP
VPropN NP PP
NPP
John wrote thosewords
in
th of
NP PP
thebook
ofproverbs
Penn POS Tags
• John wrote those words in the Book of Proverbs.
[John/NNP ]t /VBDwrote/VBD
[ those/DT words/NNS ]in/IN [ the/DT Book/NN ][ the/DT Book/NN ]of/IN [ Proverbs/NNS ][ Proverbs/NNS ]
Penn Treebank• John wrote those words in the Book of Proverbs.
(S (NP-SBJ (NP John))(VP t(VP wrote
(NP those words)(PP-LOC in(NP (NP-TTL (NP the Book)(NP (NP TTL (NP the Book)
(PP of(NP Proverbs)))(NP Proverbs)))
PSG Parse Tree
Official trading in the shares will start in Pa is on No 6Paris on Nov 6. S
VPNP
NP
PP
PP
PPVAux
NAP
PPNPP
PPVAux
A
official trading will start on Nov 6in the shares in Paris
Penn POS Tagsg• Official trading in the shares will start in Paris on
N 6
[ Official/JJ trading/NN ]Nov 6.
in/IN [ the/DT shares/NNS ][ the/DT shares/NNS ]will/MD start/VB in/IN [ P i /NNP ][ Paris/NNP ]on/IN [ Nov./NNP 6/CD ]
Penn Treebank
• Official trading in the shares will start in Paris on Nov 6
( (S (NP-SBJ (NP Official trading)(PP in
Nov 6.
(PP in(NP the shares)))
(VP will(VP start(PP-LOC in
(NP Paris))(NP Paris))(PP-TMP on
(NP (NP Nov 6)( ( )
Penn POS Tag SsetAdjective: JJAdverb: RBCardinal Number: CDDeterminer: DTPreposition: INCoordinating Conjunction CCSubordinating Conjunction: INSingular Noun: NNPlural Noun: NNSPlural Noun: NNSPersonal Pronoun: PPProper Noun: NPVerb base form: VBVerb base form: VBModal verb: MDVerb (3sg Pres): VBZWh-determiner: WDTWh determiner: WDTWh-pronoun: WP
Diff b tDifference between constituency and dependencyconstituency and dependency
Constituency GrammarConstituency GrammarCategorical Uses part of speechContext Free Grammar (CFG)Basic elements Phrases
Dependency Grammar
FunctionalContext Free GrammarContext Free GrammarBasic elements Units of Predication/ Modification/ Complementation/Modification/ Complementation/ Subordination/ Co-ordination
Bridge between Constituency and Dependency parse
Constituency uses phrasesDependencies consist of Head-modifier combinationThis is a cricket bat.This is a cricket bat.
Cricket (Category: N, Functional: Adj)Bat (Category: N, Functional: N)
For languages which are free word order we use g gdependency parser to uncover the relations between the words.Raam ne Shaam ko dekha . (Ram saw Shyam)Sh k R d kh (R Sh )Shaam ko Ram ne dekha. (Ram saw Shyam)Case markers cling to the nouns they subordinate
Example of CG and DG output
Birds Fly.Birds Fly.
S S
NP VP Subject Predicate
N VBirds fly
Birds fly
Birds fly
Birds fly
Some probabilistic parsers and why they are used
Stanford, Collins, Charniack, RASPWhy Probabilistic parsersWhy Probabilistic parsers
For a single sentence we can have multiple parsesparses. Probability for the parse is calculated and then the parse with the highest probabilitythen the parse with the highest probability is selected.This is needed in many applications of NLP,This is needed in many applications of NLP, that need parsing.