![Page 1: Syntactic Parsing - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0207-syntactic... · Syntactic Parsing Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February](https://reader035.vdocument.in/reader035/viewer/2022070112/6052d2be2582613c6338bcf2/html5/thumbnails/1.jpg)
SyntacticParsing
Prof.SameerSinghCS295:STATISTICALNLP
WINTER2017
February7,2017
BasedonslidesfromNathanSchneider,NoahSmith,MarineCarpuat,DanJurafsky,andeveryoneelsetheycopiedfrom.
![Page 2: Syntactic Parsing - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0207-syntactic... · Syntactic Parsing Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February](https://reader035.vdocument.in/reader035/viewer/2022070112/6052d2be2582613c6338bcf2/html5/thumbnails/2.jpg)
Outline
CS295:STATISTICALNLP(WINTER2017) 2
SyntacticParsing
ContextFreeGrammars
Parsing:CKYAlgorithm
![Page 3: Syntactic Parsing - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0207-syntactic... · Syntactic Parsing Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February](https://reader035.vdocument.in/reader035/viewer/2022070112/6052d2be2582613c6338bcf2/html5/thumbnails/3.jpg)
Outline
CS295:STATISTICALNLP(WINTER2017) 3
SyntacticParsing
ContextFreeGrammars
Parsing:CKYAlgorithm
![Page 4: Syntactic Parsing - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0207-syntactic... · Syntactic Parsing Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February](https://reader035.vdocument.in/reader035/viewer/2022070112/6052d2be2582613c6338bcf2/html5/thumbnails/4.jpg)
LimitationsofSequenceTags
CS295:STATISTICALNLP(WINTER2017) 4
JohnSmithshotBillinhispajamas.
Whathappened?Whoshotwho?Whowaswearingthepajamas?
Usinghttp://nlp.stanford.edu:8080/corenlp/process
![Page 5: Syntactic Parsing - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0207-syntactic... · Syntactic Parsing Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February](https://reader035.vdocument.in/reader035/viewer/2022070112/6052d2be2582613c6338bcf2/html5/thumbnails/5.jpg)
Constituents
CS295:STATISTICALNLP(WINTER2017) 5
• Constituentbehaveasaunitthatcanberearranged:Johntalked[tothechildren][aboutdrugs].Johntalked[aboutdrugs][tothechildren].Johntalkeddrugstothechildrenabout
• Orsubstituted/expanded:
Johntalked[tothechildrentakingthedrugs][aboutalcohol].
HarrytheHorseahigh-classspotsuchasMindy’stheBroadwaycoppersthereasonhecomesintotheHotBoxtheythreepartiesfromBrooklyn
X
arrive(s)attract(s)love(s)sit(s)
“NounphrasesappearbeforeverbsinEnglish.”
![Page 6: Syntactic Parsing - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0207-syntactic... · Syntactic Parsing Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February](https://reader035.vdocument.in/reader035/viewer/2022070112/6052d2be2582613c6338bcf2/html5/thumbnails/6.jpg)
ConstituentsandGrammars
CS295:STATISTICALNLP(WINTER2017) 6
Grammar
• Tellsyouhowtheconstituentscanbearranged• Implicitknowledgeforus(weoftencan’ttellwhysomethingiswrong)• Generateall,andonly,thepossiblesentencesofthelanguage• Differentfrommeaning:
Colorlessgreenideassleepfuriously.
• Thewordsareintherightorder,• Andthatideasaregreenandcolorless,• Andthatideassleep,• Andthatsleepingisdonefuriously,• Asopposedto:“sleepgreenfuriouslyideascolorless”
![Page 7: Syntactic Parsing - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0207-syntactic... · Syntactic Parsing Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February](https://reader035.vdocument.in/reader035/viewer/2022070112/6052d2be2582613c6338bcf2/html5/thumbnails/7.jpg)
UsesofParsing
CS295:STATISTICALNLP(WINTER2017) 7
• Grammarcheckers• Dialogsystems• Highprecisionquestionanswering• Namedentityrecognition• Sentencecompression• Extracting opinionsaboutproducts• Improvedinteractionincomputergames• Helpinglinguistsfinddata• Machinetranslation• Relationextractionsystems
[send[thetextmessagefromJames][toSharon]]
[translate[themessage][fromHindi][toEnglish]]
![Page 8: Syntactic Parsing - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0207-syntactic... · Syntactic Parsing Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February](https://reader035.vdocument.in/reader035/viewer/2022070112/6052d2be2582613c6338bcf2/html5/thumbnails/8.jpg)
Outline
CS295:STATISTICALNLP(WINTER2017) 8
SyntacticParsing
ContextFreeGrammars
Parsing:CKYAlgorithm
![Page 9: Syntactic Parsing - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0207-syntactic... · Syntactic Parsing Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February](https://reader035.vdocument.in/reader035/viewer/2022070112/6052d2be2582613c6338bcf2/html5/thumbnails/9.jpg)
BasicGrammar:RegularExpr.
CS295:STATISTICALNLP(WINTER2017) 9
• Youcancaptureindividualwords:• (man|dog|cat)
• Simplesentences:• (man|dog|cat)(ate|loves|consumed)(.|food|lunch)
• Infinitelength?Yes!• men(wholike(cats|dogs))*cry.
FiniteStateMachine Start S1
End
S2
S3men
cry
wholike
dogs
cats
ButtooweakforEnglish.
![Page 10: Syntactic Parsing - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0207-syntactic... · Syntactic Parsing Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February](https://reader035.vdocument.in/reader035/viewer/2022070112/6052d2be2582613c6338bcf2/html5/thumbnails/10.jpg)
Context-FreeGrammars
CS295:STATISTICALNLP(WINTER2017) 10
Grammar,G TerminalSymbols Non-terminalSymbols
Rules
Grammarappliesrulesrecursively..Ifwecanconstructtheinputsentence,itisinthegrammar,otherwisenot.
![Page 11: Syntactic Parsing - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0207-syntactic... · Syntactic Parsing Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February](https://reader035.vdocument.in/reader035/viewer/2022070112/6052d2be2582613c6338bcf2/html5/thumbnails/11.jpg)
CS295:STATISTICALNLP(WINTER2017) 11
ExampleCFG
![Page 12: Syntactic Parsing - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0207-syntactic... · Syntactic Parsing Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February](https://reader035.vdocument.in/reader035/viewer/2022070112/6052d2be2582613c6338bcf2/html5/thumbnails/12.jpg)
ExampleParseTree
CS295:STATISTICALNLP(WINTER2017) 12
Ipreferamorningflight.
![Page 13: Syntactic Parsing - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0207-syntactic... · Syntactic Parsing Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February](https://reader035.vdocument.in/reader035/viewer/2022070112/6052d2be2582613c6338bcf2/html5/thumbnails/13.jpg)
ExampleParseTree:Brackets
CS295:STATISTICALNLP(WINTER2017) 13
Ipreferamorningflight.
![Page 14: Syntactic Parsing - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0207-syntactic... · Syntactic Parsing Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February](https://reader035.vdocument.in/reader035/viewer/2022070112/6052d2be2582613c6338bcf2/html5/thumbnails/14.jpg)
Moredetails:NounPhrases
CS295:STATISTICALNLP(WINTER2017) 14
“allthemorningflightsfromDenvertoTampaleavingbefore10”
NP® ProperNoun
NP® Det Nominal
Nominal® Noun|NounNominal
SimpleNounPhrases
ComplexNounPhrases
![Page 15: Syntactic Parsing - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0207-syntactic... · Syntactic Parsing Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February](https://reader035.vdocument.in/reader035/viewer/2022070112/6052d2be2582613c6338bcf2/html5/thumbnails/15.jpg)
RecursiveNounPhrases
CS295:STATISTICALNLP(WINTER2017) 15
thisisthehouse
thisisthehousethatJackbuilt
thisisthecatthatlivesinthehousethatJackbuilt
thisisthedogthatchasedthecatthatlivesinthehousethatJackbuilt
thisisthefleathatbitthedogthatchasedthecatthatlivesinthehousetheJackbuilt
thisisthevirusthatinfectedthefleathatbitthedogthatchasedthecatthatlivesinthehousethatJackbuilt
![Page 16: Syntactic Parsing - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0207-syntactic... · Syntactic Parsing Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February](https://reader035.vdocument.in/reader035/viewer/2022070112/6052d2be2582613c6338bcf2/html5/thumbnails/16.jpg)
Moredetails:VerbPhrases
CS295:STATISTICALNLP(WINTER2017) 16
VP® Verb disappearVP® VerbNP preferamorningflightVP® Verb NPPP leaveBostoninthemorningVP® Verb PP leaveinthemorning
SimpleVerbPhrases
Butallverbsarenotthesame!(thisgrammarovergenerates)
Sneezed: Johnsneezed.Find: PleasefindaflighttoNY.Give: Givemeacheaperfare.Help: Canyouhelpmewithaflight?Prefer: Iprefertoleaveearlier.Told: IwastoldUnitedhasaflight.Solution:subcategorize!
![Page 17: Syntactic Parsing - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0207-syntactic... · Syntactic Parsing Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February](https://reader035.vdocument.in/reader035/viewer/2022070112/6052d2be2582613c6338bcf2/html5/thumbnails/17.jpg)
TypesofSentences
CS295:STATISTICALNLP(WINTER2017) 17
Declarative Aplaneleft.S® NPVP
Imperative Showtheplane.S® VP
Yes/noQuestions Didtheplaneleave?S® AuxNPVP
Wh-Questions Whendidtheplaneleave?S®WhNP AuxNPVP
![Page 18: Syntactic Parsing - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0207-syntactic... · Syntactic Parsing Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February](https://reader035.vdocument.in/reader035/viewer/2022070112/6052d2be2582613c6338bcf2/html5/thumbnails/18.jpg)
SourceofGrammar?
CS295:STATISTICALNLP(WINTER2017) 18
Writesymbolicgrammar(CFGoroftenricher)andlexiconS® NPVP NN® interestNP® (DT)NN NNS® ratesNP® NNNNS NNS® raisesNP® NNP VBP® interestVP® VNP VBZ® rates
Usedgrammar/proofsystemstoproveparsesfromwords
Fedraisesinterestrates0.5%inefforttocontrolinflation◦ Minimalgrammar: 36parses◦ Simple10rulegrammar: 592parses◦ Real-sizebroad-coveragegrammar: millionsofparses
Manual
NoamChomsky
![Page 19: Syntactic Parsing - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0207-syntactic... · Syntactic Parsing Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February](https://reader035.vdocument.in/reader035/viewer/2022070112/6052d2be2582613c6338bcf2/html5/thumbnails/19.jpg)
SourceofGrammar?
CS295:STATISTICALNLP(WINTER2017) 19
Fromdata!
ThePennTreebank
Buildingatreebankseemsalotslowerandlessusefulthanbuildingagrammar
Butatreebankgivesusmanythings• Reusabilityofthelabor
• Manyparsers,POStaggers,etc.• Valuableresourceforlinguistics
• Broadcoverage• Frequenciesanddistributionalinformation• Awaytoevaluatesystems
[Marcusetal.1993,ComputationalLinguistics]
![Page 20: Syntactic Parsing - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0207-syntactic... · Syntactic Parsing Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February](https://reader035.vdocument.in/reader035/viewer/2022070112/6052d2be2582613c6338bcf2/html5/thumbnails/20.jpg)
CS295:STATISTICALNLP(WINTER2017) 20
( (S(NP-SBJ (DT The) (NN move))(VP (VBD followed)(NP(NP (DT a) (NN round))(PP (IN of)(NP(NP (JJ similar) (NNS increases))(PP (IN by)(NP (JJ other) (NNS lenders)))
(PP (IN against)(NP (NNP Arizona) (JJ real) (NN estate) (NNS loans))))))
(, ,)(S-ADV(NP-SBJ (-NONE- *))(VP (VBG reflecting)(NP(NP (DT a) (VBG continuing) (NN decline))(PP-LOC (IN in)(NP (DT that) (NN market)))))))
(. .)))
![Page 21: Syntactic Parsing - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0207-syntactic... · Syntactic Parsing Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February](https://reader035.vdocument.in/reader035/viewer/2022070112/6052d2be2582613c6338bcf2/html5/thumbnails/21.jpg)
CS295:STATISTICALNLP(WINTER2017) 21
Someoftherules,withcounts40717PP→INNP33803S→NP-SBJVP22513NP-SBJ→-NONE-21877NP→NPPP20740NP→DTNN14153S→NP-SBJVP.12922VP→TOVP11881PP-LOC→INNP11467NP-SBJ→PRP11378NP→-NONE-11291NP→NN...989VP→VBGS985NP-SBJ→NN983PP-MNR→INNP983NP-SBJ→DT969VP→VBNVP
100VP→VBDPP-PRD100PRN→:NP:100NP→DTJJS100NP-CLR→NN99NP-SBJ-1→DTNNP98VP→VBNNPPP-DIR98VP→VBDPP-TMP98PP-TMP→VBGNP97VP→VBDADVP-TMPVP...10WHNP-1→WRBJJ10VP→VPCCVPPP-TMP10VP→VPCCVPADVP-MNR10VP→VBZS,SBAR-ADV10VP→VBZSADVP-TMP
4500rulesforVP!
![Page 22: Syntactic Parsing - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0207-syntactic... · Syntactic Parsing Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February](https://reader035.vdocument.in/reader035/viewer/2022070112/6052d2be2582613c6338bcf2/html5/thumbnails/22.jpg)
EvaluatingParses
CS295:STATISTICALNLP(WINTER2017) 22
Eachparsetreeisrepresentedbyalistoftuples:
Usethistoestimateprecision/recall!
![Page 23: Syntactic Parsing - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0207-syntactic... · Syntactic Parsing Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February](https://reader035.vdocument.in/reader035/viewer/2022070112/6052d2be2582613c6338bcf2/html5/thumbnails/23.jpg)
EvaluatingParses:Example
CS295:STATISTICALNLP(WINTER2017) 23
![Page 24: Syntactic Parsing - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0207-syntactic... · Syntactic Parsing Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February](https://reader035.vdocument.in/reader035/viewer/2022070112/6052d2be2582613c6338bcf2/html5/thumbnails/24.jpg)
Outline
CS295:STATISTICALNLP(WINTER2017) 24
SyntacticParsing
ContextFreeGrammars
Parsing:CKYAlgorithm
![Page 25: Syntactic Parsing - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0207-syntactic... · Syntactic Parsing Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February](https://reader035.vdocument.in/reader035/viewer/2022070112/6052d2be2582613c6338bcf2/html5/thumbnails/25.jpg)
TheParsingProblem
CS295:STATISTICALNLP(WINTER2017) 25
Givensentencex andgrammarG,
Recognition “Proof”isadeduction,validparsetree.Issentencex inthegrammar?Ifso,proveit.
ParsingEvenwithsmallgrammars,bruteforcegrowsexponentially!
Showoneormorederivationsforx inG.
“Bookthatflight”
![Page 26: Syntactic Parsing - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0207-syntactic... · Syntactic Parsing Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February](https://reader035.vdocument.in/reader035/viewer/2022070112/6052d2be2582613c6338bcf2/html5/thumbnails/26.jpg)
TopDownParsing
CS295:STATISTICALNLP(WINTER2017) 26
“Bookthatflight”ConsidersonlyvalidtreesButareinconsistentwiththewords!
![Page 27: Syntactic Parsing - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0207-syntactic... · Syntactic Parsing Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February](https://reader035.vdocument.in/reader035/viewer/2022070112/6052d2be2582613c6338bcf2/html5/thumbnails/27.jpg)
Bottom-upParsing
CS295:STATISTICALNLP(WINTER2017) 27
BuildsonlyconsistenttreesButmostofthemareinvalid(don’tgoanywhere)!
“Bookthatflight”
![Page 28: Syntactic Parsing - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0207-syntactic... · Syntactic Parsing Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February](https://reader035.vdocument.in/reader035/viewer/2022070112/6052d2be2582613c6338bcf2/html5/thumbnails/28.jpg)
ChomskyNormalForm
CS295:STATISTICALNLP(WINTER2017) 28
Contextfreegrammarwhereallnon-terminalstogo:- 2non-terminals,or- Asingleterminal A® BC D® w
ConvertingtoCNF
A® BB® CDB® w
A® CDA® w
Case1
A® BCDEA® XEX® YDY® BC
Case2
![Page 29: Syntactic Parsing - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0207-syntactic... · Syntactic Parsing Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February](https://reader035.vdocument.in/reader035/viewer/2022070112/6052d2be2582613c6338bcf2/html5/thumbnails/29.jpg)
CS295:STATISTICALNLP(WINTER2017) 29
OriginalGrammar ChomskyNormalForm
![Page 30: Syntactic Parsing - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0207-syntactic... · Syntactic Parsing Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February](https://reader035.vdocument.in/reader035/viewer/2022070112/6052d2be2582613c6338bcf2/html5/thumbnails/30.jpg)
DynamicProgramming
CS295:STATISTICALNLP(WINTER2017) 30
table[i,j]=Setofallvalidnon-terminalsfortheconstituentspan(i,j)
Recursion Rule:A® BCA
B C
(i,j)
(i,k) (k,j)
IfyoufindaksuchthatBisintable[i,k],andCisintable[k,j],thenAshouldbeintable[i,j]
BasecaseRule:A® word[j]
Ashouldbeintable[j-1,j]
A (j-1,j)
word[j]
![Page 31: Syntactic Parsing - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0207-syntactic... · Syntactic Parsing Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February](https://reader035.vdocument.in/reader035/viewer/2022070112/6052d2be2582613c6338bcf2/html5/thumbnails/31.jpg)
CKYAlgorithm
CS295:STATISTICALNLP(WINTER2017) 31
Book the flight through TWA
![Page 32: Syntactic Parsing - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0207-syntactic... · Syntactic Parsing Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February](https://reader035.vdocument.in/reader035/viewer/2022070112/6052d2be2582613c6338bcf2/html5/thumbnails/32.jpg)
Outline
CS295:STATISTICALNLP(WINTER2017) 32
SyntacticParsing
ContextFreeGrammars
Parsing:CKYAlgorithm
![Page 33: Syntactic Parsing - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0207-syntactic... · Syntactic Parsing Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February](https://reader035.vdocument.in/reader035/viewer/2022070112/6052d2be2582613c6338bcf2/html5/thumbnails/33.jpg)
Upcoming…
CS295:STATISTICALNLP(WINTER2017) 33
• Homework2isdueinaweek:February13,2017• Homework1gradeswillbeavailabletonight
Homework
• Proposalisdueontonight• Only2pagesProject
• Papersummaries:February17,February28,March14• Only1 pageeachSummaries