1 semantic role labeling: english propbank ling 5200 computational corpus linguistics martha palmer
TRANSCRIPT
![Page 1: 1 Semantic Role Labeling: English PropBank LING 5200 Computational Corpus Linguistics Martha Palmer](https://reader035.vdocument.in/reader035/viewer/2022062422/56649efd5503460f94c11517/html5/thumbnails/1.jpg)
1
Semantic Role Labeling:English PropBank
LING 5200Computational Corpus LinguisticsMartha Palmer
![Page 2: 1 Semantic Role Labeling: English PropBank LING 5200 Computational Corpus Linguistics Martha Palmer](https://reader035.vdocument.in/reader035/viewer/2022062422/56649efd5503460f94c11517/html5/thumbnails/2.jpg)
2 LING 5200, 2006
Ask Jeeves – A Q/A, IR ex.
What do you call a successful movie? Tips on Being a Successful Movie Vampire ... I shall
call the police. Successful Casting Call & Shoot for ``Clash of
Empires'' ... thank everyone for their participation in the making of yesterday's movie.
Demme's casting is also highly entertaining, although I wouldn't go so far as to call it successful. This movie's resemblance to its predecessor is pretty vague...
VHS Movies: Successful Cold Call Selling: Over 100 New Ideas, Scripts, and Examples from the Nation's Foremost Sales Trainer.
Blockbuster
![Page 3: 1 Semantic Role Labeling: English PropBank LING 5200 Computational Corpus Linguistics Martha Palmer](https://reader035.vdocument.in/reader035/viewer/2022062422/56649efd5503460f94c11517/html5/thumbnails/3.jpg)
3 LING 5200, 2006
Ask Jeeves – filtering w/ POS tagWhat do you call a successful movie? Tips on Being a Successful Movie Vampire ... I shall
call the police. Successful Casting Call & Shoot for ``Clash of
Empires'' ... thank everyone for their participation in the making of yesterday's movie.
Demme's casting is also highly entertaining, although I wouldn't go so far as to call it successful. This movie's resemblance to its predecessor is pretty vague...
VHS Movies: Successful Cold Call Selling: Over 100 New Ideas, Scripts, and Examples from the Nation's Foremost Sales Trainer.
![Page 4: 1 Semantic Role Labeling: English PropBank LING 5200 Computational Corpus Linguistics Martha Palmer](https://reader035.vdocument.in/reader035/viewer/2022062422/56649efd5503460f94c11517/html5/thumbnails/4.jpg)
4 LING 5200, 2006
Filtering out “call the police”
Different senses, - different syntax, - different kinds of participants, - different types of propositions.
call(you,movie,what) ≠ call(you,police)
you movie what you police
![Page 5: 1 Semantic Role Labeling: English PropBank LING 5200 Computational Corpus Linguistics Martha Palmer](https://reader035.vdocument.in/reader035/viewer/2022062422/56649efd5503460f94c11517/html5/thumbnails/5.jpg)
5 LING 5200, 2006
WordNet – Princeton (Miller 1985, Fellbaum 1998)
On-line lexical reference (dictionary) Nouns, verbs, adjectives, and adverbs grouped
into synonym sets Other relations include hypernyms (ISA),
antonyms, meronyms Typical top nodes - 5 out of 25
(act, action, activity) (animal, fauna) (artifact) (attribute, property) (body, corpus)
![Page 6: 1 Semantic Role Labeling: English PropBank LING 5200 Computational Corpus Linguistics Martha Palmer](https://reader035.vdocument.in/reader035/viewer/2022062422/56649efd5503460f94c11517/html5/thumbnails/6.jpg)
6 LING 5200, 2006
Cornerstone: English lexical resource That provides sets of possible syntactic
frames for verbs. And provides clear, replicable sense
distinctions.
AskJeeves: Who do you call for a good electronic lexical database for English?
![Page 7: 1 Semantic Role Labeling: English PropBank LING 5200 Computational Corpus Linguistics Martha Palmer](https://reader035.vdocument.in/reader035/viewer/2022062422/56649efd5503460f94c11517/html5/thumbnails/7.jpg)
7 LING 5200, 2006
WordNet – Princeton (Miller 1985, Fellbaum 1998)
Limitations as a computational lexicon Contains little syntactic information
Comlex has syntax but no sense distinctions No explicit lists of participants Sense distinctions very fine-grained, Definitions often vague
Causes problems with creating training data for supervised Machine Learning – SENSEVAL2
Verbs > 16 senses (including call) Inter-annotator Agreement ITA 71%, Automatic Word Sense Disambiguation, WSD 63%
Dang & Palmer, SIGLEX02
![Page 8: 1 Semantic Role Labeling: English PropBank LING 5200 Computational Corpus Linguistics Martha Palmer](https://reader035.vdocument.in/reader035/viewer/2022062422/56649efd5503460f94c11517/html5/thumbnails/8.jpg)
8 LING 5200, 2006
WordNet – call, 28 senses 1. name, call -- (assign a specified, proper name to; "They named their son David"; …) -> LABEL2. call, telephone, call up, phone, ring -- (get or try to get
into communication (with someone) by telephone; "I tried to call you all night"; …)
->TELECOMMUNICATE3. call -- (ascribe a quality to or give a name of a
common noun that reflects a quality; "He called me a bastard"; …)
-> LABEL4. call, send for -- (order, request, or command to come; "She was called into the director's office"; "Call the
police!") -> ORDER
![Page 9: 1 Semantic Role Labeling: English PropBank LING 5200 Computational Corpus Linguistics Martha Palmer](https://reader035.vdocument.in/reader035/viewer/2022062422/56649efd5503460f94c11517/html5/thumbnails/9.jpg)
9 LING 5200, 2006
WordNet: - call, 28 senses
WN2 , WN13,WN28 WN15 WN26
WN3 WN19 WN4 WN 7 WN8 WN9
WN1 WN22
WN20 WN25
WN18 WN27
WN5 WN 16 WN6 WN23
WN12
WN17 , WN 11 WN10, WN14, WN21, WN24
![Page 10: 1 Semantic Role Labeling: English PropBank LING 5200 Computational Corpus Linguistics Martha Palmer](https://reader035.vdocument.in/reader035/viewer/2022062422/56649efd5503460f94c11517/html5/thumbnails/10.jpg)
10 LING 5200, 2006
WordNet: - call, 28 senses, Senseval2 groups, ITA 82%, WSD 70%
WN5, WN16,WN12 WN15 WN26
WN3 WN19 WN4 WN 7 WN8 WN9
WN1 WN22
WN20 WN25
WN18 WN27
WN2 WN 13 WN6 WN23
WN28
WN17 , WN 11 WN10, WN14, WN21, WN24,
Loud cry
Label
Phone/radio
Bird or animal cry
Request
Call a loan/bond
Visit
Challenge
Bid
![Page 11: 1 Semantic Role Labeling: English PropBank LING 5200 Computational Corpus Linguistics Martha Palmer](https://reader035.vdocument.in/reader035/viewer/2022062422/56649efd5503460f94c11517/html5/thumbnails/11.jpg)
11 LING 5200, 2006
Filtering out “call the police”
Different senses, - different syntax, - different kinds of participants, - different types of propositions.
call(you,movie,what) ≠ call(you,police)
you movie what you police
![Page 12: 1 Semantic Role Labeling: English PropBank LING 5200 Computational Corpus Linguistics Martha Palmer](https://reader035.vdocument.in/reader035/viewer/2022062422/56649efd5503460f94c11517/html5/thumbnails/12.jpg)
12 LING 5200, 2006
Proposition Bank:From Sentences to Propositions (Predicates!)
Powell met Zhu Rongji
Proposition: meet(Powell, Zhu Rongji)Powell met with Zhu Rongji
Powell and Zhu Rongji met
Powell and Zhu Rongji had a meeting
. . .When Powell met Zhu Rongji on Thursday they discussed the return of the spy plane.
meet(Powell, Zhu) discuss([Powell, Zhu], return(X, plane))
debate
consult
joinwrestle
battle
meet(Somebody1, Somebody2)
![Page 13: 1 Semantic Role Labeling: English PropBank LING 5200 Computational Corpus Linguistics Martha Palmer](https://reader035.vdocument.in/reader035/viewer/2022062422/56649efd5503460f94c11517/html5/thumbnails/13.jpg)
13 LING 5200, 2006
Semantic role labels:
break (agent(Marie), patient(LCD-projector))
cause(agent(Marie), change-of-state(LCD-projector))
(broken(LCD-projector))
agent(A) -> intentional(A), sentient(A), causer(A), affector(A)
patient(P) -> affected(P), change(P),…
Filmore, 68
Jackendoff, 72
Dowty, 91
Marie broke the LCD projector.
![Page 14: 1 Semantic Role Labeling: English PropBank LING 5200 Computational Corpus Linguistics Martha Palmer](https://reader035.vdocument.in/reader035/viewer/2022062422/56649efd5503460f94c11517/html5/thumbnails/14.jpg)
14 LING 5200, 2006
Capturing semantic roles*
Richard broke [ ARG1 the laser pointer.]
[ARG1 The windows] were broken by the hurricane.
[ARG1 The vase] broke into pieces when it toppled over.
SUBJ
SUBJ
SUBJ
*See also Framenet, http://www.icsi.berkeley.edu/~framenet/
![Page 15: 1 Semantic Role Labeling: English PropBank LING 5200 Computational Corpus Linguistics Martha Palmer](https://reader035.vdocument.in/reader035/viewer/2022062422/56649efd5503460f94c11517/html5/thumbnails/15.jpg)
15 LING 5200, 2006
Frame File example: give –
Roles: Arg0: giver Arg1: thing given Arg2: entity given to
Example: double object The executives gave the chefs a standing ovation. Arg0: The executives REL: gave Arg2: the chefs Arg1: a standing ovation
![Page 16: 1 Semantic Role Labeling: English PropBank LING 5200 Computational Corpus Linguistics Martha Palmer](https://reader035.vdocument.in/reader035/viewer/2022062422/56649efd5503460f94c11517/html5/thumbnails/16.jpg)
16 LING 5200, 2006
Annotation procedure
PTB II - Extraction of all sentences with given verb
Create Frame File for that verb Paul Kingsbury (3100+ lemmas, 4400 framesets,120K predicates) Over 300 created automatically via VerbNet
First pass: Automatic tagging (Joseph Rosenzweig) http://www.cis.upenn.edu/~josephr/TIDES/index.html#lexicon
Second pass: Double blind hand correction 84% ITA, 91% Kappa Paul Kingsbury
Tagging tool highlights discrepancies Scott Cotton
Third pass: Solomonization (adjudication) Betsy Klipple, Olga Babko-Malaya
![Page 17: 1 Semantic Role Labeling: English PropBank LING 5200 Computational Corpus Linguistics Martha Palmer](https://reader035.vdocument.in/reader035/viewer/2022062422/56649efd5503460f94c11517/html5/thumbnails/17.jpg)
17 LING 5200, 2006
NomBank Frame File example: gift(nominalizations, noun predicates, partitives, etc.Roles: Arg0: giver Arg1: thing given Arg2: entity given to
Example: double objectNancy’s gift from her cousin was a complete
surprise. Arg0: her cousin REL: gave Arg2: Nancy Arg1: gift
![Page 18: 1 Semantic Role Labeling: English PropBank LING 5200 Computational Corpus Linguistics Martha Palmer](https://reader035.vdocument.in/reader035/viewer/2022062422/56649efd5503460f94c11517/html5/thumbnails/18.jpg)
18 LING 5200, 2006
Trends in Argument Numbering
Arg0 = proto-typical agent (Dowty) Arg1 = proto-typical patient Arg2 = indirect object / benefactive /
instrument / attribute / end state Arg3 = start point / benefactive /
instrument / attribute Arg4 = end point
![Page 19: 1 Semantic Role Labeling: English PropBank LING 5200 Computational Corpus Linguistics Martha Palmer](https://reader035.vdocument.in/reader035/viewer/2022062422/56649efd5503460f94c11517/html5/thumbnails/19.jpg)
19 LING 5200, 2006
Additional tags - (arguments o adjuncts?)
Variety of ArgM’s (Arg#>4): TMP - when? LOC - where at? DIR - where to? MNR - how? PRP -why? REC - himself, themselves, each
other PRD -this argument refers to or
modifies another ADV –others
![Page 20: 1 Semantic Role Labeling: English PropBank LING 5200 Computational Corpus Linguistics Martha Palmer](https://reader035.vdocument.in/reader035/viewer/2022062422/56649efd5503460f94c11517/html5/thumbnails/20.jpg)
20 LING 5200, 2006
Inflection, etc.
Verbs also marked for tense/aspect Passive/Active Perfect/Progressive Third singular (is has does was) Present/Past/Future Infinitives/Participles/Gerunds/Finites
Modals and negations marked as ArgMs for convenience
![Page 21: 1 Semantic Role Labeling: English PropBank LING 5200 Computational Corpus Linguistics Martha Palmer](https://reader035.vdocument.in/reader035/viewer/2022062422/56649efd5503460f94c11517/html5/thumbnails/21.jpg)
21 LING 5200, 2006
Word Senses in PropBank Orders to ignore word sense not feasible for
700+ verbs Mary left the room Mary left her daughter-in-law her pearls in her will
Frameset leave.01 "move away from":Arg0: entity leavingArg1: place left
Frameset leave.02 "give":Arg0: giver Arg1: thing givenArg2: beneficiary
How do these relate to traditional word senses in WordNet?
![Page 22: 1 Semantic Role Labeling: English PropBank LING 5200 Computational Corpus Linguistics Martha Palmer](https://reader035.vdocument.in/reader035/viewer/2022062422/56649efd5503460f94c11517/html5/thumbnails/22.jpg)
22 LING 5200, 2006
WordNet: - call, 28 senses, groups
WN5, WN16,WN12 WN15 WN26
WN3 WN19 WN4 WN 7 WN8 WN9
WN1 WN22
WN20 WN25
WN18 WN27
WN2 WN 13 WN6 WN23
WN28
WN17 , WN 11 WN10, WN14, WN21, WN24,
Loud cry
Label
Phone/radio
Bird or animal cry
Request
Call a loan/bond
Visit
Challenge
Bid
![Page 23: 1 Semantic Role Labeling: English PropBank LING 5200 Computational Corpus Linguistics Martha Palmer](https://reader035.vdocument.in/reader035/viewer/2022062422/56649efd5503460f94c11517/html5/thumbnails/23.jpg)
23 LING 5200, 2006
Overlap with PropBank Framesets
WN5, WN16,WN12 WN15 WN26
WN3 WN19 WN4 WN 7 WN8 WN9
WN1 WN22
WN20 WN25
WN18 WN27
WN2 WN 13 WN6 WN23
WN28
WN17 , WN 11 WN10, WN14, WN21, WN24,
Loud cry
Label
Phone/radio
Bird or animal cry
Request
Call a loan/bond
Visit
Challenge
Bid
![Page 24: 1 Semantic Role Labeling: English PropBank LING 5200 Computational Corpus Linguistics Martha Palmer](https://reader035.vdocument.in/reader035/viewer/2022062422/56649efd5503460f94c11517/html5/thumbnails/24.jpg)
24 LING 5200, 2006
Overlap between Senseval2Groups and Framesets – 95%
WN1 WN2 WN3 WN4
WN6 WN7 WN8 WN5 WN 9 WN10
WN11 WN12 WN13 WN 14
WN19 WN20
Frameset1
Frameset2
develop
![Page 25: 1 Semantic Role Labeling: English PropBank LING 5200 Computational Corpus Linguistics Martha Palmer](https://reader035.vdocument.in/reader035/viewer/2022062422/56649efd5503460f94c11517/html5/thumbnails/25.jpg)
25 LING 5200, 2006
Sense Hierarchy (Palmer, et al, SNLU04 - NAACL04)
PropBank Framesets – ITA >90% coarse grained distinctions
20 Senseval2 verbs w/ > 1 FramesetMaxent WSD system, 73.5% baseline, 90% accuracy
Sense Groups (Senseval-2) - ITA 82% (up to 90% ITA) Intermediate level – 71% -> 74%
WordNet – ITA 71% fine grained distinctions, 60.2% -> 66%
![Page 26: 1 Semantic Role Labeling: English PropBank LING 5200 Computational Corpus Linguistics Martha Palmer](https://reader035.vdocument.in/reader035/viewer/2022062422/56649efd5503460f94c11517/html5/thumbnails/26.jpg)
26 LING 5200, 2006
Limitations to PropBank Args2-4 seriously overloaded, poor
performance VerbNet and FrameNet both provide more
fine-grained role labels WSJ too domain specific, too financial,
need broader coverage genres for more general annotation Additional Brown corpus annotation, also
GALE data FrameNet has selected instances from BNC
![Page 27: 1 Semantic Role Labeling: English PropBank LING 5200 Computational Corpus Linguistics Martha Palmer](https://reader035.vdocument.in/reader035/viewer/2022062422/56649efd5503460f94c11517/html5/thumbnails/27.jpg)
27 LING 5200, 2006
Improving generalization More data?
Can we merge FrameNet and PropBank data?, What about new words and new usages of old words?
General purpose class-based lexicons for unseen words and new usages? VerbNet, but limitations of VerbNet
Semantic classes for backoff? WordNet hypernyms; WSD example lexical sets (Patrick Hanks) verb dependencies - DIRT, (Dekang Lin), very noisy
We’re still a long way from events, inference, etc.
![Page 28: 1 Semantic Role Labeling: English PropBank LING 5200 Computational Corpus Linguistics Martha Palmer](https://reader035.vdocument.in/reader035/viewer/2022062422/56649efd5503460f94c11517/html5/thumbnails/28.jpg)
28 LING 5200, 2006
FrameNet: Telling.inform
Time In 2002,
Speaker the U.S. State Department
Target INFORMED
Addressee North Korea
Message that the U.S. was aware of this program , and regards it as a violation of Pyongyang's nonproliferation commitments
![Page 29: 1 Semantic Role Labeling: English PropBank LING 5200 Computational Corpus Linguistics Martha Palmer](https://reader035.vdocument.in/reader035/viewer/2022062422/56649efd5503460f94c11517/html5/thumbnails/29.jpg)
29 LING 5200, 2006
FrameNet/PropBank:Telling.inform
Time ArgM-TMP In 2002,
Speaker – Arg0(Informer)
the U.S. State Department
Target – REL INFORMED
Addressee –
Arg1 (informed)
North Korea
Message – Arg2(information)
that the U.S. was aware of this program , and regards it as a violation of Pyongyang's nonproliferation commitments
![Page 30: 1 Semantic Role Labeling: English PropBank LING 5200 Computational Corpus Linguistics Martha Palmer](https://reader035.vdocument.in/reader035/viewer/2022062422/56649efd5503460f94c11517/html5/thumbnails/30.jpg)
30 LING 5200, 2006
Frames File: give w/ VerbNetPropBank instances mapped to VerbNet Roles:
Arg0: giver Arg1: thing given Arg2: entity given toExample: double object The executives gave the chefs a standing
ovation. Arg0: Agent The executives REL: gave Arg2: Recipient the chefs Arg1: Theme a standing ovation
![Page 31: 1 Semantic Role Labeling: English PropBank LING 5200 Computational Corpus Linguistics Martha Palmer](https://reader035.vdocument.in/reader035/viewer/2022062422/56649efd5503460f94c11517/html5/thumbnails/31.jpg)
31 LING 5200, 2006
OntoNote AdditionsThefounderofPakistan’snuclear departmentAbdul Qadeer Khanhasadmittedhe transferrednuclear technologytoIran,Libya,and North Korea
OntoBank adds• Co-reference• Word Sense Resolution into Predicates• Entity types and predicate frames connected to nodes in ontology
NPNP
NP
NP
NP
PP
PP
VP
NP
NP
NP
NP
NP
NP
VP
S
SBAR S
VP
AdmitArg0:Arg1:
TransferArg0:Arg1:Arg2:
Founder
NationAgencyPerson
Acknowledge
TransferKnow-how
NationNation
Nation
FounderArg0:Arg1:
TechnologyArg1:
DepartmentArg1: