coarse to fine grained sense disambiguation in wikipedia
DESCRIPTION
Coarse to Fine Grained Sense Disambiguation in Wikipedia. Hui Shen [ Ohio University ] Razvan Bunescu [ Ohio University ] Rada Mihalcea [ University of North Texas ]. *SEM , June 13, 2013. Question Answering. Q: What is the composition of Earth’s atmosphere?. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/1.jpg)
Coarse to Fine Grained Sense Disambiguation in Wikipedia
Hui Shen [Ohio University]Razvan Bunescu [Ohio University]Rada Mihalcea [University of North Texas]
*SEM, June 13, 2013
![Page 2: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/2.jpg)
2
Question Answering
Q: What is the composition of Earth’s atmosphere?
As the Earth is 4.5 billion years old, it would have lost its atmosphere by now if there were no protective magnetosphere ... The atmosphere is composed of 78% nitrogen and 21% oxygen.
![Page 3: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/3.jpg)
3
KB
Question Answering
Q: What is the composition of Earth’s atmosphere?
As the Earth is 4.5 billion years old, it would have lost its atmosphere by now if there were no protective magnetosphere ... The atmosphere is composed of 78% nitrogen and 21% oxygen.
Atmosphere of Earth
![Page 4: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/4.jpg)
4
Fine to Coarse GrainedWord Sense Disambiguation
As the Earth is 4.5 billion years old, it would have lost its atmosphere by now if there were no protective magnetosphere ...The atmosphere is composed of 78% nitrogen and 21% oxygen.
AtmosphereAtmosphere (unit) Atmosphere (ambiance)
sense
A. Mars A. Earth A. Venus Stellar A.
reference specific sense
![Page 5: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/5.jpg)
5
Word Sense Disambiguation
• Select the correct sense of a word based on the context:– Use a repository of senses such as WordNet:
• Static resource, short glosses, too fine grained.– Unsupervised:
• Similarity between context and sense definition or gloss.– Supervised:
• Train on text manually tagged with word senses.– Limited amount of manually labeled data.
• Use Wikipedia for WSD:– Large sense repository, continuously growing.– Large training dataset.
![Page 6: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/6.jpg)
6
Wikipedia for WSD
’’’Palermo’’’ is a city in [[Southern Italy]], the [[capital city | capital]] of the [[autonomous area | autonomous region]] of [[Sicily]]. wiki
Palermo is a city in Southern Italy, the capital of the autonomous region of Sicily. html
capital citycapital (economics)
financial capital
human capital
capital (architecture)
![Page 7: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/7.jpg)
7
WSD Trained on Wikipedia Corpus – atmosphere –
1. Collect all WP titles that are linked from the anchor word atmosphere.
=> Atmosphere, Atmosphere of Earth, Mood (Pshychology), …
2. Create a sense repository from all titles that have sufficient support in WP.
=> {Atmosphere, Atmosphere of Earth, Atmosphere of Mars, Atmosphere of Venus, Stellar Atmosphere, Atmosphere (unit), Atmosphere (music group)}
3. For each sense, use the links as labeled examples and train a classifier to distinguish between alternative senses of the word atmosphere:– extract features from the word context.– each WP sense acts as a different label in the classification model.
![Page 8: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/8.jpg)
8
Major Shortcoming of WP Corpus for WSD
• Classification algorithms work with disjoint categories.
• Sense labels collected from WP are not disjoint!
– Many instances that are linked to Atmosphere could have been linked to more specific titles: Atmosphere of Earth or Atmosphere of Mars.
• Atmosphere category is ill defined.
Þ the learning algorithm will underperform, since it tries to separate Atmosphere examples from Atmosphere of Earth examples.
![Page 9: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/9.jpg)
9
Sense & Reference Annotations in Wikipedia
The Beagle 2 lander objectives were to characterize the physical properties of the atmosphere and surface layers.
Atmospheresense
Atmosphere of Marsreference
![Page 10: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/10.jpg)
10
Sense & Reference Annotations in Wikipedia
The Orbiter has been successfully performing scientific measurements since early 2004, namely high-resolution imaging and study of the interaction of the atmosphere with the interplanetary medium.
Atmospheresense
Atmosphere of Marsreference
![Page 11: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/11.jpg)
11
Sense & Reference Annotations in Wikipedia
Assuming the planet’s atmosphere is close to chemical equilibrium, it is predicted that 55 Cancri d is covered in a layer of water clouds.
Atmospheresense
Atmosphere of Cancri reference
no article yet in WP
![Page 12: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/12.jpg)
12
Sense & Reference Annotations in Wikipedia
An aerogravity assist, or AGA, is a spacecraft maneuver designed to change velocity when arriving at a body with an atmosphere.
Atmospheresense
generic use
![Page 13: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/13.jpg)
13
Annotation Inconsistencies in Wikipedia: Potential Causes
1. Editors may be unaware that an article exists in Wikipedia for the actual reference of a word, or for a more specific sense of the word:– end up using a link to an article describing the general sense of the
word.
2. More specific articles are introduced only in newer versions of Wikipedia:– and thus earlier annotations could not have been aware of these
more recent articles.
![Page 14: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/14.jpg)
14
Annotation Inconsistencies in Wikipedia: Potential Causes
3. Annotating words with the most specific sense or reference available in WP may require substantial cognitive effort:– editors will then choose to link to a general sense of the word.
Trajan was nominated as Consul and brought Apollodorus of Damascus with him to Rome around 91.
Romereference
Ancient Romemore specific reference
![Page 15: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/15.jpg)
15
An animal sleeps on the couch.
A cat sleeps on the couch.
A white siameze cat sleeps on the couch.
![Page 16: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/16.jpg)
16
An cat sleeps on the piece of furniture.
A cat sleeps on the couch.
A cat sleeps on the white leather couch.
![Page 17: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/17.jpg)
17
Possible Solutions (1)
1) Group overlapping senses and references into one general sense category.
Atmosphere of Earth
AtmosphereAtmosphere
of Mars
Atmosphere of Venus
Stellar Atmosphere
Atmosphere
=> {Atmosphere, Atmosphere (unit), Atmosphere (music group)}
![Page 18: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/18.jpg)
18
Possible Solutions (1)
• Straightforward to implement:– Train multiclass classifier to distinguish the disjoint categories
{Atmosphere, Atmosphere (unit), Atmosphere (music group)}
• Disadvantage is loss of disambiguation resolution:– Resulting WSD system cannot link atmosphere to the more
specific senses {Atmosphere of Earth, Atmosphere of Mars, Atmosphere of Venus, Stellar Atmosphere}
![Page 19: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/19.jpg)
19
Possible Solutions (2)
2) Keep the original sense repository, but change the definition of some sense categories such that all categories in the repository become mutually disjoint.
Atmosphere
Atm. Earth
Atm. Mars
Atm. Venus
StelarAtm.
![Page 20: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/20.jpg)
20
Possible Solutions (2)
2) Keep the original sense repository, but change the definition of some sense categories such that all categories in the repository become mutually disjoint.
Atm. Earth
Atm. Mars
Atm. Venus
StelarAtm.
Atmosphere (G)
![Page 21: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/21.jpg)
21
Possible Solutions (2)
=> {Atmosphere (G), Atmosphere of Earth, Atmosphere of Mars, Atmosphere of Venus, Stellar Atmosphere, Atmosphere (unit), Atmosphere (music group)}
Atm. Earth
Atm. Mars
Atm. Venus
StelarAtm.
Atmosphere (G)
Atmosphere (G) = generic uses, or senses/references not in WP.
![Page 22: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/22.jpg)
22
Possible Solutions (2)
• Advantage is WSD system can make more fine grained annotations:– annotate senses down to reference level i.e. Atmosphere of
Earth, Atmosphere of Mars, Stellar Atmosphere.
• Not as straightforward to train:
Atmosphere (G)
unknown set of training examplesknown set of training examples
![Page 23: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/23.jpg)
23
Proposed WSD Architecture
Atmosphere (unit) Atmosphere (music group)Atmosphere
“In global climate models, the properties of the atmosphere are specified at a number of …”
Level 1
![Page 24: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/24.jpg)
24
Proposed WSD Architecture
Atmosphere (unit) Atmosphere (music group)Atmosphere
“In global climate models, the properties of the atmosphere are specified at a number of …”
Level 1
![Page 25: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/25.jpg)
25
Proposed WSD Architecture
Atmosphere (unit) Atmosphere (music group)Atmosphere
“In global climate models, the properties of the atmosphere are specified at a number of …”
Atmosphere (G) Atmosphere (S)
Level 1
Level 2
![Page 26: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/26.jpg)
26
Proposed WSD Architecture
Atmosphere (unit) Atmosphere (music group)Atmosphere
“In global climate models, the properties of the atmosphere are specified at a number of …”
Atmosphere (G) Atmosphere (S)
Level 1
Level 2
![Page 27: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/27.jpg)
27
Proposed WSD Architecture
Atmosphere (unit) Atmosphere (music group)Atmosphere
“In global climate models, the properties of the atmosphere are specified at a number of …”
Atmosphere (G) Atmosphere (S)
A. Mars A. Earth A. Venus Stellar A.
Level 1
Level 2
Level 3
![Page 28: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/28.jpg)
28
Proposed WSD Architecture
Atmosphere (unit) Atmosphere (music group)Atmosphere
“In global climate models, the properties of the atmosphere are specified at a number of …”
Atmosphere (G) Atmosphere (S)
A. Mars A. Earth A. Venus Stellar A.
Level 1
Level 2
Level 3
![Page 29: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/29.jpg)
29
Straightforward Training WSD for L1 & L3: Use multiclass classifiers
Atmosphere (unit) Atmosphere (music group)AtmosphereLevel 1
Atmosphere, Atmosphere of Earth, Atmosphere of Mars, Atmosphere of Venus, Stellar
Atmosphere
training examples
A. Mars A. Earth A. Venus Stellar A.Level 3
![Page 30: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/30.jpg)
30
Training Level 2 Classifier
Atmosphere (G) Atmosphere (S)Level 2
Atmosphere of Earth, Atmosphere of Mars,
Atmosphere of Venus, Stellar Atmosphere
training examples training examples
?
Atmosphere (G)
unknown set of training examplesknown set of training examples
![Page 31: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/31.jpg)
31
Naïve SVM
Atmosphere (G) Atmosphere (S)Level 2
Atmosphere of Earth, Atmosphere of Mars,
Atmosphere of Venus, Stellar Atmosphere
training examples training examples Atmosphere
• 60% of the examples linked to Atmosphere should actually belong to Atmosphere (S).Þ underperforming classifier.
![Page 32: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/32.jpg)
32
Training for Level 2: Semi-supervised
Þ learning with positive and unlabeled examples.
Atmosphere (G) Atmosphere (S)Level 2
Atmosphere of Earth, Atmosphere of Mars,
Atmosphere of Venus, Stellar Atmosphere
positive training examplesunlabeled training examples Atmosphere
![Page 33: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/33.jpg)
33
Learning with Positive and Unlabeled Examples
1) Adaptation of the Biased SVM [Lee and Liu, ICML 2003]:
decision function f(x)s(x) = +1, if x positive (P)s(x) = −1, if x unlabeled (U)
![Page 34: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/34.jpg)
34
Learning with Positive and Unlabeled Examples
1) Adaptation of the Biased SVM [Lee and Liu, ICML 2003]:
• control penalties for errors on positive (CP) vs. unlabeled (CU) examples• how do we set CP and CU? want CP > CU
![Page 35: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/35.jpg)
35
1) Adaptation of Biased SVM [Lee and Liu, ICML 2003]:
– Use development data to tune CP and CU.
• [Lee and Liu, 03]
• [this work, 13]
Learning with Positive and Unlabeled Examples
)1( maxargmaxarg,,
fprecallaccUPUP CCCC
both estimated un positive and unlabeled examples from dev. data
![Page 36: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/36.jpg)
36
Learning with Positive and Unlabeled Examples
2) Weighted Samples SVM [Elkan and Noto, KDD 2008]:I. Train prob. classifier g(x) to compute p(s 1|x) on P and U.II. Train final decision function f(x) on weighted examples sampled
as follows: sample each example in P with weight 1. sample each example from U as two examples:
one positive, with weight p(y +1 | x, s 0). one negative, with weight p(y −1 | x, s 0).
both estimated based on g(x)
![Page 37: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/37.jpg)
37
Evaluation Datasets
![Page 38: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/38.jpg)
38
Evaluation Datasets
![Page 39: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/39.jpg)
39
Evaluation Datasets
![Page 40: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/40.jpg)
40
Evaluation Datasets
![Page 41: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/41.jpg)
41
Experimental Results: Level 2 Accuracy
• Trained only on WP links, tested on manual annotations.– Averaged over 4-fold cross-validation experiments.
![Page 42: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/42.jpg)
42
Experimental Results: Level 2 F-measure
• Trained only on WP links, tested on manual annotations.– Averaged over 4-fold cross-validation experiments.
![Page 43: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/43.jpg)
43
Experimental Results: Overall Accuracy
Atmosphere (unit) Atmosphere (music group)Atmosphere
... atmosphere ...
Atmosphere (G) Atmosphere (S)
A. Mars A. Earth A. Venus Stellar A.
Level 1
Level 2
Level 3
![Page 44: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/44.jpg)
44
Experimental Results: Overall Accuracy
• Leaf nodes as sense repository.
![Page 45: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/45.jpg)
45
Future Work
• Extend learning with positive and unlabeled data to multiclass:
• Automatically add fine grained links to entire Wikipedia..
Atmosphere (unit) Atmosphere (music group)Atmosphere
“In global climate models, the properties of the atmosphere are specified at a number of …”
Atmosphere (O) A. Mars A. Earth A. Venus Stellar A.
Level 1
Level 2
![Page 46: Coarse to Fine Grained Sense Disambiguation in Wikipedia](https://reader036.vdocument.in/reader036/viewer/2022062501/56816635550346895dd9a151/html5/thumbnails/46.jpg)
46
Questions
?