![Page 1: Robust Distant Supervision Relation Extraction via Deep ... · Some New York city mayors –William O’Dwyer, Vincent R. Impellitteri and Abraham Beame – were born abroad. ii](https://reader033.vdocument.in/reader033/viewer/2022041911/5e67aa2755089b3b9c241273/html5/thumbnails/1.jpg)
Robust Distant Supervision Relation Extraction via Deep Reinforcement
Learning
Pengda Qin, Weiran Xu andWilliamWang1
BUPT
![Page 2: Robust Distant Supervision Relation Extraction via Deep ... · Some New York city mayors –William O’Dwyer, Vincent R. Impellitteri and Abraham Beame – were born abroad. ii](https://reader033.vdocument.in/reader033/viewer/2022041911/5e67aa2755089b3b9c241273/html5/thumbnails/2.jpg)
Outline
•Motivation•Algorithm• Experiments•Conclusion
2
![Page 3: Robust Distant Supervision Relation Extraction via Deep ... · Some New York city mayors –William O’Dwyer, Vincent R. Impellitteri and Abraham Beame – were born abroad. ii](https://reader033.vdocument.in/reader033/viewer/2022041911/5e67aa2755089b3b9c241273/html5/thumbnails/3.jpg)
Outline
•Motivation•Algorithm• Experiments•Conclusion
3
![Page 4: Robust Distant Supervision Relation Extraction via Deep ... · Some New York city mayors –William O’Dwyer, Vincent R. Impellitteri and Abraham Beame – were born abroad. ii](https://reader033.vdocument.in/reader033/viewer/2022041911/5e67aa2755089b3b9c241273/html5/thumbnails/4.jpg)
Plain Text Corpus(Unstructured Info)
Classifier Entity-Relation Triple(Structured Info)
Relation Type withLabeled Dataset
Relation Type withoutLabeled Dataset
4
Relation Extraction
![Page 5: Robust Distant Supervision Relation Extraction via Deep ... · Some New York city mayors –William O’Dwyer, Vincent R. Impellitteri and Abraham Beame – were born abroad. ii](https://reader033.vdocument.in/reader033/viewer/2022041911/5e67aa2755089b3b9c241273/html5/thumbnails/5.jpg)
“If two entities participate in a relation, any sentencethat contains those two entities might express thatrelation.” (Mintz, 2009)
5
Distant Supervision
![Page 6: Robust Distant Supervision Relation Extraction via Deep ... · Some New York city mayors –William O’Dwyer, Vincent R. Impellitteri and Abraham Beame – were born abroad. ii](https://reader033.vdocument.in/reader033/viewer/2022041911/5e67aa2755089b3b9c241273/html5/thumbnails/6.jpg)
Data(x): <Belgium, Nijlen>Label(y): /location/contains
Target Corpus(Unlabeled)
Relation Label: /location/contains
1. Nijlen is a municipalitylocated in the Belgianprovince of Antwerp.
2. ……3. ……
6
Sentence Bag:
Distant Supervision
![Page 7: Robust Distant Supervision Relation Extraction via Deep ... · Some New York city mayors –William O’Dwyer, Vincent R. Impellitteri and Abraham Beame – were born abroad. ii](https://reader033.vdocument.in/reader033/viewer/2022041911/5e67aa2755089b3b9c241273/html5/thumbnails/7.jpg)
v Within-Sentence-Bag Level
v Entity-Pair Level
§ Hoffmann et al., ACL 2011.§ Surdean et al., ACL 2012.§ Zeng et al., ACL 2015. § Li et al., ACL 2016.
§ None
7
Wrong Labeling
![Page 8: Robust Distant Supervision Relation Extraction via Deep ... · Some New York city mayors –William O’Dwyer, Vincent R. Impellitteri and Abraham Beame – were born abroad. ii](https://reader033.vdocument.in/reader033/viewer/2022041911/5e67aa2755089b3b9c241273/html5/thumbnails/8.jpg)
§ Place_of_Death (William O’Dwyer, New York city)
8
v Entity-Pair Level
Wrong Labeling
i. Some New York city mayors – William O’Dwyer, Vincent R. Impellitteriand Abraham Beame – were born abroad.
ii. Plenty of local officials have, too, including two New York city mayors, James J. Walker, in 1932, and William O’Dwyer, in 1950.
![Page 9: Robust Distant Supervision Relation Extraction via Deep ... · Some New York city mayors –William O’Dwyer, Vincent R. Impellitteri and Abraham Beame – were born abroad. ii](https://reader033.vdocument.in/reader033/viewer/2022041911/5e67aa2755089b3b9c241273/html5/thumbnails/9.jpg)
v Most of entity pairs only have several sentences
1 Sentence55%2 Sentence
32%
Other4%
9
v Lots of entity pairs have repetitive sentences
Wrong Labeling
![Page 10: Robust Distant Supervision Relation Extraction via Deep ... · Some New York city mayors –William O’Dwyer, Vincent R. Impellitteri and Abraham Beame – were born abroad. ii](https://reader033.vdocument.in/reader033/viewer/2022041911/5e67aa2755089b3b9c241273/html5/thumbnails/10.jpg)
Outline
•Motivation•Algorithm• Experiments•Conclusion
10
![Page 11: Robust Distant Supervision Relation Extraction via Deep ... · Some New York city mayors –William O’Dwyer, Vincent R. Impellitteri and Abraham Beame – were born abroad. ii](https://reader033.vdocument.in/reader033/viewer/2022041911/5e67aa2755089b3b9c241273/html5/thumbnails/11.jpg)
Negative set
Positive set
Negative set
Positive setFalse Positive
False Positive
DS Dataset Cleaned Dataset
11
Overview
False PositiveIndicator
![Page 12: Robust Distant Supervision Relation Extraction via Deep ... · Some New York city mayors –William O’Dwyer, Vincent R. Impellitteri and Abraham Beame – were born abroad. ii](https://reader033.vdocument.in/reader033/viewer/2022041911/5e67aa2755089b3b9c241273/html5/thumbnails/12.jpg)
Sentence-Level Indicator
False-Positive Indicator
Learn a Policy to Denoise the Training Data
12
General Purpose and Offline Process
Without Supervised Information
Requirements
![Page 13: Robust Distant Supervision Relation Extraction via Deep ... · Some New York city mayors –William O’Dwyer, Vincent R. Impellitteri and Abraham Beame – were born abroad. ii](https://reader033.vdocument.in/reader033/viewer/2022041911/5e67aa2755089b3b9c241273/html5/thumbnails/13.jpg)
Negative set
Positive set
Negative set
Positive setFalse Positive
𝑅𝑒𝑤𝑎𝑟𝑑
𝐴𝑐𝑡𝑖𝑜𝑛
Classifier𝑇𝑟𝑎𝑖𝑛
False Positive
DS Dataset Cleaned Dataset
13
Overview
False PositiveIndicator
Policy-Based Agent
![Page 14: Robust Distant Supervision Relation Extraction via Deep ... · Some New York city mayors –William O’Dwyer, Vincent R. Impellitteri and Abraham Beame – were born abroad. ii](https://reader033.vdocument.in/reader033/viewer/2022041911/5e67aa2755089b3b9c241273/html5/thumbnails/14.jpg)
v State
14
v Action
v Reward§ ???
§ Sentence vector§ The average vector of previous removed sentences
§ Remove & retain
Deep Reinforcement Learning
![Page 15: Robust Distant Supervision Relation Extraction via Deep ... · Some New York city mayors –William O’Dwyer, Vincent R. Impellitteri and Abraham Beame – were born abroad. ii](https://reader033.vdocument.in/reader033/viewer/2022041911/5e67aa2755089b3b9c241273/html5/thumbnails/15.jpg)
v One relation type has an agent
15
v Sentence-level
v Split into training set and validation set
§ Positive: Distantly-supervised positive sentences§ Negative: Sampled from other relations
Deep Reinforcement Learning
![Page 16: Robust Distant Supervision Relation Extraction via Deep ... · Some New York city mayors –William O’Dwyer, Vincent R. Impellitteri and Abraham Beame – were born abroad. ii](https://reader033.vdocument.in/reader033/viewer/2022041911/5e67aa2755089b3b9c241273/html5/thumbnails/16.jpg)
RL Agent Train
16
TrainRelationClassifier
RelationClassifier
𝐹/01/
𝐹/0× +𝓡0 + ×(−𝓡0)
Noisydataset𝑃:;<0
Cleaneddataset
Cleaneddataset
Removedpart
Removedpart
𝓡0 = 𝛼(𝐹/0 - 𝐹/01/ )
RL Agent
Epoch i-1
EpochiNoisy
dataset𝑃:;<0
+𝑁:;<0
𝑁:;<0 +
Deep Reinforcement Learning
![Page 17: Robust Distant Supervision Relation Extraction via Deep ... · Some New York city mayors –William O’Dwyer, Vincent R. Impellitteri and Abraham Beame – were born abroad. ii](https://reader033.vdocument.in/reader033/viewer/2022041911/5e67aa2755089b3b9c241273/html5/thumbnails/17.jpg)
17
Positive Set Negative Set
§ Accurate
§ Steady
§ Fast
False Positive
§ Obvious
Reward
![Page 18: Robust Distant Supervision Relation Extraction via Deep ... · Some New York city mayors –William O’Dwyer, Vincent R. Impellitteri and Abraham Beame – were born abroad. ii](https://reader033.vdocument.in/reader033/viewer/2022041911/5e67aa2755089b3b9c241273/html5/thumbnails/18.jpg)
18
Positive Set Negative Set
RelationClassifier
TrainRelationClassifier
Train
Calculate
𝐹/
Epoch 𝑖
False Positive
False Positive
Positive Negative
Reward
![Page 19: Robust Distant Supervision Relation Extraction via Deep ... · Some New York city mayors –William O’Dwyer, Vincent R. Impellitteri and Abraham Beame – were born abroad. ii](https://reader033.vdocument.in/reader033/viewer/2022041911/5e67aa2755089b3b9c241273/html5/thumbnails/19.jpg)
Outline
• Motivation• Algorithm• Experiments• Conclusion
19
![Page 20: Robust Distant Supervision Relation Extraction via Deep ... · Some New York city mayors –William O’Dwyer, Vincent R. Impellitteri and Abraham Beame – were born abroad. ii](https://reader033.vdocument.in/reader033/viewer/2022041911/5e67aa2755089b3b9c241273/html5/thumbnails/20.jpg)
Evaluation on a Synthetic Noise Dataset
v Dataset: SemEval-2010 Task 8v True Positive: Cause-Effectv False Positive: Other relation types
20
v True Positive + False Positive: 1331 samples
![Page 21: Robust Distant Supervision Relation Extraction via Deep ... · Some New York city mayors –William O’Dwyer, Vincent R. Impellitteri and Abraham Beame – were born abroad. ii](https://reader033.vdocument.in/reader033/viewer/2022041911/5e67aa2755089b3b9c241273/html5/thumbnails/21.jpg)
0.64
0.645
0.65
0.655
0.66
0.665
0.67
0.675
0.68
0.685
0 10 20 30 40 50 60 70 80 90 100
F1Score
Epoch
200 FPs in 1331 Samples
21
(198/388)
(197/339)
(195/308)(180/279) (179/260)
False PositiveRemoved Part
Evaluation on a Synthetic Noise Dataset
![Page 22: Robust Distant Supervision Relation Extraction via Deep ... · Some New York city mayors –William O’Dwyer, Vincent R. Impellitteri and Abraham Beame – were born abroad. ii](https://reader033.vdocument.in/reader033/viewer/2022041911/5e67aa2755089b3b9c241273/html5/thumbnails/22.jpg)
22
0.68
0.69
0.7
0.71
0.72
0.73
0.74
0.75
0 10 20 30 40 50 60 70 80 90 100
F1Score
Epoch
0 FPs in 1331 samples
(0/258)
(0/150)
(0/121)
(0/59)(0/32)
Evaluation on a Synthetic Noise Dataset
![Page 23: Robust Distant Supervision Relation Extraction via Deep ... · Some New York city mayors –William O’Dwyer, Vincent R. Impellitteri and Abraham Beame – were born abroad. ii](https://reader033.vdocument.in/reader033/viewer/2022041911/5e67aa2755089b3b9c241273/html5/thumbnails/23.jpg)
23
vCNN+ONE, PCNN+ONE§ Distant supervision for relation extraction via piecewise convolutional
neural networks. (Zeng et al., 2015)
vCNN+ATT, PCNN+ATT§ Neural relation extraction with selective attention over instances.
(Lin et al., 2016)
Distant Supervision
vDataset: Riedel et al., 2010§ http://iesl.cs.umass.edu/riedel/ecml/
![Page 24: Robust Distant Supervision Relation Extraction via Deep ... · Some New York city mayors –William O’Dwyer, Vincent R. Impellitteri and Abraham Beame – were born abroad. ii](https://reader033.vdocument.in/reader033/viewer/2022041911/5e67aa2755089b3b9c241273/html5/thumbnails/24.jpg)
24
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4
CNN-basedCNN+ONE
CNN+ONE_RL
CNN+ATT
CNN+ATT_RL
Distant Supervision
![Page 25: Robust Distant Supervision Relation Extraction via Deep ... · Some New York city mayors –William O’Dwyer, Vincent R. Impellitteri and Abraham Beame – were born abroad. ii](https://reader033.vdocument.in/reader033/viewer/2022041911/5e67aa2755089b3b9c241273/html5/thumbnails/25.jpg)
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4
PCNN-based
PCNN+ONE
PCNN+ONE_RL
PCNN+ATT
PCNN+ATT_RL
25
Distant Supervision
![Page 26: Robust Distant Supervision Relation Extraction via Deep ... · Some New York city mayors –William O’Dwyer, Vincent R. Impellitteri and Abraham Beame – were born abroad. ii](https://reader033.vdocument.in/reader033/viewer/2022041911/5e67aa2755089b3b9c241273/html5/thumbnails/26.jpg)
Outline
•Motivation•Algorithm• Experiments•Conclusion
26
![Page 27: Robust Distant Supervision Relation Extraction via Deep ... · Some New York city mayors –William O’Dwyer, Vincent R. Impellitteri and Abraham Beame – were born abroad. ii](https://reader033.vdocument.in/reader033/viewer/2022041911/5e67aa2755089b3b9c241273/html5/thumbnails/27.jpg)
vWe propose a deep reinforcement learning methodfor robust distant supervision relation extraction.
v Our method is model-agnostic.
v Our method boost the performance of recently proposedneural relation extractors.
27
Conclusion
![Page 28: Robust Distant Supervision Relation Extraction via Deep ... · Some New York city mayors –William O’Dwyer, Vincent R. Impellitteri and Abraham Beame – were born abroad. ii](https://reader033.vdocument.in/reader033/viewer/2022041911/5e67aa2755089b3b9c241273/html5/thumbnails/28.jpg)
28
Thank you!
Q&A