robust distant supervision relation extraction via deep ... · some new york city mayors –william...
TRANSCRIPT
Robust Distant Supervision Relation Extraction via Deep Reinforcement
Learning
Pengda Qin, Weiran Xu andWilliamWang1
BUPT
Outline
•Motivation•Algorithm• Experiments•Conclusion
2
Outline
•Motivation•Algorithm• Experiments•Conclusion
3
Plain Text Corpus(Unstructured Info)
Classifier Entity-Relation Triple(Structured Info)
Relation Type withLabeled Dataset
Relation Type withoutLabeled Dataset
4
Relation Extraction
“If two entities participate in a relation, any sentencethat contains those two entities might express thatrelation.” (Mintz, 2009)
5
Distant Supervision
Data(x): <Belgium, Nijlen>Label(y): /location/contains
Target Corpus(Unlabeled)
Relation Label: /location/contains
1. Nijlen is a municipalitylocated in the Belgianprovince of Antwerp.
2. ……3. ……
6
Sentence Bag:
Distant Supervision
v Within-Sentence-Bag Level
v Entity-Pair Level
§ Hoffmann et al., ACL 2011.§ Surdean et al., ACL 2012.§ Zeng et al., ACL 2015. § Li et al., ACL 2016.
§ None
7
Wrong Labeling
§ Place_of_Death (William O’Dwyer, New York city)
8
v Entity-Pair Level
Wrong Labeling
i. Some New York city mayors – William O’Dwyer, Vincent R. Impellitteriand Abraham Beame – were born abroad.
ii. Plenty of local officials have, too, including two New York city mayors, James J. Walker, in 1932, and William O’Dwyer, in 1950.
v Most of entity pairs only have several sentences
1 Sentence55%2 Sentence
32%
Other4%
9
v Lots of entity pairs have repetitive sentences
Wrong Labeling
Outline
•Motivation•Algorithm• Experiments•Conclusion
10
Negative set
Positive set
Negative set
Positive setFalse Positive
False Positive
DS Dataset Cleaned Dataset
11
Overview
False PositiveIndicator
Sentence-Level Indicator
False-Positive Indicator
Learn a Policy to Denoise the Training Data
12
General Purpose and Offline Process
Without Supervised Information
Requirements
Negative set
Positive set
Negative set
Positive setFalse Positive
𝑅𝑒𝑤𝑎𝑟𝑑
𝐴𝑐𝑡𝑖𝑜𝑛
Classifier𝑇𝑟𝑎𝑖𝑛
False Positive
DS Dataset Cleaned Dataset
13
Overview
False PositiveIndicator
Policy-Based Agent
v State
14
v Action
v Reward§ ???
§ Sentence vector§ The average vector of previous removed sentences
§ Remove & retain
Deep Reinforcement Learning
v One relation type has an agent
15
v Sentence-level
v Split into training set and validation set
§ Positive: Distantly-supervised positive sentences§ Negative: Sampled from other relations
Deep Reinforcement Learning
RL Agent Train
16
TrainRelationClassifier
RelationClassifier
𝐹/01/
𝐹/0× +𝓡0 + ×(−𝓡0)
Noisydataset𝑃:;<0
Cleaneddataset
Cleaneddataset
Removedpart
Removedpart
𝓡0 = 𝛼(𝐹/0 - 𝐹/01/ )
RL Agent
Epoch i-1
EpochiNoisy
dataset𝑃:;<0
+𝑁:;<0
𝑁:;<0 +
Deep Reinforcement Learning
17
Positive Set Negative Set
§ Accurate
§ Steady
§ Fast
False Positive
§ Obvious
Reward
18
Positive Set Negative Set
RelationClassifier
TrainRelationClassifier
Train
Calculate
𝐹/
Epoch 𝑖
False Positive
False Positive
Positive Negative
Reward
Outline
• Motivation• Algorithm• Experiments• Conclusion
19
Evaluation on a Synthetic Noise Dataset
v Dataset: SemEval-2010 Task 8v True Positive: Cause-Effectv False Positive: Other relation types
20
v True Positive + False Positive: 1331 samples
0.64
0.645
0.65
0.655
0.66
0.665
0.67
0.675
0.68
0.685
0 10 20 30 40 50 60 70 80 90 100
F1Score
Epoch
200 FPs in 1331 Samples
21
(198/388)
(197/339)
(195/308)(180/279) (179/260)
False PositiveRemoved Part
Evaluation on a Synthetic Noise Dataset
22
0.68
0.69
0.7
0.71
0.72
0.73
0.74
0.75
0 10 20 30 40 50 60 70 80 90 100
F1Score
Epoch
0 FPs in 1331 samples
(0/258)
(0/150)
(0/121)
(0/59)(0/32)
Evaluation on a Synthetic Noise Dataset
23
vCNN+ONE, PCNN+ONE§ Distant supervision for relation extraction via piecewise convolutional
neural networks. (Zeng et al., 2015)
vCNN+ATT, PCNN+ATT§ Neural relation extraction with selective attention over instances.
(Lin et al., 2016)
Distant Supervision
vDataset: Riedel et al., 2010§ http://iesl.cs.umass.edu/riedel/ecml/
24
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4
CNN-basedCNN+ONE
CNN+ONE_RL
CNN+ATT
CNN+ATT_RL
Distant Supervision
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4
PCNN-based
PCNN+ONE
PCNN+ONE_RL
PCNN+ATT
PCNN+ATT_RL
25
Distant Supervision
Outline
•Motivation•Algorithm• Experiments•Conclusion
26
vWe propose a deep reinforcement learning methodfor robust distant supervision relation extraction.
v Our method is model-agnostic.
v Our method boost the performance of recently proposedneural relation extractors.
27
Conclusion
28
Thank you!
Q&A