robust distant supervision relation extraction via deep ... · some new york city mayors –william...

Robust Distant Supervision Relation Extraction via Deep Reinforcement

Learning

Pengda Qin, Weiran Xu andWilliamWang1

BUPT

Outline

•Motivation•Algorithm• Experiments•Conclusion

2

Outline

•Motivation•Algorithm• Experiments•Conclusion

3

Plain Text Corpus(Unstructured Info)

Classifier Entity-Relation Triple(Structured Info)

Relation Type withLabeled Dataset

Relation Type withoutLabeled Dataset

4

Relation Extraction

“If two entities participate in a relation, any sentencethat contains those two entities might express thatrelation.” (Mintz, 2009)

5

Distant Supervision

Data(x): <Belgium, Nijlen>Label(y): /location/contains

Target Corpus(Unlabeled)

Relation Label: /location/contains

1. Nijlen is a municipalitylocated in the Belgianprovince of Antwerp.

2. ……3. ……

6

Sentence Bag:

Distant Supervision

v Within-Sentence-Bag Level

v Entity-Pair Level

§ Hoffmann et al., ACL 2011.§ Surdean et al., ACL 2012.§ Zeng et al., ACL 2015. § Li et al., ACL 2016.

§ None

7

Wrong Labeling

§ Place_of_Death (William O’Dwyer, New York city)

8

v Entity-Pair Level

Wrong Labeling

i. Some New York city mayors – William O’Dwyer, Vincent R. Impellitteriand Abraham Beame – were born abroad.

ii. Plenty of local officials have, too, including two New York city mayors, James J. Walker, in 1932, and William O’Dwyer, in 1950.

v Most of entity pairs only have several sentences

1 Sentence55%2 Sentence

32%

Other4%

9

v Lots of entity pairs have repetitive sentences

Wrong Labeling

Outline

•Motivation•Algorithm• Experiments•Conclusion

10

Negative set

Positive set

Negative set

Positive setFalse Positive

False Positive

DS Dataset Cleaned Dataset

11

Overview

False PositiveIndicator

Sentence-Level Indicator

False-Positive Indicator

Learn a Policy to Denoise the Training Data

12

General Purpose and Offline Process

Without Supervised Information

Requirements

Negative set

Positive set

Negative set

Positive setFalse Positive

𝑅𝑒𝑤𝑎𝑟𝑑

𝐴𝑐𝑡𝑖𝑜𝑛

Classifier𝑇𝑟𝑎𝑖𝑛

False Positive

DS Dataset Cleaned Dataset

13

Overview

False PositiveIndicator

Policy-Based Agent

v State

14

v Action

v Reward§ ???

§ Sentence vector§ The average vector of previous removed sentences

§ Remove & retain

Deep Reinforcement Learning

v One relation type has an agent

15

v Sentence-level

v Split into training set and validation set

§ Positive: Distantly-supervised positive sentences§ Negative: Sampled from other relations

Deep Reinforcement Learning

RL Agent Train

16

TrainRelationClassifier

RelationClassifier

𝐹/01/

𝐹/0× +𝓡0 + ×(−𝓡0)

Noisydataset𝑃:;<0

Cleaneddataset

Cleaneddataset

Removedpart

Removedpart

𝓡0 = 𝛼(𝐹/0 - 𝐹/01/ )

RL Agent

Epoch i-1

EpochiNoisy

dataset𝑃:;<0

+𝑁:;<0

𝑁:;<0 +

Deep Reinforcement Learning

17

Positive Set Negative Set

§ Accurate

§ Steady

§ Fast

False Positive

§ Obvious

Reward

18

Positive Set Negative Set

RelationClassifier

TrainRelationClassifier

Train

Calculate

𝐹/

Epoch 𝑖

False Positive

False Positive

Positive Negative

Reward

Outline

• Motivation• Algorithm• Experiments• Conclusion

19

Evaluation on a Synthetic Noise Dataset

v Dataset: SemEval-2010 Task 8v True Positive: Cause-Effectv False Positive: Other relation types

20

v True Positive + False Positive: 1331 samples

0.64

0.645

0.65

0.655

0.66

0.665

0.67

0.675

0.68

0.685

0 10 20 30 40 50 60 70 80 90 100

F1Score

Epoch

200 FPs in 1331 Samples

21

(198/388)

(197/339)

(195/308)(180/279) (179/260)

False PositiveRemoved Part

Evaluation on a Synthetic Noise Dataset

22

0.68

0.69

0.7

0.71

0.72

0.73

0.74

0.75

0 10 20 30 40 50 60 70 80 90 100

F1Score

Epoch

0 FPs in 1331 samples

(0/258)

(0/150)

(0/121)

(0/59)(0/32)

Evaluation on a Synthetic Noise Dataset

23

vCNN+ONE, PCNN+ONE§ Distant supervision for relation extraction via piecewise convolutional

neural networks. (Zeng et al., 2015)

vCNN+ATT, PCNN+ATT§ Neural relation extraction with selective attention over instances.

(Lin et al., 2016)

Distant Supervision

vDataset: Riedel et al., 2010§ http://iesl.cs.umass.edu/riedel/ecml/

24

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4

CNN-basedCNN+ONE

CNN+ONE_RL

CNN+ATT

CNN+ATT_RL

Distant Supervision

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4

PCNN-based

PCNN+ONE

PCNN+ONE_RL

PCNN+ATT

PCNN+ATT_RL

25

Distant Supervision

Outline

•Motivation•Algorithm• Experiments•Conclusion

26

vWe propose a deep reinforcement learning methodfor robust distant supervision relation extraction.

v Our method is model-agnostic.

v Our method boost the performance of recently proposedneural relation extractors.

27

Conclusion

28

Thank you!

Q&A