pd3: better cross-lingual transfer by combining direct ... · pd3 i/pron love/v cats/n cats/n...

27.03.2018 | Fachbereich Informatik | UKP Lab 1

Steffen Eger*, Andreas Rückle, Iryna Gurevych

PD3: Better Cross-Lingual Transfer By Combining Direct Transfer and Annotation Projection


Argumentation Mining

● Fast-growing research field in NLP

● Different sub-tasks:

1) segmenting arguments from non-arguments in text;

2) classifying them (claim, premise, ...);

3) finding relations between arguments (support, attack)

4) Ranking arguments


Challenges for argumentation mining

● Going cross-lingual

○ I.e. train system in a source language L1 (typically: English),

then apply system to specific target language L2 of interest

○ Avoids having to redo (high) annotation costs

● Recently, several works have addressed variants of this setup:

○ Aker and Zhang, 2017; Sliwa et al. 2018;

Eger et al., 2018; Rocha et al. 2018


Task considered in our work

● We consider argumentation mining

○ On the sentence-level

○ Classifying each sentence into 4 classes:

■ Claim, MajorClaim, Premise, None

● Dataset is derived from the Persuasive Essay (PE) dataset of Stab

and Gurevych (2017); Eger et al. (2018) (bi-lingual variant)

○ But token-level annotations are mapped to the sentence-level


(Mono-lingual) Examples

● Not cooking fresh food will lead to lack of nutrition Claim

● To sum up, [...] the merits of animal experiments still outweigh the

demerits Major claim

● For example, tourism makes up one third of Czech’s economy

Premise

● I will mention some basic reasoning as follows O


Our contribution

● We explore cross-lingual argumentation mining in the low-resource

setting, i.e., having very little parallel data, ….

○ Which is likewise a hot topic concurrently (Zhang et al.,

2016; Artetxe et al., 2017; Artetxe et al.,

2018; Lample et al., 2018; Schulz et al. 2018)

● … by combining two standard cross-lingual approaches --- direct

transfer and annotation projection

7

Excursion - Cross-lingual transfer 1: Direct Transfer

L1 L2

Die Stube brenntKinder sind doof

…..

Bilingual word embeddings

I/PRON love/V children/NCats/N like/V me/PRON

…..

8

Direct Transfer

I/PRON love/V children/NCats/N like/V me/PRON

…..

L1 L2

Die Stube brenntKinder sind doof

…..

Bilingual word embeddings

9

Excursion - Cross-lingual transfer 2: Annotation Projection

L1 L2

Die Stube brenntDas Wasser läuft

…..

L1-L2

Horses eat carrots Pferde essen MöhrenSoccer is football Fußball ist Fußball

…..

I/PRON love/V cats/NCats/N like/V me/PRON

…..

10

Projection


…..

L1 L2


…..

L1-L2

Horses eat carrots Pferde essen MöhrenSoccer is football Fußball ist Fußball

…..

Train

11

Projection

L1 L2


…..

L1-L2

Horses/N eat/V carrots/N Pferde essen MöhrenSoccer/N is/V football/N Fußball ist Fußball

…..

Annotate


…..

12

Projection

L1 L2


…..

L1-L2

Horses/N eat/V carrots/N Pferde/N essen/V Möhren/NSoccer/N is/V football/N Fußball/N ist/V Fußball/N

…..

Project


…..

13

Projection

L1 L2


…..

L1-L2


…..

Project


…..

14

Projection

L1 L2


…..

L1-L2


…..

Project


…..

15

Projection

L1 L2


…..

L1-L2


…..

Train/Annotate


…..

16

PD3


…..

L1 L2


…..

L1-L2


…..

Train/Annotate

Train on bilingual

repres./Annotate

17

PD3


…..

L1 L2


…..

L1-L2


…..

Train/Annotate

Train on bilingual

repres./Annotate

18

PD3


…..

L1 L2


…..

L1-L2


…..

Train/Annotate

Train on bilingual

repres./Annotate

19

PD3: Combining Direct Transfer and Projection

• One last issue:

• Can either merge all 3 datasets

• Or use multi-task learning, taking e.g., both L1 datasets as Task1 and the L2 dataset as Task2

20

Experiments

• Bilingual data:• en: To sum up [...], the merits of animal experiments still outweigh

the demerits MajorClaim• de: Zusammenfassend kann ich bestätigen [...], dass die Vorzüge

von Tierversuchen die Nachteile [...] überwiegen MajorClaim• About 7k parallel sentences, available here:

https://github.com/UKPLab/coling2018-xling_argument_mining

● Setup:○ 2k for train (en), 0.5k for dev (en), 1.5k for test (de)○ 3k as parallel data (and further subsets thereof)

■ We also consider non-argumentative parallel data from TED○ Evaluation Metric is Macro-F1

https://github.com/UKPLab/coling2018-xling_argument_mining

21

Results - high quality bilingual embeddings

22

Results - low quality bilingual embeddings

23

Results - low quality bilingual embeddings

24

Results - non-argumentative parallel data

25

Conclusion

● Considered low-resource language transfer for ArgMin○ By combining direct transfer and annotation projection

● There are benefits, but they’re small

● Also, they diminish quickly

● True low-resource language transfer still a big challenge○ And an important avenue for the future

● Doing annotation projection using machine translation without any parallel data (Artexte et al. 2018, Lample et al. 2018) may be worthwhile to investigate prospectively


THÁNK YÕU

pd3: better cross-lingual transfer by combining direct ... · pd3 i/pron love/v cats/n cats/n...

Documents