ds-to-ps conversion

DS-to-PS conversion

Fei XiaUniversity of Washington

July 29, 2011

1

Main steps in building the treebank

• DS treebank:– Tokenization– Morphological analysis, voice, etc.– POS tagging– DS

• Propbank: adding Predicate-argument info

• Automatic DS-to-PS conversion

• Some manual check to ensure the conversion works well

2

Outline

• Important concepts

• Compatibility and consistency

• Handling inconsistency

3

Important concepts

• Linguistic phenomena

• Representation type

• Linguistic theory– Theoretical framework– Linguistic analyses

• Annotation guidelines4

Linguistic phenomena

• They are what we want to present, including– General concepts: e.g., which words form a

phrase? What types of phrases does a language have?

– Types of relations between words or phrases (e.g., subjecthood, temporal modification)

– Specific constructions (e.g., small clause)– Finer-grained distinctions (e.g., unergative vs.

unaccusative)

5

Representation type

• It is the type of mathematical object that is used to represent syntactic facts

• Examples: DS, PS

• Each representation type can decide what more specific representation devices to employ– Labels on the arcs of a tree– Use of empty nodes or coindexation between nodes

6

Linguistic theory

• It explains how linguistic phenomena are represented in the chose representation type

• It has two components:– Theoretical framework: it provides vocabulary and

constraints in which linguistic theories can be formulated: e.g., GB, LFG, LTAG, HPSG

– Linguistic analyses

7

Small clause

8

“Exceptional case-marking” analysis

9

“Raising-to-object” analysis

10

Annotation guidelines

• Guideline designers need to choose the following– Linguistic phenomena to represent– Representation type– Theoretical framework– Linguistic analyses

– Descriptions– Examples: sentences with DS or PS trees

11

Outline

• Important concepts

• Compatibility and consistency

• Handling inconsistency

12

“Exceptional case-marking” analysis

13

“Raising-to-object” analysis

14

Implicit vs. explicit information

• Certain aspects of information has to be expressed explicitly in DS, but not PS, or vice versa– Head in DS – Syntactic categories of phrases in PS

• Not explicitly providing info does not mean that corresponding concepts does not exist in DS/PS

15

Syntactic consistency

• We assume each phrase in a PS has a special word, head word, which represents the property of the phrase.

• A (DS, PS) pair is called consistent if there is a way to assign a head word to each internal node in the PS so that the resulting DS is identical to the given DS.

16

Consistent pairs

17

Inconsistent pairs

18

A real example

19

Consistency assumption

20

Definition of consistency

• A DS and a PS are consistent iff there exists a flattened version of the PS that is identical to the DS.

• If the input DS and the desired PS are consistent, the PS can be created by stretching the DS and adding syntactic labels.

21

Checking consistency

• For each (dep, head) pair in the DS– find their location in the PS and their closest

antecedent – add heads to the nodes on the path between the

leaf nodes and the antecedent

• The DS and the PS are consistent iff each node in the PS has exactly one head.

22

(Vinken, join)

(Vinken)

(join)

(join)

(join)

(board, join)

(board)

(will, join)

(29, join)

(29)

23

Outline

• Important concepts

• Compatibility and consistency

• Handling inconsistency

24

wh-movement

(who, come)come

come

come

come

come

come

come

(come, think)25

wh-movement

(who, come)

(come, think)

come

come

come

come

come | think

come | think

come

26

wh-movement

(who, come)

(come, think)

come

come

come

??

think

think

??

(you, think)27

Can DS and PS be inconsistent?• DS and PS can represent different aspects of the same

overall pictures, and still be consistent.– Info provided in PropBank: e.g., empty subject, unaccusative– Info that is in PS only: e.g., traces

• DS and PS should not choose “conflicting” analyses.– DS and PS are two images of the same underlying treebank, not

two separate treebanks.– Ex: ba-construction in Chinese: verb, prep, or something else?– Ex: free relatives: empty nominal head

• The inconsistency cases should be rare and well-motivated.

28

How to handle inconsistency?

• Detect inconsistency in (DS, PS) pairs in the guidelines

• Consult guideline designers to determine whether the inconsistency can be resolved by changing analyses

• If not, introduce DScons and ensure sufficient info is in DS for automatic conversion.

29

Two-stage conversion

• DS to DScons: by removing “inconsistency” between DS and PS.

• DScons to PS: by applying conversion rules

30

Case #1: long-distance movement

31

DSconst:DSprop:

• Other examples: extraposition• Easily detectable due to non-projectivity• Create DSconst by moving up the “moved element” and leaving a trace

• which node is the “moved element”? The one that is apart from other nodes in the subtree.

Case #2: local scrambling

32

Detectable by assuming canonical word order: k1 > k2

Need from PS/DS teams the canonical word order and what word order triggers movement

Case #3: small clause rule

33

Detectable by dependency type k2s

Need confirmation from IIIT that k2s is used only for small clause

Case 4: support verb

34

Detectable by dependency type “pof”Need confirmation from IIIT that “pof” is used only for support verb

Conclusion

• We define consistency between DS and PS

• DS and PS can be inconsistent but such cases should be rare and well-motivated.

• We will handle inconsistency with the two-stage approach

35

Conversion algorithm

36

Definition of conversion rule• A conversion rule is a (DS_pattern, PS_pattern) pair.

• Ex:

• Simplest case: – DS_pattern corresponds to only one dependency link– Decomposing DS becomes trivial– PS_pattern is a tree fragment (e.g., wh-movement)– Learning rules from (PS, DS) pairs is easy

37

Extracting rules

38

Rules extracted from the example

39

Input DS

40

41

Gluing PS segments together

42

c

c

c

43

44