statistical dependency parsing in korean: from corpus generation to automatic parsing

Statistical Dependency Parsing in Korean:From Corpus Generation To Automatic Parsing

University of Colorado at BoulderOctober 6th, 2011

[email protected]

Workshop on Statistical Parsing of Morphologically-Rich Languages12th International Conference on Parsing Technologies

Jinho D. Choi & Martha Palmer

Thursday, October 6, 2011

Dependency Parsing in Korean• Why dependency parsing in Korean?

- Korean is a flexible word order language.

2

S

NP-SBJ

AP

NP-OBJ VP

VP

��

VP

She still him loved

ADVOBJ

SBJSBJ

ADV

OBJ

SOV constructionS

NP-SBJ

AP

NP-OBJ VP

VP

��

VP

she still *T* loved

S

�Him

NP-OBJ-1


Dependency Parsing in Korean• Why dependency parsing in Korean?

- Korean is a flexible word order language.

3

- Rich morphology makes it easy for dependency parsing.

��

��

She still him

loved

SBJ OBJADV

그녀 + 는She + Aux. particle

그 + 를He + Obj. case marker


Dependency Parsing in Korean• Statistical dependency parsing in Korean

- Sufficiently large training data is required.

• Not much training data available for Korean dependency parsing.

• Constituent Treebanks in Korean

- Penn Korean Treebank: 15K sentences.

- KAIST Treebank: 30K sentences.

- Sejong Treebank: 60K sentences.

• The most recent and largest Treebank in Korean.

• Containing Penn Treebank style constituent trees.

4


Sejong Treebank• Phrase structure

- Including phrase tags, POS tags, and function tags.

- Each token can be broken into several morphemes.

5

��(��)/NP+ /JX

��/NNG+�/XSV+�/EP+�/EF

��

�� /MAG

��/NP+�/JKO��

��

!

!

!

!

S

NP-SBJ

AP

NP-OBJ VP

VP

��

VP

She still him loved Tokens are mostly separatedby white spaces.


Sejong Treebank

6

The tree consists of phrasal nodes as described inTable 2. Each token can be broken into several mor-phemes annotated with POS tags (Table 1). In the Se-jong Treebank, tokens are separated mostly by whitespaces; for instance, an item like ‘A(B+C)D’ is con-sidered as a single token because it does not containany white space in between. As a result, a token canbe broken into as many as 21 individual morphemes(e.g., ‘A’, ‘(’, ‘B’, ‘+’, ‘C’, ‘)’, ‘D’).

Notice that some phrases are annotated with func-tion tags (Table 2). These function tags show depen-dency relations between the tagged phrases and theirsiblings, so can be used as dependency labels duringthe conversion. There are three other special types ofphrase-level tags besides the ones in Table 2. X indi-cates phrases containing only case particles, endingmarkers, or punctuation. L and R indicate phrasescontaining only left and right brackets, respectively.These tags are also used to determine dependencyrelations during the conversion.

Phrase-level tags Function tagsS Sentence SBJ SubjectQ Quotative clause OBJ ObjectNP Noun phrase CMP ComplementVP Verb phrase MOD Noun modifierVNP Copula phrase AJT Predicate modifierAP Adverb phrase CNJ ConjunctiveDP Adnoun phrase INT VocativeIP Interjection phrase PRN Parenthetical

Table 2: Phrase-level tags (left) and function tags (right)in the Sejong Treebank.

3.2 Head-percolation rules

Table 3 gives the list of head-percolation rules (fromnow on, headrules), derived from analysis of eachphrase type in the Sejong Treebank. Except for thequotative clause (Q), all other phrase types try tofind their heads from the rightmost children, whichaligns with the general concept of Korean being ahead-final language. Note that these headrules donot involve the POS tags in Table 1; those POS tagsare used only for morphemes within tokens (andeach token is annotated with a phrase-level tag). It ispossible to extend the headrules to token-level andfind the head morpheme of each token; however,finding dependencies between different morphemeswithin a token is not especially interesting although

there are some approaches that have treated eachmorpheme as an individual token to parse (Chunget al., 2010).5

S r VP;VNP;S;NP|AP;Q;*

Q l S|VP|VNP|NP;Q;*

NP r NP;S;VP;VNP;AP;*

VP r VP;VNP;NP;S;IP;*

VNP r VNP;NP;S;*

AP r AP;VP;NP;S;*

DP r DP;VP;*

IP r IP;VNP;*

X|L|R r *

Table 3: Head-percolation rules for the Sejong Tree-bank. l/r implies looking for the leftmost/rightmost con-stituent. * implies any phrase-level tag. | implies a logi-cal OR and ; is a delimiter between tags. Each rule giveshigher precedence to the left (e.g., S takes the highestprecedence in VP).

Once we have the headrules, it is pretty easy to gen-erate dependency trees from constituent trees. Foreach phrase (or clause) in a constituent tree, we findthe head of the phrase using its headrules and makeall other nodes in the phrase dependents of the head.The procedure goes on recursively until every nodein the tree finds its head (for more details, see Choiand Palmer (2010)). A dependency tree generatedby this procedure is guaranteed to be well-formed(unique root, single head, connected, and acyclic);however, it does not include labels yet. Section 3.3shows how to add dependency labels to these trees.In addition, Section 3.4 describes heuristics to re-solve some of the special cases (e.g., coordinations,nested function tags).

It is worth mentioning that constituent trees inthe Sejong Treebank do not include any empty cat-egories. This implies that dependency trees gener-ated by these headrules consist of only projective de-pendencies (non-crossing edges; Nivre and Nilsson(2005)). On the other hand, the Penn Korean Tree-bank contains empty categories representing long-distance dependencies. It will be interesting to see ifwe can train empty category insertion and resolutionmodels on the Penn Korean Treebank, run the mod-

5Chung et al. (2010) also showed that recovering certainkinds of null elements improves PCFG parsing, which can beapplied to dependency parsing as well.

NNG General noun MM Adnoun EP Prefinal EM JX Auxiliary PRNNP Proper noun MAG General adverb EF Final EM JC Conjunctive PRNNB Bound noun MAJ Conjunctive adverb EC Conjunctive EM IC InterjectionNP Pronoun JKS Subjective CP ETN Nominalizing EM SN NumberNR Numeral JKC Complemental CP ETM Adnominalizing EM SL Foreign wordVV Verb JKG Adnomial CP XPN Noun prefix SH Chinese wordVA Adjective JKO Objective CP XSN Noun DS NF Noun-like wordVX Auxiliary predicate JKB Adverbial CP XSV Verb DS NV Predicate-like wordVCP Copula JKV Vocative CP XSA Adjective DS NA Unknown wordVCN Negation adjective JKQ Quotative CP XR Base morpheme SF, SP, SS, SE, SO, SW

Table 1: POS tags in the Sejong Treebank (PM: predicate marker, CP: case particle, EM: ending marker, DS: deriva-tional suffix, PR: particle, SF SP SS SE SO: different types of punctuation).

Automatic morphological analysis has been one ofthe most broadly explored topics in Korean NLP.Kang and Woo (2001) presented the KLT morpho-logical analyzer, which has been widely used in Ko-rea. The Sejong project distributed the IntelligentMorphological Analyzer (IMA), used to pre-processraw texts in the Sejong Treebank.3 Shim and Yang(2002) introduced another morphological analyzer,called Mach, optimized for speed; it takes about 1second to analyze 1.3M words yet performs very ac-curately. Han (2005) presented a finite-state trans-ducer that had been used to check morphology inthe Penn Korean Treebank. We use IMA and Machto generate fine-grained (IMA) and coarse-grained(Mach) morphologies for our experiments.

Statistical dependency parsing has been exploredrelatively little in Korean. Chung (2004) presenteda dependency parsing model using surface contex-tual information. This parser can be viewed as aprobabilistic rule-based system that gathers proba-bilities from features like lexical items, POS tags,and distances. Oh and Cha (2008) presented anotherstatistical dependency parsing model using cascadedchunking for parsing and conditional random fieldsfor learning. Our work is distinguished from theirsin mainly two ways. First, we add labels to de-pendency edges during the conversion, so parsingperformance can be evaluated on both labels andedges. Second, we selectively choose morphemesuseful for dependency parsing, which prevents gen-erating very sparse features. The morpheme selec-tion is done automatically by applying our linguisti-cally motivated rules (cf. Section 5.3).

3The system is not publicly available but can be requestedfrom the Sejong project (http://www.sejong.or.kr).

Han et al. (2000) presented an approach for han-dling structural divergence and recovering droppedarguments in a Korean-to-English machine transla-tion system. In their approach, they used a Koreandependency parser for lexico-structural processing.

3 Constituent-to-dependency conversion

3.1 Constituent trees in the Sejong Treebank

The Sejong Treebank contains constituent trees sim-ilar to ones in the Penn Treebank.4 Figure 2 showsa constituent tree and morphological analysis for asentence, She still loved him, in Korean.

S

NP-SBJ

AP

NP-OBJ VP

VP

��

VP

She still him loved

��(��)/NP+ /JX

��/NNG+�/XSV+�/EP+�/EF

��

�� /MAG

��/NP+�/JKO��

��

!

!

!

!

Figure 2: A constituent tree and morphological analysisfor a sentence, She still loved him, in Korean.

4The bracketing guidelines can be requested from the Se-jong project, available only in Korean.


Dependency Conversion• Conversion steps

- Find the head of each phrase using head-percolation rules.

• All other nodes in the phrase become dependents of the head.

- Re-direct dependencies for empty categories.

• Empty categories are not annotated in the Sejong Treebank.

• Skipping this step generates only projective dependency trees.

- Label (automatically generated) dependencies.

• Special cases

- Coordination, nested function tags.

7


Dependency Conversion• Head-percolation rules

- Achieved by analyzing each phrase in the Sejong Treebank.

8

The tree consists of phrasal nodes as described inTable 2. Each token can be broken into several mor-phemes annotated with POS tags (Table 1). In the Se-jong Treebank, tokens are separated mostly by whitespaces; for instance, an item like ‘A(B+C)D’ is con-sidered as a single token because it does not containany white space in between. As a result, a token canbe broken into as many as 21 individual morphemes(e.g., ‘A’, ‘(’, ‘B’, ‘+’, ‘C’, ‘)’, ‘D’).

Notice that some phrases are annotated with func-tion tags (Table 2). These function tags show depen-dency relations between the tagged phrases and theirsiblings, so can be used as dependency labels duringthe conversion. There are three other special types ofphrase-level tags besides the ones in Table 2. X indi-cates phrases containing only case particles, endingmarkers, or punctuation. L and R indicate phrasescontaining only left and right brackets, respectively.These tags are also used to determine dependencyrelations during the conversion.

Phrase-level tags Function tagsS Sentence SBJ SubjectQ Quotative clause OBJ ObjectNP Noun phrase CMP ComplementVP Verb phrase MOD Noun modifierVNP Copula phrase AJT Predicate modifierAP Adverb phrase CNJ ConjunctiveDP Adnoun phrase INT VocativeIP Interjection phrase PRN Parenthetical

Table 2: Phrase-level tags (left) and function tags (right)in the Sejong Treebank.

3.2 Head-percolation rules

Table 3 gives the list of head-percolation rules (fromnow on, headrules), derived from analysis of eachphrase type in the Sejong Treebank. Except for thequotative clause (Q), all other phrase types try tofind their heads from the rightmost children, whichaligns with the general concept of Korean being ahead-final language. Note that these headrules donot involve the POS tags in Table 1; those POS tagsare used only for morphemes within tokens (andeach token is annotated with a phrase-level tag). It ispossible to extend the headrules to token-level andfind the head morpheme of each token; however,finding dependencies between different morphemeswithin a token is not especially interesting although

there are some approaches that have treated eachmorpheme as an individual token to parse (Chunget al., 2010).5

S r VP;VNP;S;NP|AP;Q;*

Q l S|VP|VNP|NP;Q;*

NP r NP;S;VP;VNP;AP;*

VP r VP;VNP;NP;S;IP;*

VNP r VNP;NP;S;*

AP r AP;VP;NP;S;*

DP r DP;VP;*

IP r IP;VNP;*

X|L|R r *

Table 3: Head-percolation rules for the Sejong Tree-bank. l/r implies looking for the leftmost/rightmost con-stituent. * implies any phrase-level tag. | implies a logi-cal OR and ; is a delimiter between tags. Each rule giveshigher precedence to the left (e.g., S takes the highestprecedence in VP).

Once we have the headrules, it is pretty easy to gen-erate dependency trees from constituent trees. Foreach phrase (or clause) in a constituent tree, we findthe head of the phrase using its headrules and makeall other nodes in the phrase dependents of the head.The procedure goes on recursively until every nodein the tree finds its head (for more details, see Choiand Palmer (2010)). A dependency tree generatedby this procedure is guaranteed to be well-formed(unique root, single head, connected, and acyclic);however, it does not include labels yet. Section 3.3shows how to add dependency labels to these trees.In addition, Section 3.4 describes heuristics to re-solve some of the special cases (e.g., coordinations,nested function tags).

It is worth mentioning that constituent trees inthe Sejong Treebank do not include any empty cat-egories. This implies that dependency trees gener-ated by these headrules consist of only projective de-pendencies (non-crossing edges; Nivre and Nilsson(2005)). On the other hand, the Penn Korean Tree-bank contains empty categories representing long-distance dependencies. It will be interesting to see ifwe can train empty category insertion and resolutionmodels on the Penn Korean Treebank, run the mod-

5Chung et al. (2010) also showed that recovering certainkinds of null elements improves PCFG parsing, which can beapplied to dependency parsing as well.

Korean is a head-final language.

No rules to find the head morpheme of each token.


Dependency Conversion• Dependency labels

- Labels retained from the function tags.

- Labels inferred from constituent relations.

9

S

NP-SBJ

AP

NP-OBJ VP

VP

��

VP

She still him loved

ADVOBJ

SBJ

els on the Sejong Treebank, and use the automati-cally inserted and linked empty categories to gener-ate non-projective dependencies.

3.3 Dependency labels

Two types of dependency labels are derived from theconstituent trees. The first type includes labels re-tained from the function tags. When any node an-notated with a function tag is determined to be a de-pendent of some other node by our headrules, thefunction tag is taken as the dependency label to itshead. Figure 3 shows a dependency tree convertedfrom the constituent tree in Figure 2, using the func-tion tags as dependency labels (SBJ and OBJ).

�� root

OBJADV

ROOTSBJ

She still him loved

Figure 3: A dependency tree with labels, converted fromthe constituent tree in Figure 2.

The other labels (ROOT and ADV) belong to the sec-ond type; these are labels inferred from constituentrelations. Algorithm 1 shows how to infer these la-bels given constituents c and p, where c is a de-pendent of p according to our headrules. ROOT isthe dependency label of the root node (root in Fig-ure 3). ADV is for adverbials, and (A|D|N|V)MODare for (adverb, adnoun, noun, verb) modifiers, re-spectively. DEP is the default label used when nocondition is met. These inferred labels are usedwhen c is not annotated with a function tag.

There are two other special types of labels. Adependency label P is assigned to all punctuationregardless of function tags or constituent relations.Furthermore, when c is a phrase type X, a label Xis conjoined with its genetic dependency label (e.g.,CMP → X CMP). This is because the phrase type Xusually consists of morphemes detached from theirhead phrases (e.g., case particles, ending markers),so should have distinguished dependency relationsfrom other standalone tokens. Table 4 shows thedistribution of all labels in the Sejong Treebank (inpercentile).

input : (c, p), where c is a dependent of p.output: A dependency label l as c l←− p.begin

if p = root then ROOT→ lelif c.pos = AP then ADV→ lelif p.pos = AP then AMOD→ lelif p.pos = DP then DMOD→ lelif p.pos = NP then NMOD→ lelif p.pos = VP|VNP|IP then VMOD→ lelse DEP→ l

endAlgorithm 1: Getting inferred labels.

AJT 11.70 MOD 18.71 X 0.01CMP 1.49 AMOD 0.13 X AJT 0.08CNJ 2.47 DMOD 0.02 X CMP 0.11INT 0.09 NMOD 13.10 X CNJ 0.02OBJ 8.95 VMOD 20.26 X MOD 0.07PRN 0.15 ROOT 8.31 X OBJ 0.07SBJ 11.74 P 2.42 X SBJ 0.09

Table 4: Distribution of all dependency labels (in %).

3.4 Coordination and nested function tagsBecause of the conjunctive function tag CNJ, identi-fying coordination structures is relatively easy. Fig-ure 4 shows constituent and dependency trees for asentence, I and he and she left home, in Korean.

S

NP-SBJ

��I_and

NP-CNJ NP-SBJ

NP-CNJ NP-SBJ

VP

NP-OBJ VP

�� he_and she home left

OBJSBJ

CNJ CNJ

Figure 4: Constituent (top) and dependency (bottom)trees for, I and he and she left home, in Korean.

According to our headrules, she becomes the headof both I and he, which is completely acceptable butcan cause long-distance dependencies when the co-ordination chain becomes long. Instead, we make


Dependency Conversion• Coordination

- Previous conjuncts as dependents of the following conjuncts.

• Nested function tag

- Nodes with nested f-tags become the heads of the phrases.

10

S

NP-SBJ

��I_and

NP-CNJ NP-SBJ

NP-CNJ NP-SBJ

VP

NP-OBJ VP

�� he_and she home left

OBJSBJ

CNJ CNJ


Dependency Parsing• Dependency parsing algorithm

- Transition-based, non-projective parsing algorithm.

• Choi & Palmer, 2011.

- Performs transitions from both projective and non-projective dependency parsing algorithms selectively.

• Linear time parsing speed in practice for non-projective trees.

• Machine learning algorithm

- Liblinear L2-regularized L1-loss support vector.

11

Jinho D. Choi & Martha Palmer. 2011. Getting the Most out of Transition-based Dependency Parsing. In Proceedings of ACL:HLT’11


Dependency Parsing• Feature selection

- Each token consists of multiple morphemes (up to 21).

- POS tag feature of each token?

• (NNG & XSV & EP & EF & SF) vs. (NNG | XSV | EP | EF | SF)

• Sparse information vs. lack of information.

12

��

��

��

��/NNP+��/NNG+�/JKO!Hodong + Prince + JKO

��/NNG+�/XSV+�/EP+�/EF+./SF!Love + XSV + EP + EF + .

��/NNP+ �/NNG+�/JX!Nakrang + Princess + JXNakrang_��

Hodong_��

��

Happy medium?


Dependency Parsing• Morpheme selection

13

λ1, that is wi−1. When the oracle predicts there is notoken in λ1 that has a dependency relation with wj ,wj is removed from β and added to λ1 along with allother tokens in λ2. The procedure is repeated withthe first token in β, that is wj+1. The algorithm ter-minates when there is no token left in β.

5.2 Machine learning algorithmWe use Liblinear L2-regularized L1-loss SVM forlearning (Hsieh et al., 2008), applying c = 0.1(cost), e = 0.1 (termination criterion), B = 0 (bias).

5.3 Feature extractionAs mentioned in Section 3.1, each token in our cor-pora consists of one or many morphemes annotatedwith different POS tags. This morphology makesit difficult to extract features for dependency pars-ing. In English, when two tokens, wi and wj , arecompared for a dependency relation, we extract fea-tures like POS tags of wi and wj (wi.pos, wj .pos),or a joined feature of POS tags between two tokens(wi.pos+wj .pos). Since each token is annotated witha single POS tag in English, it is trivial to extractthese features. In Korean, each token is annotatedwith a sequence of POS tags, depending on how mor-phemes are segmented. It is possible to join all POStags within a token and treat that as a single tag (e.g.,NNP+NNG+JX for the first token in Figure 5); how-ever, these tags usually cause very sparse vectorswhen used as features.

�� /NNP+��/NNG+�/JKO!

Hodong + Prince + JKO

��/NNG+�/XSV+�/EP+�/EF+./SF�� !

Love + XSV + EP + EF + .

�� /NNP+ �/NNG+�/JX!

Nakrang + Princess + JX

�� Nakrang_�� Hodong_��

Figure 5: Morphological analysis for a sentence, Princess

Nakrang loved Prince Hodong., in Korean.

An alternative is to extract the POS tag of only thehead morpheme for each token. This prevents the

sparsity issue, but we discover that no matter howwe choose the head morpheme, it prunes out toomany other morphemes helpful for parsing. Thus,as a compromise, we decide to select certain typesof morphemes and use only these as features. Ta-ble 6 shows the types of morphemes used to extractfeatures for our parsing models.

FS The first morphemeLS The last morpheme before JO|DS|EMJK Particles (J* in Table 1)DS Derivational suffixes (XS* in Table 1)EM Ending markers (E* in Table 1)PY The last punctuation, only if there is no other

morpheme followed by the punctuation

Table 6: Types of morphemes in each token used to ex-tract features for our parsing models.

Figure 6 shows morphemes extracted from the to-kens in Figure 5. For unigrams, these morphemescan be used either individually (e.g., the POS tag ofJK for the 1st token is JX) or jointly (e.g., a joinedfeature of POS tags between LS and JK for the 1sttoken is NNG+JX) to generate features. From ourexperiments, features extracted from the JK and EMmorphemes are found to be the most useful.

��

��/NNG � � �/XSV �/EF �/SF��/NNP��/NNG �/JKO � � ��/NNP �/NNG �/JX � � �

Figure 6: Morphemes extracted from the tokens in Fig-ure 5 with respect to the types in Table 6.

For n-grams where n > 1, it is not obvious whichcombinations of these morphemes across differenttokens yield useful features for dependency parsing.Trying out every possible combination is not practi-cal; thus, we restrict our search to only joined fea-tures of two morphemes between wi and wj , whereeach morpheme is taken from a different token. Itis possible to extend these features to another level,which we will explore in the future.

Table 7 shows a set of individual features ex-tracted from unigrams. wi is the last token in λ1

and wj is the first token in β as described in Sec-

��/NNP+�/NNG+�/JKOHodong + Prince + JKO

��/NNG+ /XSV+/EP+�/EF+./SFLove + XSV + EP + EF + .

��/NNP+��/NNG+�/JXNakrang + Princess + JX






��/NNG+�/XSV+�/EP+�/EF+./SF�� !













��







Dependency Parsing• Feature extraction

- Extract features using only important morphemes.

• Individual POS tag features of the1st and 3rd tokens.: NNP1, NNG1, JK1, NNG3, XSV3, EF3

• Joined features of POS tags between the 1st and 3rd tokens.: NNP1_NNG3, NNP1_XSV3, NNP1_EF3, JK1_NNG3, JK1_XSV3

- Tokens used: wi, wj, wi±1, wj±1

14

��/NNP+�/NNG+�/JKOHodong + Prince + JKO

��/NNG+ /XSV+/EP+�/EF+./SFLove + XSV + EP + EF + .

��/NNP+��/NNG+�/JXNakrang + Princess + JX






��/NNG+�/XSV+�/EP+�/EF+./SF�� !













��







Experiments• Corpora

- Dependency trees converted from the Sejong Treebank.

- Consists of 20 sources in 6 genres.

• Newspaper (NP), Magazine (MZ), Fiction (FI), Memoir (ME), Informative Book (IB), and Educational Cartoon (EC).

- Evaluation sets are very diverse compared to training sets.

• Ensures the robustness of our parsing models.

15

tion 5.1 (they also represent the i’th and j’th tokens

in a sentence, respectively). ‘m’ and ‘p’ indicate the

form and POS tag of the corresponding morpheme.

FS LS JK DS EM PY

wi m,p m,p m,p m,p m

wj m,p m,p m,p m,p m

wi−1 m m,p p m,p m

wi+1 m m,p m

wj−1 m p m,p m,p m

wj+1 m,p m m p p m

Table 7: Individual features from unigrams.

Table 8 shows a set of joined features extracted from

unigrams, wi and wj . For instance, a joined feature

of forms between FS and LS for the first token in

Figure 5 is Nakrang+Princess.

FS LS JK

JK m+m m+m

EM m+m m+m m+m

Table 8: Joined features from unigrams, wi and wj .

Finally, Table 9 shows a set of joined features ex-

tracted from bigrams, wi and wj . Each column and

row represents a morpheme in wi and wj , respec-

tively. ‘x’ represents a joined feature of POS tags

between wi and wj . ‘y’ represents a joined feature

between a form of wi’s morpheme and a POS tag of

wj’s morpheme. ‘z’ represents a joined feature be-

tween a POS tag of wi’s morpheme and a form of

wj’s morpheme. ‘*’ and ‘+’ indicate features used

only for fine-grained and coarse-grained morpholo-

gies, respectively.

FS LS JK DS EM

FS x y,z x∗,y

+,z z z

LS x x, z x∗,y

+x,z x

JK x∗,y,z

+x,y x

∗,y

+,z+

x,y+

x∗,y

+

DS x,z y x x x, z

EM x z y, z z x, z+

Table 9: Joined features from bigrams, wi and wj .

A few other features such as dependency labels of

[wj , the rightmost dependent of wi, and the left-

most dependent of wj] are also used. Note that we

are considering far fewer tokens than most other de-

pendency parsing approaches (only wi, wj , wi±1,

wj±1). We expect to achieve higher parsing ac-

curacy by considering more tokens; however, this

small span of features still gives promising results.

6 Experiments

6.1 CorporaThe Sejong Treebank contains 20 different corpora

covering various topics. For our experiments, we

group these corpora into 6 genres: Newspaper (NP),

Magazine (MZ), Fiction (FI), Memoir (ME), Infor-

mative Book (IB), and Educational Cartoon (EC).

NP contains newspapers from five different sources

talking about world, politics, opinion, etc. MZ con-

tains two magazines about movies and educations.

FI contains four fiction texts, and ME contains two

memoirs. IB contains six books about science, phi-

losophy, psychology, etc. EC contains one cartoon

discussing world history.

Table 10 shows how these corpora are divided into

training, development, and evaluation sets. For the

development and evaluation sets, we pick one news-

paper about art, one fiction text, and one informa-

tive book about trans-nationalism, and use each of

the first half for development and the second half for

evaluation. Note that these development and evalu-

ation sets are very diverse compared to the training

data. Testing on such evaluation sets ensures the ro-

bustness of our parsing model, which is very impor-

tant for our purpose because we are hoping to use

this model to parse various texts on the web.

NP MZ FI ME IB EC

T 8,060 6,713 15,646 5,053 7,983 1,548

D 2,048 - 2,174 - 1,307 -

E 2,048 - 2,175 - 1,308 -

Table 10: Number of sentences in training (T), develop-

ment (D), and evaluation (E) sets for each genre.

6.2 EvaluationsTo set an upper bound, we first build a parsing

model based on gold-standard morphology from the

Sejong Treebank, which is considered fine-grained

morphology. To compare the impact of fine-grained

vs. coarse-grained morphologies, we train two other

# of sentences in each set


Experiments• Morphological analysis

- Two automatic morphological analyzers are used.

• Intelligent Morphological Analyzer

- Developed by the Sejong project.

- Provides the same morphological analysis as their Treebank.

• Considered as fine-grained morphological analysis.

• Mach (Shim and Yang, 2002)

- Analyzes 1.3M words per second.

- Provides more coarse-grained morphological analysis.

16

Kwangseob Shim & Jaehyung Yang. 2002. A Supersonic Korean Morphological Analyzer. In Proceedings of COLING’02


Experiments• Evaluations

- Gold-standard vs. automatic morphological analysis.

• Relatively low performance from the automatic system.

- Fine vs. course-grained morphological analysis.

• Differences are not too significant.

- Robustness across different genres.

17

Gold, fine-grained Auto, fine-grained Auto, coarse-grainedLAS UAS LS LAS UAS LS LAS UAS LS

NP 82.58 84.32 94.05 79.61 82.35 91.49 79.00 81.68 91.50FI 84.78 87.04 93.70 81.54 85.04 90.95 80.11 83.96 90.24IB 84.21 85.50 95.82 80.45 82.14 92.73 81.43 83.38 93.89

Avg. 83.74 85.47 94.57 80.43 83.01 91.77 80.14 82.89 91.99

Table 11: Parsing accuracies achieved by three models (in %). LAS - labeled attachment score, UAS - unlabeledattachment score, LS - label accuracy score

parsing models, based on the output of IMA, [auto,fine-grained], and the output of Mach, [auto, coarse-grained]. All parsing accuracies achieved by thesethree models are provided in Table 11.

The [gold, fine-grained] model shows over threepoints improvement on the average LAS comparedto the [auto, fine-grained] model. The [auto, fine-grained] morphology gives an F1-score of 89.59%for morpheme segmentation, and a POS taggingaccuracy of 94.66% on correctly segmented mor-phemes; our parsing model is expected to performbetter as the automatic morphological analysis im-proves. On the other hand, the differences betweenthe [auto, fine-grained] and [auto, coarse-grained]models are small. More specifically, the differencebetween the average LAS achieved by these twomodels is statistically significant (McNemar, p =0.01); however, the difference in the average UASis not statistically significant, and the average LS isactually higher for the [auto, coarse-grained] model.

These results seem to confirm our hypothesis, “amore fine-grained morphology is not necessarily abetter morphology for dependency parsing”; how-ever, more careful studies need to be done to verifythis. Furthermore, it is only fair to mention that the[auto, fine-grained] model uses a smaller set of fea-tures than the [auto, coarse-grained] model (Table 9)because many lexical features can be replaced withPOS tag features without compromising accuracy forthe [auto, fine-grained] model, but not for the [auto,coarse-grained] model.

It is interesting to see that the numbers are usuallyhigh for LS, which shows that our models success-fully learn labeling information from morphemessuch as case particles or ending markers. All threemodels show robust results across different gen-res although the accuracies for NP are significantly

lower than the others. We are currently working onthe error analysis of why our models perform worseon NP and how to improve accuracy for this genrewhile keeping the same robustness across the others.

7 Conclusion and future work

There has been much work done on automatic mor-phological analysis but relatively less work done ondependency parsing in Korean. One major reasonis the lack of training data in dependency structure.Here, we present head-percolation rules and heuris-tics to convert constituent trees in the Sejong Tree-bank to dependency trees. We believe these head-rules and heuristics can be beneficial for those whowant to build statistical dependency parsing modelsof their own.

As a pioneer of using this dependency Treebank,we focus our effort on feature extraction. Becauseof rich morphology in Korean, it is not intuitive howto extract features from each token that will be use-ful for dependency parsing. We suggest a rule-basedway of selecting important morphemes and use onlythese as features to build dependency parsing mod-els. Even with a small span of features, we achievepromising results although there is still much roomfor improvement.

We will continuously develop our parsing modelby testing more features, treating morphemes as in-dividual tokens, adding deterministic rules, etc. Weare planning to use our parsing model to improveother NLP tasks in Korean such as machine transla-tion and sentiment analysis.

Acknowledgments

Special thanks are due to Professor Kong Joo Leeof Chungnam National University and KwangseobShim of Shungshin Women’s University. We grate-


Conclusion• Contributions

- Generating a Korean Dependency Treebank.

- Selecting important morphemes for dependency parsing.

- Evaluating the impact of fine vs. coarse-grained morphological analysis on dependency parsing.

- Evaluating the robustness across different genres.

• Future work

- Increase the feature span beyond bigrams.

- Find head morphemes of individual tokens.

- Insert empty categories.

18


Acknowledgements• Special thanks are due to

- Professor Kong Joo Lee of Chungnam National University.

- Professor Kwangseob Shim of Shungshin Women’s University.

• We gratefully acknowledge the support of the National Science Foundation Grants CISE-IIS-RI-0910992, Richer Representations for Machine Translation. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.

19


statistical dependency parsing in korean: from corpus generation to automatic parsing

Technology