an attempt at unsupervised learning of hierarchical dependency parsing

6

An Attempt at Unsupervised Learning of Hierarchical Dependency Parsing via the Dependency Model with Valence (DMV)

Upload: ernst

Post on 07-Jan-2016

28 views

Category:

Documents

3 download

Report

Download

Embed Size (px):

DESCRIPTION

An Attempt at Unsupervised Learning of Hierarchical Dependency Parsing. via the Dependency Model with Valence (DMV). Motivation. Dependency Parsing: Search Query Refinement Statistical Machine Translation Unsupervised Learning: Availability of Large Quantities of Data. DMV. - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: An Attempt at Unsupervised Learning of Hierarchical Dependency Parsing

An Attempt at Unsupervised Learning of Hierarchical Dependency

Parsing

via the Dependency Model with Valence (DMV)

Page 2: An Attempt at Unsupervised Learning of Hierarchical Dependency Parsing

Motivation

• Dependency Parsing:• Search Query Refinement• Statistical Machine Translation

• Unsupervised Learning:• Availability of Large Quantities of Data

Page 3: An Attempt at Unsupervised Learning of Hierarchical Dependency Parsing

DMV

• Pick a Direction (left or right)• Generate the first child, or stop;• Generate more children, until stop.• Repeat in the other direction.• Recurse…

• Porder• Pstop• Pattach

Page 4: An Attempt at Unsupervised Learning of Hierarchical Dependency Parsing

EM

• Inside-Outside Algorithm:• Inside: Pi(i,X,j) = P(X derives i…j)• Outside: Po(i,X,j) = P(S derives 0…iXj…l)

• Re-Estimation:• Frequency of sub-tree (i,X,j)=Pi(i,X,j)*Po(i,X,j)

Page 5: An Attempt at Unsupervised Learning of Hierarchical Dependency Parsing

Evaluation

• Head-percolation of Penn Treebank parses;• % edges correct (directed or undirected) in the

best (P)CFG parse…

• Zero Knowledge: 14.4 (29.9)• Adjacent Word Heuristic: 33.6 • Klein & Manning: 43.2 (63.7)• Oracle: 75.5 (77.5)• - Pattach: 60.0 (63.3) - Pstop: 53.9 (57.7)• - PstopA: 50.0 (54.8) - PstopN: 12.5 (30.8)

Page 6: An Attempt at Unsupervised Learning of Hierarchical Dependency Parsing

EM

• Didn’t work out… always made things worse, even when initialized with very good solutions.

• If started using Zero Knowledge, then after 1 iteration already gets 18.4 (38.4), then worsens.

• If started using an Ad-Hoc Harmonic for Pattach, then 21.5 (47.1) after 1 iteration, then worse, and similarly even for the Oracle solution…

• Summary:• - DMV – useful, simple, extensible model;• - EM – more thorough debugging needed.

Dependency Grammar and Dependency Parsing - …stp.lingfil.uu.se/~nivre/docs/05133.pdfDependency Grammar and Dependency Parsing Joakim Nivre 1 Introduction ... Thus, in the sentence

Statistical Dependency Parsing - Uppsala Universitynivre/docs/StatState.pdf · I Methods for statistical dependency parsing I Chart parsing techniques ... Eisner 2000]: I Split head

Improving dependency parsing using word clusters · dependency parsing. Dependency-based approaches to syntactic parsing are applied to a wide range of languages and new methods are

Dependency Parsing - Cornell University

Statistical Dependency Parsing Korean - Verbs Indexverbs.colorado.edu/~mpalmer/Ling7800/LeePaper.pdfStatistical Dependency Parsing Korean LINGUISTICS ... High LS implies that models

Bare-Bones Dependency Parsing - Uppsala Universitynivre/docs/BareBones.pdf · I Parsing methods for bare-bones dependency parsing I Chart parsing techniques ... [Kuhlmann and Satta

Recent Advances in Dependency Parsingnivre/docs/eacl4.pdf · unsupervised dependency parsing with richer contexts and smoothing. In Proceedings of Human Language Technologies: The

Concavity and Initialization for Unsupervised Dependency ...kgimpel/talks/gimpel... · Unsupervised Dependency Parsing Kevin Gimpel Noah A. Smith 1. lti Unsupervised learning in NLP

Unsupervised Dependency Parsing

Unsupervised Dependency Parsing David Mareček Institute of Formal and Applied Linguistics Charles University in Prague Doctoral thesis defense September

Dependency Parsing

Dependency Parsing - Oregon State Universityclasses.engr.oregonstate.edu/.../notes/dependency-parsing-lecture.pdf · Outline Introduction Dependency Parsing Formal definition Parsing

NLP Programming Tutorial 12 - Dependency Parsing · 3 NLP Programming Tutorial 12 – Dependency Parsing Two Types of Parsing Dependency: focuses on relations between words …

A Framework for Unsupervised Dependency Parsing using a Soft … Framework for... · 2013. 12. 8. · Keywords: Unsupervised dependency parsing, bilexical grammars, soft-EM al-gorithm

Dependency Parsing by Belief Propagation

Punctuation: Making a Point - Stanford NLP Groupnlp.stanford.edu/pubs/punctuation-slides.pdf · Punctuation: Making a Point in Unsupervised Dependency Parsing Valentin I. Spitkovsky

Dependency Parsing - UMD

Dependency Parsing - cs.princeton.edu

Feature Embedding for Dependency Parsing

Partial Dependency Parsing for Irish

Bare-Bones Dependency Parsing - Uppsala Universitystp.lingfil.uu.se/~nivre/docs/BareBones.pdf · I Parsing methods for bare-bones dependency parsing I Chart parsing ... Eisner 2000]:

Syntax and parsing an overview - folk.uio.nofolk.uio.no/liljao/inf5830/inf5830_syntax_handout.pdf · Syntax and parsing (today) Dependency grammar (10/10) Dependency parsing (17/10)

Unsupervised Partial Parsing: Thesis defense

Lecture 19: Dependency Grammars and Dependency Parsing

DEPENDENCY PARSING FRAMENET, SEMANTIC …nlp.cs.rpi.edu/course/fall16/lecture4.pdf · DEPENDENCY PARSING，FRAMENET, SEMANTIC ROLE LABELING, SEMANTIC PARSING HengJi [email protected] September

Unsupervised Semantic Parsing

Methods in Unsupervised Dependency Parsingrasooli/papers/candidacy2016.pdf · Fully Unsupervised Parsing Models Syntactic Transfer Models Conclusion Methods in Unsupervised Dependency

Dependency Parsing (3) - University Of Maryland · Dependency Parsing: what you should know •Transition-based dependency parsing •Shift-reduce parsing •Transition systems: arc

Methods in Unsupervised Dependency Parsing

Punctuation: Making a Point - Stanford NLP GroupPunctuation: Making a Point in Unsupervised Dependency Parsing Valentin I. Spitkovsky withDaniel Jurafsky (StanfordUniversity) andHiyan

Exploiting Reducibility in Unsupervised Dependency Parsing David Mareček and Zdeněk Žabokrtský Institute of Formal and Applied Linguistics Charles University

Exploiting Reducibility in Unsupervised Dependency Parsing

CHAPTER 14 Dependency Parsing

Unsupervised Dependency Parsing - cuni.czmarecek/papers/2012_marecek... · 2012. 10. 2. · Abstract Unsupervised dependency parsing is an alternative approach to identifying rela-tions

Dependency Parsing 2 - University Of Maryland · Data-driven dependency parsing Goal:learn a good predictor of dependency graphs Input: sentence Output: dependency graph/tree G =