Towards Parsing Unrestricted Text into PropBank Predicate-Argument

Structures

ACL4 ProjectNCLT Seminar Presentation, 7th June 2006

Conor Cafferkey

Project Overview

Open research problem:● Integrating syntactic parsing and semantic role labeling

(SRL)

Approach● Retraining a history-based generative lexicalized parser

(Bikel, 2002)● Semantically-enriched training corpus (Penn Treebank +

PropBank-derived semantic role annotations)

Treebank Syntactic Bracketing Style

Treebank Syntactic Bracketing Style

Semantic Roles

● Relationship that a syntactic constituent has with a predicate

● Predicate-argument relations● PropBank (Palmer et al., 2005)

PropBank Predicate-Argument Relations

Frameset: hate.01

ARG0: experiencer

ARG1: target

PropBank Argument Types

● ARG0 - ARG5: arguments associated with a verb predicate, defined in the PropBank Frames scheme.

● ARGM-XXX: adjunct-like arguments of various sorts, where XXX is the type of the adjunct. Types include locative (LOC), temporal (TMP) , manner (MNR), etc.

● ARGA: causative agents.● rel: the verb of the proposition.

Current Approaches

● Semantic role labeling (SRL) task:– Identify, given a verb:

● which nodes of the syntactic tree are arguments of that verb, and

● what semantic role each such argument plays with regard to the verb.

Current Approaches

● “Pipelined” approach● Parsing → Pruning → ML-techniques → post-

processing

● CoNLL-2005 (Carreras and Márquez, 2005)– SVM, Random Fields, Random Forests, …

– Various lexical parameters

An Integrated Approach to Semantic Parsing

● Integrate syntactic and semantic parsing● Retrain parser using semantically-enriched

corpus (Treebank + PropBank-derived semantic roles)

● Parser itself performs semantic role labeling (SRL)

Project Components

● “Off-the-shelf”:– Parser (Bikel, 2002) emulating Collins’ (1999) model 2– Penn Treebank Release 2 (Marcus et al., 1993)– PropBank 1.0 (Palmer, 2005)

● Written for project (mainly in Python):– Scripts to annotate Treebank with PropBank data– Script to generate new head-finding rules for Bikel’s parser– SRL evaluation scripts– Utility scripts (pre-processing, etc.)

Appending Semantic Roles to Treebank Syntactic Category Labels

wsj/15/wsj_1568.mrg 16 2 gold hate.01 vn--a 0:1-ARG0 2:0-rel 3:1-ARG1

Syntactic Bracketing Evaluation

• Parseval measures (Black, et al., 1992)

filetestintsconstituenofnumber

filetestintsconstituencorrectofnumberprecision

filegoldintsconstituenofnumber

filetestintsconstituencorrectofnumberrecall

Syntactic Bracketing Evaluation

recallprecision

recallprecisionscoref

2

● Harmonic mean of precision and recall:

Baseline Syntactic Bracketing Performance

Parsing Section 00, trained with sections 02-21 of Penn Treebank (1918 sentences)

Parse Time: 114:41

Semantically-Augmented Treebanks

● N: augment node labels with ARGNs only● N-C: augment node label with conflated ARGNs

only● M: augment node labels with ARGMs only● M-C: augment node labels with conflated

ARGMs only● NMR: augment node labels with ARGNs,

ARGMs and rels

Syntactic Bracketing Evaluation

Parsing Section 00, trained with sections 02-21 of Penn Treebank (1918 sentences)

Semantic Evaluation

filetestinlabelsroleofnumber

filetestinlabelsrolecorrectofnumberprecision

filegoldinlabelsroleofnumber

filetestinlabelsrolecorrectofnumberrecall

Semantic Evaluation

● Evaluating by terminal number and height● Evaluating by terminal span

● How strictly to evaluate?

Semantic Role Labeling Evaluation

Parsing Section 00, trained with sections 02-21 of Penn Treebank (1918 sentences)

Semantic Role Labeling Evaluation

Parsing Section 00, trained with sections 02-21 of Penn Treebank (1918 sentences)

Syntactic Nodes that Play Multiple Semantic Roles

Adding More Information

● Co-index the semantic role labels with governing predicate (verb)

● i.e. include the appropriate roleset name in each semantic label augmentation

Co-indexing the Semantic Augmentations

Adding More Information

● Data sparseness● Time efficiency● Need to make some sort of generalizations

● “Syntacto-semantic” verb classes● VerbNet (Kipper et al., 2002)

Co-indexing with VerbNet classes

Future Ideas

● Integrate the (un co-indexed) output from the re-trained parser into a pipelined SRL system

● Syntactic parsing informed by semantic roles?– Recoding the parser to take better advantage of the

semantic roles

– Reranking n-best parser outputs based on semantic roles

Summary

● Retrained a history-based generative lexicalized parser with semantically-enriched corpus– Corpus annotation

– Generating head-finding rules

● Evaluated parser’s performance– Syntactic parsing (evalb)

– Semantic parsing (SRL)

References● Bikel, Daniel M. 2002. Design of a Multi-lingual, Parallel-processing Statistical

Parsing Engine. In Proceedings of HLT2002, San Diego, California.

● Black, Ezra, Frederick Jelinek, John D. Lafferty, David M. Magerman, Robert L. Mercer and Salim Roukos. 1992. Towards History-based Grammars: Using Richer Models for Probabilistic Parsing. In Proceedings DARPA Speech and Natural Language Workshop, Harriman, New York, pages 134-139. Morgan Kaufmann.

● Carreras, Xavier and Lluís Màrquez. 2005. Introduction to the CoNLL-2005 Shared Task: Semantic Role Labeling. In Proceedings of CoNLL-2005, pages152-164.

● Collins, Michael John. 1999. Head-driven Statistical Models for Natural Language Parsing. Ph.D. thesis, University of Pennsylvania, Philadelphia.

References● Kipper, Karin, Hoa Trang Dang and Martha Palmer. 2000. Class-

Based Construction of a Verb Lexicon. In Proceedings of Seventeenth National Conference on Artificial Intelligence, Austin, Texas.

● Marcus, Mitchell P., Beatrice Santroini and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of English: the Penn Treebank. Computational Linguistics, 19(2):313-330.

● Palmer, Martha, Daniel Gildea and Paul Kingsbury. 2005. The Proposition Bank: An Annotated Corpus of Semantic Roles. Computational Linguistics, 31(1):71-106.

● Yi, Szu-ting and Martha Palmer. 2005. The integration of syntactic parsing and semantic role labeling. In Proceedings of CoNLL-2005, pages 237-240.

http://student.dcu.ie/~cafferc2/