elaborazione del linguaggio naturale

54
ELABORAZIONE DEL LINGUAGGIO NATURALE SINTASSI: GRAMMATICHE CONTEXT-FREE

Upload: aurora

Post on 23-Feb-2016

60 views

Category:

Documents


0 download

DESCRIPTION

ELABORAZIONE DEL LINGUAGGIO NATURALE. SINTASSI: GRAMMATICHE CONTEXT-FREE. Syntax. Syntax: from Greek syntaxis “setting out together, arrangement’’ Refers to the way words are arranged together, and the relationship between them. Distinction: Prescriptive grammar: how people ought to talk - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: ELABORAZIONE DEL LINGUAGGIO NATURALE

ELABORAZIONE DEL LINGUAGGIO NATURALE

SINTASSI: GRAMMATICHE CONTEXT-FREE

Page 2: ELABORAZIONE DEL LINGUAGGIO NATURALE

Syntax

• Syntax: from Greek syntaxis “setting out together, arrangement’’

• Refers to the way words are arranged together, and the relationship between them.

• Distinction:– Prescriptive grammar: how people ought to talk– Descriptive grammar: how they do talk

• Goal of syntax is to model the knowledge of that people unconsciously have about the grammar of their native language

Page 3: ELABORAZIONE DEL LINGUAGGIO NATURALE

key ideas of syntax

• Constituency• Subcategorization• Grammatical relations

Plus one part we won’t have time for:• Movement/long-distance dependency

Page 4: ELABORAZIONE DEL LINGUAGGIO NATURALE

Context-Free Grammars (CFG)

• Capture constituency and ordering– Ordering:

• What are the rules that govern the ordering of words and bigger units in the language?

– Constituency:How words group into units and how the various kinds of units behave

Page 5: ELABORAZIONE DEL LINGUAGGIO NATURALE

Constituency• E.g., Noun phrases (NPs)

• Three parties from Brooklyn• A high-class spot such as Mindy’s• The Broadway coppers• They• Harry the Horse• The reason he comes into the Hot Box

• How do we know these form a constituent?

Page 6: ELABORAZIONE DEL LINGUAGGIO NATURALE

Constituency (II)– They can all appear before a verb:

• Three parties from Brooklyn arrive…• A high-class spot such as Mindy’s attracts…• The Broadway coppers love…• They sit

– But individual words can’t always appear before verbs:• *from arrive…• *as attracts…• *the is• *spot is…

– Must be able to state generalizations like:• Noun phrases occur before verbs

Page 7: ELABORAZIONE DEL LINGUAGGIO NATURALE

Constituency (III)

• Preposing and postposing:– On September 17th, I’d like to fly from Atlanta to Denver– I’d like to fly on September 17th from Atlanta to Denver– I’d like to fly from Atlanta to Denver on September 17th.

• But not:– *On September, I’d like to fly 17th from Atlanta to Denver– *On I’d like to fly September 17th from Atlanta to Denver

Page 8: ELABORAZIONE DEL LINGUAGGIO NATURALE

Indicating constituents: brackets, trees

• [S [NP [PRO I]] [VP [V prefer] [NP [Det a] [Nom [N morning] [N flight] ] ] ] ] S

NP VP

NP

VerbPro

Nom

Det NounNoun

I prefer morninga flight

Page 9: ELABORAZIONE DEL LINGUAGGIO NATURALE

NLE 11

Beyond regular languages: Context-Free Grammars

S NP VPNP Det NominalNominal NounVP V

Det theDet aNoun flightV left

Page 10: ELABORAZIONE DEL LINGUAGGIO NATURALE

CFGs: set of rules

• S -> NP VP– This says that there are units called S, NP, and VP

in this language– That an S consists of an NP followed immediately

by a VP– Doesn’t say that that’s the only kind of S– Nor does it say that this is the only place that NPs

and VPs occur

Page 11: ELABORAZIONE DEL LINGUAGGIO NATURALE

Generativity

• As with FSAs you can view these rules as either analysis or synthesis machines– Generate strings in the language– Reject strings not in the language– Impose structures (trees) on strings in the language

• How can we define grammatical vs. ungrammatical sentences?

Page 12: ELABORAZIONE DEL LINGUAGGIO NATURALE

Derivations

• A derivation is a sequence of rules applied to a string that accounts for that string– Covers all the elements in the string– Covers only the elements in the string

Page 13: ELABORAZIONE DEL LINGUAGGIO NATURALE

Derivations as Trees

S

NP VP

NP

VerbPro

Nom

Det NounNoun

I prefer morninga flight

Page 14: ELABORAZIONE DEL LINGUAGGIO NATURALE

CFGs more formally• A context-free grammar has 4 parameters

(“is a 4-tuple”)1) A set of non-terminal symbols (“variables”) N

2) A set of terminal symbols (disjoint from N)

3) A set of productions P, each of the form• A -> • Where A is a non-terminal and is a string of symbols from the infinite set

of strings ( N)*

4) A designated start symbol S

Page 15: ELABORAZIONE DEL LINGUAGGIO NATURALE

Defining a CF language via derivation

• A string A derives a string B if – A can be rewritten as B via some series of rule applications

• More formally:– If A -> is a production of P– and are any strings in the set ( N)*– Then we say that

• A directly derives or A – Derivation is a generalization of direct derivation– Let 1, 2, … m be strings in ( N)*, m>= 1, s.t.

• 1 2, 2 3… m-1 m • We say that 1derives m or 1* m

– We then formally define language LG generated by grammar G• A set of strings composed of terminal symbols derived from S• LG = {w | w is in * and S * w}

Page 16: ELABORAZIONE DEL LINGUAGGIO NATURALE

NLE 21

What `context free’ means

Page 17: ELABORAZIONE DEL LINGUAGGIO NATURALE

NLE 22

Derivations and languages

• The language LG GENERATED by a CFG grammar G is the set of strings of TERMINAL symbols that can be derived from the start symbol S using the production rules in G– LG = {w | w is in * and S derives w}

• The strings in LG are called GRAMMATICAL

• The strings not in LG are called UNGRAMMATICAL

Page 18: ELABORAZIONE DEL LINGUAGGIO NATURALE

NLE 23

Grammar development

• One of the most basic skills in NLE is the ability to write a CFG for some fragment of a language (e.g., the dates)

• We’ll briefly cover some of the issues to be addressed when writing small CFG grammars

Page 19: ELABORAZIONE DEL LINGUAGGIO NATURALE

NLE 24

An example lexicon

Page 20: ELABORAZIONE DEL LINGUAGGIO NATURALE

NLE 25

An example grammar

Page 21: ELABORAZIONE DEL LINGUAGGIO NATURALE

NLE 26

A simple parse tree

Page 22: ELABORAZIONE DEL LINGUAGGIO NATURALE

NLE 27

Basic types of phrases

• Sentences• Noun Phrases• Verb phrases• Prepositional phrases

Page 23: ELABORAZIONE DEL LINGUAGGIO NATURALE

NLE 28

Basic types of sentences

Page 24: ELABORAZIONE DEL LINGUAGGIO NATURALE

NPs• NP -> Pronoun

– I came, you saw it, they conquered

• NP -> Proper-Noun– Los Angeles is west of Texas– John Hennessy is the president of Stanford

• NP -> Det Noun– The president

• NP -> Nominal

• Nominal -> Noun Noun– A morning flight to Denver

Page 25: ELABORAZIONE DEL LINGUAGGIO NATURALE

NLE 30

Noun phases: premodifiers• NP (Det) (Card) (Ord) (Quant) (AP) Nominal• Det: Determiners

– a flight– Optional: I’m looking for flights to Denver

• Card: Cardinal numbers (one stop)• Ord: Ordinal numbers (the first flight)• Quantifiers: most flights to Denver leave in the morning• AP (Adjectives): three very expensive seats

Page 26: ELABORAZIONE DEL LINGUAGGIO NATURALE

NLE 31

Noun phases: postmodifiers

• Nominal Noun• Nominal Nominal PP (PP) (PP)• Nominal Nominal GerundVP• Nominal Nominal RelClause

Page 27: ELABORAZIONE DEL LINGUAGGIO NATURALE

NLE 32

Types of postnominal modifiers

Page 28: ELABORAZIONE DEL LINGUAGGIO NATURALE

PPs

• PP -> Preposition NP– From LA– To the store– On Tuesday morning– With lunch

Page 29: ELABORAZIONE DEL LINGUAGGIO NATURALE

NLE 34

Recursion

• Nominal Nominal PP (PP) (PP)– Is an example of RECURSIVE rule

• Other examples:– NP NP PP– VP VP PP

• Recursion a powerful device, but could have bad consequences (see lectures on parsing)

Page 30: ELABORAZIONE DEL LINGUAGGIO NATURALE

NLE 35

Recursion and VP attachment

Page 31: ELABORAZIONE DEL LINGUAGGIO NATURALE

Recursion

[[Flights] [from Denver]][[[Flights] [from Denver]] [to Miami]][[[[Flights] [from Denver]] [to Miami]] [in February]][[[[[Flights] [from Denver]] [to Miami]] [in February]] [on a

Friday]]Etc.

NP -> NP PP

Page 32: ELABORAZIONE DEL LINGUAGGIO NATURALE

NLE 37

Coordination• NP NP and NP

– John and Mary left• VP VP and VP

– John talks softly and carries a big stick• S S and / but / S

– Kim is a lawyer but Sandy is reading medicine.• In fact, probably English has a

– XP XP and XP rule

Page 33: ELABORAZIONE DEL LINGUAGGIO NATURALE

Implications of recursion and context-freeness

• If you have a rule like– VP -> V NP

– It only cares that the thing after the verb is an NPIt doesn’t have to know about the internal affairs of that NP

Page 34: ELABORAZIONE DEL LINGUAGGIO NATURALE

The point• VP -> V NP• (I) hate

flights from Denverflights from Denver to Miamiflights from Denver to Miami in Februaryflights from Denver to Miami in February on a Fridayflights from Denver to Miami in February on a Friday under $300flights from Denver to Miami in February on a Friday under $300 with lunch

Page 35: ELABORAZIONE DEL LINGUAGGIO NATURALE

Problems

• Agreement• Subcategorization• Movement (for want of a better term)

Page 36: ELABORAZIONE DEL LINGUAGGIO NATURALE

Agreement• This dog• Those dogs

• This dog eats• Those dogs eat

• *This dogs• *Those dog

• *This dog eat• *Those dogs eats

Page 37: ELABORAZIONE DEL LINGUAGGIO NATURALE

Possible CFG Solution• S -> NP VP• NP -> Det Nominal• VP -> V NP• …

• SgS -> SgNP SgVP• PlS -> PlNp PlVP• SgNP -> SgDet SgNom• PlNP -> PlDet PlNom• PlVP -> PlV NP• SgVP ->SgV Np• …

Page 38: ELABORAZIONE DEL LINGUAGGIO NATURALE

CFG Solution for Agreement

• It works and stays within the power of CFGs• But it’s ugly• And it doesn’t scale all that well

Page 39: ELABORAZIONE DEL LINGUAGGIO NATURALE

Subcategorization• Sneeze: John sneezed• *John sneezed the book

• Say: You said [United has a flight]S

• Prefer: I prefer [to leave earlier]TO-VP

• *I prefer United has a flight

• Give: Give [me]NP[a cheaper fare]NP

• Help: Can you help [me]NP[with a flight]PP

• *Give with a flight

Page 40: ELABORAZIONE DEL LINGUAGGIO NATURALE

Subcategorization

• Subcat expresses the constraints that a predicate (verb for now) places on the number and syntactic types of arguments it wants to take (occur with).

Page 41: ELABORAZIONE DEL LINGUAGGIO NATURALE

So?

• So the various rules for VPs overgenerate

– They permit the presence of strings containing verbs and arguments that don’t go together

– For example:– VP -> V NP – therefore

Sneezed the book is a VP since “sneeze” is a verb and “the book” is a valid NP

Page 42: ELABORAZIONE DEL LINGUAGGIO NATURALE

Possible CFG Solution

• VP -> V• VP -> V NP• VP -> V NP PP• …

• VP -> IntransV• VP -> TransV NP• VP -> TransVwPP NP PP• …

Page 43: ELABORAZIONE DEL LINGUAGGIO NATURALE

Forward Pointer

• It turns out that verb subcategorization facts will provide a key element for semantic analysis (determining who did what to who in an event).

Page 44: ELABORAZIONE DEL LINGUAGGIO NATURALE

Movement

• Core example– My travel agent booked the flight

– [[My travel agent]NP [booked [the flight]NP]VP]S

• i.e. “book” is a straightforward transitive verb. It expects a single NP arg within the VP as an argument, and a single NP arg as the subject.

Page 45: ELABORAZIONE DEL LINGUAGGIO NATURALE

Movement

• What about?– Which flight do you want me to have the travel

agent book?• The direct object argument to “book” isn’t

appearing in the right place. It is in fact a long way from where its supposed to appear.

• And note that it’s separated from its verb by 2 other verbs.

Page 46: ELABORAZIONE DEL LINGUAGGIO NATURALE

CFGs: a summary• CFGs appear to be just about what we need to account for a lot of basic

syntactic structure in English.

• But there are problems– That can be dealt with adequately, although not elegantly, by staying within

the CFG framework.

• There are simpler, more elegant, solutions that take us out of the CFG framework (beyond its formal power).Syntactic theories: HPSG, LFG, CCG, Minimalism, etc.

Page 47: ELABORAZIONE DEL LINGUAGGIO NATURALE

Other syntactic stuff

• Grammatical relations– Subject

• I booked a flight to New York• The flight was booked by my agent

– Object• I booked a flight to New York

– Complement• I said that I wanted to leave

Page 48: ELABORAZIONE DEL LINGUAGGIO NATURALE

Dependency parsing• Word to word links instead of constituency• Based on the European rather than American traditions• But dates back to the Greeks• The original notions of Subject, Object and the progenitor of

subcategorization (called ‘valence’) came out of Dependency theory.• Dependency parsing is quite popular as a computational model

since relationships between words are quite useful

Page 49: ELABORAZIONE DEL LINGUAGGIO NATURALE

Dependency parsing

Bills on ports and immigration were submitted by Senator Brownback

NPS

NPNNP NNP

PPIN

VPVP

VBNVBD

NNCCNNSNPIN

NP PP

NNS

submitted

Bills were Brownback

Senator

nsubjpass auxpass agent

nnprep_onports

immigration conj_and

Parse tree:Nesting of multi-word constituents

Typed dep parse:Grammatical relations between individual words

Page 50: ELABORAZIONE DEL LINGUAGGIO NATURALE

Why are dependency parses useful?

• Example: multi-document summarization

Need to identify sentences from different documents that each say roughly the same thing:

phrase structure trees of paraphrasing sentences which differ in word order can be significantly differentbut dependency representations will be very similar

Page 51: ELABORAZIONE DEL LINGUAGGIO NATURALE

NLE 57

CFGs vs Regular languages

• For many applications, finite state languages (the languages defined by FA) are appropriate

• Limitation of FAs: cannot count– I.e., cannot check A n B n

• Example of construction showing that English is CF: long-distance dependencies– Which film did Kim say the director who we just

met _ recommended _?

Page 52: ELABORAZIONE DEL LINGUAGGIO NATURALE

NLE 58

The Chomsky Hierarchy• Finite-state languages (type 3)

– A bC | Cb (a single NT on the right)• Context-free languages (type 2)

– A BB• Context-sensitive languages (type 1)

– CAC BB• Recursively enumerable languages

– Every language that can be specified by a finite algorithm

Page 53: ELABORAZIONE DEL LINGUAGGIO NATURALE

NLE 59

Readings

• Jurafsky and Martin, chapter 9• The chapters on context-free languages in

– The Free Dictionary: http://encyclopedia.thefreedictionary.com/Context-free%20language

– Wikipedia: http://en.wikipedia.org/wiki/Context-free_grammar

Page 54: ELABORAZIONE DEL LINGUAGGIO NATURALE

ACKNOWLEDGMENTS

• Many of these slides borrowed from Jim Martin