csci 5832 natural language processing
DESCRIPTION
CSCI 5832 Natural Language Processing. Jim Martin Lecture 18. Today 3/20. Semantics. Transition. First we did words (morphology) Then simple sequences of words Then we looked at true syntax Now we’re moving on to meaning. Where some would say we should have started to begin with. Example. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/1.jpg)
04/19/23 1
CSCI 5832Natural Language Processing
Jim Martin
Lecture 18
![Page 2: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/2.jpg)
04/19/23 2
Today 3/20
• Semantics
![Page 3: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/3.jpg)
04/19/23 3
Transition
• First we did words (morphology)
• Then simple sequences of words
• Then we looked at true syntax
• Now we’re moving on to meaning. Where some would say we should have started to begin with.
![Page 4: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/4.jpg)
04/19/23 4
Example
Even if this is the right tree, what does that tell us about the meaning?
![Page 5: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/5.jpg)
04/19/23 5
Meaning
• Language is useful and amazing because it allows us to encode/decode… Descriptions of the world What we’re thinking What we think about what other people think
• Don’t be fooled by how natural and easy it is… In particular, you never really… Utter word strings that match the world Say what you’re thinking Say what you think about what other people think
![Page 6: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/6.jpg)
04/19/23 6
Meaning
• You’re simply uttering linear sequences of words such that when other people read/hear and understand them they come to know what you think of the world.
![Page 7: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/7.jpg)
04/19/23 7
Meaning Representations
• We’re going to take the same basic approach to meaning that we took to syntax and morphology
• We’re going to create representations of linguistic inputs that capture the meanings of those inputs.
• But unlike parse trees and the like these representations aren’t primarily descriptions of the structure of the inputs…
![Page 8: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/8.jpg)
04/19/23 8
Meaning Representations
• In most cases, they’re simultaneously descriptions of the meanings of utterances and of some potential state of affairs in some world.
![Page 9: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/9.jpg)
04/19/23 9
Meaning Representations
• What could this mean… representations of linguistic inputs that
capture the meanings of those inputs
• For us it means Representations that permit or facilitate
semantic processing
![Page 10: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/10.jpg)
04/19/23 10
Semantic Processing
• Ok, so what does that mean?
• Representations that Permit us to reason about their truth
(relationship to some world) Permit us to answer questions based on their
content Permit us to perform inference (answer
questions and determine the truth of things we don’t actually know)
![Page 11: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/11.jpg)
04/19/23 11
Semantic Processing
• Touchstone application is often question answering Can a machine answer questions involving
the meaning of some text or discourse? What kind of representations do we need to
mechanize that process?
![Page 12: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/12.jpg)
04/19/23 12
Semantic Processing
• We’re going to discuss 2 ways to attack this problem (just as we did with parsing) There’s the principled, theoretically motivated
approach… Computational/Compositional Semantics
• Chapters 17 and 18
And there are limited, practical approaches that have some hope of being useful Information extraction
• Chapter 22
![Page 13: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/13.jpg)
04/19/23 13
Semantic Analysis
• Compositional Analysis Create a FOL representation that accounts for
all the entities, roles and relations present in a sentence. Similar to our approach to full parsing
• Information Extraction Do a superficial analysis that pulls out only
the entities, relations and roles that are of interest to the consuming application. Similar to chunking
![Page 14: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/14.jpg)
04/19/23 14
Information Extraction (preview)
Investigators worked leads Monday in Riverside County where the car was reported stolen and reviewed security tape from Highway 241 where it was abandoned, said city of Anaheim spokesman John Nicoletti.
![Page 15: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/15.jpg)
04/19/23 15
Information ExtractionNamed Entities
• Investigators worked leads Monday in Riverside County where the car was reported stolen and reviewed security tape from Highway 241 where it was abandoned, said city of Anaheim spokesman John Nicoletti.
Investigators worked leads [Monday] in [Riverside County] where the car was reported stolen and reviewed security tape from [Highway 241] where it was abandoned, said city of [Anaheim] spokesman [John Nicoletti].
![Page 16: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/16.jpg)
04/19/23 16
Information ExtractionEvents
• Investigators worked leads Monday in Riverside County where the car was reported stolen and reviewed security tape from Highway 241 where it was abandoned, said city of Anaheim spokesman John Nicoletti.
Investigators worked leads Monday in Riverside County where the car was reported stolen and reviewed security tape from Highway 241 where it was abandoned, said city of Anaheim spokesman John Nicoletti.
![Page 17: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/17.jpg)
04/19/23 17
Information ExtractionEvents
• Investigators worked leads Monday in Riverside County where the car was reported stolen and reviewed security tape from Highway 241 where it was abandoned, said city of Anaheim spokesman John Nicoletti.
Investigators worked leads Monday in Riverside County where the car was reported stolen and reviewed security tape from Highway 241 where it was abandoned, said city of Anaheim spokesman John Nicoletti.
![Page 18: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/18.jpg)
04/19/23 18
Break
• Today’s material is a mixture of Chapters 17 and 18.
• The next quiz will be pushed back to 4/15 and will cover 17,18,19,20
![Page 19: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/19.jpg)
04/19/23 19
Break
• I’m not normally this scatterbrained.
• We’re going to finish delivery of the book by next week (assuming we can make acrobat do the right things).
• I’ll be less disorganized when you get back from break...
![Page 20: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/20.jpg)
04/19/23 20
Representational Schemes
• We’re going to make use of First Order Logic (FOL) as our representational framework Not because we think it’s perfect Many of the alternatives turn out to be either
too limiting or They turn out to be notational variants
![Page 21: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/21.jpg)
04/19/23 21
FOL
• Allows for… The analysis of truth conditions
Allows us to answer yes/no questions
Supports the use of variables Allows us to answer questions through the use of
variable binding
Supports inference Allows us to answer questions that go beyond what
we know explicitly
![Page 22: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/22.jpg)
04/19/23 22
FOL
• This choice isn’t completely arbitrary or driven by the needs of practical applications
• FOL reflects the semantics of natural languages because it was designed that way by human beings
• In particular…
![Page 23: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/23.jpg)
04/19/23 23
Meaning Structure of Language
• The semantics of human languages… Display a basic predicate-argument structure Make use of variables Make use of quantifiers Use a partially compositional semantics
![Page 24: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/24.jpg)
04/19/23 24
Predicate-Argument Structure
• Events, actions and relationships can be captured with representations that consist of predicates and arguments to those predicates.
• Languages display a division of labor where some words and constituents function as predicates and some as arguments.
![Page 25: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/25.jpg)
04/19/23 25
Predicate-Argument Structure
• Predicates Primarily Verbs, VPs, PPs, Sentences Sometimes Nouns and NPs
• Arguments Primarily Nouns, Nominals, NPs, PPs But also everything else; as we’ll see it
depends on the context
![Page 26: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/26.jpg)
04/19/23 26
Example
• Mary gave a list to John.• Giving(Mary, John, List)• More precisely
Gave conveys a three-argument predicate The first arg is the subject The second is the recipient, which is
conveyed by the NP in the PP The third argument is the thing given,
conveyed by the direct object
![Page 27: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/27.jpg)
04/19/23 27
Not exactly
• The statement The first arg is the subject
can’t be right.
• Subjects can’t be givers.
• We mean that the meaning underlying the subject phrase plays the role of the giver.
![Page 28: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/28.jpg)
04/19/23 28
Better
• Turns out this representation isn’t quite as useful as it could be. Giving(Mary, John, List)
• Better would be
€
∃e,y Giving(e)^Giver(e,Mary)^Given(e,y)
^Givee(e,John)^List(y)
![Page 29: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/29.jpg)
04/19/23 29
Predicates
• The notion of a predicate just got more complicated…• In this example, think of the verb/VP providing a
template like the following
• The semantics of the NPs and the PPs in the sentence plug into the slots provided in the template
€
∃e,x,y,zGiving(e)^Giver(e,x)^Given(e,y)^Givee(e,z)
![Page 30: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/30.jpg)
04/19/23 30
Semantic Analysis
• Semantic analysis is the process of taking in some linguistic input and assigning a meaning representation to it. There a lot of different ways to do this that
make more or less (or no) use of syntax We’re going to start with the idea that syntax
does matter The compositional rule-to-rule approach
![Page 31: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/31.jpg)
04/19/23 31
Compositional Analysis
• Principle of Compositionality The meaning of a whole is derived from the
meanings of the parts
• What parts? The constituents of the syntactic parse of the
input
• What could it mean for a part to have a meaning?
![Page 32: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/32.jpg)
04/19/23 32
Example
• AyCaramba serves meat
),()^,()^( MeateServedAyCarambaeServereServinge∃
![Page 33: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/33.jpg)
04/19/23 33
Compositional Analysis
![Page 34: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/34.jpg)
04/19/23 34
Augmented Rules
• We’ll accomplish this by attaching semantic formation rules to our syntactic CFG rules
• Abstractly
• This should be read as the semantics we attach to A can be computed from some function applied to the semantics of A’s parts.
)}.,....({... 11 semsemfA nn αααα→
![Page 35: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/35.jpg)
04/19/23 35
Example
• Easy parts… NP -> PropNoun NP -> MassNoun PropNoun -> AyCaramba MassMoun -> meat
• Attachments
{PropNoun.sem}
{MassNoun.sem}
{AyCaramba}
{MEAT}
![Page 36: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/36.jpg)
04/19/23 36
Example
• S -> NP VP• VP -> Verb NP• Verb -> serves
• {VP.sem(NP.sem)}• {Verb.sem(NP.sem)• ???
),()^,()^( xeServedyeServereServingeyx ∃λλ
![Page 37: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/37.jpg)
04/19/23 37
Lambda Forms
• A simple addition to FOL Take a FOPC sentence
with variables in it that are to be bound.
Allow those variables to be bound by treating the lambda form as a function with formal arguments
)(xxPλ
)(
))((
SallyP
SallyxxPλ
![Page 38: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/38.jpg)
04/19/23 38
Example
![Page 39: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/39.jpg)
04/19/23 39
Example
![Page 40: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/40.jpg)
04/19/23 40
Example
![Page 41: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/41.jpg)
04/19/23 41
Example
![Page 42: CSCI 5832 Natural Language Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56812d1b550346895d9211c0/html5/thumbnails/42.jpg)
04/19/23 42
Next Time
Finish reading 17 and 18 for next time.Have a good break.