1 natural language processing chapter 14. 2 transition first we did words (morphology) then we...

1

Natural Language Processing

Chapter 14

2

Transition

• First we did words (morphology)• Then we looked at syntax• Now we’re moving on to meaning.

3

Semantics

• What things mean• Approach: meaning representations

which bridge the gap from linguistic forms to knowledge of the world

• Serve the practical purposes of a program doing semantic processing

4

Semantic Processing

• Representations that allow a system to– Answer questions– Determine truth– Perform inference– …

5

Types of Meaning Representations

• First-order predicate calculus• Semantic networks• Conceptual dependency• Frame-based representations

• See lecture for examples…

6

Verifiability

• Does Spice Island serve vegetarian food?

• Serves(spiceisland,vegetarianfood)• Verifiability: the system’s ability to

compare representations to facts in memory

7

Current Focus in Class

• Conventional meanings of words• Ignore context• Literal meaning

8

Ambiguity

• I want to eat someplace that’s close to Pitt

• Mary kissed her husband and Joan did too

• I baked the cake on the table• Old men and women go to the park• Every student ate a sandwich

9

Canonical Form

• Does Spice Island have vegetarian dishes?

• Do they have vegetarian food at Spice Island?

• Are vegetarian dishes served at Spice Island?

• Does Spice Island serve vegetarian fare?• Canonical form: inputs that mean the

same thing should have the same meaning representations

10

Canonical Form

• Simplifies reasoning• Makes representations more compact

(fewer different representations)• BUT: makes semantic analysis harder

– Need to figure out that “have” and “serve” mean the same thing in the previous examples; same for the various phrases for vegetarian food

– BUT: can perform word sense disambiguation; use a single representation for all senses in a synset

11

Representational Schemes

• We’re going to make use of First Order Predicate Calculus (FOPC) as our representational framework– Not because we think it’s perfect– All the alternatives turn out to be

either too limiting or too complicated, or

– They turn out to be notational variants

12

Knowledge Based Agents

• Central component: knowledge base, or KB.

• A set of sentences in a knowledge representation language

• Generic Functions– TELL (add a fact to the knowledge base)– ASK (get next action based on info in KB)

• Both often involve inference, which is?• Deriving new sentences from old

13

Fundamental Concepts of logical representation and reasoning

• Information is represented in sentences, which must have correct syntax( 1 + 2 ) * 7 = 21 vs. 2 ) + 7 = * ( 1 21

• The semantics of a sentence defines its truth with respect to each possible world

• W is a model of S means that sentence S is true in world W

• What do the following mean?– X |= Y – X entails Y– Y logically follows from X

14

Entailment

• A |= B• In all worlds in which A is true, B

must be true as well• All models of A are models of B• Whenever A is true, B must be true

as well• A entails B• B logically follows from A

15

Inference

KB |-i A

Inference algorithm i can derive A from KB

i derives A from KBi can derive A from KBA can be inferred from KB by i

16

Propositional Logic Syntax

• Sentence -> AtomicSent | complexSentAtomicSent -> true|false| P, Q, R …ComplexSent ->sentence | ( sentence sentence ) |( sentence sentence ) |( sentence sentence ) |( sentence sentence ) |( sentence )

[no predicate or function symbols]

17

Propositional Logic Sentences

• If there is a pit at [1,1], there is a breeze at [1,0]P11 B10

• There is a breeze at [2,2], if and only if there is a pit in the neighborhoodB22 ( P21 P23 P12 P32 )

• There is no breeze at [2,2]B22

18

Semantics of Prop Logic

• In model-theoretic semantics, an interpretation assigns elements of the world to sentences, and defines the truth values of sentences

• Propositional logic: easy! Assign T or F to each proposition symbol; then assign truth values to complex sentences in the obvious way

19

Propositional Logic

• A ^ B is true if both A and B are true

• A v B is true if one or both A and B are true

• P Q equiv ~P v Q. Thus, P Q is false if P is true and Q is false. Otherwise, P Q is true.

• ~A is true if A is false

20

Proofs

• A derivation• A sequence of applications of (usually

sound) rules of inference• Reasoning by Search• Example KB = AB, BC, DE, EF, D• Forward chaining: Add A, infer B, infer C• Backward chaining:F? E? D? Yes…• Sound but not complete inference

procedures

21

Resolution

• Resolution allows a sound and complete inference mechanism (search-based) using only one rule of inference

• Resolution rule:– Given: P1 P2 P3 … Pn, and P1 Q1 … Qm

– Conclude: P2 P3 … Pn Q1 … Qm

Complementary literals P1 and P1 “cancel out”

For your information only; resolution won’t be on exam

22

Resolution

• Any complete search algorithm, applying only the resolution rule, can derive any conclusion entailed by any KB in propositional logic.

• Refutation completeness: Given A, we cannot use resolution to generate the consequence A v B. But we can answer the question, is A v B true. I.e., resolution can be used to confirm or refute a sentence

Again, for your information only; will not be on the exam

23

Unsound (but useful) Inference

• Where there is smoke, there is fire• Example KB: Fire Smoke, Smoke• Abduction: conclude Fire• Unsound:

– Example KB1: Fire Smoke, DryIce Smoke, Smoke

– DryIce rather than Fire could be true

24

Propositional Logic FOPC

B11 (P12 v P21)B23 (P32 v P 23 v P34 v P 43) …“Internal squares adjacent to pits are

breezy”:All X Y (B(X,Y) ^ (X > 1) ^ (Y > 1) ^

(Y < 4) ^ (X < 4)) (P(X-1,Y) v P(X,Y-1) v P(X+1,Y) v

(X,Y+1))

25

FOPC Worlds

• Rather than just T,F, now worlds contain:

• Objects: the gold, the wumpus, people, ideas, … “the domain”

• Predicates: holding, breezy, red, sisters

• Functions: fatherOf, colorOf, plusOntological commitment

26

FOPC Syntax

• Add variables and quantifiers to propositional logic

27

Sentence AtomicSentence | (Sentence Connective Sentence) | Quantifier Variable, … Sentence | ~SentenceAtomicSentence Predicate(Term,…) | Term = TermTerm Function(Term,…) | Constant | VariableConnective | ^ | v | Quantifier all, existsConstant john, 1, …Variable A, B, C, XPredicate breezy, sunny, redFunction fatherOf, plus

Knowledge engineering involves deciding what types of things Should be constants, predicates, and functions for your problem

28

Examples

• Everyone likes chocolate X (person(X) likes(X, chocolate))

• Someone likes chocolate X (person(X) ^ likes(X, chocolate))

• Everyone likes chocolate unless they are allergic to it X ((person(X) ^ allergic (X, chocolate)) likes(X, chocolate))

29

Quantifiers

• All X p(X) means that p holds for all elements in the domain

• Exists X p(X) means that p holds for at least one element of the domain

• In well-formed FOPC, all variables are bound by (in the scope of) a quantifier

30

Nesting of Variables

1. Everyone likes some kind of food2. There is a kind of food that

everyone likes3. Someone likes all kinds of food4. Every food has someone who likes

it

31

Answers

Everyone likes some kind of foodAll P (person(P) Exists F (food(F) and

likes(P,F)))There is a kind of food that everyone likes Exists F (food(F) and (All P (person(P)

likes(P,F))))Someone likes all kinds of food Exists P (person(P) and (All F (food(F)

likes(P,F)))) Every food has someone who likes it All F (food (F) Exists P (person(P) and

likes(P,F)))

32

Semantics of FOPC: Interpretation

• Specifies which objects, functions, and predicates are referred to by which constant symbols, function symbols, and predicate symbols.

33

Example

• 3 people: John, Sally, Bill• John is tall• Sally and Bill are short• John is Bill’s father• Sally is Bill’s sister• Interpretation 1 (others are possible):

– “John”, “Sally”, and “Bill” as you think– “person” {John, Sally,Bill}– “short” {Sally,Bill}– “tall” {John}– “sister” {<Sally,Bill>} A 2-ary predicate– “father” {<Bill,John>} A 1-ary function

34

Exampletall(father(bill)) ^ ~sister(sally,bill)

• Assign meanings to terms:– “bill” Bill; “sally” Sally; “father(bill)”

John

• Assign truth values to atomic sentences– Tall(father(bill)) is T because John is in the set

assigned to “tall”– ~sister(sally,bill) is F because <Sally,Bill> is in

the set assigned to “sister”– So, sentence is false, because T ^ F is F

35

Determining Truth Values

• Exist X tall(X) : true, because the set assigned to “tall” isn’t {}

• All X short(X) : false, because there are objects that are not in the set assigned to “short”

36

Representational Schemes

• What are the objects, predicates, and functions?

37

Choices:Functions vs Predicates

• Rep-Scheme 1: tall(fatherOf(bob)).

• Rep-Scheme 2:Exists X (fatherOf(bob,X) ^ tall(X) ^ (All Y (fatherOf(bob,Y) X = Y)))

• “fatherOf” in both cases is assigned a set of 2-tuples: {<b,bf>,<t,tf>,…}

• But {<b,bf>,<t,tf>,<b,bff>,…} is possible if it is a predicate

38

Choices: Predicates versus Constants

• Rep-Scheme 1: D = {a,b,c,d,e}. red: {a,b,c}. pink: {d,e}. Some true sentences:

red(a). red(b). pink(d). ~(All X red(X)). All X (red(X) v pink(X)).

But what if we want to say that pink is pretty?

39

Choices: Predicates versus Constants

• Rep-Scheme 2: D = {a,b,c,d,e,red,pink} colorof:

<a,red>,<b,red>,<c,red>,<d,pink>,<e,pink>}

pretty: {pink} primary: {red}

• Some true sentences: colorOf(a,red). colorOf(b,red). colorOf(d,pink).

~(All X colorOf(X,red)). All X (colorOf(X,red) v colorOf(X,pink)).

pretty(pink). primary(red).We have reified predicates pink and red: made them

into objects

40

Inference with Quantifiers• Universal Instantiation:

– Given X (person(X) likes(X, sun))– Infer person(john) likes(john,sun)

• Existential Instantiation:– Given x likes(x, chocolate)– Infer: likes(S1, chocolate)– S1 is a “Skolem Constant” that is not

found anywhere else in the KB and refers to (one of) the individuals who likes sun.

41

FOPC

• This choice isn’t completely arbitrary or driven by the needs of practical applications

• FOPC reflects the semantics of natural languages because it was designed that way by human beings

• In particular…

42

Meaning Structure of Language

• The semantics of human languages…– Display a basic predicate-argument

structure– Make use of variables– Make use of quantifiers– Use a partially compositional

semantics

43

Predicate-Argument Structure

• Events, actions and relationships can be captured with representations that consist of predicates and arguments to those predicates.

• Languages display a division of labor where some words and constituents function as predicates and some as arguments.

44

Example

• Mary gave a list to John• Giving(Mary, John, List)• More precisely

– Gave conveys a three-argument predicate

– The first arg refers to the subject (the giver)

– The second is the recipient, which is conveyed by the NP in the PP

– The third argument refers to the thing given, conveyed by the direct object

45

Is this a good representation?

• John gave Mary a book for Susan– Giving (john,mary,book,susan)

• John gave Mary a book for Susan on Wednesday – Giving (john,mary,book,susan,wednesday)

• John gave Mary a book for Susan on Wednesday in class– Giving (john,mary,book,susan,wednesday,inClass)

• John gave Mary a book for Susan on Wednesday in class after 2pm– Giving

(john,mary,book,susan,wednesday,inClass,>2pm)

46

Reified Representation

• Exist b,e (ISA(e,giving) ^ agent(e,john) ^ beneficiary(e,sally) ^ patient(e,b) ^ ISA(b,book))

• “That happened on Sunday”• Add later (assuming S2 is the

skolem for e): – happenedOnDay(s2,sunday)

47

Representing Time in Language

• I ran to Oakland• I am running to Oakland• I will run to Oakland• Now, all represented the same:• Exist w (ISA(w,running) ^

agent(w,speaker) ^ dest(w,oakland))

48

Representing Time

• Events are associated with points or intervals in time.

1. Exist w,i (ISA(w,running) ^ agent(w,speaker) ^ dest(w,oakland) ^ interval(w,i) ^ precedes(i,now))

2. … member(i,now)3. … precedes(now,i)

49

Determining Temporal Relations is complex (largely unsolved)

• Ok, so for Christmas, we fly to Dallas then to El Paso (refers to the future, but the tense is present)

• Let’s see, flight 1390 will be at the gate an hour now (refers to an interval starting in the past using the future tense)

• I take the bus in the morning but the incline in the evening (habitual – not a specific morning or evening)

50

Reference Point

• Flight 2020 arrived• Flight 2020 had arrived • What’s the difference?• What do you expect in the second

example?

51

Reference Point

• Reichenbach (1947) introduced notion of Reference point (R), separated out from Speech time (S) and Event time (E)

• Example:– When Mary's flight departed, I made the

call– When Mary's flight departed, I had made

the call

• Departure event specifies reference point.

52

Reichenbach Applied to Tenses

S S R,S

S,R,E S,R S

We refer to the S,R,E notation as a Basic Tense Structure (BTS)

53

Aspect: A property of Verb Phrases

• Combo of several qualities of events• Statives: express a property of the experiencer;

do not imply anything about the time bounds– “I like the Yankees in the World Series”– “I want to go first class”– “I have the flu”

• Tests for statives: – cannot be used as commands: *“Want to go first

class!”– Odd in progressive: “I am wanting to go first class”

• Subinterval property: a subinterval of having the flue is also having the flue

54

Aspect (continued)

• Activities: have a time span, but not necessarily an end point– Running, playing the piano, looking for an

umbrella

• Tests: (progressive and commands are fine)– Odd when modified by temporal expressions

with “in”: *He was running in 5 minutes

• Subinterval property: e.g., any subinterval of playing the piano is playing the piano

55

Aspect (continued)

• Accomplishments: have a well-defined end point, and cause a new state of affairs to exist

• “I booked a reservation” • Test: “in” adverbials ok: “I booked a

reservation in 5 minutes”• Test and property: Cannot be modified by

“stopped” without changing the meaning• What happens when you change

“booked” to the progressive?

56

Aspect (continued)

• Achievements: occur at a singular point in time; cause a new state of affairs to exist– “Lost an umbrella”, “noticed the

picture”, “arrived at the station”

• Test: cannot occur over a time span– *I lost an umbrella for three months

• Note: “I noticed the picture for three months” has an iterative meaning (this is not a simple achievement)