cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ......
TRANSCRIPT
![Page 1: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/1.jpg)
1
Understanding Natural Language14 14.0 The Natural Language
UnderstandingProblem
14.1 Deconstructing Language: A SymbolicAnalysis
14.2 Syntax
14.3 Syntax and Knowledgewith ATN parsers
14.4 Stochastic Tools forLanguage Analysis
14.5 Natural LanguageApplications
14.6 Epilogue and References
14.7 Exercises
Additional source used in preparing the slides:Patrick H. Winston’s AI textbook, Addison Wesley, 1993.
![Page 2: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/2.jpg)
2
Chapter objective
• Give a brief introduction to deterministic techniques used in understanding natural language
![Page 3: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/3.jpg)
3
An early natural language understanding system: SHRDLU (Winograd, 1972)
![Page 4: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/4.jpg)
4
It could converse about a blocks world
• What is sitting on the red block?
• What shape is the blue block on the table?
• Place the green cylinder on the red brick.
• What color is the block on the red block? Shape?
![Page 5: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/5.jpg)
5
The problems
• Understanding language is not merely understanding the words: it requires inference about the speaker’s goals, knowledge, assumptions. The context of interaction is also important.
• Do you know where Rekhi 309 is?• Yes. • Good, then please go there and pick up the documents.
• Do you know where Rekhi 309 is?• Yes, go up the stairs and enter the semi-circular section.• Thank you.
![Page 6: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/6.jpg)
6
The problems (cont’d)
• Implementing a natural language understanding program requires that we represent knowledge and expectations of the domain and reason effectively about them.
• nonmonotonicity• belief revision• metaphor• planning• learning• …
Shall I compare thee to a summer’s day?Thou art more lovely and more temperate:Rough winds do not shake the darling buds of May,And summer’s lease hath all too short a date:
Shakespeare’s Sonnet XVIII
![Page 7: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/7.jpg)
7
The problems (cont’d)
• There are three major issues involved in understanding natural language:
• A large amount of human knowledge is assumed.• Language is pattern based. Phoneme, word, and
sentence orders are not random.• Language acts are products of agents embedded in
complex environments.
![Page 8: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/8.jpg)
8
SHRDLU’s solution
• Restrict focus to a microworld :blocks world
• Constrain the language:use templates
• Do not deal with problems involving commonsense reasoning:still can communicate meaningfully
![Page 9: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/9.jpg)
9
Linguists’ approach
• Prosody: rhythm and intonation of language
• Phonology: sounds that are combined
• Morphology: morphemes that make up words
• Syntax: rules for legal phrases and sentences
• Semantics: meaning of words, phrases, sentences
• Pragmatics: effects on the listener
• World knowledge: background knowledge of the physical world
![Page 10: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/10.jpg)
10
Stages of language analysis
1. Parsing: analyze the syntactic structure of a sentence
2. Semantic interpretation: analyze the meaning of a sentence
3. Contextual/world knowledge representation: Analyze the expanded meaning of a sentence
For instance, consider the sentence:
Tarzan kissed Jane.
![Page 11: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/11.jpg)
11
The result of parsing would be:
![Page 12: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/12.jpg)
12
The result of semantic interpretation
![Page 13: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/13.jpg)
13
The result of contextual/world knowledge interpretation
![Page 14: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/14.jpg)
14
Can also represent questions:Who loves Jane?
![Page 15: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/15.jpg)
15
Parsing using Context-Free Grammars
A bunch of rewrite rules:
1. sentence ↔ noun_phrase verb_phrase2. noun_phrase ↔ noun3. noun_phrase ↔ article noun4. verb_phrase ↔ verb5. verb_phrase ↔ verb noun_phrase6. article ↔ a7. article ↔ the8. noun ↔ man9. noun ↔ dog10. verb ↔ likes11. verb ↔ bites
these are the terminals
these are the symbols of the grammar
these are the nonterminals
![Page 16: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/16.jpg)
16
Parsing
• It is the search for a legal derivation of the sentence.
• sentence → noun_phrase verb_phrase→ article noun verb_phrase→ The noun verb_phrase→ The man verb_phrase→ The man verb noun_phrase→ The man bites noun_phrase→ The man bites article noun→ The man bites the noun→ The man bites the dog
• Each intermediate form is a sentential form .
![Page 17: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/17.jpg)
17
Parsing (cont’d)
• The result is a parse tree. A parse tree is a structure where each node is a symbol from the grammar. The root node is the starting nonterminal, the intermediate nodes are nonterminals, the leaf nodes are terminals.
• “Sentence” is the starting nonterminal.
• There are two classes of parsing algorithms• top-down parsers: start with the starting symbol and try
to derive the complete sentence• bottom-up parsers: start with the complete sentence and
attempt to find a series of reductions to derive the start symbol
![Page 18: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/18.jpg)
18
The parse tree
![Page 19: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/19.jpg)
19
Parsing is a search problem
• Search for the correct derivation
• If a wrong choice is made, the parser needs to backtrack
• Recursive descent parsers maintain backtrack pointers
• Look-ahead techniques help determine the proper rule to apply
• We’ll study transition network parsers (and augmented transition networks)
![Page 20: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/20.jpg)
20
Transition networks
![Page 21: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/21.jpg)
21
Transition networks (cont’d)
• It is a set of finite-state machines representing the rules in the grammar• Each network corresponds to a single nonterminal• Arcs are labeled with either terminal or nonterminal symbols• Each path from the start state to the final state corresponds to a rule for that nonterminal• If there is more than one rule for a nonterminalthere are multiple paths from the start to the goal (e.g., noun_phrase)
![Page 22: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/22.jpg)
22
The main idea
• Finding a successful transition through the network corresponds to replacement of the nonterminal with the RHS
• Parsing a sentence is a matter of traversing the network:
• If the label of the transition (arc) is a terminal, it must match the input, and the input pointer advances
• If the label of the transition (arc) is a nonterminal, the corresponding transition network is invoked recursively
• If several alternative paths are possible, each must be tried (backtracking)---very much like nondeterministic finite automaton---until a successful path is found
![Page 23: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/23.jpg)
23
Parsing the sentence “Dog bites.”
![Page 24: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/24.jpg)
24
Notes
• A “successful parse” is the complete traversal of the net for the starting nonterminal fromsinitial to sfinal .
• If no path works, the parse “fails.” It is not a valid sentence (or part of sentence).
• The following algorithm would be called using
parse(sinitial )
• It would start with the net for “sentence.”
![Page 25: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/25.jpg)
25
The algorithm
Function parse(grammar_symbol);
beginsave pointer to current location in input stream;casegrammar_symbol is a terminal;if grammar_symbol matches the next word in the input stream then
return(success)else begin
reset input streamreturn(failure)
end;
![Page 26: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/26.jpg)
26
The algorithm (cont’d)
… case …
…
grammar_symbol is a nonterminal;beginretrieve the transition network labeled by grammar_symbolstate := start state of network;if transition(state) returns successthen return(success)
else beginreset input stream;return (failure)
endend
end.
![Page 27: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/27.jpg)
27
The algorithm (cont’d)
Function transition(current_state);begincasecurrent_state is a final state:return (success)
current_state is not a final state:while there are unexamined transitions out of current_statedo begingrammar_symbol := the label on the next unexamined transitionif parse(grammar_symbol) returns (success)then beginnext_state := state at the end of the transition;if transition(next_state) returns (success);then return(success)
endend
return(failure)end
end.
![Page 28: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/28.jpg)
28
Modifications to return the parse tree
1. Each time the function parse is called with a terminal symbol as argument and that terminal matches the next symbol of input, it returns a tree consisting of a single leaf node labeled with that symbol.
2. When parse is called with a nonterminal, N, it calls transition. If transition succeeds, it returns an ordered set of subtrees. Parse combines these into a tree whose root is N and whose children are the subtrees returned by transition.
![Page 29: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/29.jpg)
29
Modifications to return the parse tree (cont’d)
3. In searching for a path through a network, transition calls parse on the label of each arc. On success, parse returns a tree representing a parse of that symbol. Transition saves these subtrees in an ordered set and, on finding a path through the network, returns the ordered set of parse trees corresponding to the sequence of arc labels on the path.
![Page 30: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/30.jpg)
30
Comments of transition networks
• They capture the regularity in the sentence structure
• They exploit the fact that only a small vocabulary is needed in a specific domain
• If a sentence “doesn’t make sense”, it might be caught by the domain information. For instance, the answer to both of the following questions is “there is none”
• “Pick up the blue cylinder”• “Pick up the red blue cylinder”
![Page 31: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/31.jpg)
31
The Chomsky Hierarchy and CFGs
• A CFG: a single nonterminal is allowed on the left-hand side.
• CFGs are not powerful enough to represent natural language
• Simply add plural nouns to the dogs world grammar:
noun ↔ mennoun ↔ dogsverb ↔ bitesverb ↔ like
“A men bites a dogs” will be a legal sentence
![Page 32: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/32.jpg)
32
Options to deal with context
• Extend CFGs
• Use context-sensitive grammars (CSGs)With CSGs the only restriction is that the RHS is at least as long as the LHS
• Note that the one higher class, recursively enumerable languages or Turing recognizable languages is not an usually regarded as an option
![Page 33: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/33.jpg)
33
A context-sensitive grammar
sentence ↔ noun_phrase verb_phrasenoun_phrase ↔ article number nounnoun_phrase ↔ number nounnumber ↔ singularnumber ↔ pluralarticle singular ↔ a singulararticle singular ↔ the singulararticle plural ↔ the pluralsingular noun ↔ dog singularsingular noun ↔ man singularplural noun ↔ men pluralplural noun ↔ dogs pluralsingular verb_phrase ↔ singular verbplural verb_phrase ↔ plural verb
![Page 34: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/34.jpg)
34
A context-sensitive grammar (cont’d)
singular verb ↔ bitessingular verb ↔ likesplural verb ↔ biteplural verb ↔ like
![Page 35: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/35.jpg)
35
“The dogs bite”
sentence → noun_phrase verb_phrase→ article plural noun verb_phrase→ The plural noun verb_phrase→ The dogs plural verb_phrase→ The dogs plural verb→ The dogs bite
![Page 36: cs4811-ch14a-nlunilufer/classes/cs4811/2005-spring/lecture... · • metaphor • planning • ... use templates • Do not deal with ... • It is the search for a legal derivation](https://reader035.vdocument.in/reader035/viewer/2022062604/5fc611ba79ca866bca5fd19a/html5/thumbnails/36.jpg)
36
CSGs for practical parsing
1. The number of rules and nonterminals in the grammar increase drastically.
2. They obscure the phrase structure of the language that is so clearly represented in the context-free rules
3. By attempting to handle more complicated checks for agreement and semantic consistency in the grammar itself, they lose many of the benefits of separating the syntactic and semantic components of language
4. CSGs do not address the problem of building a semantic representation of the text