111 artificial intelligence cs 165a tuesday, november 6, 2007 inference in fol (ch 9) 1
TRANSCRIPT
111
Artificial Intelligence
CS 165A
Tuesday, November 6, 2007
Inference in FOL (Ch 9)
1
2
George Boole (1815-1864)
• More than 100 years later, he didn’t know about Leibniz, but proceeded to bring to life part of Leibniz’ dream
• His insight: Logical relationships are expressible as a kind of algebra– Letters represent classes (rather than numbers)
– So logic can be viewed as a form of mathematics
• Published The Laws of Thought
British
• He extended Aristotle's simple syllogisms to a broader range of reasoning– Syllogism: Premise_1, Premise_2 Conclusion– His logic: Propositional logic
3
Gottlob Frege (1848-1925)
• He provided the first fully developed system of logic that encompassed all of the deductive reasoning in ordinary mathematics.
• He intended for logic to be the foundation of mathematics – all of mathematics could be based on, and derived from, logic
• In 1879 he published Begriffsschrift, subtitled “A formula language, modeled upon that of arithmetic, for pure thought”
– This can be considered the ancestor of all current computer programming languages
– Made the distinction between syntax and semantics critical
German
• He invented what we today call predicate calculus (or first-order logic)
4
Notes
• HW#3 due Thursday
• Midterm exam one week from today– Short-answer conceptual questions and HW-like questions
– Topics: Reading, lectures, material through chapter 8 What AI is, main issues and areas in AI Problem solving and intelligent agents Blind search, informed search, adversarial search Logic and inference in propositional calculus Basics of first-order logic
– Closed book
– You may bring one 8.5”x11” piece of paper with your notes Some formulas, procedures will be given (posted in advance)
5
Reminder
• Term– Constant, variable, function( )
• Atomic sentence– Predicate( ), term1 = term2
• Literal– An atomic sentence or a negated atomic sentence
• Sentence– Atomic sentence, sentences with quantifiers and/or connectives
6
Simple example of inference in FOL
Bob is a buffalo
Pat is a pig
Buffaloes outrun pigs
Does Bob outrun Pat?
Buffalo(Bob)
Pig(Pat)
Buffalo(x) Pig(y) Outrun(x,y)
KB entails Outrun(Bob, Pat)?
KB0 |– Buffalo(Bob) Pig(Pat)
(And-Introduction)
KB1 |– Buffalo(Bob) Pig(Pat) Outrun(Bob, Pat)
(Universal Instantiation) [coming soon]
KB2 |– Outrun(Bob, Pat)
(Modus Ponens)
KB1
KB0
KB2
KB3
KB0
S
Review
7
Using FOL to express knowledge
• One can express the knowledge of a particular domain in first-order logic
• Example: The “kinship domain”– Objects: people– Properties: gender, family relationships– Unary predicates: Male, Female, MotherOf, FatherOf– Binary predicates: Parent, Sibling, Brother, Sister, Son,
Daughter, Father, Mother, Uncle, Aunt, Grandparent, Grandfather, Grandmother, Husband, Wife, Spouse, Brother-in-law, Stepmother, etc….
– Functions: MotherOf, FatherOf…
• Note: There is usually (always?) more than one way to specify knowledge
8
Kinship domain
• Write down what we know (what we want to be in the KB)– One’s mother is one’s female parent
m, c Mother(m, c) Female(m) Parent(m, c) m, c TheMotherOf(c) = m Female(m) Parent(m, c)
– One’s husband is one’s male spouse w, h Husband(h, w) Male(h) Spouse(h, w)
– One is either male or female x Male(x) Female(x)
– Parent-child relationship p, c Parent(p, c) Child(c, p)
– Grandparent-grandchild relationship g, c Grandparent(g, c) p Parent(g, p) Parent(p, c)
– Etc…
• Now we can reason about family relationships. (How?)
9
Kinship domain (cont.)
Assertions (“Add this sentence to the KB”)
TELL( KB, m, c Mother(c) = m Female(m) Parent(m, c) )
TELL( KB, w, h Husband(h, w) Male(h) Spouse(h, w) )
TELL( KB, x Male(x) Female(x) )
TELL( KB, Female(Mary) Parent(Mary, Frank) Parent(Frank, Ann) )– Note: TELL( KB, S1 S2 ) TELL( KB, S1) and TELL( KB, S2)
(because of and-elimination and and-introduction)
Queries (“Does the KB entail this sentence?”)
ASK( KB, Grandparent(Mary, Ann) ) True
ASK( KB, x Child(x, Frank) ) True– But a better answer would be { x / Ann }– This returns a substitution or binding
10
Implementing ASK: Inference
• We want a sound and complete inference algorithm so that we can produce (or confirm) entailed sentences from the KB
KB KB i
• The resolution rule, along with a complete search algorithm, provides a complete inference algorithm to confirm or refute a sentence in propositional logic(Sec. 7.5)
– Based on proof by contradiction (refutation)
• Refutation: To prove that the KB entails P, assume P and show a contradiction:
(KB P False) (KB P)Prove this!
11
Inference in First-Order Logic
• Inference rules for propositional logic:– Modus ponens, and-elimination, and-introduction, or-introduction,
resolution, etc.
– These are valid for FOL also
• But since these don’t deal with quantifiers and variables, we need new rules, especially those that allow for substitution (binding) of variables to objects– These are called lifted inference rules
12
Substitution and variable binding
• Notation for substitution:– SUBST ( Binding list, Sentence )
Binding list: { var / ground term, var / ground term, … } “ground term” = term with no variables
– SUBST( {var/gterm}, Func (var) ) = Func (gterm) SUBST (, p)
– Examples: SUBST ( {x/Mary}, FatherOf (x) ) = FatherOf (Mary) SUBST ( {x/Joe, y/Lisa}, Siblings (x,y) ) = Siblings (Joe, Lisa)
13
Three new inference rules using SUBST(, p)
• Universal Instantiation
)},/({
gvSUBST
v
• Existential Instantiation
)},/({
kvSUBST
v k – constant that does not appear elsewhere in the knowledge base
g – ground term
• Existential Introduction
)},/({
vgSUBSTvv – variable not in g – ground term in
14
To Add to These Rules
15
Universal Instantiation – examples
)},/({
gvSUBST
v g – ground term
x Sleepy(x)– SUBST({x/Joe}, )
Sleepy(Joe)
x Mother(x) Female(x)– SUBST({x/Mary}, )
Mother(Mary) Female(Mary)– SUBST({x/Dad}, )
Mother(Dad) Female(Dad)
x, y Buffalo(x) Pig(y) Outrun(x,y)– SUBST({x/Bob}, )
y Buffalo(Bob) Pig(y) Outrun(Bob,y)
16
Existential Instantiation – examples
)},/({
kvSUBST
v k – constant that does not appear elsewhere in the knowledge base
x BestAction(x)– SUBST({x/B_A}, )
BestAction(B_A)– “B_A” is a constant; it is not in our universe of actions
y Likes(y, Broccoli)– SUBST({y/Truman}, )
x Likes(Truman, Broccoli)– “Truman” is a constant; it is not in our universe of people
17
Existential Introduction – examples
)},/({
vgSUBSTvv – variable not in g – ground term in
• Likes(Jim, Broccoli)– SUBST({Jim/x}, )
x Likes(x, Broccoli)
x Likes(x, Broccoli) Healthy(x)– SUBST({Broccoli/y}, )
y x Likes(x, y) Healthy(x)
18
What’s our goal here?
• Formulate a search process:– Initial state
KB– Operators
Inference rules– Goal test
KB contains S
• What is a node?– KB + new sentences (generated by applying the inference rules)– In other words, the new state of the KB
• What kind of search to use?– I.e., which node to expand next?
• How to apply inference rules? – Need to match the premise pattern
19
Applying inference rules
• FOL query– ASK(KB, Outruns(Bob, Pat))
– ASK(KB, x BestAction(x, t))
– ASK(KB, x Equals(x, AuntOf(HusbandOf(Mary))))
• Proof – From axioms (KB) to hypothesis (S)– Does the KB entail a sentence S – if so, what steps were taken?
• We need to apply inference rules and appropriate substitutions to reach the desired sentence– Answer: Yes, SUBST({x/Grab}, Action(x, t))
– Answer: Yes, the KB entails S (and here’s how…)
• Reasoning systems may require the solution path, not just the final answer
Path = proof
Path shows reasoning (here’s why I think you have cancer)
20
Concerns
• Is the inference procedure sound and complete?
• Is it efficient?– We need to be concerned about the branching factor
– Generalized modus ponens (GMP) is one way of reducing branching factor
• How do we know which operators (inference rules) are valid at any particular time?– Inference rule: Pig(x) Cuddly(x)
– KB: Pig(Bob)
Pig(x) and Pig(Bob) are unified by the substitution = {x/Bob}
21
Problem of branching factor
• 10 inference rules– Branching factor can be huge!
• Common inference pattern:– And-Introduction
– Universal Instantiation
– Modus ponens
• Let’s introduce a new inference rule which reduces the branching factor by combining these three steps into one– Generalized Modus Ponens
Buffaloes outrun pigs example
22
Generalized Modus Ponens
• For atomic sentences pi , pi' , and q , where there is a substitution such that SUBST(, pi') = SUBST(, pi) for all i, then
),(
)(,,,, 2121
qSUBST
qpppppp nn
• GMP features:– Efficient – it combines several inferences into one
– Sensible – substitutions are purposeful (unification)
– Efficient – when sentences are in a certain canonical form (Horn clauses), this is the only inference rule necessary (!!!)
Specific facts General rule
Instantiated conclusion
23
Generalized Modus Ponens Example
• Go from– Buffalo(Bob), Pig(Pat), (Buffalo(x) Pig(y) Outrun(x,y))
to
– Outrun(Bob, Pat)
• More generally, go from– p1', p2', (p1 p2
q)
to
– SUBST(, q)
where SUBST(, pi') = SUBST(, pi)
• Reminder: What is ?
24
Unification
• Unification takes two atomic sentences p and q and returns a substitution that would make p and q look the same (or else it fails)– UNIFY(p, q) = where SUBST(, p) = SUBST(, q) is the unifier of the sentences p and q
• Examples– p = Knows(John, x) q = Knows(John, Jane)
= {x/Jane}
– p = Knows(John, x) q = Knows(y, Fred) = {x/Fred, y/John}
– p = Knows(John, x) q = Knows(y, MotherOf(y)) = {x/MotherOf(John), y/John}
– p = Knows(John, x) q = Knows(x, Mary) = fail
– p = Knows(John, x1) q = Knows(x2, Mary)
= {x1/Mary, x2/John}
25
From GMP to GR
• GMP requires sentences in Horn clause
• Prolog is based on this: Horn clause form and GMP inference– Horn clause: an implication of positive literals
• It is “reasonably powerful,” but – Cannot handle disjunctions or negations
– Many legal sentences cannot be expressed in a Horn clause
– I.e., this form is incomplete
• It turns out we can do better by putting the sentences of the KB into a different format and using just one inference rule, which is both complete and sound!
– KB Format: Conjunctive normal form (CNF)
– Inference Rule: Generalized (first-order) resolution
26
Note: Completeness
• An inference procedure i is complete iff– KB i whenever KB
– “The procedure finds the needle in the haystack when there is one”
• An inference procedure using GMP is incomplete– There are sentences entailed by the KB that the procedure cannot
infer
• A complete inference procedure exists: Generalized Resolution– Generalized resolution will produce if KB entails – However, what if is not entailed by the KB? Will resolution tell us
this? No, there exists no certain procedure to show this in general
– Related to the famous “Halting problem”
27
Resolution
rp
rqqp
,
Simple resolution
Let’s rewrite it:rp
rqqp
,
ca
cbba
,or:
Remember: p q p q
So resolution is really the “chaining” of implications….
Review
28
Generalized (first-order) resolution
• Generalized resolution is the basis for a complete inference procedure– Can derive new implications (P Q)
GMP can only derive atomic conclusions (Q)
– Any sentence can be put into its normal form
• Just like “regular” resolution except– Uses UNIFY/SUBST rather than =
– Deals with more than 2 disjuncts
• GR is complete, but semidecidable– Not always able to show that a sentence is not entailed by the KB
29
Reminder: Atomic sentences vs. Literals
• Atomic sentences– Predicate and its arguments (terms)
P(x), Q(y), Siblings(Bill, z), Equal(x, y)
• Literals– Atomic sentences and negated atomic sentences
P(x), Q(y), Siblings(Bill, z), Equal(x, y)
30
Two versions of Generalized Resolution
kj qandpexceptnm
nk
mj
qqppSUBST
qqq
ppp
),( 11
1
1
Disjunctions
For literals pi and qi , where UNIFY(pj , qk) =
kj qandpexceptnnnn
nkn
nnj
qqrrssppSUBST
qqqss
rrppp
),(4231
43
21
1111
11
11
Implications
For atomic sentences pi, qi, ri, si , where UNIFY(pj , qk) =