artificial general intelligence (agi) building thinking machines © 2007 general intelligence...
TRANSCRIPT
Artificial general intelligence(AGI)
“building thinking machines”
© 2007 General Intelligence Research Group
AGI vs “narrow” AI
• examples of narrow AI:– face recognition– spam filtering– data mining– Google
Common objections
• intelligence is not well-defined• it’s too hard• computing power is not there yet• no unifying theory of AI• we don’t understand the brain• etc…
All this is bull shit!
AI pioneers• Alan Turing (1912-1954)• John von Neumann (1903-1957)
John McCarthy (1927-) Marvin Minsky (1927-)
Implications of AGI
• complete automation• ethical issues• “Technological Singularity”
Vernor Vinge (1944-)Ray Kurzweil (1948-)
Representative AGI projects
Cyc
• most-funded AI project in history ($10s of millions)• based on predicate logic• complete ontology• millions of facts, concepts
Doug Lenat (1950-)
Soar• Allen Newell (1927-1992) John E Laird
• based on production rules & rete algorithm• learning – “chunking”
Novamente• Ben Goertzel (1966-)• probabilistic logic based on
“uncertain probabilities”• graph-based
knowledge representation• genetic algorithms
for learning• robot living in virtual reality• 2007 book:
Artificial General Intelligence
NARS• Non-Axiomatic Reasoning System• Pei Wang• can learn from experience• work with insufficient
knowledge and resources• unified cognition:
reasoning, learning,planning, etc…
• 2006 book:Rigid Flexibility
SNePS
• Semantic Network Processing System• Stuart C Shapiro• extends first-order logic• belief revision /
assumption-basedtruth maintenance
• natural languageunderstanding
AIXI
• Marcus Hutter• highly abstract• based on Kolmogorov
complexity theory• KC is incomputable• learning may take
forever!
Polyscheme
• Nick Cassimatis• integrates multiple methods
of representation, reasoning,and problem-solving
• procedural substrate• not “one model”
CAM-brain
• Hugo de Garis (1947-)• neural network• evolvable hardware• cellular automata• currently at
Wuhan University
SAIL• John Weng neural network-based• navigates and learns from environment
autonomously
Jeff Hawkins (1957-)
• inventor of “Palm Pilot”• founded Redwood
Neuroscience Institute• 2005 book:
On Intelligence• HTM (Hierarchical
Temporal Memory)• neurally-inspired
Brain-inspired
AI
visual cortex
Wiring of 6-layer cortex
Neurally-inspired AI
• feedforward neural network• Jeff Hawkins’ approach• problem:
invariant recognition:translation,rotation,scaling
Statistical learning
• takes place in a vector space• requires many examples• target = manifold• difficult to learn
concepts withvariableseg:On(apple,table),On(car,road), etc…
“Spatial” pattern recognition
ANN,SVM,PCA,Clustering,etc…
Logic-based vision
• visual features logical representation
Logical-vision exampleQuadrilateral() :-∃e1:edge
e∃ 2:edgee∃ 3:edgee∃ 4:edgev∃ 1:vertexv∃ 2:vertexv∃ 3:vertexv∃ 4:vertex
Connects(e1,v1,v2) ^Connects(e2,v2,v3) ^Connects(e3,v3,v4) ^Connects(e4,v4,v1)
“Syntactic” pattern recognition
predicate logic formula: featurei relation1(feature1, feature2, …) ^
relation2(feature3, feature4, …) ^…
Spatial interpretation?
Logic-based AI
Avoid reinventing the wheel!
Logic-based AI
• first-order predicate logic (Prolog)• common objections:
“brittle”“rigid”“binary” “not numerical”“just a theorem prover”
• probabilistic / fuzzy logic• non-deductive mechanisms
eg: abduction, induction
Modules
• perception (eg vision)• pattern recognition• inference• natural language• learning• truth maintenance• planning
Architecture
Pattern recognition• “neural characteristics” “soft computing”• Prolog:
chair(X) :- leg1, leg2, leg3, leg4, seat, back, horizontal(seat), vertical(back),...
leg1
chair
leg2 leg3 leg4 … ...
fuzzy values
Pattern recognition– “chairs”
more chairs
still more chairs
Pattern recognition
• how humans recognize “concepts”?• [Michalski 1989] “2-tiered approach”
rule-based vs instance-based• Prolog:
chair :- chair1
chair :- chair2
chair :- chair3
...chair :- (rule for general chair)
Probabilistic logic• classical resolution [JA Robinson 1965]• Bayesian networks [eg Judea Pearl]
Resolution algorithm
• try to resolve formulas repeatedly until no more can be resolved
P V Q ~P V R
Q V R
Bayesian network
• propositional
First-order Bayes net
BeltStatus(belt) RoomTemp(room)
EngineStatus(machine)
Bayesian vs classical logic
• Conditional Probability Table (CPT) classical
Bayesian (A ^ B)A B
C
A B CT T 1.0T F 0.0F T 0.0F F 0.0
A B CT T TT F FF T FF F F
KBMC• Knowledge-Based Model Construction• [Wellman et al 1992]• generate Bayesian networks “on-the-fly” to
answer specific queries
KB
KBMC example
KBMC example
Belief bases vs belief sets
• belief set = Cn( belief base )
set of consequences• belief sets are too large to manipulate• for AGI, must use belief base
Fuzzy logic• “John’s girl friend is probably very pretty”
• fuzziness probability• Lotfi Zadeh (1921-)
1965 fuzzy sets1973 fuzzy logic
Confidence
• Example:A. 10 girls, 5 have long hairB. 1000 girls, 500 have long hair
p = 0.5but A and B are not the sameB has higher confidence
• used in Pei Wang’s NARS logic
Probabilistic-fuzzy inference
( P, C, Z )n ( P, C, Z ) x1 x2 . . .
Ps and Zs can be point-valued or interval-valued
probabilityconfidencefuzziness
Probability intervals
• Example:marry fool [p = 0.8]! marry loser [p = 0.7]
p( fool V loser ) =0.7 + 0.1 * p( marry ) [ 0.7, 0.8 ]
unknown
Conditional probability table (CPT)
• All permutations of fuzzy values
• Or, store in a“distribution-free”format?
a b Cz1 … (P1, C1, Z1)
z2 … (P2, C2, Z2)
z3 … (P3, C3, Z3)
z4 … (P4, C4, Z4)
… … …
“Rules of thought”
• “If cats have claws,and Juney is a cat,then Juney has claws.”
• P,x,y P(x) ^ isa(y,x) P(y)• modus ponens: • syllogisms
Q Q P P,
reasoning
deduction retroduction
induction abduction
Abduction
• “finding explanations”• eg glass is wet it was raining• algorithm:
reverse of deduction (eg resolution)• very high complexity
(within the arithmetical complexity class )0 2
Abduction algorithm
Induction vs abduction
• abduction: answer = ground literalseg “grass is wet” “it was raining”
• induction: answer = general formulaeeg daughter(X,Y) :- father(Y,X) ^ female(Y)
Induction• learning general patterns statistically• ILP (Inductive Logic Programming)[Stephen Muggleton]1990s
Induction example
Given data:
male(mary) = falsefemale(mary) = truemother(mary, louise) = truefather(mary, bob) = truedaughter(bob, mary) = true
daughter(X,Y) :- father(Y,X) ^ female(Y)
Natural language
• unifying framework• language = knowledge-based inference• [Jerry R Hobbs] “Abduction as Interpretation”
eg “The Boston office called.”• “apple pie” “door knob” “street hawker”
• all we need is a lot of rules• can inductively learn the rules
Belief maintenance
• Truth Maintenance System (TMS)• belief revision• to attain “consistency”• avoid “cognitive dissonance”
Truth maintenance
justifications
Belief revision
• “Epistemic entrenchment” Belief Base• [Mary-Anne Williams
1995]
entrenchmentranking
“Click” feeling
Perhaps an effect of successful inference, abduction, or belief revision?
Paraconsistency
• holding 2 contradictory beliefs in the knowledge base at the same time
Associative memory
• knowledge base = database• special indexing to allow associative recall• hard disk = long-term memory• RAM = working memory
Planning
Conclusions
• “neural” is problematic• “blank slate” is problematic• “logic-based” is very promising
Agendafor Logic-based AI
1. design probabilistic-fuzzy logic2. develop algorithms for:– abduction–belief maintenance
3. acquire common sense knowledge
“Web 2.0”-style collaboration• branching• voting• commercial• problem: too few members
Thank you
• [Aliseda 2006] Abductive Reasoning: Logical Investigations into Discovery and Explanation. Synthese Library Series vol 330, Springer
• [Antoniou 1997] Nonmonotonic Reasoning, MIT Press• [Cussens 2001] Integrating probabilistic and logical reasoning. In David
Corfield and Jon Williamson eds Foundations of Bayesianism, volume 24 of Applied Logic Series, pages 241-260. Kluwer, Dordrecht
• [2000 Flach & Kakas eds] Induction and Abduction, Springer Applied Logic Series #18
• [Haddawy 1994] Generating Bayesian networks from probability logic knowledge, in Proceedings of the 10th conference on uncertainty in AI, 1994.
• [Hobbs 200?] Abduction as Interpretation• [Jaeger 1997] Relational Bayesian networks. In Proceedings of the 13th Annual
Conference on Uncertainty in AI (UAI-97), p266-273, San Francisco, CA, 1997, Morgan Kaufman Publishers
• [Kakas, Kowalski, Toni 1992] Abductive Logic Programming, Journal of Logic and Computation 2(6):719-770. http://citeseer.ist.psu.edu/kakas93abductive.html
• [Laskey 2006] MEBN: A logic for open-world probabilistic reasoning. GMU C4I Center Technical Report C4I-06-01. George Mason Univ, USA.
• [Milch & Russell 2007] First-Order Probabilistic Languages: Into the Unknown In ILP: Proceedings of the 16th International Conference on Inductive Logic Programming. Berlin: Springer
• [Michalski 1989] Two-tiered concept meaning, inferential matching, and conceptual cohesiveness. In Vosniadou & Ortony eds, Similarity and analogical reasoning, p122-145. Cambridge University Press, New York.
• [Muggleton 1996] Stochastic logic programs. In de Raedt, ed, Advances in Inductive Logic Programming, p254-264, IOS Press 1996.
• [Ngo, Haddawy, & Helwig 1995] A theoretical framework for context-sensitive temporal probability model construction with application to plan projection. In Proceedings of the 11th Annual Conference on Uncertainty in Artificial Intelligence (UAI-95), p419-426, Montreal, Quebec, Canada.
• [Norvig & Russell 2003] Artificial Intelligence: A Modern Approach, Prentice Hall.
• [Poole 1993] Probabilistic horn abduction and Bayesian networks, Artificial Intelligence, 64(1), 81-129, 1993
• [Poole 2003] First-order probabilistic inference, Proc, IJCAI-03, Acapulco, August 2003, p985-991
• [Wellman, Breese, Goldman 1992] From knowledge bases to decision models. Knowledge Engineering Review 7(1): 35-52
• [Williams 1995] Changing nonmonotonic reasoning inference relations, in Proceedings of the second world conference on the fundamentals of AI, 469-482, Ankgor, Paris, 1995