selmer bringsjord, bettina schimanski dept. of cognitive science / dept. of computer science rair...

Selmer Bringsjord, Bettina Schimanski

Dept. of Cognitive Science / Dept. of Computer Science

RAIR (Rensselaer Artificial Intelligence and Reasoning) Lab

Rensselaer Polytechnic Institute

Troy, NY 12180 USA

selmer@rpi.edu ; schimb@rpi.edu

www.rpi.edu/~brings ; www.bettinaschimanski.com

www.cs.rpi.edu/~schimb/peri

IJCAI 2003 Conference

August 15, 2003

Can you solve this puzzle?

S/SS/S

Could you solve it in under one Could you solve it in under one second?second?

S/SS/S

A New Kind of AIA New Kind of AIAssume the ‘A’ part isn’t the problem: we know Assume the ‘A’ part isn’t the problem: we know what an artifact is.what an artifact is.

There is no agreement on what human intelligence is. There is no agreement on what human intelligence is. - Two notorious conferences. See - Two notorious conferences. See The g FactorThe g Factor..

But we can agree that one great success story of But we can agree that one great success story of psychology is testing, and prediction on the basis of psychology is testing, and prediction on the basis of it. (it. (The Big TestThe Big Test))

Psychometric AI offers a simple but radical answer:Psychometric AI offers a simple but radical answer:– (Naïve definition): (Naïve definition): AI is the field devoted to building

intelligent artificial agents, i.e., agents capable of solid performance on intelligence tests.

Don’t confuse this with: “Some Don’t confuse this with: “Some humanhuman is is intelligent…”intelligent…”

An Answer to: What is AI?

S/BS/B

Psychometric AI is not completely Psychometric AI is not completely newnew

Early roots: PAI was Early roots: PAI was implicitly entertained implicitly entertained with Evan’s 1968 with Evan’s 1968 ANALOGY programANALOGY program

However, PAI can not However, PAI can not be based on tests be based on tests which consist solely of which consist solely of geometric analogiesgeometric analogies

B/BB/B

“If one were offered a machine purported to be intelligent, what would be an appropriate method of evaluating this claim? The most obvious approach might be to give the machine an IQ test … However, [good performance on tasks seen in IQ tests would not] be completely satisfactory because the machine would have to be specially prepared for any specific task that it was asked to perform. The task could not be described to the machine in a normal conversation (verbal or written) if the specific nature of the task was not already programmed into the machine. Such considerations led many people to believe that the ability to communicate freely using some form of natural language is an essential attribute of an intelligent entity.” (Fischler & Firschein 1990, p. 12)

Objection 1Objection 1

B/BB/B

Our RebuttalOur Rebuttal

As we’ve already implied, PERI will be able to solve many puzzles, without being pre-engineered. Have built theorem prover-based agents to infallibly crack not only geometric analogies, but RPM items they have never seen beforeHumans will communicate with PERI in natural languageTests of mental ability = much more than IQ tests

B/SB/S

Intelligence Tests: Narrow vs. BroadIntelligence Tests: Narrow vs. BroadSpearman’sview of intelligence

Thurstone’s view ofintelligence

S/SS/S

Narrow Test: RPMNarrow Test: RPM

=========== start of search ===========given clause #1: (wt=2) 10 [] R1(a11).given clause #2: (wt=2) 11 [] R1(a12).given clause #3: (wt=2) 12 [] R1(a13)....given clause #4: (wt=2) 13 [] R2(a21).given clause #278: (wt=16) 287 [para_into,64.3.1,3.3.1] R2(x)| -R3(a23)| -EmptyBar(y)| -R3(x)| -EmptyBar(x)| -T(a23)| -R3(y)| -T(y).given clause #279: (wt=16) 288 [para_into,65.3.1,8.3.1] R2(x)| -R3(a23)| -StripedBar(y)| -R3(x)| -StripedBar(x)| -EmptyBar(a23)| -R3(y)|-EmptyBar(y).Search stopped by max_seconds option.============ end of search ============

Correct!

S/BS/B

Broad Test: WAISBroad Test: WAISWechsler Adult Intelligent ScaleWechsler Adult Intelligent Scale

WAIS includes many sub-testsWAIS includes many sub-tests

B/BB/B

Broad Test: WAISBroad Test: WAISWechsler Adult Intelligent ScaleWechsler Adult Intelligent Scale

WAIS includes many sub-testsWAIS includes many sub-tests

The Comprehension sub-test is so difficult The Comprehension sub-test is so difficult it could be a motivator for the CYC dreamit could be a motivator for the CYC dream

-Deals with ordinary conversationDeals with ordinary conversation

Ex: “Why are the tires of Ex: “Why are the tires of automobiles made of rubber, automobiles made of rubber,

rather than, say, plastic?”rather than, say, plastic?”

B/BB/B

Broad Test: WAISBroad Test: WAISWechsler Adult Intelligent ScaleWechsler Adult Intelligent Scale

WAIS includes many sub-testsWAIS includes many sub-tests

Picture ArrangementPicture Arrangement

- Deals with many mental facets- Deals with many mental facets

B/BB/B

Broad Test: WAISBroad Test: WAISWechsler Adult Intelligent ScaleWechsler Adult Intelligent Scale

WAIS includes many sub-testsWAIS includes many sub-tests

Block DesignBlock Design

- PERI has already - PERI has already cracked thiscracked this

B/BB/B

Broad Test: WAISBroad Test: WAISWechsler Adult Intelligent ScaleWechsler Adult Intelligent Scale

WAIS includes many sub-testsWAIS includes many sub-tests

PERI should specifically solve all of the WAISPERI should specifically solve all of the WAIS

- From there, we move to all other established tests- From there, we move to all other established tests

B/BB/B

Improved Definition of PAIImproved Definition of PAI

Psychometric AI is the field devoted to Psychometric AI is the field devoted to building information-processing entities building information-processing entities capable of at least solid performance on capable of at least solid performance on all all established, established, validatedvalidated tests of tests of intelligence intelligence and mental ability, and mental ability, a class of a class of tests thattests that includes IQ tests, tests of includes IQ tests, tests of reasoning, of creativity, mechanical ability, reasoning, of creativity, mechanical ability, and so on.and so on.

B/BB/B

PERIPERIPscyhometric Experimental Robotic IntelligencePscyhometric Experimental Robotic Intelligence

Scorbot-ER IX Scorbot-ER IX

Sony B&W XC55 Video Sony B&W XC55 Video CameraCamera

Cognex MVS-8100M Cognex MVS-8100M Frame GrabberFrame Grabber

Dragon Naturally Dragon Naturally Speaking SoftwareSpeaking Software

NL NL (CARMEL & RealPro?)(CARMEL & RealPro?)

BH8-260 BarrettHand BH8-260 BarrettHand Dexterous 3-Finger Dexterous 3-Finger Grasper SystemGrasper System

PERI was not designed to simulate

PERI was not designed to simulate

how a human thinks

how a human thinks

- AI, not cognitive modeling

- AI, not cognitive modeling

B/BB/B

Block Design TaskBlock Design Task(Not actual blocks from WAIS)(Not actual blocks from WAIS)

PERI can solve it in under one second of CPU time and not just ones nd not just ones from the WAIS but any Block Design task given to itfrom the WAIS but any Block Design task given to it

Place blocks together so that all edges match in color

(Blocks courtesy of Binary Arts Corporation)B/BB/B

PERI in actionPERI in action

B/SB/S

Objection 2Objection 2

Isn’t PAI is idiosyncratic?Isn’t PAI is idiosyncratic?– PAI is a generalization of a longstanding PAI is a generalization of a longstanding

answer to the “What is AI?” questionanswer to the “What is AI?” question– Is the answer that appeals to the TT and its Is the answer that appeals to the TT and its

relativesrelatives– AI is the field devoted to building artificial AI is the field devoted to building artificial

entities able to pass TTTentities able to pass TTT– Tests are clean and crisp – have determinate Tests are clean and crisp – have determinate

starts and ends, yielding concrete verdictsstarts and ends, yielding concrete verdicts– PAI extends and clarifies Turing’s approachPAI extends and clarifies Turing’s approach

S/SS/S

Objections 3 & 4Objections 3 & 4

Don’t TT and TTT subsume PAI?Don’t TT and TTT subsume PAI?– PAI uses Divide and ConquerPAI uses Divide and Conquer– All topics are fair game in TT and TTTAll topics are fair game in TT and TTT– Both are hugely ambitious goalsBoth are hugely ambitious goals

AI has Applications…?AI has Applications…?– An agent able to pass these tests will have An agent able to pass these tests will have

the capacity to provide the desired the capacity to provide the desired applicationsapplications

S/SS/S

Objections 5 & 6Objections 5 & 6

But PAI will only tap logicist AI?But PAI will only tap logicist AI?– PAI will unite logicist and sub-symbolic PAI will unite logicist and sub-symbolic

approachesapproaches

But AI researchers don’t do PAI?But AI researchers don’t do PAI?– PAI is descriptive and prescriptivePAI is descriptive and prescriptive– Developers will inevitably modularize their Developers will inevitably modularize their

challenges, building sub-agents and negotiate challenges, building sub-agents and negotiate tests associated with these modulestests associated with these modules

– AI applications probably themselves AI applications probably themselves constitute miniature testsconstitute miniature tests

S/SS/S

Objections 7 & 8Objections 7 & 8

Behaving like humans vs. doing well on IQ testsBehaving like humans vs. doing well on IQ tests– Create “smart” agents and then “de-smart” them Create “smart” agents and then “de-smart” them

laterlater

Performing well on “any possibly existing” Performing well on “any possibly existing” psychometric test makes PAI ill-definedpsychometric test makes PAI ill-defined– The challenge is to create an agent with the The challenge is to create an agent with the

capacity to crack tests he has never seen beforecapacity to crack tests he has never seen before– These tests must be validated via ordinary These tests must be validated via ordinary

statistico-mathematical standards used in statistico-mathematical standards used in psychometricspsychometrics

S/BS/B

Narratological Reasoning & Narratological Reasoning & Thwarting Terrorism…Thwarting Terrorism…

Terrorists struggle to make stories real.Their behavior can be anticipated, and thus

thwarted.

We need computers that can imagine futureevents in a (twisted) narrative.

Threat anticipation (Picture Arrangement)Threat anticipation (Picture Arrangement)

Predictive power done quickly ….Predictive power done quickly ….B/BB/B

Picture ArrangementPicture Arrangement

B/BB/B

Picture ArrangementPicture Arrangement

Currently untouchable AI -- but we shall see.

B/BB/B

B/SB/S

RAIR Web and R&D

PERI

Savant

Slate

Super TeachingCDsCDs

PAI

MARMML

Advanced Synthetic Characters

S/SS/S

Future of PAI / Questions?Future of PAI / Questions?

Efforts to crack IQ tests should lead to Efforts to crack IQ tests should lead to systems able to generate new items for systems able to generate new items for other established tests, and new testsother established tests, and new testsMay help determine what tests a machine May help determine what tests a machine can solve, and those it has more trouble can solve, and those it has more trouble than a human = securitythan a human = securityOnce successful: Is PERI a genius or does Once successful: Is PERI a genius or does human intelligence go well beyond what can human intelligence go well beyond what can be tested for?be tested for?Divides Strong and Weak AIDivides Strong and Weak AI

S/SS/S