data on trial: lessons from the turing test minds & machines

23
Data on Trial: Lessons from The Turing Test Minds & Machines

Upload: makayla-wesson

Post on 11-Dec-2015

215 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Data on Trial: Lessons from The Turing Test Minds & Machines

Data on Trial:Lessons from The Turing Test

Minds & Machines

Page 2: Data on Trial: Lessons from The Turing Test Minds & Machines

“I believe that in about fifty years’ time it will be possible to programme computers, with astorage capacity of about 109, to make them playthe imitation game so well that an averageinterrogator will not have more than 70 per centchance of making the right identification after5 minutes of questioning”

-Alan Turing (1950)

Page 3: Data on Trial: Lessons from The Turing Test Minds & Machines

The Imitation Game, or …The Turing Test!

Interrogator

Machine

Human

Page 4: Data on Trial: Lessons from The Turing Test Minds & Machines

A Definition of Intelligence?

• Some commentators see the Turing Test as a definition of intelligence.

• Many people have commented on the shortcomings of the Turing Test as a definition of intelligence:– This definition would be philosophical behaviorism,

which has obvious problems. In short, we think that being intelligent causes the behavior, but doesn’t consist in the behavior.

– This definition would define intelligence based on the subjective judgment of whomever happens to be the interrogator, however long the conversation was, and what the conversation was about.

Page 5: Data on Trial: Lessons from The Turing Test Minds & Machines

Not a Definition

• Turing himself clearly did not intend to propose a definition of intelligence. For example, in his paper he readily acknowledges that one could have intelligent beings not being able to pass the test simply by not having a human-like intellect:– “May not machines carry out something which ought

to be described as thinking but which is very different from what a man does? This objection is a very strong one, but at least we can say that if, nevertheless, a machine can be constructed to play the imitation game satisfactorily, we need not be troubled by this objection”

Page 6: Data on Trial: Lessons from The Turing Test Minds & Machines

A Sufficient Condition for Intelligence?

• Many commentators interpret this statement as saying that if a machine passes the Turing Test, then it is intelligent, i.e. that passing the Turing Test is a sufficient condition for intelligence (since intelligence is a necessary condition to pass it), but not a necessary one (and hence it is not a definition).

• In logic:– P I– But not: I P

Page 7: Data on Trial: Lessons from The Turing Test Minds & Machines

Not a Great Criterion Either

• As a sufficient condition for being intelligent, the Turing Test suffers from some of the same problems as before: such a criterion would still amount to a subjective judgment based on imprecisely defined behavioral criteria.

• In short, this seems to be a rather sloppy criterion!

• Now, would Turing really not have anticipated this (rather straightforward) objection? Hmm…

Page 8: Data on Trial: Lessons from The Turing Test Minds & Machines

Eliza

• A psychotherapist program developed by Joseph Weizenbaum in 1966.

• Many people conversing with Eliza had no idea that they weren’t talking to a human.

• Did Eliza pass the Turing Test?• Or is it just easy being a psychotherapist?• Eliza wasn’t really tested in the format that Turing

proposed.• Still, it is interesting that humans were quick to attribute

human-level intelligence to such a simple program.• Maybe in a real Turing Test a relatively simple computer

program can ‘trick’ the interrogator as well?

Page 9: Data on Trial: Lessons from The Turing Test Minds & Machines

The Loebner Competition

• Multiple judges rank-order multiple humans and multiple computer programs from ‘most likely to be human’ to ‘least likely to be human’.

• Loebner promises $100,000 for the first computer program to be ‘indistinguishable from a human’.

• Thus far, Loebner is still a rich man!• So maybe it is difficult to pass the test. • Maybe we could say that if something does pass

the test, then there is at least a good chance for it to be intelligent.

• Maybe, but this is not very exciting.

Page 10: Data on Trial: Lessons from The Turing Test Minds & Machines

“Contrary Views”

• In his paper Turing goes over a list of “Contrary Views on the Main Question”:

• Machines:– can only do what they’re told to do (Lady Lovelace)– can’t learn– can’t be creative– can’t make mistakes– can’t … (fill in the blank)

• Turing: Our mistakes are that:– We generalize from existing (special-purpose) machines

(Turing-machines are general-purpose)– We equate level of mechanics with level of functioning

(emergent behavior; emergent properties)

Page 11: Data on Trial: Lessons from The Turing Test Minds & Machines

Another Question

• If Turing’s point of his article was to propose a test or criteria for intelligence, then why are none of these objections about the validity of this test?

• At best, these seem to be objections to the claim that machines can pass this test.

• Hmm, so what is Turing’s real point of the paper?

Page 12: Data on Trial: Lessons from The Turing Test Minds & Machines

Turing’s Argument for AI?

• Some commentators see Turing’s paper as an argument for the possibility of AI:– Machines (computers) can pass the Turing Test– Anything that passes the Turing Test

is intelligent– Therefore, machines (computers) can be intelligent

Page 13: Data on Trial: Lessons from The Turing Test Minds & Machines

Can Machines pass the Turing Test?

• As we saw, Turing thinks that this is true (or at least, he expresses confidence that machines will do well in the Imitation Game).

• However, he never really makes clear why he thinks this is so.

• You would think that Turing would have made some effort to argue for this (rather controversial) claim if it is used as a premise of the earlier argument. Hmm…

• Presumably, Turing thinks that passing the test requires nothing more than some kind of information processing ability, which is exactly what computers do.

Page 14: Data on Trial: Lessons from The Turing Test Minds & Machines

A Puzzle

• But wait, can’t we then just argue as follows: – Intelligence requires nothing more than some kind of

information processing ability – Computers can have this information processing

ability– Therefore, computers can be intelligent

• Indeed, this is exactly how proponents of AI make the argument today.

• So why didn’t Turing make this very argument? Why bring in the game?

Page 15: Data on Trial: Lessons from The Turing Test Minds & Machines

The Puzzle Again … And A Little Question

• Going back to the “Contrary Views”, AI opponents think machines can’t do certain things, but Turing thinks they can. Now, if the issue is whether or not computers can be intelligent, isn’t that indeed the crucial, make-or-break issue?

• The Turing Test doesn’t seem to be able to shed any more light on this issue, so: why bring in the game?

• Oh, and why the strange set-up of the Turing-Test anyway? Why did Turing ‘pit’ a machine against a human in some kind of contest? Why not have the interrogator simply interact with a machine, see what it is or is not able to do, and determine whether or not the machine is intelligent based on those interactions?

• If we are so concerned about what machines can and cannot do, why not simply do:

Page 16: Data on Trial: Lessons from The Turing Test Minds & Machines

The Super-Simplified Turing Test!!

Interrogator Machine

Page 17: Data on Trial: Lessons from The Turing Test Minds & Machines

Answer: Bias

• The mere knowledge that we are dealing with a machine will bias our judgment as to whether that machine can think or not, as we may bring certain preconceptions about machines to the table.

• Moreover, knowing that we are dealing with a machine will most likely lead us to raise the bar for intelligence: it can’t write a sonnet? Ha, I knew it!

• By shielding the interrogator from the interrogated, such a bias and bar-raising is eliminated in the Turing-Test.

• OK, but still, why not:

Page 18: Data on Trial: Lessons from The Turing Test Minds & Machines

The Simplified Turing Test

Interrogator Machine or Human

Note: this is exactly how many commentators talk about the Turing Test

Page 19: Data on Trial: Lessons from The Turing Test Minds & Machines

Level the Playing Field

• Since we know we might be dealing with a machine, we still raise the bar for the entity on the other side being intelligent.

• Through his set-up of the test, Turing made sure that the bar for being intelligent wouldn’t be raised any higher for machines than we do for fellow humans.

Page 20: Data on Trial: Lessons from The Turing Test Minds & Machines

Back to The Puzzle• OK, fine, but we are still left with our basic puzzle: why bring

up any test at all? Why not simply consider the questions as to whether machines can or cannot have certain abilities head-on, and try and define some more precise and objective criteria for intelligence, instead of having this be determined by such a sloppy Game?

• I believe that the answer to the little question provides us with an answer to our puzzle: the convoluted set-up wasn’t merely a practical consideration to eliminate bias in some strange game, but rather the whole point of his article!

• That is, I believe the point of Turing’s article wasn’t to make any argument for the possibility of AI, but rather that if we put a label ‘intelligent being’ on other human beings based on their behavior then, just to be fair, we should do the same for machines, whether we are correct in any such attributions or not.

Page 21: Data on Trial: Lessons from The Turing Test Minds & Machines

‘Imitation Game’ vs ‘Turing Test’

• I think it is likely that Turing never intended to propose any kind of test for machine intelligence (let alone propose a definition!).

• At best, Turing would say that ‘passing the test’ means that we should call that entity intelligent, correct or not.

• In other words, Turing’s point was about language use!• Talking about language use, I think we really should no

longer refer to the Turing Test as the Turing ‘Test’. • Interesting fact: In his original article Turing uses the

word ‘pass’ or ‘passing’ 0 times, ‘test’ 4 times, and ‘game’ 37 times.

Page 22: Data on Trial: Lessons from The Turing Test Minds & Machines

In Turing’s Words

The original question, “Can machines think?”, I believeto be too meaningless to deserve discussion. NeverthelessI believe that at the end of the century the use of words and general educated opinion will have altered so much that onewill be able to speak of machines thinking without expectingto be contradicted.

-Alan Turing (1950)

Page 23: Data on Trial: Lessons from The Turing Test Minds & Machines

Oh, and another thing• I believe that seeing Turing’s contribution as laying out a test,

and our obsession to try and pass that test (or at least thinking about AI that way) has been (and still is) detrimental to the field.

• E.g. In “Essentials of Artificial Intelligence”, Ginsberg defines AI as “the enterprise of constructing a physical symbol system that can reliably pass the Turing Test”

• But trying to pass the test encourages building cheap tricks to convince the interrogator that he/she is dealing with a human, which is exactly what we have seen with Eliza, Parry, and the modern-day Alice and Jabberwacky.

• This kind of work has advanced the field of AI … exactly zilch!

• Can we please stop talking about a ‘Test’?!?• Thank you!