http://www.cs.tufts.edu/comp/150BBR/timetable.htmlhttp://www.cs.tufts.edu/comp/150BBR/timetable.htmlhttp://www.cs.tufts.edu/comp/150BBR/syllabus.htmlhttp://www.cs.tufts.edu/comp/150BBR/syllabus.html

COMP150COMP150Behavior-Based RoboticsBehavior-Based Robotics

For class use only, do not distributeFor class use only, do not distribute

Robo-ethicsRobo-ethics

A nascent interdisciplinary approach started recently towards developing A nascent interdisciplinary approach started recently towards developing

ethical foundations ethical foundations for autonomous robotsfor autonomous robots

Term originally coinded by Gianmarco Veruggio with applications to where Term originally coinded by Gianmarco Veruggio with applications to where

it is ethical to use robots (military, surgery, elder care) etc., it now has it is ethical to use robots (military, surgery, elder care) etc., it now has

been widened to include inqueries into what it taks for robots to behave been widened to include inqueries into what it taks for robots to behave

ethically (i.e., it includes new areas in AI such as machine ethics)ethically (i.e., it includes new areas in AI such as machine ethics)

The goal for robo-ethics is to develop principles for robot deployment and The goal for robo-ethics is to develop principles for robot deployment and

robot behaviors that ethically sound and socially acceptablerobot behaviors that ethically sound and socially acceptable

Topics include (autonomous) Topics include (autonomous) military robotsmilitary robots (that can use lethal force), (that can use lethal force),

medical robotsmedical robots (e.g., robot surgeons, elder care robots), (e.g., robot surgeons, elder care robots), service robots service robots

(e.g., robot assistants, robot tutors, robot nannies, cleaning robots), and (e.g., robot assistants, robot tutors, robot nannies, cleaning robots), and

personal robots personal robots (e.g., robo-companions, robot pets, sex robots). (e.g., robo-companions, robot pets, sex robots).

From: “The Inherent Dangers of Unidirectional From: “The Inherent Dangers of Unidirectional Emotional Bonds between Humans and Social Robots”Emotional Bonds between Humans and Social Robots”

Claim:Claim: social robots will be – and already are – causing us to social robots will be – and already are – causing us to

automatically develop automatically develop unidirectional emotional bondsunidirectional emotional bonds with them; not even with them; not even

hardened roboticists are or will be immune!hardened roboticists are or will be immune!

These emotional bonds can be beneficial to us, but can also be These emotional bonds can be beneficial to us, but can also be harmfulharmful

and and psychologically dangerouspsychologically dangerous, and can, moreover, can be , and can, moreover, can be exploitedexploited (very (very

much in the way that computers can be exploited by hackers)much in the way that computers can be exploited by hackers)

Social robots could create psychological dependencies that would have Social robots could create psychological dependencies that would have

seriousserious consequences for human societies consequences for human societies

We need to start NOW: (1) to investigate the potential dangers, (2) We need to start NOW: (1) to investigate the potential dangers, (2)

attempt to mitigate them in our designs, and (3) develop principles that attempt to mitigate them in our designs, and (3) develop principles that

future law-makers can use to impose clear restrictions on the types of future law-makers can use to impose clear restrictions on the types of

social robots that can be deployedsocial robots that can be deployed

The robots are here!The robots are here!

Social robots are differentSocial robots are different

Industrial Robots Social Robotsfor industrial production for personal use and/or entertainmentrestricted to factory environment not restricted to any environmentmachine-like appearance life-like appearancevery task-specific programming more open-ended programmingmounted or limited mobility unconstrained mobility

DDifferent from industrial robots, which have been around for ifferent from industrial robots, which have been around for

decades, many of these new kinds of robots are decades, many of these new kinds of robots are mobilemobile and, to and, to

varying degrees, varying degrees, autonomousautonomous

Different from industrial robots, social robots are built Different from industrial robots, social robots are built

specifically to connect with humans at a social level; and this specifically to connect with humans at a social level; and this

level often involves human emotions and feelings!level often involves human emotions and feelings!

Overall social robots are different from industrial in important Overall social robots are different from industrial in important

ways:ways:

Social robots are differentSocial robots are different ““A sociable robot is able to A sociable robot is able to communicatecommunicate and and interact with usinteract with us, , understandunderstand

and even and even relate to usrelate to us, in a personal way. It is a robot that is socially , in a personal way. It is a robot that is socially

intelligent intelligent in a human-like wayin a human-like way.” (Breazeal, 2002).” (Breazeal, 2002)

And, in fact, it is possible that even a hardened roboticist likeAnd, in fact, it is possible that even a hardened roboticist like

“Breazeal experienced what might be called a “Breazeal experienced what might be called a maternal connection maternal connection to Kismet; to Kismet;

she certainly describes she certainly describes a sense of connectiona sense of connection with it as with it as more than “mere” more than “mere”

machinemachine. When she graduated from MIT and left the AI Laboratory where she . When she graduated from MIT and left the AI Laboratory where she

had done her doctoral research, the tradition of academic property rights had done her doctoral research, the tradition of academic property rights

demanded that Kismet be left behind in the laboratory that had paid for its demanded that Kismet be left behind in the laboratory that had paid for its

development. What she left behind was the robot 'head' and its attendant development. What she left behind was the robot 'head' and its attendant

software. Breazeal described software. Breazeal described a sharp sense of lossa sharp sense of loss.” (Turkle, 2006).” (Turkle, 2006)

Thus, different from industrial robots, social robots push – Thus, different from industrial robots, social robots push – and are and are

designed and intended to push –designed and intended to push – our “Darwinian buttons” our “Darwinian buttons”

Only good intentions...Only good intentions...

Therapy robotsTherapy robots

Robots animalsRobots animals

Robotic dollsRobotic dolls

... BUT: Are all robots good?... BUT: Are all robots good?

The fact that social robots are already becoming part of our personal lives, The fact that social robots are already becoming part of our personal lives,

requires us to reflect on what exactly happens in our interactions with those requires us to reflect on what exactly happens in our interactions with those

robots, at multiple levelsrobots, at multiple levels

For example, we need to understand what For example, we need to understand what emotional impactemotional impact these robots these robots

will have on us, not only in short term interactions, but in the will have on us, not only in short term interactions, but in the long termlong term!!

While robots certainly can have benefits for humans (e.g., health benefits), While robots certainly can have benefits for humans (e.g., health benefits),

they can also do significant harmthey can also do significant harm

This is due to the This is due to the unidirectional emotional attachment unidirectional emotional attachment relationships that relationships that

humans are likely to form with robots, especially when they interacting with humans are likely to form with robots, especially when they interacting with

them on a regular basisthem on a regular basis

From robo-pets, to robo-friends, to robo-lovers – it's already a fact, not From robo-pets, to robo-friends, to robo-lovers – it's already a fact, not

fiction, and there is much more to come...fiction, and there is much more to come...

The personification of robots...The personification of robots... It is well-known that people have a tendency to name inanimate things It is well-known that people have a tendency to name inanimate things

they like and use frequently (e.g., their car)they like and use frequently (e.g., their car)

Naming, however, comes at a price: it automatically generates a kind of Naming, however, comes at a price: it automatically generates a kind of

intimacy and connectedness to the named objectintimacy and connectedness to the named object

Different from cars, robots are perceived as autonomous agents with the Different from cars, robots are perceived as autonomous agents with the

degree of autonomy determing the extent to which it is perceived as degree of autonomy determing the extent to which it is perceived as

human-like (e.g., Kiesler and Hinds, 2004) – hence, naming only re-human-like (e.g., Kiesler and Hinds, 2004) – hence, naming only re-

inforces what the self-propelled behavior of a robot already does: inforces what the self-propelled behavior of a robot already does:

prompting the inscription of intentionality into an artifact!prompting the inscription of intentionality into an artifact!

And as a result, people will form beliefs that robot are capable of having And as a result, people will form beliefs that robot are capable of having

mental statesmental states, including social emotional states (such as empathy, worry, , including social emotional states (such as empathy, worry,

caring, etc.)caring, etc.)

How do you feel about this?How do you feel about this?

Is it OK to tease robots?Is it OK to tease robots?

Is it OK to date robots?Is it OK to date robots?Is it OK to date robots?Is it OK to date robots?

Is it OK to torture robots?Is it OK to torture robots?

REMEMBER: robots do not care!REMEMBER: robots do not care! The main problem is that The main problem is that robots do not carerobots do not care and this is a problem that we and this is a problem that we

can grasp can grasp intellectuallyintellectually, but not , but not emotionallyemotionally!!

And it is the lack of emotional understanding that the social bond is And it is the lack of emotional understanding that the social bond is

unidirectional, which will ultimately cause us harmunidirectional, which will ultimately cause us harm

There are many examples of how humans anthropomorphize robots, There are many examples of how humans anthropomorphize robots,

project their own mentality into them, and form what look like deep project their own mentality into them, and form what look like deep

emotional unidirectional relationships with thememotional unidirectional relationships with them

Examples include: Examples include:

military robots used in combatmilitary robots used in combat

robot pets used by childrenrobot pets used by children

health care robots used for elder carehealth care robots used for elder care

service robots used in the householdservice robots used in the household

From Garreau’s: “Bots on the Ground”From Garreau’s: “Bots on the Ground”

The most effective way to find and destroy a land mine is to step on it. This has bad results, The most effective way to find and destroy a land mine is to step on it. This has bad results, of course, if you're a human. But not so much if you're a robot and have as many legs as a of course, if you're a human. But not so much if you're a robot and have as many legs as a centipede sticking out from your body.centipede sticking out from your body.

That's why Mark Tilden, a robotics physicist at the Los Alamos National Laboratory, built That's why Mark Tilden, a robotics physicist at the Los Alamos National Laboratory, built something like that. At the Yuma Test Grounds in Arizona, the autonomous robot, 5 feet long something like that. At the Yuma Test Grounds in Arizona, the autonomous robot, 5 feet long and modeled on a stick-insect, strutted out for a live-fire test and worked beautifully, he says. and modeled on a stick-insect, strutted out for a live-fire test and worked beautifully, he says. Every time it found a mine, blew it up and lost a limb, it picked itself up and readjusted to Every time it found a mine, blew it up and lost a limb, it picked itself up and readjusted to move forward on its remaining legs, continuing to clear a path through the minefield. Finally it move forward on its remaining legs, continuing to clear a path through the minefield. Finally it was down to one leg. Still, it pulled itself forward. Tilden was ecstatic. The machine was was down to one leg. Still, it pulled itself forward. Tilden was ecstatic. The machine was working splendidly. The human in command of the exercise, however -- an Army colonel -- working splendidly. The human in command of the exercise, however -- an Army colonel -- blew a fuse.blew a fuse.

The colonel ordered the test stopped.The colonel ordered the test stopped.

Why? asked Tilden. What's wrong?Why? asked Tilden. What's wrong?

The colonel just could not stand the pathos of watching the burned, scarred and crippled The colonel just could not stand the pathos of watching the burned, scarred and crippled machine drag itself forward on its last leg.machine drag itself forward on its last leg.

This test, he charged, was inhumane.This test, he charged, was inhumane.

Ted Bogosh recalls one day in Camp Victory, near Baghdad, when he was a Marine master Ted Bogosh recalls one day in Camp Victory, near Baghdad, when he was a Marine master sergeant running the robot repair shop.sergeant running the robot repair shop.

That day, an explosive ordnance disposal technician walked through his door. The EODs, as That day, an explosive ordnance disposal technician walked through his door. The EODs, as they are known, are the people who -- with their robots -- are charged with disabling Iraq's they are known, are the people who -- with their robots -- are charged with disabling Iraq's most virulent scourge, the roadside improvised explosive device. In this fellow's hands was a most virulent scourge, the roadside improvised explosive device. In this fellow's hands was a small box. It contained the remains of his robot. He had named it Scooby-Doo.small box. It contained the remains of his robot. He had named it Scooby-Doo.

"There wasn't a whole lot left of Scooby," Bogosh says. The biggest piece was its 3-by-3-by-"There wasn't a whole lot left of Scooby," Bogosh says. The biggest piece was its 3-by-3-by-4-inch head, containing its video camera. On the side had been painted "its battle list, its 4-inch head, containing its video camera. On the side had been painted "its battle list, its track record. This had been a really great robot." The veteran explosives technician looming track record. This had been a really great robot." The veteran explosives technician looming over Bogosh was visibly upset. He insisted he did not want a new robot. He wanted Scooby-over Bogosh was visibly upset. He insisted he did not want a new robot. He wanted Scooby-Doo back.Doo back.

"Sometimes they get a little emotional over it," Bogosh says. "Like having a pet dog. It "Sometimes they get a little emotional over it," Bogosh says. "Like having a pet dog. It attacks the IEDs, comes back, and attacks again. It becomes part of the team, gets a name. attacks the IEDs, comes back, and attacks again. It becomes part of the team, gets a name. They get upset when anything happens to one of the team. They identify with the little robot They get upset when anything happens to one of the team. They identify with the little robot quickly. They count on it a lot in a mission."quickly. They count on it a lot in a mission."

From Garreau’s: “Bots on the Ground”From Garreau’s: “Bots on the Ground”

It's common for a soldier to cut out a magazine picture of a woman, tape it to the antenna It's common for a soldier to cut out a magazine picture of a woman, tape it to the antenna and name the bot something like "Cheryl," says Paul Varian, a former Army chief warrant and name the bot something like "Cheryl," says Paul Varian, a former Army chief warrant officer who has served three tours in Iraq with the Robotic Systems Joint Project Office. officer who has served three tours in Iraq with the Robotic Systems Joint Project Office. "There's an awful lot of picture-taking," he says. One guy who married just before "There's an awful lot of picture-taking," he says. One guy who married just before deployment wanted his wife to see the gal who was his constant companion. It was a deployment wanted his wife to see the gal who was his constant companion. It was a PackBot. "One Guard unit got so attached to a development model that we gave it to them. It PackBot. "One Guard unit got so attached to a development model that we gave it to them. It was pretty beat up. They put it in a place of honor in their museum."was pretty beat up. They put it in a place of honor in their museum."

"When we first got there, our robot, his name was Frankenstein" says Sgt. Orlando Nieves, "When we first got there, our robot, his name was Frankenstein" says Sgt. Orlando Nieves, an EOD from Brooklyn. "He'd been in a couple of explosions and he was made of pieces and an EOD from Brooklyn. "He'd been in a couple of explosions and he was made of pieces and parts from other robots." Not only did the troops promote him to private first class, they parts from other robots." Not only did the troops promote him to private first class, they awarded him an EOD badge -- a coveted honor. "It was a big deal. He was part of our team, awarded him an EOD badge -- a coveted honor. "It was a big deal. He was part of our team, one of us. He did feel like family."one of us. He did feel like family."

Humans respond so readily to Kismet, created by Cynthia Breazeal, that graduate students Humans respond so readily to Kismet, created by Cynthia Breazeal, that graduate students working in the lab at night have been known to put up a curtain between themselves and the working in the lab at night have been known to put up a curtain between themselves and the bot, Brooks reports. They couldn't stand the way it seemed to gaze around and stare at bot, Brooks reports. They couldn't stand the way it seemed to gaze around and stare at them. It broke their concentration. These humans are as sophisticated about robots as them. It broke their concentration. These humans are as sophisticated about robots as anyone on Earth. Yet even they are freaked by Kismet's lifelike behavior. "We're anyone on Earth. Yet even they are freaked by Kismet's lifelike behavior. "We're programmed biologically to respond to certain sorts of things," Brooks explains.programmed biologically to respond to certain sorts of things," Brooks explains.

From Garreau’s: “Bots on the Ground”From Garreau’s: “Bots on the Ground”

Even the Roomba does the trick!Even the Roomba does the trick! People seem to develop, over time, a strong sense of gratitude for robots People seem to develop, over time, a strong sense of gratitude for robots

that work for themthat work for them

And they can develop a sense of, if not urge for, reciprocation (i.e., that And they can develop a sense of, if not urge for, reciprocation (i.e., that

they want to do something nice for the robot), even though the robot does they want to do something nice for the robot), even though the robot does

not even know that they are there!not even know that they are there!

Some roomba users, for example (Sung et al., 2007), do things like:Some roomba users, for example (Sung et al., 2007), do things like:

introduce the robot to their parentsintroduce the robot to their parents

bring the Roomba along on travelsbring the Roomba along on travels

clean the floor for the Roombaclean the floor for the Roomba

““I can’t imagine not having him any longer. He’s my BABY!!... When I write I can’t imagine not having him any longer. He’s my BABY!!... When I write

emails about him which I’ve done that as well, I just like him, I call him emails about him which I’ve done that as well, I just like him, I call him

Roomba baby…He’s a sweetie.” (Sung et al., 2007)Roomba baby…He’s a sweetie.” (Sung et al., 2007)

It does not help when...It does not help when... when researchers who work on emotions say that their robots when researchers who work on emotions say that their robots have have

emotions, emotions, implement implement emotions, etc. and use otherwise suggestive emotions, etc. and use otherwise suggestive language (e.g., during research presentations) that any of the control language (e.g., during research presentations) that any of the control processes in these artificats could be similar to identically named human processes in these artificats could be similar to identically named human emotion processesemotion processes

when companies (like I-Robot, Hasbro, and others) say that their toys are when companies (like I-Robot, Hasbro, and others) say that their toys are “recreating the emotions of a cat, an infant, etc.” and when they even give “recreating the emotions of a cat, an infant, etc.” and when they even give their toys names with attributes like “alive” (e.g., Hasbro's “baby alive”) their toys names with attributes like “alive” (e.g., Hasbro's “baby alive”) and “real”and “real”

when companies like I-Robot create a Facebook page for their PackBot when companies like I-Robot create a Facebook page for their PackBot and use the word “I” in a narrative-type style to recount events as if the and use the word “I” in a narrative-type style to recount events as if the robot had experienced themrobot had experienced them

NOTE that all of this helps create and maintain NOTE that all of this helps create and maintain the false beliefthe false belief that there is that there is “somebody at home” in our current robots...“somebody at home” in our current robots...

Instead we should...Instead we should... make it part of a robot's design (appearance make it part of a robot's design (appearance andand behavior alike) behavior alike) that the that the

robot continuously signals, unmistakenly and clearly, to the human robot continuously signals, unmistakenly and clearly, to the human that it that it

is a machineis a machine, , that it does not have emotionsthat it does not have emotions, , that it cannot reciprocatethat it cannot reciprocate

Of course, this is no guarantee that people will not fall for it, but at least Of course, this is no guarantee that people will not fall for it, but at least

they will not fall for it as easilythey will not fall for it as easily

Moreover, it will reduce the likelihood that robots can use a human's Moreover, it will reduce the likelihood that robots can use a human's

attachment to their advantage (e.g., a malicious robot controller might be attachment to their advantage (e.g., a malicious robot controller might be

able to get people to engage in all kinds of irrationalities by threating to able to get people to engage in all kinds of irrationalities by threating to

end its relation with the human)end its relation with the human)

The challenge will be to walk the fine line between making interactions The challenge will be to walk the fine line between making interactions

with robots easier and more natural, while clearly instilling in humans the with robots easier and more natural, while clearly instilling in humans the

belief that robots are man-made machines with no internal life (at least the belief that robots are man-made machines with no internal life (at least the

present ones)present ones)