Introduction to AI Robotics (MIT Press) Chapter 1 1
1 From Teleoperation to Autonomy
• Define Intelligent Robot
• Be able to describe at least two differences between AI and engineering approaches to robotics
• Be able to describe the difference between telepresence and semi-autonomous control
• Have some feel for the history and societal impact of robotics
History-AI-EngineeringTeleop-Motivation-Components-Problems-AlternativesCase StudiesProgrammingSummaryReview
Introduction to AI Robotics (MIT Press) Chapter 1 2
1 Intelligent Robot
• Mechanical creature which can function autonomously
– Mechanical= built, constructed
– Creature= think of it as an entity with its own motivation, decision making processes
– Function autonomously= can sense, act, maybe even reason; doesn’t just do the same thing over and over like automation
History-AI-EngineeringTeleopCase StudiesProgrammingSummaryReview
Introduction to AI Robotics (MIT Press) Chapter 1 3
1 What are Robots?• Autonomous mechanical creatures
– Capek 1921: R.U.R.
• Intelligent because teleoperation doesn’t work, doesn’t scale
• Physically situated, but now software agents or softbots– Principles from robotics influenced AI
community, esp. planning
– Combines programming, networks, operating systems, algorithms, … everything about CS into a system (the ultimate software engineering project)
www.fradulent.org/rur.htm
History-AI-EngineeringTeleopCase StudiesProgrammingSummaryReview
Introduction to AI Robotics (MIT Press) Chapter 1 4
1 Robots Constantly in the Press
www.sony.com
www.irobot.com
courtesy of Honda
courtesy of MIT AI Lab
History-AI-EngineeringTeleopCase StudiesProgrammingSummaryReview
Introduction to AI Robotics (MIT Press) Chapter 1 5
1 Less Famous Cousins at WTC
Inuktun microTracks
½ iRobot PackBot
Introduction to AI Robotics (MIT Press) Chapter 1 6
1 Why Robots? Dirty, Dangerous, Dull Tasks
• JV2010, TRADOC, JFCOM, all branches even down to the organic level– Reconnaissance, MOUT, denial of area, consequence management,
logistics, demining
Replace Humans with Robots
www.friendlymachines.com
History-AI-EngineeringTeleopCase StudiesProgrammingSummaryReview
Introduction to AI Robotics (MIT Press) Chapter 1 7
1 Why Robots? Better Than Bio
• Robots at WTC…– voids smaller than person
could enter
– voids on fire or oxygen depleted
• NBC Response– Lose ½ cognitive attention
with each level of protection• Level A=12.5% of normal
ability
Do Things that Living Things Can’t
Void on fire
Void:1’x2.5’x60’History-AI-EngineeringTeleopCase StudiesProgrammingSummaryReview
Introduction to AI Robotics (MIT Press) Chapter 1 8
1 Major Robot Modalities: UAV, UGV, UUV
• Unmanned Aerial Vehicles– drones since Vietnam: Global Hawk, UCAV
– easy: nothing to hit
– hard: mission sensing, human-in-the-loop control
• Unmanned Ground Vehicles– since 1967
– easy: can always stop and think, a priori maps
– hard: perceiving, e.g., light vegetation vs. wall
• Unmanned Underwater Vehicles– ROVs since 1960s
– easy: run tethers
– hard: platform operation in unfriendly environment
Mobility (platform), Perception,Communications +HRI, Control (Intelligence), Power
History-AI-EngineeringTeleopCase StudiesProgrammingSummaryReview
Introduction to AI Robotics (MIT Press) Chapter 1 9
1 A Brief History…
Introduction to AI Robotics (MIT Press) Chapter 1 10
1 Industrial Manipulators
• “Tommy” type of robots: deaf, dumb, and blind
• High precision, fast repetition
• Usually no sensing of the environment– Welding can be off by an inch…
History-AI-EngineeringTeleopCase StudiesProgrammingSummaryReview
Introduction to AI Robotics (MIT Press) Chapter 1 11
1 3 Ways of Controlling a Robot
• “RC-ing”– you control the robot
– you can view the robot and it’s relationship to the environment
– ex. radio controlled cars, bomb robots
– operator isn’t removed from scene, not very safe
HistoryTeleop-Motivation-Components-Problems-AlternativesCase StudiesProgrammingSummaryReview
Introduction to AI Robotics (MIT Press) Chapter 1 12
1 3 Ways of Controlling a Robot
• teleoperation– you control the robot
– you can only view the environment through the robot’s eyes
– don’t have to figure out AI
HistoryTeleop-Motivation-Components-Problems-AlternativesCase StudiesProgrammingSummaryReview
Introduction to AI Robotics (MIT Press) Chapter 1 13
1 3 Ways of Controlling a Robot
• semi- or full autonomy– you might control the robot sometimes
– you can only view the environment through the robot’s eyes
– ex. Sojouner with different modes
– human doesn’t have to do everything
HistoryTeleop-Motivation-Components-Problems-AlternativesCase StudiesProgrammingSummaryReview
Introduction to AI Robotics (MIT Press) Chapter 1 14
1 Components of a Telesystem(after Uttal 89)
• Local – display
– Local control device
• Communication• Remote
– sensor
– mobility
– effector
– power
• Local – display
– Local control device
• Communication• Remote
– sensor
– mobility
– effector
– power
Display
Control
Sensor
Mobility
Effector
Power
Communi-cation
Local
Remote
Introduction to AI Robotics (MIT Press) Chapter 1 15
1 Example
Local
Remote
HistoryTeleop-Motivation-Components-Problems-AlternativesCase StudiesProgrammingSummaryReview
Introduction to AI Robotics (MIT Press) Chapter 1 16
1 Typical Run
HistoryTeleop-Motivation-Components-Problems-AlternativesCase StudiesProgrammingSummaryReview
Introduction to AI Robotics (MIT Press) Chapter 1 17
1 Problems That You Saw• no feedback, couldn’t really tell that the robot was
stuck but finally got free– robot doesn’t have “proprioception” or internal sensing to tell
you what the flippers were doing. No crunching noises, no pose widget to show the flippers
• no localization, mapping-> no idea how far traveled• partial solution: better instrumentation (but can’t do dead
reckoning well)
– operator doesn’t have an external viewpoint to show itself relative to the environment
• solution: two robots, one to spot the other
• communications dropout, even though ~3 meters away• lighting conditions went from dark to very bright
– hard for computer vision or human to adjust
HistoryTeleop-Motivation-Components-Problems-AlternativesCase StudiesProgrammingSummaryReview
Introduction to AI Robotics (MIT Press) Chapter 1 18
1 But good for unmodeled events
HistoryTeleop-Motivation-Components-Problems-AlternativesCase StudiesProgrammingSummaryReview
Introduction to AI Robotics (MIT Press) Chapter 1 19
1 Communications is Important:DarkStar+7 seconds=DarkSpot
• 7 second communications lag (satellite relay)
• “interruption” lag on part of operator
HistoryTeleop-Motivation-Components-Problems-AlternativesCase StudiesProgrammingSummaryReview
Introduction to AI Robotics (MIT Press) Chapter 1 20
1 Predator:~7:1 human to robot ratio
• 4 people to control it (52-56 weeks of training)– one for flying
– two for instruments
– one for landing/takeoff
• plus maintenance, sensor processing and routing
• lack of self-awareness– in Kosovo, come along side in helicopter and shoot down
Leo’s unofficialPredator page
HistoryTeleop-Motivation-Components-Problems-AlternativesCase StudiesProgrammingSummaryReview
Introduction to AI Robotics (MIT Press) Chapter 1 21
1 Summary of Teleop Problems
• cognitive fatigue
• communications dropout
• communications bandwidth
• communications lag
• too many people to run one robot (hidden cost)
HistoryTeleop-Motivation-Components-Problems-AlternativesCase StudiesProgrammingSummaryReview
Introduction to AI Robotics (MIT Press) Chapter 1 22
1 Telesystems Best Suited For:
• the tasks are unstructured and not repetitive
• the task workspace cannot be engineered to permit the use of industrial manipulators
• key portions of the task require dexterous manipulation, especially hand-eye coordination, but not continuously
• key portions of the task require object recognition or situational awareness
• the needs of the display technology do not exceed the limitations of the communication link (bandwidth, time delays)
• the availability of trained personnel is not an issue
HistoryTeleop-Motivation-Components-Problems-AlternativesCase StudiesProgrammingSummaryReview
Introduction to AI Robotics (MIT Press) Chapter 1 23
1 Teleop Improvements: Telepresence
• Telepresence– improves human control, reduces simulator sickness
and cognitive fatigue by providing sensory feedback to the point that teleoperator feels they are “present” in robot’s environment
– increases demands on bandwidth
HistoryTeleop-Motivation-Components-Problems-AlternativesCase StudiesProgrammingSummaryReview
Introduction to AI Robotics (MIT Press) Chapter 1 24
1 Teleop Improvements:Supervisory Control
• Semi-autonomous– Supervisory Control
• human is involved, but routine or “safe” portions of the task are handled autonomously by the robot
• is really a type of mixed-initiative
• Shared Control/ Guarded Control– human initiates action, interacts with remote by adding
perceptual inputs or feedback, and interrupts execution as needed
– robot may “protect” itself by not bumping into things
• Traded Control– human initiates action, does not interact
HistoryTeleop-Motivation-Components-Problems-AlternativesCase StudiesProgrammingSummaryReview
Introduction to AI Robotics (MIT Press) Chapter 1 25
1 Teleop Improvements:Mixed-Initiative
• Levels of Initiative
– do only what told to do (teleoperation)
– recommend or augment (cognitive augmentation)
– act and report
– act on own and supervise itself (autonomy)
HistoryTeleop-Motivation-Components-Problems-AlternativesCase StudiesProgrammingSummaryReview
Introduction to AI Robotics (MIT Press) Chapter 1 26
1 “No Hands Across America”
• 1994
• CMU NavLab
• Pittsburgh to San Diego– 2897 miles total
– 2849 autonomously
• Autonomous or Mixed-Initiative?
HistoryTeleop-Motivation-Components-Problems-AlternativesCase StudiesProgrammingSummaryReview
Introduction to AI Robotics (MIT Press) Chapter 1 27
1 Mixed-Initiative Matching Game
Level of Initiative
• teleoperation
• cognitive augmentation
• act and report
• act on own
Application
• janitorial robot• medical robot
(telemedicine)• high flying surveillance
drone• combat aerial vehicle• resupply system for
bringing water to fire fighters
• guard dog robot• “nurse” robot
HistoryTeleop-Motivation-Components-Problems-AlternativesCase StudiesProgrammingSummaryReview
Introduction to AI Robotics (MIT Press) Chapter 1 28
1 Collaborative Teleoperation
Urban is stuck, Inuktun can’t help from current perspective
1. Driven off 3rd floor2. Hoisted to 2nd floor by tether3. Has better view, changing
configuration & rocking extend view
mpg: June 2, 2000 SRDR Miami Beach: view from Inuktun as it falls mpg: June 2, 2000 SRDR Miami Beach: view from Inuktun from hoisted position
1
2
3
still: June 2, 2000 SRDR Miami Beach
Introduction to AI Robotics (MIT Press) Chapter 1 29
1 2000 AAAI Mobile Robot
• 2 robots helping each other reduced collision errors, sped up time navigating confined space, righting
HistoryTeleopCase StudiesProgrammingSummaryReview
Introduction to AI Robotics (MIT Press) Chapter 1 30
1 Example:Mixed-Initiative & Collab. Teleop
• 9/2000 DARPA Tactical Mobile Robots demonstration
• Robot used an intelligent assistant agent to look for signs of snipers hiding in urban rubble
– motion– skin color – difference in color– thermal (IR camera)
• Human navigated mother robot using viewpoint of 2nd robot (not in picture)
• Once deposited the human moved the daughter robot, and either saw a sniper or was alerted by the agent
HistoryTeleopCase StudiesProgrammingSummaryReview
Introduction to AI Robotics (MIT Press) Chapter 1 31
1 AI provides the “other stuff”
• knowledge representation• understanding natural langugage• learning• planning and problem solving• inference• search• vision
HistoryTeleopCase StudiesProgrammingSummaryReview
Introduction to AI Robotics (MIT Press) Chapter 1 32
1 Example User Expectation of AI
• Proposed Goal: 1:1 soldier:any robot, where 1 soldier is responsible for 1 or more active robots but does not have to pay continuous attention to them.
4 specialists:1 vehicle
1 specialist:1 vehicle
1 specialist:1 modality 1 specialist 1 soldier
1 specialist:n vehicles
MAV-UGVcooperativemonitoring
Flocks of MAVs
UAVs astheater assets
MAVs asorganic
assets
Field recon-figurable
UUVs
Young Frankenstein
Introduction to AI Robotics (MIT Press) Chapter 1 33
1 More Reasonable Expectionsagents with“tactical”autonomy,toolkits
Mass-produceddedicated
agents
Field-reconfigurable
agents
Cooperating“pack” or “herd”
agents
Vehicle success isstill based on human,but robot is “in front”
Human intermittentattention as teamcoordinator, notwith individuals
Human primaryresponsibility as a
tool builder, expertadvisor. Peer-level
communication
Consolidation
Dedicated Autonomy Systems
Reconfigurable AutonomySystems
HistoryTeleopCase StudiesProgrammingSummaryReview
Introduction to AI Robotics (MIT Press) Chapter 1 34
1 Programming Notes• You always need telesystem or human intervention as a
backup – at some point a human will need to take control– embed in your design
• “Roboticists automate what is easy and leave the rest to the human”- Don Norman
• The user interface is absolutely critical– User interface make up 60% of commercial code– Useful= is the program purpose useful?
• usually given to designer via specifications and requirements
– Usable= can a human use it efficiently?• designer must conduct usability studies • avoid “if I can use it, some one else will”
HistoryTeleopCase StudiesProgrammingSummaryReview
Introduction to AI Robotics (MIT Press) Chapter 1 35
1 Example of How an “Internal” Display Can Hurt
• gamer joystick plus laptop with video & audio• robot state: battery, comms, orientation, camera, encoders• was not used on rubble pile at WTC because it scared off rescuers: too complicated, too long to boot,
too toy– now integrated with Land Warrior– used in Afghanistan
iRobot PackBotvideo, FLIR, 2 way audio
HistoryTeleopCase StudiesProgrammingSummaryReview
Introduction to AI Robotics (MIT Press) Chapter 1 36
1 Summary
• Teleoperation arose a partial solution to autonomy– cognitive fatigue, high comms bandwidth, long
delays, and many:one human to robot ratios– Telepresence tries to reduce cognitive fatigue
through enhanced immersive environments– Semi-autonomy tries to reduce fatigue, bandwidth by
delegating portions of the task to robot• mixed-initiative
• Teleop isn’t simple and improvements aren’t just “better user interfaces”
HistoryTeleopCase StudiesProgrammingSummaryReview
Introduction to AI Robotics (MIT Press) Chapter 1 37
1 Review Questions
• What is an intelligent robot?
• What is the difference between engineering and AI robotics?
• What are 3 types of control?
• What are the parts of a telesystem?
• What are problems with teleoperation?
• What’s the difference between telepresence and semi-autonomous control?
• What are the levels of initiative (mixed-initiative)?
• What are alternatives to traditional teleoperation?
HistoryTeleopCase StudiesProgrammingSummaryReview