robots for urban search and rescue

Robots for Urban Robots for Urban Search and RescueSearch and Rescue

Game-Based Design of Game-Based Design of Human-Robot Interfaces for Human-Robot Interfaces for

USRUSR

CHI2004CHI2004

2

Robotic Search and RescueRobotic Search and Rescue

► At least 3 used to search At least 3 used to search the rubble of the World the rubble of the World Trade CenterTrade Center

►Not very sophisticated botsNot very sophisticated bots

wimpey

researcher and three grad students who did this experimentationAll of the robots had microphones to detect voices or other sounds of possible survivors. Some carry thermal cameras for body heatcolor searching cameras distinctive from the gray dust of the debrisnot yet sophisticated enough to roam the rubble. too big and heavy to maneuver the terrainrubble is in vertical piles, which the robots were not capable of climbing

3

Other robotic workOther robotic work

► Life-threatening Life-threatening work, like mine-work, like mine-clearingclearing

► De-mining resulted De-mining resulted in at least 500 in at least 500 deaths from 1996 to deaths from 1996 to 2002, according to a 2002, according to a database maintained database maintained by the Journal of by the Journal of Mine Action. Mine Action.

wimpey

Chiba Univ. in Japan showed this guy offthe Comet III, six legged spider-like action,weighs 1 ton.

4

RoboCupRescueRoboCupRescue

► Simulation Project, Robotics and Simulation Project, Robotics and Infrastructure Project, Infrastructure Project, Simulation League and Robot Simulation League and Robot LeagueLeague

► Integration of these activities Integration of these activities creates digitally-empowered creates digitally-empowered international rescue brigades in international rescue brigades in the future. the future.

wimpey

League Vision for 2050:minimize risk to search and rescue personnel increasing victim survival rates with robots that can: autonomously negotiate compromised/collapsed structures find victims and ascertain their condition produce practical maps of their locations deliver sustenance and communications provide structural shoring ...allowing human rescuers to quickly locate and extract victims. Competition Mission: evaluate mobile robots in a challenging and realistic environment Develop performance metrics Foster collaboration among robotic researchers

5

Inspirational Interface Inspirational Interface

►First Person ShootersFirst Person Shooters►Besides point and click fragging, Besides point and click fragging,

players can …players can … quickly assess the environmentquickly assess the environment make time-critical decisionsmake time-critical decisions control low-level decisionscontrol low-level decisions

This maps very closely with the requirements This maps very closely with the requirements and goals of Urban Search and Rescueand goals of Urban Search and Rescue

wimpey

low level decisions include - moving to new locations- looking at a point of interest

6

brothers? cousins?brothers? cousins?

►Searching a maze or building for Searching a maze or building for enemiesenemies

►Vs.Vs.►Using a robot to explore an arena and Using a robot to explore an arena and

find victimsfind victims

wimpey

in each you have...specifically defined goal,outcome is uncertain,task is emotionally appealing,physics of the "game" are those of real life,and the user understands them intuitively

7

HRIHRI

►Human-Robot Interface awarenessHuman-Robot Interface awareness Human side: location, activities, status, Human side: location, activities, status,

and surroundings of the robotand surroundings of the robot Robot side: human commands it will Robot side: human commands it will

receive, and physics it operates under receive, and physics it operates under

8

awarenesses IIawarenesses II

►From the FPS user interface, we derive From the FPS user interface, we derive the ideas of …the ideas of … Surroundings awarenessSurroundings awareness Status awarenessStatus awareness

wimpey

so from this we have the idea of ...

wimpey

local landmarks, knowledge about how the robot can move around safely

wimpey

state of the robot, power level, orientation, camara posistion

9

The robot designThe robot design

►Magellan Pro robotMagellan Pro robot► 450 MHz PIII 450 MHz PIII ► 16 sensor panels16 sensor panels► Pan-tilt-zoom camaraPan-tilt-zoom camara►Wireless microphoneWireless microphone

wimpey

each panel with bump sensor, IR sensor, and sonar

10

wimpey

So what do you have in a typical FPS display...-large viewing area displaying the game world-status area around perimeter for information regarding health, equipment, ammo-a radar mini-map showing enemies-a separate screen containing a map of the game level (usually for strategic decision-making)

11

Interface incorporationInterface incorporation

►Most control via Most control via keyboardkeyboard

►No “mouselook” No “mouselook” due to hardware due to hardware limitationslimitations

►WASD for robot WASD for robot directiondirection

► UHJK for visionUHJK for vision

wimpey

much like Eric had said about professional computer users, pressing keys a whole lot faster than moving the pointer around

wimpey

gamers often used to WASD directional padcamara was implemented likewise, with UHJK mapping

wimpey

depth perception added to the operator via ground-plane bars, as if laying on the ground at .5 meter intervals

wimpey

operator can set "interest points" by clicking on an image.this has the effect of orienting the robot on local landmarks

12

► Victim interest point Victim interest point ►Dialog window for Dialog window for

entering victim dataentering victim data► Saved time - didn’t have Saved time - didn’t have

to fill out paper formto fill out paper form

► Additions:Additions: Creates on the fly Creates on the fly

webpage of victim datawebpage of victim data

13

Hip InfoHip Info

►Dynamically created webpage could be Dynamically created webpage could be sent to rescue workers equipped with sent to rescue workers equipped with PDAsPDAs

► Instant access to …Instant access to … victim's statusvictim's status image of the victimimage of the victim surrounding areasurrounding area a map to the victima map to the victim

wimpey

In a real urban search and rescue, this information could be shared with recue workers equipped with PDAs-instant access to victim's status, image of the victim and surrounding structure, a map to the victim

14

End GameEnd Game

►Results:Results: Keyboard design idea good and quick, but Keyboard design idea good and quick, but

with a high learning curvewith a high learning curve Large rotational movements had bad Large rotational movements had bad

effectseffects Short-changed surroundings awarenessShort-changed surroundings awareness

wimpey

the trade-off:takes a while to master the keyboard, but more important than usability for a wide range of inexperienced usershad motion errors, disoriented the operator with lagging video, smaller movements matched more of the underlying robot capabilitiesmade for more accurate maps, making victim data web pages more usefuldidnt have to worry about robotic behaviors kicking in (like obstacle avoidance of an obstace that's not really there, only a sensor error from turning too fast)sometimes the operator would be driving the robot with the camera off-center

15

The FutureThe Future

►Harder CompetitionsHarder Competitions More real-life problems, like More real-life problems, like

communication blackoutscommunication blackouts

►With more autonomy comes … more With more autonomy comes … more robots searching!robots searching! RedundancyRedundancy Faster searchFaster search

►Another point of view design neededAnother point of view design needed

wimpey

redundency, if you lose a robot, it's ok, there's a whole team leftfaster coverage of arena means faster search

16

““real time strategy” gamesreal time strategy” games(RTS in nerdspeak)(RTS in nerdspeak)

wimpey

FPS design effective for managing a single avatarhow would you handle multiple robots, though?

wimpey

cloud riding perspectivevisual markers attached to units (status)view of arena is the central window, rather than view from a single agentstatus indicated changes in iconic representations (the blood splot)effective control of multiple units

17

RTS UI requirementsRTS UI requirements

►Accurate mapping and localizationAccurate mapping and localization►Autonomous navigation and searchingAutonomous navigation and searching►Sharing of information amongst robotsSharing of information amongst robots►Ability to identify relevant visual Ability to identify relevant visual

featuresfeatures

wimpey

We are not quite there yet

18

Game OverGame Over(Groan)(Groan)

Credits:Credits:Maxwell, Ward, Heckel, Game-Based Design of Human-Maxwell, Ward, Heckel, Game-Based Design of Human-

Robot Interfaces for Urban Search and RescueRobot Interfaces for Urban Search and RescueNat’l Geographic Today news articleNat’l Geographic Today news article(http://news.nationalgeographic.com/news/(http://news.nationalgeographic.com/news/

2001/09/0914_TVdisasterrobot2001/09/0914_TVdisasterrobot.html).html)

Blizzard Entertainment (Starcraft)Blizzard Entertainment (Starcraft)Microsoft Games (Halo)Microsoft Games (Halo)RoboCupRescue - RoboCupRescue - http://www.isd.mel.nist.gov/RoboCup2003/background.htmhttp://www.isd.mel.nist.gov/RoboCup2003/background.htmPCWorld.com – mine robots news PCWorld.com – mine robots news

((http://www.pcworld.com/news/article/0,aid,110127,00.http://www.pcworld.com/news/article/0,aid,110127,00.aspasp))

robots for urban search and rescue

Documents

robot interfaces

robot leagueintegration

mobile robots

usrchi2004robotic search

goals of urban search

fps user interface

bump sensor

ir sensor