![Page 1: The Mobile Agents 2005 Field Test at MDRS: Planning for Exploration William J. Clancey NASA Ames Research Center Chief Scientist, Human-Centered Computing](https://reader037.vdocument.in/reader037/viewer/2022110205/56649ca75503460f94969169/html5/thumbnails/1.jpg)
The Mobile Agents The Mobile Agents 2005 Field Test at 2005 Field Test at
MDRS:MDRS:Planning for ExplorationPlanning for Exploration
William J. ClanceyWilliam J. ClanceyNASA Ames Research
CenterChief Scientist, Human-Centered
ComputingIntelligent Systems Division
![Page 2: The Mobile Agents 2005 Field Test at MDRS: Planning for Exploration William J. Clancey NASA Ames Research Center Chief Scientist, Human-Centered Computing](https://reader037.vdocument.in/reader037/viewer/2022110205/56649ca75503460f94969169/html5/thumbnails/2.jpg)
Brahms: Bill Clancey, Maarten Sierhuis, Ron van Hoof, Mike Scott, Serge Yentus
MEX: Rick Alena, John Ossenfort, Charles Lee, Ed Walker, Hugo Notario
RIALIST: John Dowding, Jim HieronymusGeology: Abby Semple & Brent GarryERA: Jeff Graham, Rob Hirsh, Eduardo Herrera, Bill
VreugdeRST: Shannon Rupert + 12 participating scientistsNREN: Ray Gilstrap, Kevin BassSciOrg: Dan Berrios, Rich Keller
MA 2005 Participants
![Page 3: The Mobile Agents 2005 Field Test at MDRS: Planning for Exploration William J. Clancey NASA Ames Research Center Chief Scientist, Human-Centered Computing](https://reader037.vdocument.in/reader037/viewer/2022110205/56649ca75503460f94969169/html5/thumbnails/3.jpg)
QuickTime™ and aH.263 decompressor
are needed to see this picture.
Origins of MA Design:Historical & Analog Missions
Apollo’s CapCom, a virtual assistant on the moon
Biologists navigating in Haughton Crater (1999)
Crew struggles with GPS during MDRS5 simulation
QuickTime™ and aSorenson Video decompressorare needed to see this picture.
![Page 4: The Mobile Agents 2005 Field Test at MDRS: Planning for Exploration William J. Clancey NASA Ames Research Center Chief Scientist, Human-Centered Computing](https://reader037.vdocument.in/reader037/viewer/2022110205/56649ca75503460f94969169/html5/thumbnails/4.jpg)
WILLIAM J. CLANCEY, PI & MAARTEN SIERHUIS, PROJECT MANAGER
Human-Systems Integration
• Device Control: sensors/instruments, tools, robots, systems (life, power, computing)
• Data Flow: Queue, sample, store, associate, share, display
• Work Management: Monitor, advise (loc, time, activity), interpret, alert
Voice-Commanded Toolkit
![Page 5: The Mobile Agents 2005 Field Test at MDRS: Planning for Exploration William J. Clancey NASA Ames Research Center Chief Scientist, Human-Centered Computing](https://reader037.vdocument.in/reader037/viewer/2022110205/56649ca75503460f94969169/html5/thumbnails/5.jpg)
Technical Features
• Multi-Agent System
– Functional & personal
– Distributed control & data-processing
• Wireless Computing
– Platform independent
– Tested over 5 km
• Modular Configurations
– Plug & play (Java API)
– E.g., 5 robots, 3 cameras, 5 biosensors, 2 computer sensors, dGPS, headsets
Astronaut Backpack System
![Page 6: The Mobile Agents 2005 Field Test at MDRS: Planning for Exploration William J. Clancey NASA Ames Research Center Chief Scientist, Human-Centered Computing](https://reader037.vdocument.in/reader037/viewer/2022110205/56649ca75503460f94969169/html5/thumbnails/6.jpg)
Mobile Agents Utah Field Test 2005
• 50 Participants; 17 days; 2 NASA centers & several universities
• Multi-robot, auto-recon, & freeform exploration scenarios
• More proactive ERA design
• More proactive RST
• More robust, faster, reusable agent system (> 60)
• Improved alerts, queries (> 90), & auto-associations
• EVA crew less reliant on HabCom crew member
Geologists Exploring with Mobile Agents
![Page 7: The Mobile Agents 2005 Field Test at MDRS: Planning for Exploration William J. Clancey NASA Ames Research Center Chief Scientist, Human-Centered Computing](https://reader037.vdocument.in/reader037/viewer/2022110205/56649ca75503460f94969169/html5/thumbnails/7.jpg)
New MA’05 Operations
• Interacting with the robots to determine their status: Whom are you following? Do you still have comms?
• Establish dependence on robot relay: Boudreaux, join me.
• Contextual robot move commands: Come here. Go there.
• Directed photos: Take a picture of work site 5.
• Astronaut naming of “work stations" (default 1 m areas) within work sites: Call this location 9 with a radius of 5 m.
• Astronauts can refer to names teammates create.
• Requesting locations and navigation information: Where is work site 5? Where am I? List all the locations near work site 6. Where is the next activity?
![Page 8: The Mobile Agents 2005 Field Test at MDRS: Planning for Exploration William J. Clancey NASA Ames Research Center Chief Scientist, Human-Centered Computing](https://reader037.vdocument.in/reader037/viewer/2022110205/56649ca75503460f94969169/html5/thumbnails/8.jpg)
New ERA Hardware
Instrument Arm Robotically Deployed Relay (RDR)
RDR Pin Assembly Thibodeaux (2nd ERA)
![Page 9: The Mobile Agents 2005 Field Test at MDRS: Planning for Exploration William J. Clancey NASA Ames Research Center Chief Scientist, Human-Centered Computing](https://reader037.vdocument.in/reader037/viewer/2022110205/56649ca75503460f94969169/html5/thumbnails/9.jpg)
ERA Spiral Recon Plan
MDRS
![Page 10: The Mobile Agents 2005 Field Test at MDRS: Planning for Exploration William J. Clancey NASA Ames Research Center Chief Scientist, Human-Centered Computing](https://reader037.vdocument.in/reader037/viewer/2022110205/56649ca75503460f94969169/html5/thumbnails/10.jpg)
Mobile Agents Pooh’s CornerComms Network (2005)
Rick Alena, NASA Ames,March 2005
![Page 11: The Mobile Agents 2005 Field Test at MDRS: Planning for Exploration William J. Clancey NASA Ames Research Center Chief Scientist, Human-Centered Computing](https://reader037.vdocument.in/reader037/viewer/2022110205/56649ca75503460f94969169/html5/thumbnails/11.jpg)
Plan B: ATV with ERA Agent & Operator Display
![Page 12: The Mobile Agents 2005 Field Test at MDRS: Planning for Exploration William J. Clancey NASA Ames Research Center Chief Scientist, Human-Centered Computing](https://reader037.vdocument.in/reader037/viewer/2022110205/56649ca75503460f94969169/html5/thumbnails/12.jpg)
Mobile Agents MDRS38Configuration
AstroAgent
HabCom
Agent
ERAAgent
ERA
Hab Crew
Astro
RST
![Page 13: The Mobile Agents 2005 Field Test at MDRS: Planning for Exploration William J. Clancey NASA Ames Research Center Chief Scientist, Human-Centered Computing](https://reader037.vdocument.in/reader037/viewer/2022110205/56649ca75503460f94969169/html5/thumbnails/13.jpg)
Collaborative Interplanetary Science
EVA astronaut 2
ERA Brahms VM
ERA agent
Comm. agent
ERA
MDRS Crew
SpaceSuit_2 Brahms VM
SpaceSuit_1 agent
Astro_1 agent
EVA astronaut 1
SpaceSuit_1 Brahms VM
SpaceSuit_1 agent
Astro_1 agent
SUNY BuffaloRST
Mars SocietyRST
AZ
CA
RST Facilitator
UK/NY
Compendium
MeetingReplay Hab Brahms VM
CapCom agent
ScienceOrganizer
agent
CompendiumAagent
ScienceOrganizerAagent
ScienceOrganizerCompendium
Maarten Sierhuis, RIACS/NASA Ames
![Page 14: The Mobile Agents 2005 Field Test at MDRS: Planning for Exploration William J. Clancey NASA Ames Research Center Chief Scientist, Human-Centered Computing](https://reader037.vdocument.in/reader037/viewer/2022110205/56649ca75503460f94969169/html5/thumbnails/14.jpg)
EVA Data in Science Organizer (WkStn9)
![Page 15: The Mobile Agents 2005 Field Test at MDRS: Planning for Exploration William J. Clancey NASA Ames Research Center Chief Scientist, Human-Centered Computing](https://reader037.vdocument.in/reader037/viewer/2022110205/56649ca75503460f94969169/html5/thumbnails/15.jpg)
From EVA to Desktop
From: [email protected]: MDRS New ImageCollection:
image_collection6Date: April 15, 2005 5:36:55 PM PDTTo: [email protected]: [email protected],
New ImageCollection: image_collection6
EVA Plan: Day_5_Astro_EVA_2_To_Beyond_Poohs_Corner_PlanActivity: WorkInRavine
Creator: AstroOneObject ID: AstroOneModel_IMAGE_COLLECTION_3
TimeStamp: 04/16/2005 01:08:07Latitude: 3824.201417 NORTHLongitude: 11046.8688492 WEST
Link for RST: https://marst.arc.nasa.gov/mdrs2005/183971. . .
![Page 16: The Mobile Agents 2005 Field Test at MDRS: Planning for Exploration William J. Clancey NASA Ames Research Center Chief Scientist, Human-Centered Computing](https://reader037.vdocument.in/reader037/viewer/2022110205/56649ca75503460f94969169/html5/thumbnails/16.jpg)
Science Organizer Data Map
Generated During EVA
Dan Berrios, NASA Ames
![Page 17: The Mobile Agents 2005 Field Test at MDRS: Planning for Exploration William J. Clancey NASA Ames Research Center Chief Scientist, Human-Centered Computing](https://reader037.vdocument.in/reader037/viewer/2022110205/56649ca75503460f94969169/html5/thumbnails/17.jpg)
MDRS IT Network
OpsConWin2003 Server
HabObs
LCDDisplay
MonitorDisplay
WLANAP900+
DirectWay56 Kbps
HabcomLaptop
MEX-NETWLAN Root
Printer
ERA Workstations
MDRS Top FloorNetwork
ShorelinePhone
Processor
MEX WorkStations
LCDDisplay
LCDDisplay
Hub
TP 390
MDRS LowerNetwork
Hub
MDRS WLANDHCP
SW
NREN Satellite Link Network
Phone Lines
HubMA Workstations
Hub
ScienceOrganizer
ERAWorkStations
MEX-Hab(g)WLAN Root
Ext HabWired Network
Rick Alena, NASA Ames,March 2005
![Page 18: The Mobile Agents 2005 Field Test at MDRS: Planning for Exploration William J. Clancey NASA Ames Research Center Chief Scientist, Human-Centered Computing](https://reader037.vdocument.in/reader037/viewer/2022110205/56649ca75503460f94969169/html5/thumbnails/18.jpg)
ISP/IT NETWORK(TAZ to MDRS internal hubs)
MEX Net(hubs to Tropos & EVA Computers)
KaOS Registry(agent locator)
Mobile Agents ComputersMobile Agents Computers(AstroPacks, ERAs, HabCom)(AstroPacks, ERAs, HabCom)
Agents(Personal, Loc, Nav, Plan, Comm)
Peripheral Devices(dGPS, biosens, cameras, robots, db, email)
Mobile Agents Systems Architecture
![Page 19: The Mobile Agents 2005 Field Test at MDRS: Planning for Exploration William J. Clancey NASA Ames Research Center Chief Scientist, Human-Centered Computing](https://reader037.vdocument.in/reader037/viewer/2022110205/56649ca75503460f94969169/html5/thumbnails/19.jpg)
Experiment Observations(Day 5 EVA 2 Beyond Pooh’s Corner)
• Enables independent, simultaneous work, which geologists prefer
• But agents talk over astro conversations– Astro cannot hear that partner
is listening to agent– Astro interleaves response with
conversation– Results in “What did you say?”
• Ready for next level:Voice loop management, alternative feedback, protocol personalization
Overall: System Responsive and
processed data appropriately
![Page 20: The Mobile Agents 2005 Field Test at MDRS: Planning for Exploration William J. Clancey NASA Ames Research Center Chief Scientist, Human-Centered Computing](https://reader037.vdocument.in/reader037/viewer/2022110205/56649ca75503460f94969169/html5/thumbnails/20.jpg)
Example Good Results
Where is Boudreaux?
Boudreaux, stop.
How much time is left?
What is the current activity?
(Astro2) Where is Astronaut 1?
Where am I?
Warning: Time exceeded.
Sorry: Can’t do that.
Warning: Battery is low.
![Page 21: The Mobile Agents 2005 Field Test at MDRS: Planning for Exploration William J. Clancey NASA Ames Research Center Chief Scientist, Human-Centered Computing](https://reader037.vdocument.in/reader037/viewer/2022110205/56649ca75503460f94969169/html5/thumbnails/21.jpg)
Some Not So Good Results
Misunderstood “okay” as volume change command.
Boudreaux is 1.1794857 E 7 meters North-East-East from you.
Thibodeaux, join me (OK)
Astro2 remarks to partner that join command worked; Astro 1 is listening to his agent and didn’t understand; Astro2 repeats what she said, but agent hears this as the command itself!
![Page 22: The Mobile Agents 2005 Field Test at MDRS: Planning for Exploration William J. Clancey NASA Ames Research Center Chief Scientist, Human-Centered Computing](https://reader037.vdocument.in/reader037/viewer/2022110205/56649ca75503460f94969169/html5/thumbnails/22.jpg)
DesertRATS (Sept 05)
• Ames, JSC, Glenn @ Meteor Crater area
• Scout Rover = ERA• 2 Pressurized Suits &
Helmet-mounted display• Power, pressure, coolant
systems data• Standard commands:
Move, Go To, Photo, Print label, Where is…?
• EVA plan monitored
Scout Rover: Mobile Agents allows
voice commanding
![Page 23: The Mobile Agents 2005 Field Test at MDRS: Planning for Exploration William J. Clancey NASA Ames Research Center Chief Scientist, Human-Centered Computing](https://reader037.vdocument.in/reader037/viewer/2022110205/56649ca75503460f94969169/html5/thumbnails/23.jpg)
For more information…
• http://www.marssociety.org/MDRS/fs04– “Mission Info”– Daily reports– Photographs
• http://bill.clancey.name– 1st Space Exploration Conf. – Simulating Activities– Field Science Ethnography– Automating CapCom– Roles for Agent Assistants in
Field Science
• Brahms: www.agentisolutions.com
Special thanks to Abby Semple and Brent Garry