principles of autonomy and decision making 1 brian williams and nicholas roy 16.410/16.413 september...
TRANSCRIPT
![Page 1: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/1.jpg)
Principles of Autonomy and Decision Making
1
Brian Williams and
Nicholas Roy
16.410/16.413
September 8th, 2004
![Page 2: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/2.jpg)
Today’s AssignmentToday’s Assignment
• Read Chapters 1 and 2 of AIMA– “Artificial Intelligence: A Modern Approach”
by Stuart Russell and Peter Norvig– 2nd Edition (not 1st Edition!!)– AIMA is available at the Coop
• Read Chapters 1 and 2 of AIMA– “Artificial Intelligence: A Modern Approach”
by Stuart Russell and Peter Norvig– 2nd Edition (not 1st Edition!!)– AIMA is available at the Coop
![Page 3: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/3.jpg)
OutlineOutline• Objectives
• Agents and Their Building Blocks
• Agent Paradigms
• Principles for Building Agents:– Modeling Formalisms– Algorithmic Principles
• Building an Agent: Fall 03 Projects
• Objectives
• Agents and Their Building Blocks
• Agent Paradigms
• Principles for Building Agents:– Modeling Formalisms– Algorithmic Principles
• Building an Agent: Fall 03 Projects
![Page 4: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/4.jpg)
Course Objective 1: Principles of AgentsCourse Objective 1: Principles of Agents
16.410/13: To learn the modeling and algorithmic building blocks for creating reasoning and learning agents:
• To formulate reasoning problems.• To describe, analyze and demonstrate
reasoning algorithms.• To model and encode knowledge used by
reasoning algorithms.
16.410/13: To learn the modeling and algorithmic building blocks for creating reasoning and learning agents:
• To formulate reasoning problems.• To describe, analyze and demonstrate
reasoning algorithms.• To model and encode knowledge used by
reasoning algorithms.
![Page 5: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/5.jpg)
Course Objective 2: Building Agents
Course Objective 2: Building Agents
16.413: To appreciate the challenges of building a state of the art autonomous explorer:
Fall 03, goals were:• To model and encode knowledge needed to solve
a state of the art challenge.• To work through the process of autonomy systems
integration.• To assess the promise, frustrations and challenges
of using (b)leading art technologies.
Fall 04, stay tuned.
16.413: To appreciate the challenges of building a state of the art autonomous explorer:
Fall 03, goals were:• To model and encode knowledge needed to solve
a state of the art challenge.• To work through the process of autonomy systems
integration.• To assess the promise, frustrations and challenges
of using (b)leading art technologies.
Fall 04, stay tuned.
![Page 6: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/6.jpg)
OutlineOutline• Objectives
• Agents and Their Building Blocks
• Agent Paradigms
• Principles for Building Agents:– Modeling Formalisms– Algorithmic Principles
• Building an Agent: Fall 03 Projects
• Objectives
• Agents and Their Building Blocks
• Agent Paradigms
• Principles for Building Agents:– Modeling Formalisms– Algorithmic Principles
• Building an Agent: Fall 03 Projects
![Page 7: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/7.jpg)
courtesy JPL
``Our vision in NASA is to open the Space Frontier . . . We must establish a virtual presence, in space, on planets, in aircraft and spacecraft.’’ - Daniel S. Goldin, NASA Administrator, May 29, 1996
1. Mission-Oriented Agents1. Mission-Oriented Agents
![Page 8: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/8.jpg)
MESSENGER mission to Mercury
MESSENGER mission to Mercury
VenusSample Return
VenusSample Return
Comet NucleusSample Return
Primitive Bodies MissionsPrimitive Bodies Missions
Pluto/Kuiper Express
Europa Orbiter
EuropaLander
Neptune Orbiter
Titan Explorer
Inner and Outer Planets Missions
Courtesy of JPL
![Page 9: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/9.jpg)
![Page 10: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/10.jpg)
Mars Exploration Rovers – Jan. 2004Mars Exploration Rovers – Jan. 2004
Mission Objective: Learn about ancient water and climate on Mars. • For each rover, analyze a total of 6-12 targets
– Targets = natural rocks, abraded rocks, and soil
• Drive 200-1000 meters per rover
• Take 1-3 panoramas both with Pancam and mini-TES
• Take 5-15 daytime and 1-3 nightime sky observations with mini-TES
Mission Objective: Learn about ancient water and climate on Mars. • For each rover, analyze a total of 6-12 targets
– Targets = natural rocks, abraded rocks, and soil
• Drive 200-1000 meters per rover
• Take 1-3 panoramas both with Pancam and mini-TES
• Take 5-15 daytime and 1-3 nightime sky observations with mini-TES
Mini-TESPancam
Navcam
Rock Abrasion ToolMicroscopic Imager
Mossbauer spectrometer
APXS
![Page 11: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/11.jpg)
Activity NameDurati
on 10 11 12 13 14 15 16 17 18 19 20 21 22 23 0 1 2 3 4 5 6 7 8 9
10 11 12 13 14 15 16 17 18 19 20 21 22 23 0 1 2 3 4 5 6 7 8 9
DTE4.500.75
DTE period DFE
Night Time Rover Operations 16.97 Night Time Rover OperationsSleep Wakeup
Pre-Comm Session Sequence Plan Review
Current Sol Sequence Plan Review1.501.50
Current Sol Sequence Plan Review Current Sol Sequence Plan Review
Prior Sol Sequence Plan Review 2.00 Prior Sol Sequence Plan Review
Real-TIme Monitoring 4.500.75
Real-TIme Monitoring Real-TIme Monitoring
Downlink Product Generation... 2.75 Downlink Product Generation
Tactical Science Assessment/Observation Planning
5.00Tactical Science Assessment/Observation Planning
Science DL Assessment Meeting 1.00Science DL Assessment Meeting
Payload DL/UL Handoffs 0.50 Payload DL/UL Handoffs
Tactical End-of-Sol Engr. Assessment & Planning
5.50Tactical End-of-Sol Engr. Assessment & Planning
DL/UL Handover Meeting 0.50DL/UL Handover Meeting
Skeleton Activity Plan Update 2.50Skeleton Activity Plan Update
SOWG Meeting 2.00 SOWG Meeting
Uplink Kickoff Meeting 0.25 Uplink Kickoff Meeting
Activity Plan Integration & Validation 1.75Activity Plan Integration & Validation
Activity Plan Approval Meeting 0.50 Activity Plan Approval Meeting
Build & Validate Sequences 2.25 Build & Validate Sequences
UL1/UL2 Handover 1.00 UL1/UL2 Handover
Complete/Rework Sequences 2.50 Complete/Rework Sequences
Margin 1 0.75Margin 1
Command & Radiation Approval 0.50 Command & Radiation Approval
Margin 2 1.25 Margin 2
Radiation 0.50 Radiation
MCT Team 7.004.00
rover operation (7 hr)
tactical sci assess & obs planning (5 hr)
tactical end-of-sol eng assess (5 hr)
SOWG mtg (2 hr)
sequence development (5.5 hr)
activity integration & validation (3.5 hr)
sequence integration & validation (4 hr)
One day in the life of a Mars rover
![Page 12: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/12.jpg)
Agent Building BlocksAgent Building Blocks
• Activity Planning• Execution/Monitoring
• Activity Planning• Execution/Monitoring
![Page 13: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/13.jpg)
Cassini Maps Titan courtesy JPL
• 7 year cruise
• ~ 150 - 300 ground operators
•~ 1 billion $
• 7 years to build
Agents As Engineers
•150 million $
•2 year build
• 0 ground ops
Affordable Missions
![Page 14: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/14.jpg)
Four launches in 7 months
Mars Climate Orbiter: 12/11/98Mars Polar Lander: 1/3/99
Stardust: 2/7/99 QuickSCAT: 6/19/98courtesy of JPL
![Page 15: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/15.jpg)
Mars Polar Lander
Launch: 1/3/99 courtesy of JPL
Spacecraft require a good physical commonsense…
![Page 16: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/16.jpg)
Traditional spacecraft commanding
GS,SITURN,490UA,BOTH,96-355/03:42:00.000; CMD,7GYON, 490UA412A4A,BOTH, 96-355/03:47:00:000, ON; CMD,7MODE, 490UA412A4B,BOTH, 96-355/03:47:02:000, INT; CMD,6SVPM, 490UA412A6A,BOTH, 96-355/03:48:30:000, 2; CMD,7ALRT, 490UA412A4C,BOTH, 96-355/03:50:32:000, 6; CMD,7SAFE, 490UA412A4D,BOTH, 96-355/03:52:00:000, UNSTOW; CMD,6ASSAN,490UA412A6B,BOTH, 96-355/03:56:08:000, GV,153,IMM,231,
GV,153; CMD,7VECT, 490UA412A4E,BOTH, 96-355/03:56:10.000, 0,191.5,6.5,
0.0,0.0,0.0,96-350/00:00:00.000,MVR;
SEB,SCTEST,490UA412A23A,BOTH, 96-355/03:56:12.000, SYS1,NPERR; CMD,7TURN, 490UA412A4F,BOTH, 96-355/03:56:14.000, 1,MVR; MISC,NOTE, 490UA412A99A,, 96-355/04:00:00.000, ,START OF TURN;, CMD,7STAR, 490UA412A406A4A,BOTH 96-355/04:00:02.000, 7,1701,
278.813999,38.74; CMD,7STAR, 490UA412A406A4B,BOTH,96-355/04:00:04.000, 8,350,120.455999,
-39.8612; CMD,7STAR, 490UA412A406A4C,BOTH,96-355/04:00:06.000, 9,875,114.162,
5.341; CMD,7STAR, 490UA412A406A4D,BOTH,96-355/04:00:08.000, 10,159,27.239,
89.028999; CMD,7STAR, 490UA412A406A4E,BOTH,96-355/04:00:10.000, 11,0,0.0,0.0; CMD,7STAR, 490UA412A406A4F,BOTH,96-355/04:00:12.000, 21,0,0.0,0.0;
Whats a better paradigm?
![Page 17: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/17.jpg)
Houston, we have a problem ...
courtesy of NASA
• Quintuple fault occurs (three shorts, tank-line and pressure jacket burst, panel flies off) – Diagnosis.
• Mattingly works in ground simulator to identify new sequence handling severe power limitations.– Planning & Resource Allocation
• Mattingly identifies novel reconfiguration, exploiting LEM batteries for power.– Reconfiguration and Repair
• Swaggert & Lovell work on Apollo 13 emergency rig lithium hydroxide unit. – Execution
![Page 18: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/18.jpg)
Remote Agent on Deep Space 1
Started: January 1996Launch: Fall 1998
![Page 19: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/19.jpg)
Remote Agent
• Goal-directed
• First time correct
• projective• reactive
• Commonsense models
• Heavily deductive
Scripts
component models
GoalsGoals
Diagnosis Diagnosis & Repair& Repair
Mission Mission DescriptionDescription ExecutiveExecutive
Planner/Planner/SchedulerScheduler
Mission-levelactions &resources
![Page 20: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/20.jpg)
Remote Agent ExperimentMay 17-18th experiment• Generate plan for course correction and thrust • Diagnose camera as stuck on
– Power constraints violated, abort current plan and replan• Perform optical navigation• Perform ion propulsion thrust
May 21th experiment.• Diagnose faulty device and
– Repair by issuing reset. • Diagnose switch sensor failure.
– Determine harmless, and continue plan. • Diagnose thruster stuck closed and
– Repair by switching to alternate method of thrusting. • Back to back planning
May 17-18th experiment• Generate plan for course correction and thrust • Diagnose camera as stuck on
– Power constraints violated, abort current plan and replan• Perform optical navigation• Perform ion propulsion thrust
May 21th experiment.• Diagnose faulty device and
– Repair by issuing reset. • Diagnose switch sensor failure.
– Determine harmless, and continue plan. • Diagnose thruster stuck closed and
– Repair by switching to alternate method of thrusting. • Back to back planning
See rax.arc.nasa.gov
![Page 21: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/21.jpg)
Course Objective 2Course Objective 2
Plan
ExecuteMonitor &Diagnosis
![Page 22: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/22.jpg)
Agent Building BlocksAgent Building Blocks
• Activity Planning• Execution/Monitoring• Diagnosis• Repair• Scheduling• Resource Allocation
• Activity Planning• Execution/Monitoring• Diagnosis• Repair• Scheduling• Resource Allocation
![Page 23: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/23.jpg)
2. Mobile Agents
Target
Day 4During the
DayScience Activities
Day 1Long-Distance Traverse (<20-50 meters)
Day 2Initial Position; Followed by “Close Approach”
During the DayAutonomous On-Board Navigation
Changes, as needed
Day 2 Traverse Estimated Error Circle
Day 3Science Prep(if Required)
Day 2 Traverse Estimated Error Circle
![Page 24: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/24.jpg)
Multi-Vehicle Path PlanningMulti-Vehicle Path Planning
![Page 25: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/25.jpg)
Nomad Antarctic Explorer
Nomad Antarctic Explorer
Images courtesy of D. Apostopolous, CMU
Of 100 rock samples, Nomad correctly classified 3 as meteorites and incorrectly classified a 4th.
![Page 26: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/26.jpg)
GroundhogGroundhog
Movie courtesy of S. Thrun, Stanford University
![Page 27: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/27.jpg)
Movie courtesy of S. Thrun, Stanford University
![Page 28: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/28.jpg)
Movie courtesy of S. Thrun, Stanford University
![Page 29: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/29.jpg)
![Page 30: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/30.jpg)
![Page 31: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/31.jpg)
Agent Building BlocksAgent Building Blocks
• Activity Planning• Execution/Monitoring• Diagnosis• Repair• Scheduling• Resource Allocation
• Activity Planning• Execution/Monitoring• Diagnosis• Repair• Scheduling• Resource Allocation
• Path Planning• Localization• Map Building
• Path Planning• Localization• Map Building
![Page 32: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/32.jpg)
Plan
ExecuteMonitor &Diagnosis
Locate inWorld
Navigate
Map
![Page 33: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/33.jpg)
3. Agile Agents3. Agile Agents
![Page 34: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/34.jpg)
![Page 35: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/35.jpg)
Agent Building BlocksAgent Building Blocks
• Activity Planning• Execution/Monitoring• Diagnosis• Repair• Scheduling• Resource Allocation
• Activity Planning• Execution/Monitoring• Diagnosis• Repair• Scheduling• Resource Allocation
• Path Planning• Localization• Map Building• Trajectory Design• Policy Construction
• Path Planning• Localization• Map Building• Trajectory Design• Policy Construction
![Page 36: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/36.jpg)
3. Human-Robot Interaction3. Human-Robot Interaction
![Page 37: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/37.jpg)
Agent Building BlocksAgent Building Blocks
• Activity Planning• Execution/Monitoring• Diagnosis• Repair• Scheduling• Resource Allocation
• Activity Planning• Execution/Monitoring• Diagnosis• Repair• Scheduling• Resource Allocation
• Path Planning• Localization• Map Building• Trajectory Design• Policy Construction• Plan Adaptation• Dialogue Management• People Tracking
• Path Planning• Localization• Map Building• Trajectory Design• Policy Construction• Plan Adaptation• Dialogue Management• People Tracking
![Page 38: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/38.jpg)
NASA Exploration Initiative• NASA has developed a bold vision focused on robotic and
combined human-robotic exploration – Response to critical need to augment human presence in space missions
with automated, closely cooperating robotic devices
– Significant cost reduction and safety improvement
![Page 39: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/39.jpg)
Challenge• Autonomous humanoid robots
– Can execute tasks intended for humans
• Human-robot interaction– Understand human tasks
![Page 40: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/40.jpg)
Example: orbit assembly and repair• Robonaut – Humanoid robot for EVA assistance
![Page 41: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/41.jpg)
Example: surface exploration• ERA – EVA robotic assistant follows astronaut and
helps with sample collection, instrument placement
![Page 42: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/42.jpg)
Example Mission Scenario: Task Execution
• Robot walks to its sample area
• Begins collecting samples
• Walks back to astronaut– Stumbles over unseen rock along the way, but
recovers using appropriate limb motions
![Page 43: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/43.jpg)
OutlineOutline• Objectives
• Agents and Their Building Blocks
• Agent Paradigms
• Principles for Building Agents:– Modeling Formalisms– Algorithmic Principles
• Building an Agent: Fall 03 Projects
• Objectives
• Agents and Their Building Blocks
• Agent Paradigms
• Principles for Building Agents:– Modeling Formalisms– Algorithmic Principles
• Building an Agent: Fall 03 Projects
![Page 44: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/44.jpg)
Agent ParadigmsAgent Paradigms
![Page 45: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/45.jpg)
Model-based AgentsModel-based Agents
World Model
![Page 46: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/46.jpg)
Reflexive AgentsReflexive Agents
![Page 47: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/47.jpg)
Goal-Oriented AgentsGoal-Oriented Agents
![Page 48: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/48.jpg)
Utility-Based AgentsUtility-Based Agents
![Page 49: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/49.jpg)
OutlineOutline• Objective
• Agents and Their Building Blocks
• Agent Paradigms
• Principles for Building Agents:– Modeling Formalisms– Algorithmic Principles
• Building an Agent: Fall 03 Projects
• Objective
• Agents and Their Building Blocks
• Agent Paradigms
• Principles for Building Agents:– Modeling Formalisms– Algorithmic Principles
• Building an Agent: Fall 03 Projects
![Page 50: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/50.jpg)
Building Blocks for Agent Paradigms
Building Blocks for Agent Paradigms
• Extensive Reasoning
• Extensive Optimization
• Extensive Learning
• Extensive Reasoning
• Extensive Optimization
• Extensive Learning
![Page 51: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/51.jpg)
Models Underlying The Building Blocks Models Underlying The Building Blocks
• Activity Planning
• Execution/Monitoring
• Diagnosis
• Repair
• Scheduling
• Activity Planning
• Execution/Monitoring
• Diagnosis
• Repair
• Scheduling
• Resource Allocation• Global Path Planning• Task Assignment• Trajectory Design• Policy Construction
• Resource Allocation• Global Path Planning• Task Assignment• Trajectory Design• Policy Construction
Consistency-based:
• State Spaces
• Rules, First Order Logic
• Strips Operators
• Constraint Networks
• Propositional Logic
Consistency-based:
• State Spaces
• Rules, First Order Logic
• Strips Operators
• Constraint Networks
• Propositional Logic
Probabilistic & Utility-based:
• Weighted Graphs
• Linear Programs
• Mixed Integer Programs
• Markov Decision Processes
• Graphical Models
Probabilistic & Utility-based:
• Weighted Graphs
• Linear Programs
• Mixed Integer Programs
• Markov Decision Processes
• Graphical Models
Models:
Building Blocks:
![Page 52: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/52.jpg)
Probability and Decision TheoryProbability and Decision Theory
Sondik, 1971
States S1
Rewards R1
S2
T(sj|ai, si)
Z2
b1Beliefs
Z1Observations
a1Actions
O(zj|si)
b2
HiddenObservable
![Page 53: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/53.jpg)
Probability ModelsProbability Models
• Bayes Rule
• Graphical Models
• Bayes Rule
• Graphical Models
)(
)()|()|(
zp
xpxzpzxp
)(
)()|()|(
zp
xpxzpzxp
p(x)
p(z)
p(y) p(w)
p(v)
![Page 54: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/54.jpg)
Algorithm Instances Of Building Blocks
Algorithm Instances Of Building Blocks
• Activity Planning– Graphplan, SatPlan, Partial
Order Planning• Execution/Monitoring• Diagnosis
– Constraint Suspension• Repair
– Rule-based• Scheduling
– CSP-based• Resource Allocation
– LP-based
• Activity Planning– Graphplan, SatPlan, Partial
Order Planning• Execution/Monitoring• Diagnosis
– Constraint Suspension• Repair
– Rule-based• Scheduling
– CSP-based• Resource Allocation
– LP-based
• Global Path Planning– Roadmap
• Task Assignment• Trajectory Design
– MILP• Policy Construction
– MDP– Reinforcement Learning
• Global Path Planning– Roadmap
• Task Assignment• Trajectory Design
– MILP• Policy Construction
– MDP– Reinforcement Learning
![Page 55: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/55.jpg)
Modeling Cooperative Path Planningas a Mixed Integer Program
Modeling Cooperative Path Planningas a Mixed Integer Program
![Page 56: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/56.jpg)
Cooperative Path Planning:MILP Encoding: Fuel Equation
Cooperative Path Planning:MILP Encoding: Fuel Equation
min = JT = min q’wi + r’vi + p’wNmin = JT = min q’wi + r’vi + p’wNwi, vi wi, vi i=1
N-1
i=1
N-1
slack control vector weighting vectors
slack state vector
past-horizon
terminal cost termtotal fuel calculated over all time instants i
![Page 57: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/57.jpg)
Cooperative Path Planning:MILP Encoding: ConstraintsCooperative Path Planning:
MILP Encoding: Constraints• sij <= wij, etc. State Space Constraints
• si+1 = Asi + Bui State Evolution Equation
• xi <= xmin + Mti1
-xi <= -xmax + Mti2
yi <= ymin + Mti3 Obstacle Avoidance
-yi <= -ymax + Mti4 (for all time i)
tik <= 3 (t introduce IP element)
• Similar equation for Collision Avoidance (for all pairs of vehicles)
• sij <= wij, etc. State Space Constraints
• si+1 = Asi + Bui State Evolution Equation
• xi <= xmin + Mti1
-xi <= -xmax + Mti2
yi <= ymin + Mti3 Obstacle Avoidance
-yi <= -ymax + Mti4 (for all time i)
tik <= 3 (t introduce IP element)
• Similar equation for Collision Avoidance (for all pairs of vehicles)
![Page 58: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/58.jpg)
Uninformed Search:
• Depth First, Breadth First
• Iterative Deepening.
• Backtrack Search
• Backtrack w Forward checking
• Conflict-directed Search
Uninformed Search:
• Depth First, Breadth First
• Iterative Deepening.
• Backtrack Search
• Backtrack w Forward checking
• Conflict-directed Search
Informed Search:
• Single Source Shortest Bath
• Best First Search (A*, Hill Climbing, …)
• Simplex
• Branch and Bound
Informed Search:
• Single Source Shortest Bath
• Best First Search (A*, Hill Climbing, …)
• Simplex
• Branch and Bound
Algorithm Principles Underlying Building Blocks
Algorithm Principles Underlying Building Blocks
![Page 59: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/59.jpg)
Deduction:
• Unification
• Unit Clause Resolution
• Arc Consistency.
• Gaussian Elimination
Relaxation
• Value Iteration
• Reinforcement Learning
Deduction:
• Unification
• Unit Clause Resolution
• Arc Consistency.
• Gaussian Elimination
Relaxation
• Value Iteration
• Reinforcement Learning
Divide and Conquer• Branching• Sub-goaling• Variable Splitting• Dynamic Programming• Uninformed & Informed
Abstraction:• Conflicts• Bounding
Divide and Conquer• Branching• Sub-goaling• Variable Splitting• Dynamic Programming• Uninformed & Informed
Abstraction:• Conflicts• Bounding
Algorithm Principles Underlying Building Blocks
Algorithm Principles Underlying Building Blocks
![Page 60: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/60.jpg)
OutlineOutline• Objectives
• Agents and Their Building Blocks
• Agent Paradigms
• Principles for Building Agents:– Modeling Formalisms– Algorithmic Principles
• Building an Agent: Fall 03 Projects
• Objectives
• Agents and Their Building Blocks
• Agent Paradigms
• Principles for Building Agents:– Modeling Formalisms– Algorithmic Principles
• Building an Agent: Fall 03 Projects
![Page 61: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/61.jpg)
16.413 Project: Example of a Model-based Agent:
• Goal-directed
• First time correct
• projective• reactive
• Commonsense models
• Heavily deductive
Scripts
component models
GoalsGoals
TitanTitanDiagnosis Diagnosis & Repair& Repair
Mission Mission DescriptionDescription
KirkKirkExecutiveExecutive
EuropaEuropaPlanner/Planner/
SchedulerScheduler
Mission-levelactions &resources
![Page 62: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/62.jpg)
16.410/3 Student Teams
• Ground Science Planning for Rovers– Jessica Marquez and Julie Arnold
• Onboard Planning and Execution on Rovers
– Stephen Licht, Andrew Vaughn, Steve Paschall
• Model Based Diagnosis and Execution on Rovers
– Lars Blackmore, Steve Block, Thomas Leauté, Emily Fox
• Model Based Execution on SPHERES I
– Mehdi Alighanbari, Tsoline Mikaelian, Martin Ouimet, Mike Voightmann
• Model Based Execution on SPHERES II
– Robert Effinger, Jacomo Corbo, Jonathon Histon, Sameera Ponda
ME
RS
PH
ER
ES
![Page 63: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/63.jpg)
Rover Testbed Setup
Differential drive
Laser range scannerSonar sensors
Wheel encoders(odometry)
Stereo camera
Inclinometer
GPS receiver
Compass
Antennas for wireless LAN
rFLEXcontroller
rFLEXscreen
Sonarcontrolboard
Sonar sensors
SICK LMS 200 laser scanner
Onboard PC
Firewire card
ttyR ports
Stereo camera
Serial port
Inclinometer
802.11a wireless network adapter
Ethernet card
Left motor
Right motor
• Sensors give information on motion and environment.
• Onboard PC allows for real-time computation and command processing.
Figures from Seung Chung’s Project Description handout
![Page 64: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/64.jpg)
Simple Slipping Scenario
• Initialize in all-stop state
• Command ‘go’ : successful driving
• Command ‘stop’ : successful stopping
• Command ‘go’ : slips
• Command ‘stop’ : successful stopping
• Command ‘go’ : successful driving
![Page 65: Principles of Autonomy and Decision Making 1 Brian Williams and Nicholas Roy 16.410/16.413 September 8 th, 2004](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f015503460f94c16ecd/html5/thumbnails/65.jpg)
Course Objective 1: Principles of Agents
16.410/13: To learn the modeling and algorithmic building blocks for creating reasoning and learning agents:
• To formulate reasoning problems.• To describe, analyze and demonstrate reasoning
algorithms.• To model and encode knowledge used by reasoning
algorithms.