continuous time and resource uncertainty cse 574 lecture spring ’03 stefan b. sigurdsson

Continuous Time and Resource Uncertainty

CSE 574 LectureSpring ’03

Stefan B. Sigurdsson

(Big Mars Rover Picture)

Lecture Overview

Context– Classical planning– The Mars Rover domain– Relaxing the assumptions– Q: What’s so different?

InnovationDiscussion

(Shakey Picture)

Slide shamelessly lifted from http://www.cs.nott.ac.uk/~bsl/G53DIA/Slides/Deliberative-architectures-I.pdf

STRIPS-Like Planning

Propositional logicClosed world assumptionFinite and staticComplete knowledgeDiscrete timeNo exogenous effects

World Description

Attainment – “Win or lose”Conjunctions of positive literals

Goal Description

Conjunctive preconditionSTRIPS operatorsConj. effect (add/delete)InstantaneousSequentialDeterministic

Actions

Plan…

(Big Mars Rover Picture)

The Mars Rover Domain

Robot control, with…– Positioning and navigation– Complex choices (goals and actions)– Rich utility model– Continuous time and concurrency – Uncertain resource consumption– Metric quantities– Very high stakes!

But alone in a finite, static universe

Resources? Metric Quantities?

What are those?

Various flavors:– Exclusive (camera arm)– Shared (OS scheduling) – Metric quantity (fuel, power, disk space)

Uncertainty

Alright, Whatsit Really Mean?

Is This Really A Planning Problem?Better suited to OR/DT-type scheduling?

– Time, resources, metric quantities, concurrency, complicated goals/rewards…

Complex, inter-dependent activities– Select, calibrate, use, reuse, recalibrate sensors– OR-type scheduling can’t handle rich choices

Insight: Maybe we can borrow some tricks?

Can Planners Scale Up?

Large plans– Sequences of ~ 100 actions

Where do we start?– POP? – MDP? – Graph/SATplan?

Can Planners Scale Up?

Large plans– Sequences of ~ 100 actions

Where do we start?– POP? (Branch factors are too big)– MDP? – Graph/SATplan?

Can Planners Scale Up?

Large plans– Sequences of ~ 100 actions

Where do we start?– POP? (Branch factors are too big)– MDP? (Complete policy is too large)– Graph/SATplan?

Can Planners Scale Up?

Large plans– Sequences of ~ 100 actions

Where do we start?– POP? (Branch factors are too big)– MDP? (Complete policy is too large)– Graph/SATplan? (Discrete representations)

Which Extensions First?

Metric quantities– Time– Resources

Resource UncertaintyConcurrency

What about non-determinism? Reasonable for Graphplan?

A (Very Incomplete)Research Timeline

1971 STRIPS (Fikes/Nilson)1989 ADL (Pednault)1991 PEDESTAL (McDermott)1992 UCPOP (Penberthy/Weld) 1992 SENSp (Etzioni et al.) CNLP (Peot/Smith)1993 Buridan (Kushmerick et al.)1994 C-Buridan (Draper et al.) JIC Scheduling (Drummond et al.) HSTS (Muscettola) Zeno (Penb./Weld) Softbots (Weld/Etzioni) MDP (Williamson/Hanks)1995 DRIPS (Haddawy et al.) IxTeT (Laborie/Ghallab)1997 IPP (Koehler et al.)

Not implemented ADL impl.

SensingConformant

Contingent

Planning + schedulingMetric time/resources

Safe planningDec. theory goalsUncertain utility

Shared resources

1998 PGraphplan (Blum/Langford) Weaver (Blythe) PUCCINI (Golden) CGP (Smith/Weld) SGP (Weld et al.) Pgraphplan (Blum/Langford)1999 Mahinur (Onder/Pollack) ILP-PLAN (Kautz/Walzer) TGP (Smith/Weld) LPSAT (Wolfman/Weld)2000 T-MDP (Boyan/Littman) HSTS/RA (Jónsson et al.)

Since then?

Uncertain/dynamicSensing

Conformant

ContingentResources

Resources

Domain Assumptions

Expressive logicNon-determinism

ObservationGoal modelPlan utility

Durative actionsComplex concurrence

Continuous timeMetric quantitiesBranching factor

Resource uncertaintyResource constraints

Goal selectionSafe planning

Exogenous events

STR

IPS

UC

POP

CG

PC

NLP

SEN

SpB

urid

anW

eave

rC

-Bur

idan

MD

PPO

-MD

PS-

MD

PT-

MD

PF-

MD

PLP

SAT

Mar

s Rov

er

Classical

Bleeding edge

Select contingencies

Serialized goals?

Brain-teaser: Domain Spec

State space S– Cartesian product of continuous and discrete axes

(time, position, achievements, energy…)

Initial state si– Probability distribution

Domain theory– Concurrent, non-deterministic, uncertain

What else?(S, si, , …)

Brain-teaser: Kalman Filters

Curiously missing from the paper we read (?)

1983 Kalman filters paper: Voyager enters Jupiter orbit through a 30 second window after 11 years in space

Hugh Durrant-Whyte’s robots

Why not for the Mars Rover?

Context Summary

Complex, exciting domainPushes the planning envelope

– Expression– Scaling

Where do we start?

Lecture Overview

ContextInnovation

– Just-in-case planning– Incremental contingency planning

Discussion

Just-In-Case Planning

Motivated by domain characteristics– Metric quantities – Large branch factors

Implications – Not plan, not policy– Expanded plan

What about concurrency?

Branch Heuristics

Most probable failure point (scheduling)Highest utility branch point (planning)

What is the intrinsic difference?

When To Execute A Contingency?

Incremental Contingency Planning AlgorithmInput: Domain description and master planOutput: Highest-utility branch pointAlgorithm:

– Compute value, estimate resources during master plan– Approximate branch point utilities– Select highest-utility branch point– Solve w/ new initial, goal conditions– Repeat while necessary

Branch Utility Approximation

… without constructing plan– Construct a plan graph– Back-propagate utility functions through plan

graph, instead of regression searching– Compute branch point utilities throughout input

plan

Back-Propagating Distributions

Mausam:

“Some parts of the paper are tersely written, which make it a little harder to understand. I got quite confused in the discussion of utility propagation. It would have been nicer had they given some theorems about the soundness of their method.”

Well, me too

Back-Propagating Distributions

A

C

D

B

E

(1, 5)

(3, 3)

(10, 15)

(10, 15)

(2, 2)

p

s

q

r

t

g

g’

A

C

D

B

E

(1, 5)

(3, 3)

(10, 15)

(10, 15)

(2, 2)

p

s

q

r

t

g

g’

1

5

Back-Propagating Distributions

A

C

D

B

E

(1, 5)

(3, 3)

(10, 15)

(10, 15)

(2, 2)

p

s

q

r

t

g

g’

1

5

Back-Propagating Distributions

5

15

A

C

D

B

E

(1, 5)

(3, 3)

(10, 15)

(10, 15)

(2, 2)

p

s

q

r

t

g

g’

1

5

5

15

5

25

Back-Propagating Distributions

A

C

D

B

E

(1, 5)

(3, 3)

(10, 15)

(10, 15)

(2, 2)

p

s

q

r

t

g

g’

1

5

5

15

12

5

25

Back-Propagating Distributions

A

C

D

B

E

(1, 5)

(3, 3)

(10, 15)

(10, 15)

(2, 2)

p

s

q

r

t

g

g’

1

5

5

15

12

r

12

t

12

5

25

Back-Propagating Distributions

A

C

D

B

E

(1, 5)

(3, 3)

(10, 15)

(10, 15)

(2, 2)

p

s

q

r

t

g

g’

1

15

5

5

15

12

rr

12

t

12

5

25

Back-Propagating Distributions

A

C

D

B

E

(1, 5)

(3, 3)

(10, 15)

(10, 15)

(2, 2)

p

s

q

r

t

g

g’

1

15

5

5

15

12

rr

12

t

12

15

t

Back-Propagating Distributions

5

25

15

A

C

D

B

E

(1, 5)

(3, 3)

(10, 15)

(10, 15)

(2, 2)

p

s

q

r

t

g

g’

1

15

5

5

15

12

rr

12

t

12

5

251

15

t+

Back-Propagating Distributions

15

t

25

6

A

C

D

B

E

(1, 5)

(3, 3)

(10, 15)

(10, 15)

(2, 2)

p

s

q

r

t

g

g’

1

15

5

5

15

12

rr

12

t

12

Back-Propagating Distributions

15

5

25+

15

t

15

t

25

6

15

5

251

15

t+

25

6

A

C

D

B

E

(1, 5)

(3, 3)

(10, 15)

(10, 15)

(2, 2)

p

s

q

r

t

g

g’

1

15

5

5

15

12

rr

12

t

12

Back-Propagating Distributions

15

5

25+

15

t

15

t

25

6

15

5

251

15

t+

25

6

15

r+

A

C

D

B

E

(1, 5)

(3, 3)

(10, 15)

(10, 15)

(2, 2)

p

s

q

r

t

g

g’

1

15

5

5

15

12

rr

12

t

12

Back-Propagating Distributions

15

5

25+1

5

t

25

6

15

5

251

15

t+

25

6

15+

15

t

A

C

D

B

E

(1, 5)

(3, 3)

(10, 15)

(10, 15)

(2, 2)

p

s

q

r

t

g

g’

1

15

5

5

15

12

rr

12

t

12

Back-Propagating Distributions

15

5

25+

5

t

15

t

25

6

15

5

251

15

t+

25

6

15+

1 18+

A

C

D

B

E

(1, 5)

(3, 3)

(10, 15)

(10, 15)

(2, 2)

p

s

q

r

t

g

g’

1

15

5

5

15

12

rr

12

t

12

Back-Propagating Distributions

15

5

25+

5

t

15

t

25

6

15

5

251

15

t+

25

6

15+

1 18+

15 25

A

C

D

B

E

(1, 5)

(3, 3)

(10, 15)

(10, 15)

(2, 2)

p

s

q

r

t

g

g’

1

15

5

5

15

12

rr

12

t

12

Back-Propagating Distributions

15

5

25+

5

t

15

t

25

6

15

5

251

15

t+

25

6

15+

1 18+

8

15 25

A

C

D

B

E

(1, 5)

(3, 3)

(10, 15)

(10, 15)

(2, 2)

p

s

q

r

t

g

g’

1

15

5

5

15

12

rr

12

t

12

Back-Propagating Distributions

15

5

25+

5

t

15

t

25

6

15

5

251

15

t+

25

6

15+

1 18+

8

15 25

18

A

C

D

B

E

(1, 5)

(3, 3)

(10, 15)

(10, 15)

(2, 2)

p

s

q

r

t

g

g’

1

15

5

5

15

12

rr

12

t

12

Back-Propagating Distributions

15

5

25+

5

t

15

t

25

6

15

5

251

15

t+

25

6

15+

1 18+

8

15 25

18

(CDE)

A

C

D

B

E

(1, 5)

(3, 3)

(10, 15)

(10, 15)

(2, 2)

p

s

q

r

t

g

g’

1

15

5

5

15

12

rr

12

t

12

Back-Propagating Distributions

15

5

25+

5

t

15

t

25

6

15

5

251

15

t+

25

6

15+

1 18+

8

15 25

18

(CDE)

A

C

D

B

E

(1, 5)

(3, 3)

(10, 15)

(10, 15)

(2, 2)

p

s

q

r

t

g

g’

1

15

5

5

15

12

rr

12

t

12

Back-Propagating Distributions

5

t

15

5

251

15

t+

25

6

15+

1 18+

15 25

[(CDE)(ABDE)]

[(DCE)(AB)(DABE)]

A

C

D

B

E

(1, 5)

(3, 3)

(10, 15)

(10, 15)

(2, 2)

p

s

q

r

t

g

g’

1

15

5

5

15

12

rr

12

t

12

Back-Propagating Distributions

5

t

15

5

251

15

t+

25

6

15+

1 18+

15 25

18

(CDE, ABDE)

6

25

16

6

25 26

(DCE, AB, DABE)

5

Utility Estimation

p

s

18

(CDE, ABDE)

6

25

16

6

25 26

(DCE, AB, DABE)

5

Utility Estimation

p

s

18

(CDE, ABDE)

6

25

16

6

25 26

(DCE, AB, DABE)

5

16

6

25

(DCE, ABDE)

MAX operator:

Utility Estimation

p

s

18

(CDE, ABDE)

6

25

16

6

25 26

(DCE, AB, DABE)

5

16

6

25

(DCE, ABDE)

MAX operator:

(Then combine w/Monte Carlo results)

Lecture Overview

ContextInnovationDiscussion

– Q: Evaluation? Inference?

Evaluation

Optimal branch selection? (Greedy…)

Incremental Contingencies…

Sometimes adding one contingency at a timeis non-optimal

Examples?

Incremental Contingencies…

1.0

0

Rain

Shine

0.5

0.5

0

1.0

Work Go clim

bing

Exercis

e

Sometimes adding one contingency at a timeis non-optimal

Evaluation

Optimal branch selection?What else?

Inference

Where can we take these ideas?What can we add to them?

Inference

Where can we take these ideas?What can we add to them?

Optimal branch selectionOptimistic branchingMutexes in plan graphNoisy/costly sensors