Download - Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto
![Page 1: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/1.jpg)
Planning with LoopsSome New Results
Yuxiao (Toby) Hu and Hector LevesqueUniversity of Toronto
![Page 2: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/2.jpg)
2
Outline
• Introduction
• Representing plans with loops– FSA plan: a type of finite state controller
• Constructing plans with loops– Search in the space of FSA plans
• Potential for improvements
• Conclusion
![Page 3: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/3.jpg)
3
The Planning Problem• Finitely many functional fluents, f1,...,fm,
whose domain may be finite or infinite;• Finitely many actions, a1,...,an (world-
changing or sensing);• Initial state: the possible values of each
fluent;• Goal: achieve some goal condition in all
contingencies.
Like contingent planning: solve a class of problems
Incomplete initial state with possibly infinite cases
![Page 4: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/4.jpg)
4
Motivating Example(a variant of striped-tower in Srivastava et al. 2008)
Fluents:– stackA (a list of block colors)– stackB (a list of block colors)– stackC (a list of block colors)– hand (empty/red/blue)
World changing actions:– pickA, pickB, putB, putC;
Sensing actions– testA? (empty/nonempty)– testB? (empty/nonempty)– testH? (red/blue)
![Page 5: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/5.jpg)
5
Motivating Example(a variant of striped-tower in Srivastava et al. 2008)
• Initially: – stackA=[blue,red,red,blue]– stackB=[ ]– stackC=[ ]– hand=empty
• Goal:– stackA=[ ]– stackB=[ ]– striped(stackC)– hand=empty
striped(X) is trueiff
X=[red,blue,…,red,blue]
![Page 6: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/6.jpg)
6
Example 1
A linear solution1.pickA;2.putB;3.pickA;4.putC;5.pickB;6.putC;7.pickA;8.putC;9.pickA10.putC.
![Page 7: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/7.jpg)
7
Example 2
![Page 8: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/8.jpg)
8
Example 2 (cont.)pickA;CASE testH? OF - red: putC; pickA; CASE testH? OF -red: putB; ... ... -blue: putC; ... ... - blue: putB; pickA; CASE testH? OF -red: putC; ... ... -blue: putB; ... ...
![Page 9: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/9.jpg)
9
Example 3
Question: Is there a generalized plan solving all problems in this class?
![Page 10: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/10.jpg)
10
Some of the Existing Approaches
• KPLANNER (Levesque 2005)generates robot programs by winding found conditional plans.
• Aranda (Srivastava et al. 2008)obtains generalized plans by winding an abstracted example plan.
• loopDISTILL (Winner and Veloso 2007)learns a “dsPlanner” by merging matching sub-plans of an example partial-order plan.
![Page 11: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/11.jpg)
11
Outline
• Introduction
• Representing plans with loops– FSA plan: a type of finite state controller
• Constructing plans with loops– Search in the space of FSA plans
• Potential for improvements
• Conclusion
![Page 12: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/12.jpg)
12
Plan Representation – Robot Programs
A robot program is defined inductively by–nil;–seq(A,P);–case(A,[R1:P1,…,Rn:Pn]);–loop(P,Q).
(Levesque 1996, 2005)
KPLANNER (Levesque 2005) uses the robot program representation.
![Page 13: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/13.jpg)
13
Robot Program – Some ExamplespickA;CASE testH? OF - red: putC; pickA; CASE testH? OF -red: putB; ... ... -blue: putC; ... ... - blue: putB; pickA; CASE testH? OF -red: putC; ... ... -blue: putB; ... ...
pickA;putB;pickA;putC;pickB;putC;pickA;putC;pickAputC.
![Page 14: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/14.jpg)
14
Plan Representation
An FSA plan is a directed graph• Each node represents a program state
o One unique "start" state o One unique "final" stateo Non-final state associated with an
action• Each edge is associated with a
sensing resulto Sensing result of world-changing
actions can be omitted
![Page 15: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/15.jpg)
15
Plan Execution(for a single complete initial world)
1. Use the "start state" as current program state;
2. If current state is the "final state", then stop;
3. Execute action associated to the current state;
4. Follow the edge with returned sensing result;
5. Make the node pointed to by this edge the current program state, and repeat from Step 2.
![Page 16: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/16.jpg)
16
Robot Program vs. FSA Plan
It can be shown that all robot programs can be represented by equivalent FSA plans.
What about the reverse direction?
![Page 17: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/17.jpg)
17
Robot Program vs. FSA Plan
feel?thirsty
drink
hungry
eat
go2bed
sleep
CASE feel? OF
- thirsty:
drink;
go2bed;
sleep;
- hungry:
eat;
go2bed;
sleep
![Page 18: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/18.jpg)
18
Robot Program vs. FSA Plan
get?instruction suggestion
follow? think?
succeed
fail
workable
unworkableok
revise?fail
![Page 19: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/19.jpg)
19
Outline
• Motivation
• Representing plans with loops– FSA plan: a type of finite state controller
• Constructing plans with loops– Search in the space of FSA plans
• Potential for improvements
• Conclusion
![Page 20: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/20.jpg)
20
Generating Plans with Loops
1. Start with the smallest FSA plan with only one non-final state.
2. If the current program state is final, the goal must be satisfied.
3. Otherwise, execute the action associated to the current program state, non-deterministically pick an applicable one if none is associated.
4. For each possible sensing result of the action, follow the transition and change the current state to the transition target. If no transition is associated to the sensing result, non-deterministically pick one for it.
5. Repeat from step 2.
![Page 21: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/21.jpg)
21
Search in the Space of FSA Plans… …
(Possible values of stackA)
![Page 22: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/22.jpg)
22
Search in the Space of FSA Plans… …
![Page 23: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/23.jpg)
23
Search in the Space of FSA Plans
pickA
… …
![Page 24: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/24.jpg)
24
Search in the Space of FSA Plans
testA?
… …
empty
![Page 25: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/25.jpg)
25
Search in the Space of FSA Plans
testA?empty
nonempty
… …
… …
![Page 26: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/26.jpg)
26
Search in the Space of FSA Plans
testA?empty
nonempty
… …
![Page 27: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/27.jpg)
27
Search in the Space of FSA Plans
testA?empty
nonempty
… …
pickB
![Page 28: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/28.jpg)
28
Search in the Space of FSA Plans
testA?empty
nonempty
… …
testA?
![Page 29: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/29.jpg)
29
Search in the Space of FSA Plans
testA?empty
nonempty
… …
pickA
![Page 30: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/30.jpg)
30
Search in the Space of FSA Plans
testA?empty
nonempty
… …
pickA
![Page 31: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/31.jpg)
31
Search in the Space of FSA Plans
testA?empty
nonempty
… …
pickA
![Page 32: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/32.jpg)
32
Search in the Space of FSA Plans
testA?empty
nonempty
… …
pickA
![Page 33: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/33.jpg)
33
Search in the Space of FSA Plans
testA?empty
nonempty
pickA
![Page 34: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/34.jpg)
And finally…
![Page 35: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/35.jpg)
35
Experimental Results
(Using the same pruning rules.)
![Page 36: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/36.jpg)
36
Experimental Results
Statistics for Aranda are estimation from figures in (Srivastava et al. 08) without redoing their experiments.
![Page 37: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/37.jpg)
37
Potential for Improvements
• We have formulated planning with loops as a search problem, and obtained a baseline implementation.
• It is using blind depth-first (iterative deepening) search, and does not scale well without effective pruning rules.
• However, the baseline implementation is a good starting point for adding heuristics and trying other improvement.
![Page 38: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/38.jpg)
38
Heuristics for Action Selection
• We tried a variant of the additive heuristics (Bonet and Geffner 2001):– Assume all fluents are independent;– Count the number of action steps to change each
fluent to a value that may satisfy the goal;– Sum of steps across all fluents used as goal distance;– Try successor states with shorter distance first
![Page 39: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/39.jpg)
39
Heuristics for Action Selection
• Appear relatively effective, but still limited:– The domain of each fluent may be infinite, so
exponential BFS was used;– There are two non-deterministic choices in the search
algorithm (choice of action and choice of transition). Greedily improving only one does not always lead to a good decision.
![Page 40: Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto](https://reader035.vdocument.in/reader035/viewer/2022062222/56649cd95503460f949a338d/html5/thumbnails/40.jpg)
40
Discussion
• Planning with loops is an interesting problem
• We formally defined FSA plans, representation for loopy plans. [And a logical account for planning problems, FSA plans and correctness.]
• We formulate planning with loops as a search problem.
• FSA planner, the baseline implementation, outperforms KPLANNER, and is good starting point for heuristics and other improvements.