cs b659: intelligent robotics
DESCRIPTION
CS b659: Intelligent Robotics. Planning Under Uncertainty. Offline learning (e.g., calibration). Prior knowledge (often probabilistic) . Sensors. Perception (e.g., filtering, SLAM). Decision-making (control laws, optimization, planning). Actions. Actuators. Dealing with Uncertainty. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/1.jpg)
CS B659: INTELLIGENT ROBOTICSPlanning Under Uncertainty
![Page 2: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/2.jpg)
Perception(e.g., filtering,
SLAM)
Prior knowledge (often probabilistic)
Sensors
Decision-making(control laws, optimization,
planning)
Offline learning(e.g., calibration)
Actuators
Actions
![Page 3: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/3.jpg)
DEALING WITH UNCERTAINTY Sensing uncertainty
Localization error Noisy maps Misclassified objects
Motion uncertainty Noisy natural processes Odometry drift Imprecise actuators Uncontrolled agents (treat as state)
![Page 4: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/4.jpg)
MOTION UNCERTAINTY
g
obstacle
obstacle
obstacle
s
![Page 5: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/5.jpg)
DEALING WITH MOTION UNCERTAINTY Hierarchical strategies
Use generated trajectory as input into a low-level feedback controller
Reactive strategies Approach #1: re-optimize when the state has
diverged from the planned path (online optimization) Approach #2: precompute optimal controls over state
space, and just read off the new value from the perturbed state (offline optimization)
Proactive strategies Explicitly consider future uncertainty Heuristics (e.g., grow obstacles, penalize nearness) Markov Decision Processes Online and offline approaches
![Page 6: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/6.jpg)
PROACTIVE STRATEGIES TO HANDLE MOTION UNCERTAINTY
g
obstacle
obstacle
obstacle
s
![Page 7: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/7.jpg)
DYNAMIC COLLISION AVOIDANCE ASSUMING WORST-CASE BEHAVIORS Optimizing
safety in real-time under worst case behavior model
T=1
T=2
TTPF=2.6
Robot
![Page 8: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/8.jpg)
MARKOV DECISION PROCESS APPROACHES
Alterovitz et al 2007
![Page 9: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/9.jpg)
DEALING WITH SENSING UNCERTAINTY Reactive heuristics often work well
Optimistic costs Assume most-likely state Penalize uncertainty
Proactive strategies Explicitly consider future uncertainty Active sensing Reward actions that yield information gain Partially Observable Markov Decision Processes
(POMDPs)
![Page 10: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/10.jpg)
10
Assuming no obstacles in the unknown region and taking the shortest path to the goal
![Page 11: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/11.jpg)
11
Assuming no obstacles in the unknown region and taking the shortest path to the goal
![Page 12: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/12.jpg)
12
Assuming no obstacles in the unknown region and taking the shortest path to the goal
Works well for navigation because the space of all maps is too huge, and certainty is monotonically nondecreasing
![Page 13: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/13.jpg)
13
Assuming no obstacles in the unknown region and taking the shortest path to the goal
Works well for navigation because the space of all maps is too huge, and certainty is monotonically nondecreasing
![Page 14: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/14.jpg)
14
What if the sensor was directed (e.g., a camera)?
![Page 15: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/15.jpg)
15
What if the sensor was directed (e.g., a camera)?
![Page 16: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/16.jpg)
16
What if the sensor was directed (e.g., a camera)?
![Page 17: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/17.jpg)
17
What if the sensor was directed (e.g., a camera)?
![Page 18: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/18.jpg)
18
What if the sensor was directed (e.g., a camera)?
![Page 19: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/19.jpg)
19
What if the sensor was directed (e.g., a camera)?
![Page 20: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/20.jpg)
20
What if the sensor was directed (e.g., a camera)?
![Page 21: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/21.jpg)
21
What if the sensor was directed (e.g., a camera)?
![Page 22: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/22.jpg)
22
What if the sensor was directed (e.g., a camera)?
![Page 23: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/23.jpg)
23
What if the sensor was directed (e.g., a camera)?
![Page 24: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/24.jpg)
24
What if the sensor was directed (e.g., a camera)?
At this point, it would have made sense to turn a bit more to see more of the unknown map
![Page 25: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/25.jpg)
ANOTHER EXAMPLE
? Locked?
Key?Key?
![Page 26: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/26.jpg)
ACTIVE DISCOVERY OF HIDDEN INTENT
No motion
Perpendicular motion
![Page 27: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/27.jpg)
MAIN APPROACHES TO PARTIAL OBSERVABILITY Ignore and react
Doesn’t know what it doesn’t know Model and react
Knows what it doesn’t know Model and predict
Knows what it doesn’t know AND what it will/won’t know in the future
• Better decisions• More components in
implementation• Harder
computationally
![Page 28: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/28.jpg)
UNCERTAINTY MODELS Model #1: Nondeterministic uncertainty
f(x,u) -> a set of possible successors Model #2: Probabilistic uncertainty
P(x’|x,u): a probability distribution over successors x’, given state x, control u
Markov assumption
![Page 29: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/29.jpg)
NONDETERMINISTIC UNCERTAINTY : REASONING WITH SETS x’ = x + e, [-a,a]
t=0
t=1-a a
t=2-2a 2a
Belief State: x(t) [-ta,ta]
![Page 30: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/30.jpg)
UNCERTAINTY WITH SENSING Plan = policy (mapping from states to
actions) Policy achieves goal for every possible
sensor result in belief stateMove
Sense1 2
3 4
Observations should be chosen wisely to keep branching factor low
Outcomes
Special case: fully observable state
![Page 31: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/31.jpg)
TARGET TRACKING
targetrobot
The robot must keep a target in its field of view
The robot has a prior map of the obstacles
But it does not know the target’s trajectory in advance
![Page 32: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/32.jpg)
TARGET-TRACKING EXAMPLE Time is discretized into small
steps of unit duration At each time step, each of
the two agents moves by at most one increment along a single axis
The two moves are simultaneous
The robot senses the new position of the target at each step
The target is not influenced by the robot (non-adversarial, non-cooperative target)
targetrobot
![Page 33: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/33.jpg)
TIME-STAMPED STATES (NO CYCLES POSSIBLE)
State = (robot-position, target-position, time) In each state, the robot can execute 5 possible actions :
{stop, up, down, right, left} Each action has 5 possible outcomes (one for each possible action
of the target), with some probability distribution[Potential collisions are ignored for simplifying the presentation]
([i,j], [u,v], t)
• ([i+1,j], [u,v], t+1)• ([i+1,j], [u-1,v], t+1)• ([i+1,j], [u+1,v], t+1)• ([i+1,j], [u,v-1], t+1)• ([i+1,j], [u,v+1], t+1)
right
![Page 34: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/34.jpg)
REWARDS AND COSTSThe robot must keep seeing the target as long as possible Each state where it does not see the target is
terminal
The reward collected in every non-terminal state is 1; it is 0 in each terminal state[ The sum of the rewards collected in an execution run is exactly the amount of time the robot sees the target]
No cost for moving vs. not moving
![Page 35: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/35.jpg)
EXPANDING THE STATE/ACTION TREE
...
horizon 1 horizon h
![Page 36: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/36.jpg)
ASSIGNING REWARDS
...
horizon 1 horizon h
Terminal states: states where the target is not visible
Rewards: 1 in non-terminal states; 0 in others
But how to estimate the utility of a leaf at horizon h?
![Page 37: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/37.jpg)
ESTIMATING THE UTILITY OF A LEAF
...
horizon 1 horizon h
Compute the shortest distance d for the target to escape the robot’s current field of view
If the maximal velocity v of the target is known, estimate the utility of the state to d/v [conservative estimate]
targetrobotd
![Page 38: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/38.jpg)
SELECTING THE NEXT ACTION
...
horizon 1 horizon h
Compute the optimal policy over the state/action tree using estimated utilities at leaf nodes
Execute only the first step of this policy
Repeat everything again at t+1… (sliding horizon)
![Page 39: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/39.jpg)
PURE VISUAL SERVOING
![Page 40: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/40.jpg)
Computing and Using a Policy
![Page 41: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/41.jpg)
NEXT WEEK More algorithm details
![Page 42: CS b659: Intelligent Robotics](https://reader035.vdocument.in/reader035/viewer/2022062501/568166dc550346895ddb0526/html5/thumbnails/42.jpg)
FINAL INFORMATION Final presentations
20 minutes + 10 minutes questions Final reports due 5/3
Must include a technical report Introduction Background Methods Results Conclusion
May include auxiliary website with figures / examples / implementation details
I will not review code