probabilistic approaches to reasoning and control: towards autonomous interactive mobile robots...
TRANSCRIPT
![Page 1: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/1.jpg)
Probabilistic approaches to reasoning and control:
Towards autonomous interactive mobile robots
Joelle PineauCarnegie Mellon University
TAMALE Seminar
March 28, 2003
![Page 2: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/2.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
Our vision of robotic-assisted health-care
Moving thingsaround
Moving thingsaround
Enabling use of remote
health services
Enabling use of remote
health services
Supportinginter-personal
communication
Supportinginter-personal
communication
Calling for helpin emergencies
Calling for helpin emergencies
Monitoring Rx adherence
& safety
Monitoring Rx adherence
& safety
Providinginformation
(TV, weather)
Providinginformation
(TV, weather)
Management support of
ADLs
Management support of
ADLsReminding
to eat, drink, & take meds
Reminding to eat, drink, & take meds
Providing physical
assistance
Providing physical
assistance
Linking the caregiver to resources
Linking the caregiver to resources
![Page 3: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/3.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
Introducing Pearl: A mobile robotic assistant for elderly people and nurses
cameras
sonars
handle bars
mobile base
carrying tray
LCD mouth
touchscreen
microphone& speakers
laser
![Page 4: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/4.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
What are the challenges?
• Interaction with the environment:
– navigating robustly
– handling dynamic obstacles
• Interaction with individuals:
– communicating by speech
– providing cognitive reminders
– interpreting and satisfying user requests
![Page 5: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/5.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
System Overview
Cognitive supportNavigation Communication
High-level controller
![Page 6: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/6.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
System Overview
Cognitive support Communication
High-level controller
• Localization and map building(Burgard et al., 1999)
• People detection and tracking(Montemerlo et al., 2002)
Navigation
![Page 7: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/7.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
Navigation and people tracking
![Page 8: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/8.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
System Overview
Navigation Communication
High-level controller
• Autominder system (Pollack et al., 2002)
Cognitive support
![Page 9: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/9.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
• Speech recognition: Sphinx system(Ravishankar, 1996)
• Speech synthesis: Festival system(Black et al., 1999)
System Overview
Cognitive supportNavigation
High-level controller
Communication
![Page 10: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/10.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
Speech recognition with Sphinx
![Page 11: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/11.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
The role of the top-level controller
Cognitive supportNavigation Communication
ACTION SELECTION - based on the trade-off between:
- goals from different modules;
- goals with varying costs / rewards;
- reducing uncertainty versus accomplishing goals.
High-level controller
![Page 12: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/12.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
Types of uncertainty in robotics
• Cause #1: Non-deterministic effects of actions
• Cause #2: Partial and noisy sensor information
• Cause #3: Inaccurate model of the world and the user
![Page 13: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/13.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
Robot control under uncertainty using Partially Observable Markov Decision Processes
State
User + Environment + Robot
Action={ say-weather,update-appointment,clarify-query}
Speech=“today”Belief State
e.g. request-weather-today
e.g. P(st=weather-today)=0.5 P(st=appointment-today )=0.5
![Page 14: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/14.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
Existing applications of POMDPs
– Maintenance scheduling
» Puterman, 1994
– Robot navigation
» Koenig & Simmons, 1995;
Roy & Thrun, 1999
– Helicopter control
» Bagnell & Schneider, 2001;
Ng et al., 2002
– Dialogue modeling
» Roy, Pineau & Thrun, 2000;
Peak&Horvitz, 2000
– Preference elicitation
» Boutilier, 2002
![Page 15: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/15.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
Graphical Model Representation
POMDP is n-tuple { S, A, , T, O, R }:
What goes on: st-1 st
at-1 at
T(s,a,s’) = state-to-state transition probabilitiesO(s,a,o) = observation generation probabilitiesR(s,a) = Reward function
S = state setA = action set = observation set
What we see: ot-1 ot-1
Belief update:
Ss
jijii
j
tt sbsasToasOsb 1,,,,
![Page 16: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/16.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
Understanding the belief state
• A belief is a probability distribution over states
Where Dim(B) = |S|-1
– E.g. Let S={s1, s2}
P(s1)
0
1
![Page 17: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/17.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
Understanding the belief state
• A belief is a probability distribution over states
Where Dim(B) = |S|-1
– E.g. Let S={s1, s2, s3}
P(s1)
P(s2)
0
1
1
![Page 18: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/18.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
• A belief is a probability distribution over states
Where Dim(B) = |S|-1
– E.g. Let S={s1, s2, s3 , s4}
Understanding the belief state
P(s1)
P(s2)
0
1
1
P(s3)
![Page 19: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/19.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
Exact planning for POMDPs
• Simple problem: |S|=2, |A|=3, ||=2 Iteration # hyper-planes 0 1
P(s1)
V0(b)
b
![Page 20: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/20.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
Exact planning for POMDPs
• Simple problem: |S|=2, |A|=3, ||=2 Iteration # hyper-planes 0 1 1 3
P(s1)
V1(b)
b
![Page 21: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/21.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
Exact planning for POMDPs
• Simple problem: |S|=2, |A|=3, ||=2 Iteration # hyper-planes 0 1 1 3
P(s1)
V1(b)
b
![Page 22: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/22.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
Exact planning for POMDPs
• Simple problem: |S|=2, |A|=3, ||=2 Iteration # hyper-planes 0 1 1 3 2 27
P(s1)
V2(b)
b
![Page 23: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/23.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
Exact planning for POMDPs
• Simple problem: |S|=2, |A|=3, ||=2 Iteration # hyper-planes 0 1 1 3 2 27 3 2187
P(s1)
V2(b)
b
![Page 24: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/24.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
Exact planning for POMDPs
• Simple problem: |S|=2, |A|=3, ||=2 Iteration # hyper-planes 0 1 1 3 2 27 3 2187 4 14,348,907
P(s1)
V2(b)
b
![Page 25: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/25.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
Properties of exact planning
• Value function is always piecewise-linear convex
• Many hyper-planes can be pruned away
P(s1)
V2(b)
b
|S|=2, |A|=3, ||=2 Iteration # hyper-planes
0 1 1 3 2 5 3 9 4 7 5 13 10 27 15 47 20 59
…
![Page 26: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/26.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
Is pruning sufficient?
|S|=20, |A|=6, ||=8
Iteration # hyper-planes0 11 5
2 213 3 ?????
…
Not for this problem!
![Page 27: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/27.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
Certainly not for this problem!
Physiotherapy
Patientroom
Robothome
|S|=576, |A|=19, |O|=17
State Features: {RobotLocation, ReminderGoal, UserLocation, UserMotionGoal,
UserStatus, UserSpeechGoal}
![Page 28: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/28.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
The two curses of POMDP planning
• The curse of dimensionality:
– the dimension of each hyper-plane = # of states
• The curse of history:
– the number of hyper-planes grows
exponentially with the planning horizon
||1
2 |||||| nAS
|| n
Complexity of POMDP value iteration:
dimensionality history
![Page 29: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/29.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
Methods to solve POMDPs
Complexity
Performance
QMDP
MDP
FIB
Grid
O(S2A) O(S2AO )O(S2AO) O(S2AB) T
POMDP
New methods?
Objective: Find a policy, (b), which maximizes reward.
![Page 30: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/30.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
New approach: A hierarchy of POMDPs
Idea: Exploit domain knowledge to divide one POMDP into many smaller ones.
Motivation: Smaller action sets help overcome the curse of history.
Assumption: We are given POMDP M = {S,A,,b,T,O,R} and hierarchy H
Act
ExamineHealth Navigate
MoveVerifyPulse
ClarifyGoal
North South East West
VerifyMeds
subtask
abstract action
primitive action
![Page 31: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/31.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
PolCA+: Planning with a hierarchy of POMDPs
Navigate
Move ClarifyGoal
South East WestNorth
AMove = {N,S,E,W}
ACTIONSNorthSouthEastWest
ClarifyGoalVerifyPulseVerifyMeds
ACTIONSNorthSouthEastWest
ClarifyGoalVerifyPulseVerifyMeds
Step 1: Select the action set
![Page 32: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/32.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
PolCA+: Planning with a hierarchy of POMDPs
Navigate
Move ClarifyGoal
South East WestNorth
AMove = {N,S,E,W}
SMove = {X,Y}
STATE FEATURESX-positionY-position
X-goalY-goal
HealthStatus
STATE FEATURESX-positionY-position
X-goalY-goal
HealthStatus
ACTIONSNorthSouthEastWest
ClarifyGoalVerifyPulseVerifyMeds
ACTIONSNorthSouthEastWest
ClarifyGoalVerifyPulseVerifyMeds
Step 1: Select the action set
Step 2: Minimize the state set
![Page 33: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/33.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
PolCA+: Planning with a hierarchy of POMDPs
Navigate
Move ClarifyGoal
South East WestNorth
AMove = {N,S,E,W}
SMove = {X,Y}
STATE FEATURESX-positionY-position
X-goalY-goal
HealthStatus
STATE FEATURESX-positionY-position
X-goalY-goal
HealthStatus
ACTIONSNorthSouthEastWest
ClarifyGoalVerifyPulseVerifyMeds
ACTIONSNorthSouthEastWest
ClarifyGoalVerifyPulseVerifyMeds
PARAMETERS
{bh,Th,Oh,Rh}
PARAMETERS
{bh,Th,Oh,Rh}
Step 1: Select the action set
Step 2: Minimize the state set
Step 3: Choose parameters
![Page 34: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/34.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
PolCA+: Planning with a hierarchy of POMDPs
Navigate
Move ClarifyGoal
South East WestNorth
AMove = {N,S,E,W}
SMove = {X,Y}
STATE FEATURESX-positionY-position
X-goalY-goal
HealthStatus
STATE FEATURESX-positionY-position
X-goalY-goal
HealthStatus
ACTIONSNorthSouthEastWest
ClarifyGoalVerifyPulseVerifyMeds
ACTIONSNorthSouthEastWest
ClarifyGoalVerifyPulseVerifyMeds
PLAN
h
PLAN
h
PARAMETERS
{bh,Th,Oh,Rh}
PARAMETERS
{bh,Th,Oh,Rh}
Step 1: Select the action set
Step 2: Minimize the state set
Step 3: Choose parameters
Step 4: Plan task h
![Page 35: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/35.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
Results on small dialogue domain
-120
-100
-80
-60
-40
-20
0
0.01 0.1 1 10 100 1000 10000 100000 1000000
Time (secs)
R
POMDPPolCA-D1PolCA-D2FIBQMDP
|S|=12, |A|=20, |O|=3
![Page 36: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/36.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
Achieving a flexible trade-off
Planning time
Reward
QMDP
FIB
POMDP
PolCA+ D2
PolCA+ D1
![Page 37: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/37.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
PolCA+ in the Nursebot domain
• Goal: A robot is deployed in a nursing home, where it provides reminders to elderly users and accompanies them to appointments.
![Page 38: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/38.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
Sample scenario
![Page 39: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/39.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
Comparing user performance
0.1 0.10.18
![Page 40: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/40.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
The effects of confirmation actions
-2000
2000
6000
10000
14000
0 400 800 1200
Time Steps
Cu
mu
lativ
e R
ew
ard
PolCA+
PolCA
QMDP
![Page 41: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/41.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
0
500
1000
1500
2000
2500
3000
3500
4000
4500
NoAbs PolCA PolCA+
# S
tate
ssubInform
subMove
subContact
subRest
subAssist
subRemind
act
Addressing the curse of dimensionality
![Page 42: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/42.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
Ongoing work
• New POMDP approximation techniques.
• Parameter estimation for adaptation to user-specific speech patterns and preferences.
• Exploration of emotion and personality types using a new head.
• Addition of an arm for object manipulation.
• Addition of weight-bearing bars for assisted walking.
![Page 43: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/43.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
Summary
• We have developed a first prototype robot able to serve as a mobile nursing assistant for elderly people.
• The top-level controller uses a hierarchical variant of POMDPs to select actions.
– PolCA+ addresses both the curse of dimensionality and the curse of history.
• Lessons learned during our experiments:
– Uncertainty is crucial when dealing with people
– Probabilistic techniques are necessary to reason about uncertainty.
– Real belief tracking and planning really matters!
![Page 44: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/44.jpg)
Project information: www.cs.cmu.edu/~nursebotNavigation software: www.cs.cmu.edu/~carmenPapers and more: www.cs.cmu.edu/~jpineau
Joint work with: Michael Montemerlo, Martha Pollack, Nicholas Roy, Sebastian Thrun
![Page 45: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/45.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
The Nursebot project in its early days
![Page 46: Probabilistic approaches to reasoning and control: Towards autonomous interactive mobile robots Joelle Pineau Carnegie Mellon University TAMALE Seminar](https://reader036.vdocument.in/reader036/viewer/2022062805/5697bff61a28abf838cbe511/html5/thumbnails/46.jpg)
Probabilistic approaches to reasoning and control for interactive mobile robots Joelle Pineau
Autominder System