assistance for the elderly in robocare
DESCRIPTION
Project “ RoboCare : A multi-agent system with fixed and robotic intelligent components” MIUR Law 449/97(yr 00) – 2003-2006. Assistance for the Elderly in RoboCare. Riccardo Rasconi ISTC-CNR [PST] Institute for Cognitive Science and Technology National Research Council of Italy - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Assistance for the Elderly in RoboCare](https://reader035.vdocument.in/reader035/viewer/2022081519/56814064550346895dabdd0e/html5/thumbnails/1.jpg)
Planning & Scheduling TeamIstc-Cnr
Assistance for the Elderly in RoboCare
Riccardo Rasconi
ISTC-CNR [PST]
Institute for Cognitive Science and Technology
National Research Council of Italy
Planning and Scheduling Team
http://pst.istc.cnr.it
Project “Project “RoboCareRoboCare: A multi-agent system with fixed and robotic : A multi-agent system with fixed and robotic intelligent components” MIUR Law 449/97(yr 00) – 2003-2006intelligent components” MIUR Law 449/97(yr 00) – 2003-2006
Joint work withAmedeo Cesta,Gabriella Cortellessa, Federico Pecora
Workshop on Telematics and Robotics
for the Quality of Life of the Elderly
28/09/2009
![Page 2: Assistance for the Elderly in RoboCare](https://reader035.vdocument.in/reader035/viewer/2022081519/56814064550346895dabdd0e/html5/thumbnails/2.jpg)
Planning & Scheduling TeamIstc-Cnr
• A distributed system– Software Agents– Robotic Agents– Human Agents
• All cooperate to provide services for human assistance
http://robocare.istc.cnr.it
The RoboCare Project’s Goal
![Page 3: Assistance for the Elderly in RoboCare](https://reader035.vdocument.in/reader035/viewer/2022081519/56814064550346895dabdd0e/html5/thumbnails/3.jpg)
Planning & Scheduling TeamIstc-Cnr
The Different Research Aspects Involved
Active Supervision Framework
Human-RobotInteraction
Personalized Intelligent Assistance
Acceptability Issues
Distributed H/S Infrastructure
Robust MobileRobotic Skills
http://robocare.istc.cnr.it
![Page 4: Assistance for the Elderly in RoboCare](https://reader035.vdocument.in/reader035/viewer/2022081519/56814064550346895dabdd0e/html5/thumbnails/4.jpg)
Planning & Scheduling TeamIstc-Cnr
The RoboCare Domestic Environment
People Localization & TrackingPosture Recognition
Robot Mobility+
User Interaction
PDA
ADL Monitoring
![Page 5: Assistance for the Elderly in RoboCare](https://reader035.vdocument.in/reader035/viewer/2022081519/56814064550346895dabdd0e/html5/thumbnails/5.jpg)
Planning & Scheduling TeamIstc-Cnr
Multiple Intelligent Systems
A number of base services provide the building blocks for higher-level assistive behavior
• Posture recognition
• Person localization
• Mobile robotic platform
• User interaction front-end
• PDA interface
• Daily activity monitor
![Page 6: Assistance for the Elderly in RoboCare](https://reader035.vdocument.in/reader035/viewer/2022081519/56814064550346895dabdd0e/html5/thumbnails/6.jpg)
Planning & Scheduling TeamIstc-Cnr
The robotic platform evolution
2004 2005 2006
![Page 7: Assistance for the Elderly in RoboCare](https://reader035.vdocument.in/reader035/viewer/2022081519/56814064550346895dabdd0e/html5/thumbnails/7.jpg)
Planning & Scheduling TeamIstc-Cnr
Vision Sensors: People Tracking System
Luca Iocchi and G. Riccardo Leone work from Univ. Rome “La Sapienza”
[Bahadori et al, Applied AI, 2007]
![Page 8: Assistance for the Elderly in RoboCare](https://reader035.vdocument.in/reader035/viewer/2022081519/56814064550346895dabdd0e/html5/thumbnails/8.jpg)
Planning & Scheduling TeamIstc-Cnr
Caregiver vs. Active SupervisionFramework interface
Contextual Knowledge Component: Non-Intrusive Activity Supervision
Caregiver specifications compiled into scheduling problems
(a temporal constraint network)
compilation
T-REXscheduling problem
physician
family members
behavioral requirements(in terms of daily activities)
[Pecora et al, ISSEJ, 2006]
![Page 9: Assistance for the Elderly in RoboCare](https://reader035.vdocument.in/reader035/viewer/2022081519/56814064550346895dabdd0e/html5/thumbnails/9.jpg)
Planning & Scheduling TeamIstc-Cnr
Representing temporal prescriptions as a schedule
• Activities and their mutual temporal constraints represented as a Simple Temporal Network
• Dispatched for execution and monitored
• Constraint violation triggers interaction
![Page 10: Assistance for the Elderly in RoboCare](https://reader035.vdocument.in/reader035/viewer/2022081519/56814064550346895dabdd0e/html5/thumbnails/10.jpg)
Planning & Scheduling TeamIstc-Cnr
Schedule Execution Monitoring
• Data are continuously retrieved from the stereo cameras (more about this issue later in the talk);
• Activity status is updated at each execution step;• All constraint violations are detected;
behavioral pattern
time nowtime nowtime nowtime nowtime nowtime nowtime nowtime nowtime nowtime now
tmaximum allowed distance
time nowtime nowtime nowtime now
![Page 11: Assistance for the Elderly in RoboCare](https://reader035.vdocument.in/reader035/viewer/2022081519/56814064550346895dabdd0e/html5/thumbnails/11.jpg)
Planning & Scheduling TeamIstc-Cnr
Using the Scheduler’s Temporal Knowledge to generate contextualized dialogues
breakfast cooking lunch
time now
t
medicine
You should hurry up taking your after-lunch medicine!
executed executed executed
![Page 12: Assistance for the Elderly in RoboCare](https://reader035.vdocument.in/reader035/viewer/2022081519/56814064550346895dabdd0e/html5/thumbnails/12.jpg)
Planning & Scheduling TeamIstc-Cnr
breakfast cooking lunch
time now
t
medicine
You should wait a little longer before having lunch!
executed executed
Using the Scheduler’s Temporal Knowledge to generate contextualized dialogues
![Page 13: Assistance for the Elderly in RoboCare](https://reader035.vdocument.in/reader035/viewer/2022081519/56814064550346895dabdd0e/html5/thumbnails/13.jpg)
Planning & Scheduling TeamIstc-Cnr
breakfast cooking lunch
time now
t
medicine
Maybe you should cook yourself something warm to
eat!
executed NOT executed
Using the Scheduler’s Temporal Knowledge to generate contextualized dialogues
![Page 14: Assistance for the Elderly in RoboCare](https://reader035.vdocument.in/reader035/viewer/2022081519/56814064550346895dabdd0e/html5/thumbnails/14.jpg)
Planning & Scheduling TeamIstc-Cnr
The Proactive Interactor
Talking head
Speechrecognition
Interaction Manager
• What is behind the interaction?
A set of active services ….
Simple I/O Engine
Input/Output Channels
Knowledge for Interaction
![Page 15: Assistance for the Elderly in RoboCare](https://reader035.vdocument.in/reader035/viewer/2022081519/56814064550346895dabdd0e/html5/thumbnails/15.jpg)
Planning & Scheduling TeamIstc-Cnr
The User Interaction Agent (1/2)
Speech Recognition
![Page 16: Assistance for the Elderly in RoboCare](https://reader035.vdocument.in/reader035/viewer/2022081519/56814064550346895dabdd0e/html5/thumbnails/16.jpg)
Planning & Scheduling TeamIstc-Cnr
The User Interaction Agent (2/2)
Verbalizations Synthesis
• Simple synthesis of Speech Acts is performed by analyzing the information contained inthe Constraint Violation DB and in the Environment Status DB
![Page 17: Assistance for the Elderly in RoboCare](https://reader035.vdocument.in/reader035/viewer/2022081519/56814064550346895dabdd0e/html5/thumbnails/17.jpg)
Planning & Scheduling TeamIstc-Cnr
Generating environment-coherent behavior
• Coordination of multiple services is achieved by solving a Multi-Agent Coordination (MAC) problem
• The MAC problem is cast as a Distributed Constraint Optimization Problem (DCOP)
• The DCOP is solved by the ADOPT-N algorithm, an extension of the ADOPT (Asynchronous Distributed Optimization) algorithm for dealing with n-ary constraints
![Page 18: Assistance for the Elderly in RoboCare](https://reader035.vdocument.in/reader035/viewer/2022081519/56814064550346895dabdd0e/html5/thumbnails/18.jpg)
Planning & Scheduling TeamIstc-Cnr
Agents, variables and soft constraints
• Through cost functions, soft constraints are used to prefer (for the monitored person) healthy states and avoid dangerous states
• Cost functions are modeled so as to reflect the desiderata of system behavior
• Detailed description in [Pecora & Cesta, Comp. Int. 2007]
![Page 19: Assistance for the Elderly in RoboCare](https://reader035.vdocument.in/reader035/viewer/2022081519/56814064550346895dabdd0e/html5/thumbnails/19.jpg)
Planning & Scheduling TeamIstc-Cnr
Possible Assistant/assisted interactions
• On-demand interaction(Person takes initiative)– Question / answering
• Proactive interaction(RoboCare takes initiative)– Danger– Warning
The RoboCare Environment as a Mixed-Initiative System
![Page 20: Assistance for the Elderly in RoboCare](https://reader035.vdocument.in/reader035/viewer/2022081519/56814064550346895dabdd0e/html5/thumbnails/20.jpg)
Planning & Scheduling TeamIstc-Cnr
Managing interaction in RoboCare
ProactiveOn-demand
![Page 21: Assistance for the Elderly in RoboCare](https://reader035.vdocument.in/reader035/viewer/2022081519/56814064550346895dabdd0e/html5/thumbnails/21.jpg)
Planning & Scheduling TeamIstc-Cnr
Some examples of interaction
![Page 22: Assistance for the Elderly in RoboCare](https://reader035.vdocument.in/reader035/viewer/2022081519/56814064550346895dabdd0e/html5/thumbnails/22.jpg)
Planning & Scheduling TeamIstc-Cnr
Proactive Warning
• Feedback from sensors is a key activator• Explanation triggered by T-REX temporal knowledge
![Page 23: Assistance for the Elderly in RoboCare](https://reader035.vdocument.in/reader035/viewer/2022081519/56814064550346895dabdd0e/html5/thumbnails/23.jpg)
Planning & Scheduling TeamIstc-Cnr
Proactive Alarm for Danger
• A reactive routine is activated with a precompiled plan– (go-to-place; try-interaction; call-emergency)
![Page 24: Assistance for the Elderly in RoboCare](https://reader035.vdocument.in/reader035/viewer/2022081519/56814064550346895dabdd0e/html5/thumbnails/24.jpg)
Planning & Scheduling TeamIstc-Cnr
On Demand Question-Answering
• Query to the temporalized knowledge in T-REX
• Very simple additional internal query capabilities
![Page 25: Assistance for the Elderly in RoboCare](https://reader035.vdocument.in/reader035/viewer/2022081519/56814064550346895dabdd0e/html5/thumbnails/25.jpg)
Planning & Scheduling TeamIstc-Cnr
Related work
• Intelligent assistants– Same domain and similar technologies:
• Autominder [Pollack et-al, 2003], PEAT [Levinson 1997], PEARL [Pineau et al. 2003; Pollack 2005], I.L.S.A. [Haigh, Kiff, & Ho 2006]
– Capability integration: • Similar problems addressed with CALO, CMradar,
etc. … although project scale quite different!
![Page 26: Assistance for the Elderly in RoboCare](https://reader035.vdocument.in/reader035/viewer/2022081519/56814064550346895dabdd0e/html5/thumbnails/26.jpg)
Planning & Scheduling TeamIstc-Cnr
Conclusions
• RoboCare has addressed (among others) – the open challenge of integrating diversified intelligent capabilities to create a
proactive monitoring assistant for everyday life in a domestic environment
• Highlighted in this work– particular use of the internal knowledge of a constraint-based scheduler (the
temporal constraint network) as well as its capability of reasoning on changes in the environment
– constraint violations determine when the system has to interact. The analysis and interpretation of the violation contribute to determine how to interact with the user
– the use of a distributed coordination algorithm to create a coherent behavior of multiple “active agents”
![Page 27: Assistance for the Elderly in RoboCare](https://reader035.vdocument.in/reader035/viewer/2022081519/56814064550346895dabdd0e/html5/thumbnails/27.jpg)
Planning & Scheduling TeamIstc-Cnr
THANK YOU!QUESTIONS?