NOAH for Cognitively

Impaired Older Adults:

Navigation and Obstacle

Avoidance Help

Pooja Viswanathan (UBC)

James Little, Alan Mackworth, Alex Mihailidis

WiML Workshop - December 6, 2010 1

Outline

• Introduction

• Methods

• Experiments

• Future Work

• Relevance

2

Introduction

3

Motivation

• Proportion of older adults in the population continues to increase

• Of the 1.5 million people residing in nursing homes, 60-80% have been diagnosed with dementia, primarily Alzheimer‟s disease (Payne et al., 2002)‏

• Older adults with cognitive impairments not allowed to operate powered wheelchairs

• Prohibition of powered wheelchair use and the lack of strength required to use manual wheelchairs reduce mobility -> social isolation, depression and increased dependence on caregivers 4

Solution

1. Detects obstacles and prevents collisions

2. Infers the user's goal location/activity and

provides automated reminders

3. Provides navigation assistance using

prompts that account for the user‟s

cognitive state

Intelligent powered wheelchair for older adults with cognitive impairment that:

5

Overview

• Wheelchair is being developed for users with mild

to moderate cognitive impairment.

• User‟s schedule, wheelchair‟s current location,

obstacles in the map, and user preferences used to

plan optimal route.

• Type and level of prompting determined

automatically for each user using information

such as wheelchair heading, errors committed,

past responsiveness to system prompts.

• Wheelchair will be tested in a long-term-care

facility with cognitively-impaired older adults. 6

Control Strategy

• The wheelchair will stop upon detecting an

obstacle and prevent movement in the direction of

the obstacle.

• User will remain in control of all other

movements.

• Passive feedback provided in the form of

reminders and directions to help user navigate to

desired goals.

• Future work can involve a greater extent of

autonomous navigation for users with lower

cognitive capacity - legal and ethical issues such as

liability will need to be addressed. 7

Methods

8

System Overview

9

The system consists of:

Nimble Rocket TM Powered

Wheelchair

Bumblebee Stereovision Camera

from Point Grey Research

Fujitsu Lifebook P7120 Laptop

(under seat)

Developed at U of T and UBC

Collision Avoidance

• Elderly residents are unsteady on their feet

• Non-contact method of collision avoidance

needed to ensure safety

• Active sensors (laser, acoustic, sonar, etc.)

are often large, expensive, power-hungry,

unsafe, and prone to cross-talk issues.

• Infrared sensor used in Mihailidis et al.,

2007 produced false alarms in natural light

10

Collision Avoidance

(a) (b) (c)

Images of a person with a cane captured using the

stereovision camera: (a) original image, (b) depth image

(brighter pixels correspond to closer objects), and (c)

occupancy grid (black denotes obstacles, white denotes free

space, and the solid grey region denotes the area outside the

camera‟s field of view).

11

Collision Avoidance

• If object detected within a specified distance

threshold, wheelchair is stopped

• Compute direction around

obstacle with greatest

amount of free space

• Prompt: “Try turning

left”

12

Mapping and Localization

• Global map created using SLAM (laser

and/or vision used)

• Current location estimated by matching

visual landmarks in incoming stereo images

with previously detected landmarks

13

Map Annotation

• Viswanathan et al. (2009, 2010)

Curious

George 14

Path Planning

• Goal locations provided to the system in the

form of the user‟s daily schedule

• Goal location, combined with the

wheelchair‟s current location, and obstacles

in the map used to construct optimal path

Lounge

Kitchen

Lounge

Kitchen

Bedroom Bedroom

15

Prompting

Fulfill the following (possibly conflicting) goals

according to the following order of priority:

• Assist in successful navigation to the desired

location (issue correct prompts as needed)

• Minimize user frustration (minimize incorrect

and excessive prompting)

• Maximize user independence (minimize

caregiver intervention)

• Maximize user understanding (issue

appropriate level of prompts) 16

Prompting

• Using POMDP similar to Boger et al., 2005

and Hoey et al., 2007

User Model

(responsiveness,

independence etc.)

Lounge Bedroom

Kitchen

17

Experiments

18

Collision Avoidance

• Experiments conducted to test efficacy of

anti-collision and prompting system

• Conducted within controlled environment

19

Collision Avoidance

• Anti-collision systems tested with the

following commonly-found objects:

– A painted white wall with a flat finish

– A light green aluminium 4-wheeled walker

– A silver aluminium walking cane

– A person who was standing still

– A person who was moving

20

Collision Avoidance

21

- 96% accuracy in preventing collisions

- 0% false alarm rate

- 100% accuracy in prompting

Collision Avoidance

Scene views of a room with windows (a). Occupancy grids produced by

stereovision (b) and infrared (c) sensors with blinds closed and opened. 22

Map Annotation

23

Place recognition

24

Indoor images from

Google and Photobucket: Segmented object images

(from LabelMe) in real

home models:

Path Planning and Prompting

• Tested using Matlab interface to collect

wheelchair position data

• Scenarios

– Independent

– Not independent, responsive

– Not independent, not responsive

– Independent/Responsive with errors

– Not Independent/not responsive with flukes

25

Path Planning and Prompting

26

“on_left”

Path Planning and Prompting • Recent work:

– Real wheelchair position data acquired using

VSLAM

– Continuous path planner used for

generalization to more complex environments

(Alton et al. 2008)

– Clinical trials to test collision avoidance system

• Varying functional abilities

• Prompts not adhered to during collisions (errors)

• Blocking of motion was found to be frustrating –

other methods? (i.e. autonomous „correction‟) Other

non-joystick interfaces?

27

Future work

– Stopping frequency

– Independence and responsiveness dynamics

(distractions?)

– Prompt levels (detailed vs. general)

– Timing – how close to turn should you prompt?

– Challenges:

• building the user behavior model!

• inconsistent sensor performance

28

Relevance • Can restore mobility in older adults who are

currently not allowed to operate powered

wheelchairs.

• Safety feature can reduce the number of

wheelchair accidents and related injuries.

• Can enhance the health care system, reduce the

burden on care-giving staff, and improve the

quality of life of older adults with cognitive

disabilities.

• Provides a real-world application to test and

enhance AI, vision and robotics techniques. 29