Using Lego Robots to Estimate Cognitive Ability in Children who have Severe DisabilitiesAlbert M. Cook1, Kim Adams1,2, Norma Albert M. Cook1, Kim Adams1,2, Norma

Harbottle1, Cheryl Harbottle1Harbottle1, Cheryl Harbottle11 Faculty of Rehabilitation Medicine, University 1 Faculty of Rehabilitation Medicine, University

of Alberta, of Alberta, 2Glenrose Rehabilitation Hospital, Edmonton, 2Glenrose Rehabilitation Hospital, Edmonton,

Alberta, CanadaAlberta, Canada

Background

• Young children who have disabilities Young children who have disabilities can control robots to perform play can control robots to perform play tasks, tasks,

• Some children are unable to Some children are unable to demonstrate cognitive abilities in demonstrate cognitive abilities in standardized testingstandardized testing

• Success with robot play tasks could Success with robot play tasks could be a proxy measurebe a proxy measure

Background• A set of cognitive skills required A set of cognitive skills required

for robot use by typically for robot use by typically developing childrendeveloping children

• Causality, coordination of multiple Causality, coordination of multiple variables, reflectivity, binary logic, variables, reflectivity, binary logic, and spatial relations and spatial relations

• This set provides a guide for This set provides a guide for comparison of performance by comparison of performance by children with disabilities children with disabilities performing robot tasksperforming robot tasks

Using a Robotic Arm, Children Who Have Physical Disabilities:

• exert control over the exert control over the environmentenvironment

• manipulate real objects (not manipulate real objects (not computer images)computer images)

• engage in turn taking activities engage in turn taking activities with a partnerwith a partner

General Themes from Teacher Interviews in previous projects

•Children’s reactions to robot were very positiveChildren’s reactions to robot were very positive•Robot tasks were more motivational (generated Robot tasks were more motivational (generated more interest and excitement) than single more interest and excitement) than single switch tasks with toys, appliances and switch tasks with toys, appliances and computer-based activitiescomputer-based activities•Vocalizations increased during & after robot Vocalizations increased during & after robot useuse•Teachers and aides underestimated the level of Teachers and aides underestimated the level of accomplishment of the children accomplishment of the children •Level of accomplishment of the children Level of accomplishment of the children changed perceptions of teachers changed perceptions of teachers and parentsand parents

Current project

Can low cost robots provide a means by which Can low cost robots provide a means by which children with severe disabilities can demonstrate children with severe disabilities can demonstrate

cognitive understanding of cognitive concepts?cognitive understanding of cognitive concepts? Group designGroup designParticipants:Participants:

Ten children ages 4 to 10 participated in the Ten children ages 4 to 10 participated in the study. Their disabilities were primarily cerebral study. Their disabilities were primarily cerebral palsy and related motor conditions. Widely palsy and related motor conditions. Widely variable motor, cognitive and language abilities, variable motor, cognitive and language abilities, non-speaking.non-speaking.

A Lego InventionA Lego Invention[1] “roverbot” vehicle was used. “roverbot” vehicle was used.

[1] Lego Invention is a registered trademark of Lego. Lego Invention is a registered trademark of Lego. http:/www.lego.comhttp:/www.lego.com

Lego Mindstorms

Methods

• Initial tasks established Initial tasks established understanding of the switch understanding of the switch operation of robotoperation of robot

• Used single switch to “playback” Used single switch to “playback” pre-stored movementspre-stored movements

• Controlled turning (left/right) and Controlled turning (left/right) and movement (go/stop) using multiple movement (go/stop) using multiple switchesswitches

The tools…

Lego Roverbot Lego Roverbot Adapted ControllerAdapted Controller

Typical roverbot tasks

• Using one or two switches to draw Using one or two switches to draw circles using the roverbot with a circles using the roverbot with a pen attached. pen attached.

• Two more switches added so that Two more switches added so that left, right, stop and go could be left, right, stop and go could be under the child’s control under the child’s control

Typical setup for robot play

Example of Robot in Use

Results

Proposed Hierarchy of Robot–Related Skills

• Based on increasing levels of cognitive skills required to complete tasks, from–Cognitive development theories–Robot use by typically developing

children

Skill Definition for robot use

Age Considerations

Lego Robot Example

1 Causality Understanding the relationship between a switch and a resulting effect

<3 yrs: "empty” switch used repeatedly[1]

Use switch to drive robot, knocking over blocks, drawing circles on paper

2 Spatial concepts-one dimension

Judgment of distance Piagetian Pre-operational (2 to 7 years)

Moving and stopping at a desired point in one dimension

3 Binary Logic Two opposite effects such as on and not on

<4 yrs: difficulty5-6 : understood[1]

Two switches with opposite effects in terms of robot action, i.e. turning robot right/left

4 Spatial concepts- multiple dimension

Judgment of distance in multiple dimensions, movement in more than one dimension

Piagetian Pre-operational (2 to 7 years)

Moving roverbot to a specific location

5 Symbolic Play Make believe with real, miniature or imaginary props(Musselwhite, 1986)

Piagetian Pre-operational (2 to 7 years)

Interactive play initiated by subject, i.e. tea party, exchanging toys.

[1] Forman, G. (1986). Observations of young children solving problems with computers and robots. J Res Childhood Educ, 1(2), 60-73.

Skill Definition for robot use

Age Considerations Lego Robot Example

6 Negation An action can be negated by its opposite

5 yrs: "what does this switch do“[1]

Difference between latch versus release to stop

7 Spatial concepts-multiple dimensions

Movement in more than one dimension to meet a functional geometric goal

Piagetian Formal Operations (age 11-12 through adult)

Drawing geometric shapes on paper using the roverbot

8 Problem solving Generation of multiple possible solutions

Piagetian Formal Operations (age 11-12 through adult)

Ability to change strategy

9 Systematic problem solving

Problem solving with a plan - not trial and error

Piagetian Formal Operations (age 11-12 through adult)

Placing objects in a particular order

10 Formulation of hypotheses

Developing independent approach to solve a problem

Piagetian Formal Operations (age 11-12 through adult)

Changing task to meet the child’s goal, i.e. pick up tree prop to decorate it

11 Solving multiple aspects of a problem simultaneously

Combination of 8-10 above

Piagetian Formal Operations (age 11-12 through adult)

Serving sushi while also controlling arm, taking orders, deciding who gets what

[1] From Forman (1986)

Summary Of Participant's Demographic Information And

standardized evaluation measures of:

Language (PPVT) Motor (GMFM)

Cognitive (Leiter) abilities

Participant Chronological Age PPVT* GMFM Leiter R Level (age) Robot Skill

L05 4 yrs 9 mo. NT** NT NT 0

L02 7 yrs 1 mo. NT 19 NT 1 (< 3 yrs)

L08 9 yrs 8 mo. NT 17 NT 1 (< 3 yrs)

L11 9 yrs 7 mo. NT 23 NT 1 (< 3 yrs)

L09 9 yrs 10 mo. NT 6 NT 0

L12 10 yrs 0 mo. NT 88 NT 5 (2 to 7 yrs)

L04 4 yrs 4 mo. NT 7 NT 7 (> 5 yrs)

L06 7 yrs 5 mo. 59/60/2/6/1 87 50 7 (> 5 yrs)

L10 10 yrs 5 mo. 63/49/1/-/- 42 58 9 (11 to 12 yrs)

L01 9 yrs 5 mo. 94/79/8/-/- 21 73 11 (11 to 12 yrs)

**Not Testable

*Raw Score /Standard Score/Percentile rank/Normal Curve Equivalent

Summary Of Change in Participant's Behavioral,

Language, and Social Skills

ParticipantReaction to robot?

Change in behavior?

Change in social skills?

Change in language skills?

L05 ~ not engaged - angry ~ no ~ no

L02 +enjoyed + attention + responds to RAs + vocalized during task

L08 o o o ~ no

L011 + aware + attention + responds to RAs ~ no

L09 o seemed to get it

o o ~ no

L012 + enjoyed + attention + responds to RAs + vocalized during task

L04 + loved +enjoyed +responds to RAs ~ no

L010 + excited + attention + other kids asked questions

+ vocalized about it before, during and after

L06 + loved + excited + mom reports more interaction

+ vocalized before task, now says 3 word sentences

L01 + enjoyed + enjoyed + vocalized with other students

+ vocalized with other students

+ positive effect, o indeterminate, ~ no effect, - negative effect

Data used to support decision re: skill achievement

Skill Category Data used to support decision

Causality Hits switch then looks at robot, expecting a movementNumber of verbal prompts decreases

Spatial concepts – one dimension (Level 2)

Able to follow steps using one switch to stop at required point

Binary (Addition of second switch) (Level 3)

Number of errors in switch activation decreases

Spatial concepts - multiple dimensions simple movement (Level 4)

Able to follow steps using two or more switches to attain a targetDiminishing errors

Symbolic Play (Level 5) Use of labels for switches, tasks, use of objects in imaginative way

Negation (Level 6) Use of two switches with opposite effects

Spatial concepts - multiple dimensions geometric goal (Level 7)

Able to follow a pattern using two or more switchesDiminishing errors

Problem Solving (Levels 8-11) Demonstration of strategies, hypothesis formation and testing

Teacher’s aide comments

““I think [the robot] makes the learning I think [the robot] makes the learning lots of fun – it also has them having lots of fun – it also has them having to think a lot more – with having to to think a lot more – with having to move it certain ways plus having to move it certain ways plus having to do the work – so for some it might be do the work – so for some it might be tricky but I think [she] enjoyed the tricky but I think [she] enjoyed the challenge”challenge”

Teacher’s aide comments

Can you see ways that robots would be Can you see ways that robots would be useful in your classroom? useful in your classroom?

““I think for some of the kids that I think for some of the kids that probably have trouble learning it’s a probably have trouble learning it’s a really interesting way of getting them really interesting way of getting them involved in the learning process.”involved in the learning process.”

Conclusions• There was significant variation in

motor, language and cognitive function in the participants

• Extremes of all three parameters were evident

• Motor ability was not directly related to cognitive ability

• Also great variation in the skill demonstrated while using the robot

ConclusionsUse of the robotic arm:Use of the robotic arm:

• Gives the child a chance to Gives the child a chance to demonstrate a range of cognitive skillsdemonstrate a range of cognitive skills

• Provides a versatile tool for Provides a versatile tool for presentation of tasks, problems and presentation of tasks, problems and learning opportunities to the childlearning opportunities to the child

• Can avoid the limitations of Can avoid the limitations of standardized test administration,standardized test administration,

e.g. verbal response or manipulation of e.g. verbal response or manipulation of objectsobjects

We gratefully Acknowledge We gratefully Acknowledge the funding of this work by the funding of this work by

the Stollery Children’s the Stollery Children’s Hospital Foundation, Hospital Foundation, Edmonton, AlbertaEdmonton, Alberta

CanadaCanada____________________________________________

We also thank Johanna Darrah, Joanne We also thank Johanna Darrah, Joanne Volden and Robin Adkins for valuable Volden and Robin Adkins for valuable

insight and adviceinsight and advice