1

Abstract—Unpredictability and complexity of social interactions are important challenges for a child with

low-functioning autism. The objective of this research is to study how a mobile robot can, by appearing more

predictable, appealing and simple than a human being, facilitate reciprocal interaction such as imitative play.

By conducting an exploratory study with four children, we found that forms of shared conventions such as

imitation of body movements and of familiar actions are higher with two children paired with a human

mediator, compared to two children paired with a robot mediator. However, the two children paired with the

robot mediator demonstrate increased shared focused attention (visual contact, physical proximity) and in

imitating smiling more than the children paired with the human mediator.

Index Terms—Mobile robotics, Pediatric rehabilitation, Low-functioning autism, Human-robot interaction.

Manuscript submitted January 25th, 2007. This work was supported in part by the Canada Research Chair (CRC), the Fonds

québécois de recherche sur la nature et les technologies (FQRNT) and the Canadian Foundation for Innovation (CFI). Audrey Duquette was a Master’s student at the Department of Psycho-Education of the Université de Sherbrooke, Québec

CANADA J1K 2R1 (e-mail: audreyduq@hotmail.com). F. Michaud holds the Canada Research Chair in Mobile Robotics and Intelligent Autonomous Systems. He is with the

Department of Electrical Engineering and Computer Engineering at the Université de Sherbrooke, Québec CANADA J1K 2R1 (phone: 819-821-8000 x 62107; fax: 819 821-7937; e-mail: Francois.Michaud@USherbrooke.ca). Contact author.

Henri Mercier is a Professor at the Department of Psycho-Education of the Université de Sherbrooke, Québec CANADA J1K 2R1 (e-mail: Henri.Mercier@USherbrooke.ca).

Exploring the Use of a Mobile Robot as an Imitation Agent with Children with Low-

Functioning Autism

Audrey Duquette, François Michaud, Henri Mercier

2

I. INTRODUCTION

Autism is one of the Pervasive Developmental Disorder subtypes (e.g., Asperger syndrome, Rett syndrome,

Childhood disintegrative disorder, Pervasive Developmental Disorder Not Otherwise Specified) described in DSM-

IV revised diagnostic tool [1], reaching 9 out of 10000 people [2]. Even though many intervention programs exist,

progress made by children with autism on overcoming their disorders is limited, and no program has yet

demonstrated how to initiate a desire within them to learn and communicate with the intent of sharing with others

instead of for satisfying primary needs [3].

This work focuses on children aged 5 that are diagnosed with low-functioning autism. These children show very

limited language skills and symbolic or pretend play [4,5]. They present similar sensory interests to 8-9 month old

typically developing children. However, their sensory play patterns are more repetitive [4], their imitation is selective

and used with an aim of increasing the stimuli [6]. They present unexploited abilities (e.g., attribute intentions to the

imitator; plan and induce imitative behaviors and understand incitation to imitate) [7,8]. They also show deficits in

sharing attention (avoids eye contact, does not respond to smiling) and conventions (poor imitation of facial

expressions and gestures such as moving the head to say no or raising a hand to say goodbye) for communicating

common interests [6]. Also noted is the quasi-absence of verbal language and pretend play [4]. These deficits are

explained by a difficulty in perceiving and understanding stimuli from their environment, affecting their

understanding of social signals (gestures, words and intentions of others) [9]. Typically developing 8-9 month old

children demonstrate communication intent from eye gaze and imitation of gestures. Thus, assuming that the

complexity of human interaction is limiting reciprocal communication of children with autism because of their

sensory impairments, we believe that children with low-functioning autism need interventions that take into account

their particular interests and their deficits in comprehension by using a predictable and simple medium, able to catch

their attention and that is also easy to understand.

Mobile robots show potential in this regard because they are predictable, simple and easy to comprehend [10] and

they can be designed in accordance of the particular interests and comprehension deficits of children with autism.

They can generate more interest and a wide variety of interplay situations compared to static objects, and also bring

into play social interactions skills (visual contact, imitation) [11]. More specifically, this exploratory study aims to

verify the research hypothesis that an animated object, more predictable and less complex than interacting with

humans, can make the child with autism demonstrate reciprocal communication, observed by: 1) the reduction of

avoidance mechanisms, namely repetitive and stereotyped play with inanimate objects; 2) an increase in shared

focused attention and shared conventions; and 3) the appearance of symbolic mode of communication like verbal

language [12]. This paper presents the results of this study, as follows. Section 2 presents a review of related works.

Section 3 explains the experimental scenario. Section 4 describes the robot designed to fulfill the requirements of the

experimental scenario. Section 5 presents the experimental methodology Section 6 describes the results obtained, and

3

the paper concludes with observations made from the results and finally by outlining future work.

2. MOBILE ROBOTS AND AUTISM

In 1976, the use of a remote controlled robot with a 7 year old boy diagnosed with autism was, we believe, the first

use of mobile robotic devices as a remedial tool for children [13]. In this work, the mobile robot was used to catalyze

communication in the boy, and positive results were reported. The AuRoRA project (Autonomous Robotic platform

as a Remedial tool for children with Autism) [14,15,16,17] is certainly now one of the best-known initiatives within

this subject area. AuRoRA investigates how a robot can become a `toy' that could possibly serve an educational or

therapeutic role for children with autism. They hope to encourage children with autism in coordinated and

synchronized interactions with the environment so as to help them develop and increase their communication and

social interaction skills [14]. Previously, the project conducted experiments with a Labo-1 mobile platform, a robot

with a rectangular body and four wheels, based on the observation that children with autism prefer predictable, stable

environments, and that they have difficulty interpreting facial expressions and other social cues. The AuRoRA

project was attempting to use the robot to bridge the gap between the complex and unpredictable world of human

social behavior and the safe predictable world of simple toys. It was shown that children with autism are capable of

distinguishing between animated and unanimated objects, expressing greater attention, laughter and vocal responses

toward a robotized platform compared with a toy truck of same dimension [14,16,18]. Other experiments conducted

by AuRoRA include a humanoid doll, Robota [19], engaged in interactions with children with autism. Robota has

been deployed in various experiments including the investigation of the possible therapeutic effects of a humanoid

robotic doll to autism therapy. Their approach was based on “the assumption that bodily interaction in imitative

interaction games is normally an important factor in a child's development of social skills and that teaching of such

skills (in a playful and exploratory context but nevertheless from an educational point of view focusing explicitly on

specific types of interactions) could help children with autism in coping with the normal dynamics of social

interactions” [20]. Robota, with its bidirectional communication capability (requests imitation and imitates) has been

used to investigate the effect of a robot's design (appearance) in facilitating and encouraging interaction of children

with autism [21]. In a longitudinal study, Robota was also used with the aim to encourage imitation and social

interaction skills. Different behavioral criteria (including Eye Gaze, Touch, and Imitation) were evaluated based on

the video data of the interactions [10]. Other uses of Robota include the study of joint attention [22] and also being

used as a salient object mediating joint attention between four children [23]. The experimental methodology followed

a case study evaluation using an analysis of interaction informed by conversation analytic principles [24]. Related to

this is the work of Kozima and Yano [25,26], suggesting that an infanoid robot (a humanoid torso designed to have a

youthful appearance) programmed to play ‘contingency-games’ could benefit both children with autism and typically

developing children in learning communication skills. Pioggia et al. [27] developed a robotic head with believable

facial display and explored its use in social skills and emotional therapy for an individual with autism. Finally, work

by Scassellati [28] is investigating the use of robots to help diagnose autism.

4

The initiative described in this paper differs from the AuRoRA project. The motivation is to determine how a

robot's motion, communication capabilities, and appearance can engage a child in interaction and once engaged, to

investigate the ability of the robot to initiate and sustain the child's interest and motivation to learn, thus facilitating in

the development of social and communication skills. We achieve this by designing and building custom-made mobile

robots with the unique needs of children diagnosed with autism in mind. Premack [29] advances the hypothesis that

the perception of intentionality in moving elements is innate to humans. An infant divides the world in two classes of

objects, i.e., whether or not it is self-propelled. For self-propelled objects, an infant has the ability to distinguish the

objects that can generate a change in their movements, with or without the assistance of another [30]. In the case of

an autonomous change (excluding cyclic movements such as those associated with a bouncing ball), Premack

suggests that infants perceive intentionality and show preference toward these objects. While adults can identify the

nature of a self-propelled object and associate intentionality to it, for infants what matters is not the object but the

type of movements it generates [31] and its velocity [32,33]. A child is also able to attribute self-propelled

capabilities once having seen the robot intentionally moving, and this association will continue even when the robot

has stopped [29]. Preliminary work [34,35,36] confirms that mobile robots, with their motion capabilities, their

appearances, their sounds and their decision-making capabilities, have the ability to not only engage, but also to hold

the attention of children with autism. This study wants to go one step further and design a mobile robot used

specifically to validate the hypothesis outlined in the introduction.

3. EXPERIMENTAL SCENARIO

Our experimental scenario consists of having a group of children interact with a robotic mobile mediator (animated

object with human-like appearance), and another group interact with a human mediator. The two mediators execute

the similar imitation play patterns involving three levels, namely facial expressions (joy, sadness, angry), body

movements (raise the arms, dance, move forward or backward), familiar actions with objects (to point to the hat and

request that it is given back, to point at the door, to point at the mediator’s picture) or familiar actions without objects

(wave hello or bye-bye, play peek-a-boo). For familiar actions, the mediator executes the actions and names it out

loud using simple words.

Based on the deficits of children with low-functioning autism and of their similarities to 8-9 month old typically

developing children, twenty activity sessions were elaborated, following the same general scenario, i.e.:

• Before entering the experimental room (a square room), the educator explains to the child that a play period is

planned by showing a picture of the mediator. This picture is either put on the door or on the wall inside the

room, depending on the activity session. The child enters the room, and 12 seconds later the educator directs the

child to the chair placed in the center. The child’s favorite toy is placed at its feet. The educator then sits on the

other chair placed near the door. A panel is placed in the room so that the mediator can hide behind it.

• Execute an activity session. Each activity session involves the three levels of imitation (facial expressions, body

movements and familiar actions). They are organized in increasing complexity (initially focussing more on facial

5

expressions and then progress onto body movements and familiar actions) to facilitate the child’s familiarization

with the mediator’s action. The activity sessions are composed of the imitative play patterns during which the

mediator asks the child to imitate the play. The twenty imitative play patterns are composed as follows (identified

by their session identification numbers): 1) 2) express happiness twice, and then anger once (after colliding with

a chair, indicating ‘no’ by nodding); 3) smile and then express happiness, twice; 4) 5) 6) say hello twice, and then

dance twice; 7) move both arms and then express happiness; 8) move both arms, and then say hello; 9) move

forward and then move backward, twice; 10) say hello twice and move forward; 11) say hello, move forward and

move backward; 12) 13) say hello, point to the hat and ask for the hat back, twice; 14) say hello, move backward,

point to the photo of the mediator; 15) say hello twice, move backward, point the photo of the mediator; 16) 17)

say hello, and express sadness twice after colliding with the panel; 18) say hello, ask the child to point the photo

of the mediator twice; 19) say hello, ask to point the photo of the mediator and play peek-a-boo twice hiding

behind the panel; 20) say hello, ask to point the photo of the mediator and play peek-a-boo four times. Imitative

play patterns in an activity session are executed in sequence, each one followed by a 12 sec observation period.

The reason why some actions are repeated twice in play patterns is to let the child integrate better the sequence of

actions in the play, in hope of getting him or her to react more. At the beginning of the activity session, the

mediator is either hiding behind the panel (2, 3, 4, 5, 6, 16, 17) or close to the child’s chair (being at a two arms

length distance for patterns 1, 7, 8, 9, 10, 11, 12, 13; or at one arm length distance for patterns 14, 15, 18, 19 20).

• At the end of each session, the mediator points the door and says “Door” to indicate to the child that the activity

is ending. The mediator then waves and says “Bye bye”, and goes behind the panel. When the educator goes

toward the child to leave the room, the robot exits from behind the panel, waves and says “Bye bye”, and again

goes back behind the panel.

In all of the activities, if the child imitates correctly the mediator’s activity, the mediator smiles and raises both its

arms and says “Happy!”.

4. TITO THE ROBOT MEDIATOR

Designing a mobile robot that can implement such activity sessions is a challenge, even if teleoperation (i.e.,

Wizard-of-Oz experimental settings) is used to navigate the robot and respond to the child's interaction. The four

major design considerations in designing such a robot are: (1) Overall Requirements - The robot should be able to

fulfill all the abilities required in the activity session; (2) Longevity Study - The robot should be able to correctly

function for a period of approximately two months; (3) Operator - The robot is to be operated by non-robotic experts;

(4) Experimental Analysis – Identifying the data that will be analyzed during evaluation can influence the design, the

choice of sensors and the capabilities of the robot.

Tito is the robot we developed to address these specifications, and is shown in Figure 1. Tito is 28 inches tall and is

colored red, yellow, and blue. Its clothes are washable and made of soft material. Tito has wheels to move, but its

6

structure shows two feet and two legs to emulate a humanoid shape. It has two arms that can move up and down

rapidly, a head that can rotate (making a ‘no’ gesture) and rise up (expressing surprise for making the peek-a-boo

gesture), a mouth (using light emitting diodes, for smiling), two eyes, a nose, hair (made from fiber optic cable that

can be illuminated) and a hat. Also, since we were interested in measuring eye gaze toward Tito, a small wireless

microphone-camera device was installed in one eye of the robot. Different parts of Tito's body can also be

illuminated, and it is able to sense if it is being shaken or if it has flipped over. Tito is sufficiently robust to sustain

rough interplay situations, and it is heavy enough so that it is difficult for a child to lift. It can sustain autonomous

action for approximately 1 hour before needing to be recharged. Tito was built in less than 6 months using modular

distributed subsystems previously designed or used on other robots (e.g., energy monitoring, locomotion control,

remote operation, sensing) [37], and a sound-generating device similar to the ones used on the RoboToys and Roball

[34,36]. The materials used to develop Tito cost approximately 2500 $CAD.

Fig. 1.Tito.

Tito generates vocal requests through pre-recorded messages, using a masculine voice with intonations that can be

interrogative, neutral or cheerful. Tito has a vocabulary of 25 words, which are used either in isolation or with

gestures and illuminated parts. The use of pre-recorded messages facilitates the use of appropriate voice intonations,

for which children with autism are more sensitive to than simply the use of words [38]. A wireless remote control

(using a Playstation video game controller) is used for teleoperation, and an on-board microcontroller enables pre-

programmed sequences of behaviors (motion and vocal messages) for Tito. Examples of pre-programmed behaviors

with vocal responses are: raising its left arm and saying “Hello!”; moving the left arm up and down three times whilst

7

saying “Bye bye”; pointing at an object whilst saying what the object is; making its hat fall and saying “Give hat”;

playing hide-and-seek and saying “Search Tito” or “Tito search”, and once found saying “Peek-a-boo!”; expressing

happiness by raising both arms, illuminating its mouth and saying “Happy!”; smiling whilst illuminating a body part;

raising one arm and saying “Arm”; moving forward or backward and saying “Forward” or “Backward”; singing and

rotating on the spot by alternatively raising each arm; or rotating its head and saying “No”. Tito also emits a sound

when it starts the execution of an experimental scenario, allowing synchronization of video data recorded with an

external camera. Tito’s activation button is hidden at the bottom of the robot so that the child is not tempted to play

with it. The timing between the interactions of the child (from sensory data and according to the experimental

scenarios) is recorded and stored internally on-board Tito.

5. EXPERIMENTAL METHODOLOGY

Our methodology consists of conducting an exploratory study following a single case protocol [39]. More

specifically, an ABA / AB’A model is adopted, where A represents the baseline level of children with autism with

the presence of the experimenter. The sample of children with autism is divided into two groups: one group paired

with the robot mediator (B), and the other with the human mediator (B’). Adopting a single case protocol for this

study is appropriate when considering the research hypothesis, i.e., studying the effect of a change in an individual

and to establish links between variables [40].

The children that participated in the trials are all diagnosed with low-functioning autism. They show a deficit in

imitation, also a severe delay in receptive and expressive language and have a sensori-motor interest that interferes

with the development of their communication skills. They can, however, have very different stereotyped repetitive

play patterns and interests. To ensure homogeneity of the population and a valid baseline level, the following

measures were taken. Children selected were diagnosed with autism as specified by the ADOS-G (Autism Diagnostic

Observation Schedule-Generic) module 1, showing severe disorders in receptive and expressive language skills and

deficits in symbolic games. Children reactions were observed and evaluated for 10 pre-determined actions of the

experimenter, using a pre-determined set of criteria. A third criterion is the result of the imitation tool PEP-R (Psycho

Educational Profile Review) [41]. These measures, supported also by the comments gathered from the educators

regularly working with the children, provided an appropriate baseline level for the composition of the sample group.

Selection of the children was done following a reasoned non-probabilistic sampling, based on the experimenter’s

judgment and in accordance to the subjects’ character [40]. All children were part of the same intervention group

from the Centre de réadaptation le Florès, Laurentides Québec. From the nine possible children around 5 years of age

available in this group, eight parents provided their consent, two were disqualified because they were able to talk

(few words), with one of them receiving private specialized care. Two others were also removed from the sample

because they attended the school only once a week. Therefore, the sample consisted of a homogeneous group of four

children addressing all criteria, 3 boys and 1 girl. Table 1 summarizes the characteristics of the participants according

8

to these selection criteria. Two of them were paired with the robot mediator (identified using labels R1 and R2 for

anonymous identification), and the other two children were paired with the human mediator (labeled E1 and E2).

Based on the similarities between the participants’ characteristics, child R1 is to be compared with child E1, and

child R2 with child E2. The overall average age is 5 years and 1 month old.

Table 1 – Population group for the trials.

Participants R1 R2 E1 E2

Age (years :months) 4 :4 5 :1 5 :5 5 :0

Communication Non-verbal Pre-verbal Non-verbal Pre-verbal

Stimulation Visual and vestibular

stimulation

Visual stimulation Visual, audible and

tactile stimulation

Visual, vestibular

and audible

stimulation

Mannerism Yes No Yes No

Imitation (PEP-R,

over 16 items)

Success = 0

Emerging = 2

Success = 5

Emerging = 6

Success = 0

Emerging = 1

Success = 4

Emerging = 5

Stimulation

(hours/week)

15 14 13.5 6

The trials were conducted in a 4.5 m 3.3 m room with a window, a chair and a toy familiar to the child placed in

the center of the room, the imitating agent (either Tito for children R1 and R2, or the experimenter playing the role of

the human mediator for children E1 and E2), the educator’s chair placed next to the door, a panel, a video camera and

a helper. The experimenter has previous clinical experience with children with autism, is 1.56 m tall and knelt during

the activity sessions. The experimenter was not known by the children prior to the trials, but they did meet a couple

of times before executing the experimental scenario.

Before executing the experimental scenario, trials were conducted with three children not part of the selected

sample of children with autism, to assess the feasibility of the experimental methodology, the validity of the

evaluation sheets, correct operation of the robot and the potential reactions from children. This enabled us to make

adjustments to the experimental scenario. For instance, the introductory phase before an activity session (explained in

Section 3) was derived from observations made in these pre-experimental trials. The time interval for each action in

an imitative play pattern was readjusted from 20 seconds (based on [42]) to 12 seconds, after observing that children

showed disinterest after this length of time. These pre-experimental trials also revealed the importance of conducting

a session that allowed the child to become familiar with the robot mediator prior to the actual experimental trials,

limiting therefore the novelty effects. This familiarization session consists of six steps: 1) the robot is off and located

in a corner of the room; 2) the robot is activated (with some illuminated parts) but remains immobile; 3) the robot is

9

activated and then proceeds to move around following the wall, without going too close to the child; 4) the robot

moves toward the child without touching and 5) touching; 6) the robot moves its arm and emits vocal messages. To

progress from one step to the other, the child had to show self-initiated interest toward Tito by going toward it and

touching it. These steps were repeated until no sign of distress or discomfort were observed. Note that when Tito was

used as the imitating agent, the experimenter remained in a corner of the room, teleoperating the robot according to

the experimental scenarios. The educator’s role during the trials is to provide assistance depending on the action in

the imitative play patterns and the child’s behavior (e.g., put the robot’s hat back if the child did not comply to the

request before 12 seconds or if he or she is having difficulties putting the hat back on).

Each child was exposed a total of 22 times to its paired mediator, at a rate of three times per week over seven

weeks and a maximum of two times per day (with a 15 minutes free play time in between). All the activity sessions

were recorded. The baseline level (A), conducted in the first and the 22nd exposure, consisted of the mediator

carrying out the following actions: say hello, express happiness, point and give back the hat, point to the mediator,

show the door to exit and say goodbye. The other twenty sessions (B/B’) were conducted by carrying out the activity

sessions following the experimental scenario described in Section 3. Each activity session is repeated twice during a

trial.

6. RESULTS

To analyze what happened during the trials, twenty-one evaluation sheets (i.e, one for the A and twenty for the

B/B’) were created, i.e., one for each activity session plus one for the baseline level. Each evaluation sheet lists the

child’s expressions or reactions observed according to the mediator’s imitative play patterns. These expressions or

reactions and the terminology were set according to the work of Camaioni and Aureli [43] and observations made

during the pre-experimental sessions. The following four categories of variables (i.e., expressions or reactions) are

observed during a session:

• Shared focused attention (4 variables): visual contact / eye gaze directed toward the mediator for longer

than 3 sec. [43], physical proximity (moving closer to two arm length distance from the mediator, with visual

contact), imitation of facial expressions (happiness, anger, sadness) or imitation of gestures that is not

directed toward the mediator. Shared focused attention here does not include joint visual attention, which

would involve having the child direct attention to an object that the mediator is looking at. Shared focused

attention means that the child shows an interest in the mediator’s eyes or body movements, or is responding

to having seen the mediator’s facial expressions or gestures by imitating them.

• Shared conventions (4 variables): imitation of facial expressions, gestures, actions and words, all directed

toward the mediator, with the purpose of sharing and communicating with the mediator. Shared conventions

occur when both the mediator and the child refer to a common point of interest through conventional means,

such as imitation of facial expressions, gestures, actions and words.

10

• Absence of sharing (7 variables): no visual contact or visual contact lasting less than 3 sec., leave the

communication area, avoid the mediator, repetitive sensori-motor play with their favorite toy, mannerisms,

ritual, aggression.

• Other phenomenon (4 variables): imitation of noise made by the mediator moving (e.g., motor noise with

Tito), imitation of words or gestures out of context, or any other behavior not listed in the previous

categories.

Dichotomic coding (presence or absence) of these variables was done over 12 sec. windowing, by two coders (98%

fidelity, based on data from the pre-experimental sessions) looking at the video footage of the sessions. If the child

exhibited both sharing and absence of sharing in the same 12 sec. window, the presence of reciprocal communication

was prioritized. This data was used to conduct chronological series analysis with cross-section multiple regression

analysis between mediators (with ** p < 0.05). The results do not reveal significant variances between the baseline

sessions (A, one conducted at the beginning of the experimental scenario and one conducted at the end of the

experimental scenario). Therefore, analysis focuses on the child’s behavior in relation to the activity session and the

child’s mediator, for which significant influences appeared. The results are presented in the following sections.

6.1 Shared focused attention

Figure 2 illustrates the chronological analysis for the shared focused attention variables. The proportion of sharing

is calculated by summing the number of occurrences of shared focused attention variables and dividing it with the

number of variables planned for the activity session. Non-verbal participants (R1 and E1) demonstrated less shared

focused attention than pre-verbal children (R2 and E2). Also, children paired with the robot mediator demonstrated

more shared focused attention than the ones paired with the human mediator.

Fig. 2.Chronological series analysis for shared focused attention variables.

Analyzing the results for the robot mediator reveals a statistically significant higher proportion in shared focused

11

attention, in comparison to the human mediator trials, for only two of the four variables (visual contact: standardized

regression coefficient =0.35**; physical proximity: =0.40**), while a smaller proportion is observed for

undirected gesture ( =-0.28**). This means that children paired with the robot exhibited more visual contact and

proximity with their mediator compared to the children paired with the experimenter, and demonstrated less gestures

that were not directed toward the robot mediator compared to the children paired with the human mediator.

Fig. 3.Occurrences of shared focused attention variables in relation to the imitative play patterns.

To further analyze this observation, we examined more closely the sessions during which children paired with the

robot mediator (R1 and R2) performed better that the children paired with the human mediator (E1 and E2) (i.e.,

sessions 1, 4, 11 to 14, 16 and 17). Figure 3 shows which imitative play patterns initiated such increases in shared

focused attention. R1 and R2 were more interested by the expression of happiness and sadness by Tito compared to

the same expression by the experimenter. This was displayed by visual contact, physical proximity and laughter

(when sadness was expressed). For actions with objects (like pointing and giving back the hat), R1 and R2 made

more visual contact and moved toward the robot, while E1 and E2 preferred to engage into a repetitive play routine

with the hat. For actions without objects, even though all children showed interest in imitative play (such as saying

hello and goodbye), R1 and R2 had more eye gaze directed toward the robot and they also moved closer to the robot.

When comparing children with the same language level (R1 with E1, and R2 with E2), we observe that children

paired with the robot mediator exhibited more shared focused attention, mostly in activities involving facial

expressions, actions with objects and actions without objects. Also, because of their severe deficits, non-verbal

12

children displayed less shared focused attention than pre-verbal children in all activities.

6.2 Shared conventions

Figure 4 illustrates the chronological analysis for shared convention variables. Again, non-verbal participants (R1

and E1) demonstrated less shared conventions than pre-verbal children (R2 and E2). Only one of the four variables

shows statistically significant results, with a decrease in the imitation of words ( =-0.44**). This indicates that

pairing a child with autism with the robot mediator had a negative influence on the imitation of words. For the other

variables, even though they revealed to be not statistically significant, they indicate that pairing a child with the robot

mediator led to an increase in facial expression (mostly happiness, for R2 compared to E2) but a decrease in the

imitation of gestures and actions. This is understandable considering Tito’s limited physical capabilities.

0

0,01

0,02

0,03

0,04

0,05

0,06

0 22 44 66 88

Exposures

Pro

port

ion

of

share

d

con

ven

tion

s

R1 E1 E2R2

Fig. 4.Chronological series analysis for shared convention variables.

Figure 5 illustrates the number of imitations exhibited for shared conventions in relation to the mediator’s imitative

play patterns. Shared conventions such as imitation of body movements and of familiar actions are higher with the

two children paired with the human mediator (E1 and E2). The two children paired with the robot mediator (R1, R2)

imitate facial expressions more than the children paired with the human mediator (E1, E2). Imitation of words mostly

appeared only for one participant (E2), who started to imitate saying “Bye bye” at the end of the sessions, even

initiating saying these words before the human mediator, two times in context and one time out of context. E2 also

imitated facial expression (smiling), some gestures (raising an arm) and some actions (hello and goodbye gestures),

but non-related to objects (contrary to R2).

13

Fig. 5.Occurrences of imitation in relation to shared convention variables.

6.3 Absence of sharing

In accordance with Figure 2 and Figure 4, periods when there was an absence of sharing were observed more with

non-verbal children (R1 and E1) compared to pre-verbal children (R2 and E2). Two variables showed statistically

significant influences, mainly that the children paired with the robot mediator (R1 and R2) left the communication

area ( = 0.38*) and initiated rituals ( = 0.17*) more compared to the children paired with the human mediator.

Leaving the communication area may be surprising because, as described in Section 6.1, R1 and R2 showed

increased proximity with the mediator. However, R1 and R2 also presented less absence of sharing than E1 and E2

(but statistically non-significant: no visual contact =-0.32; repetitive sensori-motor play =-0.03).

14

6.4 Discussion and other observations

The results indicate that non-verbal children demonstrate less interest and participation than pre-verbal children.

However, children exposed to the robotic mediator showed reduced repetitive plays with inanimate objects of interest

(their favorite toy), and no repetitive or stereotyped behavior toward the robot. These reduced behaviors can be

explained by the interest children have for the sensory properties of the robot (movements, colors, lights). Their

attention is focused on the robot rather than on the inanimate object. These results are the ones reported in [14,18]

indicating increased attention directed by children with autism towards a robot compared to an object such as a toy

truck. The trials also revealed that children exposed to the robot mediator demonstrated more visual contact and

proximity compared to children paired with the human mediator. This confirms the hypothesis that shared focused

attention is facilitated by the appealing characteristics and predictability of the robot. In fact, when the robot

expressed emotions of joy and sadness or made simple actions, children with autism reacted to the voice intonation,

the lights that represented emotions and the simple slow motion of the robot, by looking and moving toward the

robot. Tito was designed to facilitate comprehension of these interaction cues by children with autism.

For shared conventions however, the results do not confirm that interactions increased with the robot mediator.

Smaller occurrence of imitation by the children paired with the robot can be explained by many factors. First, having

two non-verbal children with very low-functioning autism does not allow us to observe use of words or imitation to

communicate. Nadel [7,8] indicates that this form of communication needs some cognitive and meta-cognitive

abilities that appear after 18 months old, such as the capacities to attribute intentions to the imitator and to understand

incitation to imitate. These abilities were not observed with the non-verbal children with autism (R1 and E1).

Looking at the pre-verbal children (R2 and E2), the child paired with the robot (R2) presented less shared

conventions in general than the child paired with human. This observation is explained by the difficulty in

understanding communication intent from gestures, actions and words made by Tito. Tito was not used to imitate the

child after the child imitated it, making the imitative plays unidirectional. Tito also lacked reciprocity compared to a

human being. The use of a different robot like Robota [11] has shown better performance in responding to the child’s

gestures. The pre-verbal child with autism exposed to the human mediator (E2) showed better imitation in familiar

actions and words (waving and saying goodbye), but used it as a means of getting out of the room and ending the

interaction.

On the other hand, in the study, the imitation of facial expression of joy (smiling) appears more with children

exposed to Tito than with children exposed to the experimenter. The results are explained by the simplicity to

comprehend the expression of the robot (illuminated smile). Smiling is the first indicator of communication named

by Spitz [44]. Tito’s capabilities were therefore sufficient to allow the pre-verbal child with autism (R2) to develop

this first form of communication. The pre-verbal child with autism exposed to Tito also showed better imitation of

actions with objects (giving back the hat to the robot, pointing toward the door) and sustained interactions with the

robot. Emergent interactions were noted with the pre-verbal child with autism paired with the robot. They were

15

observed through the imitation of gestures (handing back the hat, pointing at the door, waving good bye) and

imitation of motor noises at the same rhythm as the robot (more specifically when the robot was waving).

Furthermore, the child reproduced the same posture as the robot (e.g., by kneeling).

The Ricard and Gouin-Décarie familiarization process [45] was observed in the trials with children exposed to

robot mediator, by sharing attention to get interaction (proximity, smiling, visual contact). In general, children with

low-functioning autism do not initiate interaction appropriately. They often use adults as objects to satisfy their

primary needs, and they can be avoidant or overbearing. Further analysis of video footage revealed that the robot

appears to be an interesting way in helping children initiate contact, something that typically developing children can

do when meeting strangers. Children firstly created a distance with the robot, allowing them to initiate interactions

such as eye contact and smiles. Then children moved closer to the robot, enabling them to continue interacting with

the robot. Even when the robot hid behind the panel, children enthusiastically sought the robot. Despite at times the

children displaying ritual behavior and at times also leaving the communication area, which can be associated to an

avoidance behavior, they did so as part of such familiarization process. This pattern was not observed with the

children paired to the human mediator, nor did behavior associated with a Ricard and Gouin-Décarie familiarization

process.

Finally, another interesting observation is that children eventually made the discovery that Tito was teleoperated,

which generated enthusiastic interactions between the child and the experimenter. This concurs with Robins et al.'s

[11] observation that a robotic device reveals to be an interesting intermediate between the child and an educator.

II. CONCLUSION

The goal of this work is to address the research hypothesis that an animated object like a robot, more predictable

and less complex in its interacting modalities compared with a human, can make a child with autism demonstrate a

reduction in the use of avoidance mechanisms, namely repetitive and stereotyped play patterns with inanimate

objects, increase in shared focused attention and shared conventions; and facilitate the appearance of symbolic modes

of communication like verbal language. The results confirm only parts of the research hypothesis. Children paired

with the robot mediator showed increased shared focused attention (visual contact, physical proximity) than the

children paired with the human mediator in all types of imitation play patterns including facial expressions, body

movements, familiar actions with objects and familiar actions without objects. This validates that Tito the robot has

appealing characteristics for interacting with children with autism. However, forms of shared conventions such as

imitation of body movements and of familiar actions are higher with the two children paired with the human. This

may be explained by with the children with low-functioning autism having more difficulty in understanding the

communication intent from the limited motion capabilities of the robot.

Even though conducting an exploratory study following a single case protocol has known limitations such as the

16

difficulty to generalize results [39], it revealed to be a valuable experience in terms of validating the use of a custom-

designed mobile robot as a pedagogical tool for children with autism. Throughout the sessions, Tito was easy to use

and required no maintenance during the experiments. In addition, no session had to be interrupted because a child

felt uneasy in the presence of the mediator, or behaved inappropriately. The study gives insight into the processes

needed for decreasing children’s anguish and increasing their attention to learn or to see forms of communication

emerge. The results are very encouraging and support the continuation of work on this research question. As such we

plan to repeat these trials with a greater number of subjects and consolidate these conclusions. The role of the robot is

not to replace human educators but rather to assist them in helping a child acquire social and communication skills

that can later be exploited in social interaction with people. To do so, future work will also involve a typically

developing child serving as a model for understanding the communication intents of the robot to help children with

autism develop interaction skills.

ACKNOWLEDGMENT

The authors would like to thank the CRDI Le Florès, Laurentides, for their collaboration with this work. Thanks

also to Tamie Salter for her helpful comments on this manuscript.

REFERENCES

[1] American Psychiatric Association, DSM-IV-TR Diagnostic and Statistical Manual of Mental Disorders-IV –

Text Revision, 2000.

[2] E. Fombonne, Études épidémiologiques de l’autisme et des troubles apparentés, Psychiatrie, recherche et

intervention en santé mentale de l’enfant (PRISME), Approcher l’énigme de l’autisme, 34: 26-32, 2001.

[3] P. Doehring, Programmes d’intervention comportementale auprès des enfants autistes: quelles sont les

meilleurs? Psychiatrie, recherche et intervention en santé mentale de l’enfant (PRISME), Approcher l’énigme

de l’autisme, 34: 80-91, 2001.

[4] R. Blanc, M. Gomot, M.P. Gattegno, C. Barthélémy, J.L. Adrien, Les troubles de l'activité symbolique chez des

enfants autistes, dysphasiques et retardés mentaux et l'effet de l'étayage de l'adulte. Revue Québécoise de

Psychologie, 2002.

[5] A. M. Wetherby, C. A. Prutting, Profiles of communicative and cognitive-social abilities in autistic children,

Journal of Speech and Hearing Research, 28: 364-377, 1984.

[6] M. Lemay, L'autisme aujourd'hui. Paris:Éd. Odile Jacob, 2004.

[7] J. Nadel, Imitation and imitation recognition: Functional use in preverbal infants and nonverbal children with

autism. The Imitative Mind Development Evolution and Brain Bases. Eds Meltzoff and Prinz, 2002.

[8] J. Nadel, A. Revel, P. Andy, Ph. Gaussier, Toward communication, first imitations in infants, low-functioning

children with autism and robots. Interaction Studies, 5(1):45-74, 2004.

17

[9] M. Zilbovicius, Imagerie cérébrale et autisme infantile. Revue Cerveau et Psychologie, 2004.

[10] J. Nadel, Imitation et autisme, Revue Cerveau et Psycho, 68-71, 2004.

[11] B. Robins, K. Dautenhahn, R. te Boekhorst, A. Billard, Effects of repeated exposure to a humanoid robot on

children with autism, in Proc. Cambridge Workshop on Universal Access and Assistive Technology, Springer-

Verlag, pp. 225-236, 2004.

[12] A. Duquette, L’exploration d’une intervention avec un robot-jouet animé chez les enfants autistes, Master’s

thesis, Department of Psycho-Education, Université de Sherbrooke, 2005.

[13] S. Weir, R. Emanuel, Using LOGO to catalyse communication in an autistic child. Technical Report DAI

Research Report No. 15, University of Edinburgh, 1976.

[14] K. Dautenhahn, Design issues on interactive environments for children with autism, in Proc. International

Conference on Disability, Virtual Reality and Associated Technologies, 2000.

[15] K. Dautenhahn, I. Werry, J. Rae, P. Dickerson, P. Stribling, B. Ogden, Robotic playmates: Analysing interactive

competencies of children with autism playing with a mobile robot, in Intelligent Agents – Creating

Relationships with Computer and Robots, Kluwer Academic Publishers, 2002.

[16] K. Dautenhahn and I. Werry, A quantitative technique for analyzing robot-human interactions, in Proc.

IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pp. 1132-1138, 2002.

[17] K. Dautenhahn, I. Werry, “Towards interactive robots in autism therapy: Background, motivation and

challenges,” Pragmatics and Cognition, vol. 12, no. 1, pp. 1-35, 2004.

[18] I. Werry, K. Dautenhahn, W. Harwin, Evaluating the response of children with autism to a robot, in Proc.

Rehabilitation Engineering and Assistive Technology Society of North America (RESNA), 2001.

[19] A. Billard, Play, dreams and imitation in Robota, in Proc. Workshop on Interactive Robotics and Entertainment,

pp. 53-59, 2000.

[20] K. Dautenhahn, A. Billard, Games children with autism can play with Robota, a humanoid robotic doll, in Proc.

Cambridge Workshop on Universal Access and Assistive Technology, Springer-Verlag, pp. 179-190, 2002.

[21] B. Robins, K. Dautenhahn, R. te Boekhorst, A. Billard, Robots as assistive technology – Does appearance

matter? in Proc. IEEE International Workshop on Robot and Human Interactive Communication, 2004.

[22] B. Robins, P. Dickerson, P. Stribling, K. Dautenhahn, Robot-mediated joint attention in children with autism: A

case study in robot-human interaction, Interaction Studies: Social Behaviour and Communication in Biological

and Artificial Systems, 5(2):161-198, 2004.

[23] B. Robins, P. Dickerson, K. Dautenhahn, Robots as embodied beings – Interactionally sensitive body

movements in interactions among autistic children and a robot, Proceedings IEEE International Workshop on

Robots and Human Interactive Communication, 54-59. 2005.

18

[24] B. Robins, P. Dickerson, P. Stribling, K. Dautenhahn, Robot-mediated joint attention in therapy and education of

children with autism: can a small humanoid robot help encourage social interaction skills?, Universal Access in

the Information Society, 4(2), 2005.

[25] H. Kozima and H. Yano, Designing a robot for contingency-detection game, in Proc. International Workshop on

Robotic and Virtual Agents in Autism Therapy, 2001.

[26] H. Kozima, C. Nakagawa, Y. Yasuda, Wowing together: What facilitates social interactions in children with

autistic spectrum disorders, Proceedings International Workshop on Epigenetic Robotics, Modeling Cognitive

Development in Robotics Systems, 177-, 2006.

[27] G. Pioggia, M. Ferro, M. L. Sica, G. Dalle Mura, S. Casalini, D. De Rossi, R. Igliozzi, F. Muratori, Imitation and

learning of the emotional behaviour: Towards an android-based treatment for people with autism, Proceedings

International Workshop on Epigenetic Robotics, Modeling Cognitive Development in Robotics Systems, 119-

125, 2006.

[28] B. Scassellati, Quantitative metrics of social response for autism diagnosis, Proceedings IEEE International

Workshop on Robots and Human Interactive Communication, 2005.

[29] D. Premack, “The infant’s theory of self-propelled objects,” Cognition, 26: 1-16, 1990.

[30] M. D. Hauser, “A nonhuman primate’s expectation about object motion and destination: The importance of self-

propelled movement and animacy,” Developmental Science, 1(1): 31-37, 1998.

[31] R. Sekuler, “Visual motion perception,” Handbook of Perception: Volume 5 – Seeing, E. C. Carterette and M. P.

Friedman (Eds), Academic Press, 387-430, 1975.

[32] R. N. Aslin, S. L. Shea, “Velocity threshold in human infants: implications for the perception of motion,”

Developmental Psychology, 26: 589-598, 1990.

[33] J. L. Dannemiller, R. Freeland, “The detection of slow stimulus movement in 2- to 5- month olds,” Journal of

Experimental Child Psychology, 47: 337-355, 1989.

[34] F. Michaud and C. Théberge-Turmel, Mobile robotic toys and autism, in Socially Intelligent Agents – Creating

Relationships with Computers and Robots, K. Dautenhahn, A. Bond, L. Canamero and B. Edmonds, eds,

Kluwer Academic, pp. 125-132, 2002.

[35] F. Michaud, A. Duquette, and I. Nadeau, Characteristics of mobile robotic toys for children with pervasive

developmental disorders, in Proc. IEEE International Conference on Systems, Man, and Cybernetics, pp. 2938-

29432003.

[36] F. Michaud, T. Salter, A. Duquette, and J.-F. Laplante, Perspectives on mobile robots used as tools for pediatric

rehabilitation, to appear in Assistive Technologies, 2007.

19

[37] F. Michaud, D. Létourneau, M. Arsenault, Y. Bergeron, R. Cadrin, F. Gagnon, M.-A. Legault, M. Millette, J.-F.

Paré, M.-C. Tremblay, P. Lepage, Y. Morin, S. Caron, “Multi-modal locomotion robotic platform using leg-

track-wheel articulations,“ Autonomous Robots, Special Issue on Unconventional Robotic Mobility, 18(2): 137-

156, 2005.

[38] J.-L. Adrien, Autisme du jeune enfant. Développement psychologique et regulation de l’activité, Collection

Autisme, éd Expansion Scientifique Française, 1996.

[39] A.E. Kazdin, Statistical analysis for single-case experimental designs. Single Case Experimental Designs. Eds

Hersen and Barlow, 1976.

[40] M.-F. Fortin, Le processus de la recherche, de la conception à la réalisation, Québec Décarie Éditeur, 1996.

[41] E. Schopler, R.J. Reichler, A. Bashford, M.D. Lansing, L.M. Marcus, The Psychoeducational Profile Revised

(PEP-R). Austin Pro-Ed., 1990.

[42] T. Charman, S. Baron-Cohen, J. Swettenham, G. Baird, A. Cox, A. Drew, Testing joint attention, imitation, and

play as infancy precursors to language and theory of mind, Cognitive Development, 15: 481-498, 2000.

[43] L. Camaioni, T. Aureli, Trajectoires développementales et individuelles de la transition vers la communication

symbolique. Enfance, 3, 2002.

[44] R.A. Spitz (1976). De la naissance à la parole : la première année de vie. Paris, Presse Universitaires de France,

5e éd., 1976.

[45] M. Ricard, T. Gouin-Décarie, L’humain et l’inanimé pour l’enfant de 9-10 mois. Enfance, 45 (4), 351-360,

1990.

20

Audrey Duquette received her bachelor’s degree (’02) in psychoeducation and she has completed her Master’s degree (’05) in psychoeducation at the Université de Sherbrooke.

Her research interests cover interventions using play, imitation and mobile robot for children with autism to improve their communication and social interaction.

Since 2004, she is working at Ste-Justine Hospital as a psychoeducator performing assessments and follow-up of children between the ages of 0 to 5 years old, mainly with pervasive development

disorders and autism. Audrey Duquette is a member of OCCOPPQ (Ordre des conseillères et conseillers en orientation et psychoéducatrices et psychoéducateurs du Québec).

François Michaud (M’90) received his bachelor’s degree (’92), Master’s degree (‘93) and Ph.D. degree (‘96) in electrical engineering from the Université de Sherbrooke, Québec Canada. After completing postdoctoral work at Brandeis University, Waltham MA (’97), he became a faculty member in the Department of Electrical Engineering and Computer Engineering of the Université de Sherbrooke, and founded LABORIUS, a research laboratory working on designing intelligent autonomous systems that can assist humans in living environments. His research interests are in architectural methodologies for intelligent decision-making, design of autonomous mobile

robotics, social robotics, robot for children with autism, robot learning and intelligent systems. Prof. Michaud is the Canada Research Chairholder in Autonomous Mobile Robots and Intelligent Systems. He is a member of IEEE, AAAI and OIQ (Ordre des ingénieurs du Québec). In 2003 he received the Young Engineer Achievement Award from the Canadian Council of Professional Engineers.

Henri Mercier received his Master’s in psychoeducation and his Ph.D. in clinical sciences from the Université de Montréal, Québec Canada. He is a retired Professor from the Department of Psycho-Education of the Université de Sherbrooke and a psychotherapist. His research interest focusses on the integration of children with Pervasive Developmental Disorders and the manifestation of depression in children.