an augmented reality gamebook for children with autism spectrum disorder (asd)
TRANSCRIPT
An Augmented Reality GameBook for Children with Au8sm Spectrum Disorder (ASD) Jorge Brandão University of Minho/Algoritmi R&D Centre, Guimarães, Portugal
Pedro Cunha University of Minho/Algoritmi R&D Centre, Guimarães, Portugal
José Vasconcelos University of Minho/Algoritmi R&D Centre, Guimarães, Portugal
Vítor Carvalho University of Minho/Algoritmi R&D Centre, Guimarães, Portugal IPCA-‐EST, Barcelos, Portugal
Filomena Soares University of Minho/Algoritmi R&D Centre, Guimarães, Portugal
InternaKonal conference focused on e-‐learning in the workplace June 10-‐12, 2015, at Columbia University in New York.
• WHAT IS AUTISM
• ABSTRACT • ABOUT AUTISM
• INTRODUTION • THE GAMEBOOK • CHARACTERS • SCENARIOS • IMPLEMENTATION
• CONCLUSIONS • GAME DEMONSTRATION
SUMMARY
• Technologies, like augmented reality (AR), have the ability to catch children’s imaginaKon and to promote their aYenKon. • Children with ASD usually have difficulty to recognize facial expressions and to understand associated emoKons. • We propose to design and develop an innovaKve GameBook to assist children with ASD to recognize and acquire emoKons by engaging their aYenKon and moKvaKon. • The GameBook will contain a story that describe some scenarios and real world situaKons which will conduct the children to become involved on ficKonal contents associated with emoKons.
ABSTRACT
ABSTRACT Play with AR Avatar
Play Quiz Games
Play Memory Games
The child will have to interact with these scenarios, by playing with one 3D AR avatar with different
facial expressions
In this GameBook the child will have to idenKfy the correct
emoKonal face to the situaKon in a Quiz Game.
This GameBook the child will have to idenKfy the correct emoKonal face to the situaKon in a Quiz
Game.
x x
AUTISM SPECTRUM DISORDER IN USA
Source: Center for disease Control and PrevenKon -‐ hYp://www.cdc.gov/ncbddd/auKsm/data.html
“The AuKsm Society esKmates that the United States is facing almost $90 billion annually in costs for au8sm. (This figure includes research, insurance costs and non-‐covered expenses, Medicaid waivers for auKsm, educaKonal spending, h o u s i n g , t r a n s p o r t a K o n , employment, related therapeuKc services and caregiver costs.)” hYp://www.auKsm-‐ society.org/about-‐auKsm.
hWp://thautcast.com/drupal5/category/tags/centers-‐disease-‐control
HIGH FUNCTIONING
LOW FUNCTIONING
1 out of every 54 boys in the United States has an au8sm spectrum disorder
hYp://thautcast.com/drupal5/category/tags/centers-‐disease-‐control
1 out of every 252 girls in the United States has an au8sm spectrum disorder
hYp://thautcast.com/drupal5/category/tags/centers-‐disease-‐control
hYp://thautcast.com/drupal5/category/tags/centers-‐disease-‐control
INTRODUTION • Technology is ubiquitous and can be a safe and good way to moKvate and engage children in interacKve learning acKviKes in order to promote their cogniKve and social skills
• It is being used increasingly on a variety of pedagogical contexts, both as assisKve technologies and as tools for helping us to understand user’s moKvaKon
• It has the potenKal to provide individually tailored intervenKons that are suitable for a wide variety of abiliKes • Technological intervenKons allow its use at different speeds and locaKons, and never lose paKence with the frequent repeKKon that many people with ASD has
INTRODUCTION
• ASD children struggle with significant relaKonships and behavioural challenges that in most cases have serious implicaKons for inclusion social adulthood
• One of the most promising applicaKons in the intervenKon process of ASD children came from developing tools for promoKng children’s social and communicaKon skills
• This offers the opportunity to reduce some of the crippling personnel costs associated with auKsm care
INTRODUTION
• Children with ASD usually have difficulty to recognize facial expressions and to understand associated emoKons, to imitate or use emoKonal expressions, to understand and control their own emoKons, or to interpret emoKons or empathy with others
• Several studies show that the majority of people with ASD exhibit a natural affinity with technology and a posiKve ahtude towards computer-‐based training
INTRODUTION • Technologies, like augmented reality (AR), have the ability to catch the children’s imagina8on and to promote their aWen8on, because they can experiment ar8ficial, safe and fascina8ng environments.
• AR is an opKmal interface technology and a helpful tool to support auKsKc children capabiliKes. With AR it is possible to create more aYracKve and interacKve interfaces that can be manipulated by hand, without using convenKonal peripherals such as the keyboard and the mouse. • This characterisKc offers greater interacKon between the child and the object, raising their interest and their curiosity, providing greater adherence to the task/acKvity.
THE GAMEBOOK Given the potenKal benefits of AR we aim to design a tool based on AR in order to improve capabiliKes of ASD children Our goal is to create an innovaKve serious game based on a book style with interacKon inputs to assist children with social interacKon and communicaKon skills. We aim to promote children with ASD recognizing and acquiring emoKons by engaging their aYenKon and moKvaKon, increasing their competence on this handicap.
THE GAMEBOOK
The GameBook presents the story of Tobias’s adventure during a visit to a zoo park. The player can read the story and interact with it. The story will describe scenarios and Tobias’ interacKons with animals and real world situaKons which will conduct the children to become involved on ficKonal contents associated with emoKons.
THE GAMEBOOK
We believe that the GameBook will put the ability of the child to interact with the story and to promote his/her imagina8on and engaging.
hYp://www.tobiasadventures.in/thezoo
The GameBook can be played on any mobile technology, such as a tablet, smartphone or
laptop, with either an external web camera or an inbuilt camera.
The GameBook is available on:
GAMEBOOK CHARACTERS • The character Tobias was designed to capKvate the
children aYenKon and to empathize with them. • On his design was pretended to promote a face with
characterisKcs that allows the players to feel some aYracKon with him.
• The expressions are designed by special details in the
avatar, essenKally, with his lips and eyes. • It was given a cartoonish look to Tobias, which was
deemed by the group members as potenKally more familiar and more fun to the children than a photo-‐realisKc alternaKve.
GAMEBOOK SCENARIOS The game includes a total of 5 different scenarios, designed to involve the child
on real life situa8ons. The scenarios, all in the zoo, will promote the child’s contact with different environments and
will create some emoKonal reacKon on Tobias.
ANIVERSARY GO TO THE ZOO WITH ELEPHANT WITH LION WITH BIRD
IMPLEMENTATION OF THE GAMEBOOK To create this serious game, the following technologies were used:
The 3D flip book uses the HTML5/CSS3 and JavaScript/Jquery web languages. To develop the AR feature it was selected the ArUco® (Augmented Reality library from the University of Cordoba). This opKon was held on the easiness of use, reliability and cross-‐plakorm compaKbility.
CONCLUSIONS AND FUTURE WORK • The aim of this project was to make available a cost effec8ve tool implemented in a plagorm
usually pleasant to ASD children. • In order to test the GameBook, an exploratory study in a classroom context with ASD children
will be performed. • In the near future, a system that will allow monitoring the child’s results and these outcomes
will be available on the website of the game. • Furthermore, new stories, as for example, Tobias adventures in the Sea or in the Space, with
new characters, scenarios and situa8ons can be easily included. • It is the authors’ belief that the GameBook will increase ASD children engagement in the
learning ac8vi8es promo8ng their memory, cogni8ve and social skills and allowing them to reinforce facial expression recogni8on.
GAME DEMONSTRATION
QUESTIONS?
Jorge Brandão University of Minho/Algoritmi R&D Centre, Guimarães, Portugal E-‐mail: [email protected]
Pedro Cunha University of Minho/Algoritmi R&D Centre, Guimarães, Portugal E-‐mail: [email protected] @gmail.com
José Vasconcelos University of Minho/Algoritmi R&D Centre, Guimarães, Portugal E-‐mail: [email protected]
Vítor Carvalho University of Minho/Algoritmi R&D Centre, Guimarães, Portugal IPCA-‐EST, Barcelos, Portugal E-‐mail: [email protected]
Filomena Soares University of Minho/Algoritmi R&D Centre, Guimarães, Portugal E-‐mail: [email protected]