www.asc-inclusion.eu

A project to develop software to assist people with autism to recognise, understand and

express emotions through facial expressions, voice and body gestures

Autism & EmotionsAutism & Emotions• Difficulties recognizing and understanding mental states:

– Their own– In other people

• Aspects of difficulty include: – Facial expressions– Vocal intonation – Body gestures and body language – The integration of the above in context

• Difficulties in expressing emotions non-verbally, in all the above aspects.

• Emotion recognition and expression are not learned spontaneously >>> need to be taught explicitly.

Risk of social exclusion for individuals with autism is significantly increased without these skills.

www.ascinclusion.eu

Project IntroductionProject Introduction

• A project to develop interactive games to assist people with autism to recognise, understand and express emotions through facial expressions, voice and body gestures

• Primarily for children aged 5 to 10, high functioning

•Can serve teenagers and young adults (with proper accommodations)

•Can serve more populations that are socially challenged, such as: ADHD, Hearing impaired

Project Project PartnersPartners

www.ascinclusion.eu

Prof. Simon Baron-Cohen, head of the Autism Research Centre at Cambridge University, UK

Prof. Sven Bölte, Karolinska Institutet, Sweden

Dr. Ofer Golan, Bar-Ilan University, Israel

Prof. Peter Robinson, head of Cambridge Computer Lab, UK

Prof. Björn Schuller, Technische Universität München, Germany

Prof. Antonio Camurri, University of Genova, Italy

Autism-Europe (AE) Executive comitee

Project Method & Project Method & RationaleRationaleThe ASC-Inclusion software will:•Analyse the children’s facial expressions, vocal intonations and gestures (standard microphones and webcams);•Train children to recognise and express emotions through Interactive lessons, fun games, animations, video clips;•Personalise settings according to children’s individual needs;•Support therapists and parents with professional information, reports on children’s progress and forums to interact with other carers.

www.ascinclusion.eu

How the software will How the software will workworkInnovation meets interventionInnovation meets interventionThe program will consist of 3 state-of-the-art subsystems for: •Facial expressions•Vocal intonation•Body gestures

The subsystems will be integrated together in an online virtual world platform to provide the user with a comprehensive and motivating learning experience.

www.ascinclusion.eu

How the software will How the software will workworkFacial expressions

www.ascinclusion.eu

How the software will How the software will workworkVocal intonation

www.ascinclusion.eu

How the software will How the software will workworkBody gestures

www.ascinclusion.eu

Environment – Environment – Narrative Narrative

Environment narrative:

Complete fun learning

tasks to earn more fun

stuff!

The players are

researchers in an

international

research camp

located in the

jungle.

They research

human behaviour

and emotional

expression.

Avatar: Create your Avatar: Create your charactercharacter

Click on different shapes of mouth, eyes, eyebrows etc. to express your mood

Main area: The research Main area: The research CenterCenter

Get a task, see your research status, meet the mentor, use research tools

Motivational Elements: Motivational Elements: HomeHome

Take a canoe to your home or visit a friend’s home, design your home

Motivational Elements: Motivational Elements: GoodiesGoodies

Use the virtual coins you earned by completing “research missions” (=learning tasks) to purchase items for your home,

avatar and collections

Motivational ElementsMotivational Elements:: GamesGames

Play cool games using your virtual money, meet friends

Games: Facial expressionGames: Facial expression

Mahjong: Attention to detailsInitial level: Match happy faces.

Advanced level: Match different people who feel the same.

Games: Emotional tone Games: Emotional tone of voiceof voice

Recognise what the speakers feel according to their voice

Games: Body language Games: Body language hintshints

Edit your own movie using the hints you’ve learned

Games: Cross-modal Games: Cross-modal trainingtraining

Match picture to sound: Listen to Timmy. Click on the picture of the character who feels like Timmy.

Interactive books: Social Interactive books: Social scenesscenes

Enjoy funny stories on everyday social situationsCreate your own ending, Invent more comics stories with the characters

Testing GameTesting Game

A highly rewarding learning:Answer the questions and win a million coins!