A Multisensory and Multimodal Game for Children with Cochlear Implants
Maria Carmela SogonoSupervisor: Dr. Deborah RichardsITEC810 Information Technology ProjectDepartment of Computing, Macquarie University
2
Agenda• Problem• Related work• Methodology• Outcomes– Design principles– Instructional design– Game evaluation
• Conclusion
3
ProblemEven with hearing aids, quality is still not the same as with normal hearing
Can affect brain development in children
Problems in areas such as sound localisation
Can a multimodal, multisensory game be used to test, train, and improve sound localisation acuity?
Photo source: http://www.flickr.com/photos/adwriter/1305191984/
4
Why focus on children?
• 35:10000 children• Normal hearing is important for brain
development
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
5
Why focus on SSD?
There is less help available for children with SSD.
Photo source: http://www.firefliesandjellybeans.com/2012/05/can-you-hear-me-now.html
6
What is sound localisation?The ability to identify where sound has come from
Photo source: http://www.flickr.com/photos/reverses/5098858054/
7
Baha
Processes sound from the impaired side and sends it to the hearing ear
Boneanchored hearing aid
Photo source: http://products.cochlearamericas.com/diagram/baha-system
8
Would it be possible to improve sound localisation acuity for children with SSD and Bahas?
9
How?GamesMultisensoryMultimodal
• To appeal to the target audience• To make it easier to track, collect,
then analyse data
• Supported by perceptual motor training theory
• Supported by technological advancements
10
Can we design games to improve sound localisation acuity for children with SSD and Bahas?
Would it be possible to improve sound localisation acuity for children with SSD and Bahas?
11
Relevant work• Research studies testing sound
localisation acuity• Existing sound localisation games
developed at Macquarie University• Other games developed for
children with SSD
Photo source: http://www.flickr.com/photos/driek/5412231453/
12
Methodology1. Review of relatedliterature
13
Methodology2. Analysis of existing work
14
Methodology3. Development of design principles
15
Methodology4. Development of instructional design template
16
Methodology5. Development of game evaluation guidelines
17
Design principles1. The learning environment shall be
realistic.2. The stimulus shall be meaningful.3. The game shall start at an easy level.4. When the game level of difficulty
increases, the increase shall be incremental.
5. The game shall incorporate motivational elements.
6. The game shall provide feedback after each trial.
18
7. The game shall propose a challenge.8. The game shall make use of fantasy.9. The game shall incorporate sensory and
cognitive curiosity.10.The game shall adapt to the user’s progress.11.The game shall allow for the repetition of
tasks.12.The game shall track user data.13.The game shall use bright colours.
Design principles
19
14.The game shall be visually appealing to the target audience.
15.The game shall make use of high-quality audio.
16.The game shall allow for sound to originate from at least 2 distinct sources.
17.A non-accented, trained voice shall be used.
18.The game shall make use of multiple modalities.
Design principles
20
Interaction modalitiesModality Description TechnologyVision Visual perception Monitor
TouchscreenProjector & screen3D goggles
Audio Hearing SpeakersHaptic Deals with the sense of touch;
currently only used to provide pressure
JoystickWand
Action, manipulation
Directed by limbs and body KeyboardMouseGamepadJoystickWii controllerKinectTouchscreen
21
Instructional design templateTitle <Title of the game module>Description <Describes the game in one or two sentences>Objectives To improve skills in localization acuity through training
To track user progress whilst trainingGame Style <Describes what game elements are incorporated>
Game genres: Puzzle (Hidden object, reveal the picture) Action (Maze game, rhythm action game, shooter games) Strategy (Time management) Adventure
Background <Describes the game’s backstory>Initial Setup <Describes how the game is set up in the beginning. What a new player will see,
hear, or experience after opening the game for the first time? >Game Model <Describes game progression. What will the player have to do? What happens
when the player answers correctly or incorrectly?> Four general models of game flow: Sound identification Sound evasion Sound tracking Sound training
22
Instructional design templateSuccess Condition
<Describes how the game or a level within the game is won>
Fail Condition
<Describes how the game or a level within the game is lost>
Challenges <Describes how the game levels increase in difficulty> There are several ways to increase difficulty: Include increasing instances of the stimulus Decrease angular distance between the instances Include background noises Include additional elements onscreen to increase distraction Decrease stimulus visibility (for example, it is hiding behind
another object) Include a time limit Include a “number of times” limit
Additional Interaction
<Describes possible interaction styles that are available if the player has access to additional hardware>
23
Sample game modules
Bug KillerTreasure HuntWild Animal EvasionDance Beat
Shooter, soundidentification
Adventure, sound tracking
Maze, sound evasion
Rhythm action, sound training
Game evaluation• Game design
• Usability
• Content
24
“Does the game follow design principles and best practices?”“Is the game usable?”
“Does the game teach what it’s supposed to?”
25
Game design evaluationModule Name: <Module name>Does the module have… Rating…a realistic learning environment? <1-5>Explanation: <Describe how the game satisfies or does not satisfy this condition.>
…meaningful stimulus? <1-5>Explanation:<Describe how the game satisfies or does not satisfy this condition.>
…motivational elements? <1-5>Explanation: <Describe how the game satisfies or does not satisfy this condition.>
…feedback? <1-5>Explanation: <Describe how the game satisfies or does not satisfy this condition.>
26
Game design evaluation…a challenge? <1-5>
Explanation: <Describe how the game satisfies or does not satisfy this condition.>
…elements of fantasy? <1-5>Explanation: <Describe how the game satisfies or does not satisfy this condition.>
…adaptation to the user’s progress? <1-5>Explanation: <Describe how the game satisfies or does not satisfy this condition.>
…repetition of tasks? <1-5>Explanation: <Describe how the game satisfies or does not satisfy this condition.>
...multiple modalities? <1-5>Explanation: <Describe how the game satisfies or does not satisfy this condition.>
27
Game modules evaluated
Bug KillerTreasure HuntWild Animal EvasionDance Beat
4 3.22
3.89
3.11
28
Conclusion
29
• Augmented reality• Multiplayer games• Haptic, large scale mats• Games for adults
Recommendations