improving game accessibility with vibrotactile-enhanced hearing instruments
TRANSCRIPT
Bernd Tessendorf, Peter Derleth, Manuela Feilner, Daniel Roggen, Thomas Stiefmeier, Gerhard Tröster
Improving
Game Accessibility with
Vibrotactile-Enhanced
Hearing Instruments
13th International Conference on Computers Helping People with Special Needs July 2012, Linz, Austria
Video: http://www.ingenhindring.shdir.no/
3
comfort issue
safety issue, e.g. traffic situation
Locating sound sources
4 4
5
sight hearing haptic
~ 70% ~ 20%
Vibrotactile Feedback
6 [1] Raising awareness about space via vibro-tactile notifications, Andreas Riener, Alois Ferscha, EuroSSC, 2008, Zürich.
sight hearing haptic
~ 70% ~ 20%
Vibrotactile Feedback
7
Fast user response time
Information parallel to sound
Integration into hearing instruments
Coin shaped vibration motor
10mm
...how to integrate into hearing instruments? 8
9
Commercial FM-Receiver
10
11
Integration of Vibrotactile Feedback
Matlab USB Bit Whacker Vibration motors PWM
Xsens MT9 IMU
Potentiometer
Architecture
12
Matlab USB Bit Whacker Vibration motors PWM
Xsens MT9 IMU
Potentiometer
Architecture
13
User-in-the-loop
Flexible generation of vibration patterns
Affordable off-the-shelf components
14
Can vibrotactile feedback support HI users in gaming?
Application to Gaming
Gaming is very impressive and popular...
à gaming market 2008: 50 Billion US$, with growing tendency [1]
15 [1] PricewaterhouseCoopers: Entertainment and Media Outlook 2009–2013
Gaming is very impressive and popular...
à gaming market 2008: 50 Billion US$, with growing tendency [1]
à impulses for, e.g., GPU development due to extreme resource requirements
à impulses also for assistive technologies and HCI
16 [1] PricewaterhouseCoopers: Entertainment and Media Outlook 2009–2013
Gaming is very impressive and popular...
à gaming market 2008: 50 Billion US$, with growing tendency [1]
à impulses for, e.g., GPU development due to extreme resource requirements
à impulses also for assistive technologies and HCI
à accessible gaming: e.g. Closed Captioning
...but can also be very violent.
17 [1] PricewaterhouseCoopers: Entertainment and Media Outlook 2009–2013
DOOM 3[CC] http://vimeo.com/13340812 Nominated IGF Choice Award for Best Doom3 MOD by Reid Bryant Kimbal
3D Shooter as an evaluation scenario
• High cognitive load on user • fast gameplay, reaction time critical • sound cues are important information here • background music is present
• Reproducible daily-life scenario
• Open Source: possible to enhance with assistive technology
Explorative Evaluation with HI Users
Tactograms User Studies
20
Pre-Study: Tactograms
21
Resolution, #segments
Intensity levels
Variants of Tactograms
Patterns
22
0 0,2 0,4 0,6
Right
Left
Right
Left
Right
Left
motor 'on' motor 'off'
sec sec sec
front
right
back
Tactograms
23
Tactograms
24
16 normal hearing participants: 7 males, 9 females, age 23 – 61
Pre-study
Sessions of 90 minutes each
Measurement of accuracy and response times
25
26
Calculation of angular deviation
27
Calculation of angular deviation
User Error Rate and RMS Lateralization Error
(4 Segments, 1 Intensity) (6 Segments, 1 Intensity) (8 Segments, 1 Intensity) (12 Segments, 1 Intensity) (8 Segments, 3 Intensities) (12 Segments, 3 Intensities)
Trade-Off: Minimal quantization error (due to the encoding)
VS the number of user errors (due to misinterpretation of presented patterns)
Summary of Pre-Study Results
28
Trade-Off: Response time VS minimum lateralization error
Choosing the most suitable pattern is application-specific!
29
Vibrotactile Pattern
...to encode direction of rockets fired at the user.
2nd Pre-Experiment (4 HI users, 5 normal hearing people)
...only sounds (no visual) of rockets
Behind Left Right
No Game Music Game Music
31
2nd Pre-Experiment
32
Need to support HI users with sound localization
2nd Pre-Experiment
User Study (4 HI users, 5 normal hearing people)
• HI users with and without system • Normal hearing without system • >10h of gaming data, >4500 rockets fired
• Measure: hit rate in gameplay
34
35
sight hearing haptic
~ 70% ~ 20%
User Study
Normal hearing people outperform HI users
All HI users perform better with system than without the system
36
Conclusion
Vibrator
Integration of a vibrotactile Feedback into hearing instruments!
Outlook
• Standardize API for vibrotactile feedback, e.g. like in HTML5
• Deployment in parallel to other assistive technologies
• Confirmation of, e.g., volume change
• Alarm in emergencies
• Warning if battery is empty
13th International Conference on Computers Helping People with Special Needs July 2012, Linz, Austria
Bernd Tessendorf
[email protected] http://www.wearable.ethz.ch/
Questions? Comments? Feedback?