evaluation of a natural user interaction gameplay system using the microsoft kinect augmented with...
Upload: interactive-technologies-and-games-education-health-and-disability
Post on 13-Jan-2015
240 views
DESCRIPTION
Evaluation of a Natural User Interaction Gameplay System Using the Microsoft Kinect Augmented with Non-invasive Brain Computer Interfaces by Peter Mitchell, Dr. Brett Wilkinson, and Dr. Sean FitzgibbonTRANSCRIPT
Evaluating brain signal input for Kinect-based games
Dr Brett Wilkinson Presenting the work of:
Mr Peter Mitchell, Dr Brett Wilkinson, Dr Sean Fitzgibbon and Mr Lawrence Sambrooks
Research Overview • Used Natural User Interfaces (NUIs) through a
combination of the Microsoft Kinect and Emotiv EPOC to provide a full body Human Computer Interaction experience (HCI).
• Research completed as a pilot study to determine the usability of the combination of hardware to explore whether there is future potential for the combination.
Emotiv 2010, Arizona State University, viewed 9 September 2013, <http://lsrl.lab.asu.edu/site/?p=848> Microsoft Kinect 2012, Microsoft, viewed 9 September 2013 , <http://www.microsoft.com/en-us/kinectforwindows/>
Presentation Overview
• Background • Testing methodology • The Games
– Tile Puzzle – Street Puzzle – River Puzzle
• Results
Background on Existing Studies
VAN DE LAAR, B. L. 2009. Actual and imagined movement in BCI gaming. BOS, D.-O., REUDERINK, B., VAN DE LAAR, B., GURKOK, H., MUHL, C., POEL, M., HEYLEN, D. & NIJHOLT, A. Human-computer interaction for BCI games: Usability and user experience. Cyberworlds (CW), 2010 International Conference on, 2010. IEEE, 277-281.
• BrainBasher (van de Laar, 2009) • Used actual and imagined movement to
have participants attempt to match actions. (seen top right)
• BacteriaHunt (Bos et al., 2010) • BCI interaction used to provide a
speed modifier in combination with keyboard interaction.
• AlphaWoW (Bos et al. 2010) • Used a variety of BCI methods for
character interaction in the game World of Warcraft. Inner speech, association, and mental states.
Goals for Testing
• Does BCI input with Kinect-based games modify the experience?
• What signals are most appropriate for gameplay?
• Can individuals maintain control over their own brain waves?
Testing Approach
• 15 participants from Flinders University – Students and academics – Primarily male
• Play three puzzle games • Complete post experiment survey • Complete post experiment NASA TLX
Application Overview: Tile Puzzle
Application Overview: Tile Puzzle
• Time-based task – Freedom to explore the interaction techniques with
the Emotiv and Kinect • Concentration and Relaxation used as input
– Relax: reveal hidden image – Concentrate: hide image
• Kinect used to map movements and speech to interaction – Control cursor – Select, place, rotate tiles
Original Mock-up Example Image of initial state
Image of Calm view
Image of moving squares
Image of complete
Free Sandstone Image 2012, viewed 2/04/2012, http://www.hoskingindustries.com.au/blog/tag/grunge/page/2/ Free Sandstone Image 2012, viewed 2/04/2012, http://www.spiralgraphics.biz/packs/stone_muted/index.htm?36 Hieroglify font, http://www.fontspace.com/download/1123/e17737daec4347e0b3edd50cd5c47df6/barmee_hieroglify.zip
Initial State Relaxed State
Swapping Tiles Completed Puzzle
Finished Tile Puzzle (Video)
Street Puzzle Overview
Street Puzzle
Application Overview: Street Puzzle
• Motion – goal – control brain state • Rail-based task
– Set, randomised path • Concentration and Relaxation used as input
– Relax: slow down game time – Concentrate: speed up game time
• Kinect used to map movements to interaction – Sideway step to jump rail – Both hands used to halt motion
River Puzzle Overview
River Puzzle
Application Overview: River Puzzle
• Time-based task – Selection of appropriate game items within a set
time – The more collected the higher the score
• Concentration and Relaxation used as input – Relax: slow down game time – Concentrate: speed up game time
• Kinect used to map movements to interaction – Control cursor – Select treasure and place in inventory
Results
General Gameplay Responses
Results Continued
Ease of use with specific HCI components
Results Continued
Combination of Inputs and Marketability
Results Continued
Comparison of Percent Time Spent in EEG States
Results Continued
• Technical issues encountered with BCI equipment: – Cheap headset resulted in limited
performance – Calibration difficulties and inconsistencies – Delay between updates – Muscle movement heavily contaminated
data.
Future Work
• Look at the potential of other BCI devices • Look at the potential of other platforms • Extended evaluation to investigate if
training of state can be achieved
Conclusion
• Pilot study indicated that the technology can work together
• Developed a functional test platform • User evaluation conducted to suggest
the potential for training brain state