chapter 23 combining bci and virtual reality : scouting virtual worlds
DESCRIPTION
Chapter 23 Combining BCI and Virtual Reality : Scouting Virtual Worlds. Introduction. Before a BCI can be used for control purposes, several training sessions are necessary Operant conditioning Feed back, real-time changes to the user Machine learning - PowerPoint PPT PresentationTRANSCRIPT
Chapter 23
Combining BCI and Virtual Real-ity
: Scouting Virtual Worlds
Before a BCI can be used for control pur-poses, several training sessions are neces-sary
◦ Operant conditioning Feed back, real-time changes to the user
◦ Machine learning Adaptive algorithms to detect brain patterns
Introduction
A technology which allows a user to interact with a computer-simulated environment.
VR (Virtual Reality)
VE (Virtual Environment)◦ Allow users
to be shielded from the outside world to be able to focus on the required mental task
VR (Virtual Reality)
Then, Why VR for BCI?
Feedback
Visualization vs. VE feedback
Ron Angevin et al. (2004)◦ Control group (standard BCI
feedback) reacted faster◦ VR group achieved less error
Basic principle◦ detection & classification of motor-imagery re-
lated EEG patterns
Sensorimotor rhythms are analyzed◦ C3, Cz and Cz
Graz-BCI
VE◦ To let a user become immersed in a 3D scene
Cave◦ Multiprojection stereo-based head-tracked VE sys-
tem
Graz-BCI + VE
3D virtual environment1. Creation of a 3D model of the scene
3D modeling Software Packages Performer Maya
2. Generation of a VR-application that controls and animates the modeled scene
Virtual Research V8 HMD 640 X 480 pixels, refresh rate 60Hz
Vrjuggler + single back-projected wall + shutter glasses
Cave-like system
Graz-BCI + VE
BCI experiments◦ Require a subject in a sitting position
No positional information had to be considered
◦ Rotational information from the tracking system was ignored Rotation should be controlled by the BCI
Graz-BCI + VE
Study 1Roation in a VE by Left- and Right-Hand Motor Imagery
Imagination of left and right hand movement
Subjects◦ 2 male(23, 26 years old), 1 female(28 years old)◦ 7 months
Feedback conditions1. A standard horizontal bar graph on a desktop monitor2. Virtual conference room presented with an HMD3. Virtual pub populated with animated avatars (includ-
ing music and chatter of the avatars)
Experiment
The order of feedback conditions◦ Bar graph → HMD → Cave → HMD → bar graph
Instruction◦ To imagine left or right hand movements depending
on an acoustic cue (single or double beep)
Control◦ Either the length and the orientation of the horizontal
bar graph (in case of the standard BCI feedback)◦ Rotation angle and direction within VR
Experiment
During experiments◦ Cue at second 3◦ Feedback for 4s◦ Screen update (including rotation) 24 times/s◦ One run 40 trials
Experiment
Results
No difference between HMD and Cave
Performed well with VR than bar graph
Results
Study 2Moving Forward in a Virtual Street by Foot Mortor Im-agery
Imagination in the virtual street◦ Right hand movement: to stop◦ Foot movement to move (constant speed)◦ Walking distance is scored
CAM (Cumulative achieved mileage)
Male 23, 28 and 30 years old.
Experiment
Results
Study 3, 4Scouting through a Virtual Apartment
Asynchronous freeSpace Experi-ments
Virtual apartment on a single back-projected stereoscopic wall
Subject could decide freely where to go along predefined pathways (through the corridors or rooms)◦ Turn right, left, or straight)
System automatically guided the subject to the next junction◦ Small map in the bottom right corner of the display
Experiment
Results
Using VR◦ High classification accuracy (low error rate) can be achieved.
Subjects felt more natural in VE compared with BCI experiments with standard feedback
Each subject preferred the Cave experiments to the HMD and both were favored over BCI session on a desktop PC
Motivation seems to improve the BCI performance, but too much excitement might also distract the subject
Despite distraction from auditory and moving visual stimuli in VE, motor imagery and its classification in the ongoing EEG is still possible
Conclusion
Thank you !