IV. BCI Software

CH 20. BioSig : An Open-Source Software Library for BCI Re-

search

CH 21. BCI2000

: A General-Purpose Software Platform for BCI

2010.11.19. JO, INAH

Introduction Introduce two software packages for BCI

◦ Freely available◦ Include a set of useful tools to process EEG signals for real-time

BCI Chapter 20 : Graz BioSig toolbox.

◦ Interface to many different data formats of EEG recordings◦ Supports several-signal processing◦ Artifact reduction◦ Feature extraction◦ Classification methods

Chapter 21 : BCI2000 toolbox◦ Wadsworth center in Albany , the University of Tubingen.◦ Mainly for real-time BCI application◦ General data acquisition techniques and a general framework◦ Freely available◦ Windows platforms and C++ programming.

BioSig An Open-Source software Library for BCI research

Chapter 20.

1. Abstract Common interface to many different data formats Artifact processing, quality control Adaptive signal processing, Feature extraction methods :

very suitable for online and real-time processing. Supports handling of missing values. Includes several classification methods Overview of the current status and an outline of future pos-

sibilities. Licensed with the GNU General Public License.(GNU GPL or

GPL) Open development platform

: ideal tool for research and development.

2. OverviewCurrent, available open-source software tools

◦ BioSig, OpenEEG, Bioelectromagnetismtoolbox,

EMEGS, FieldTrip, LORETA,BCI2000◦ GPL(General Public License) : free run, copy, distribute

study, change,

and improve the software is available

A Major aim : Provide an open-source software library for BCI research

Available from http://biosig.sf.net

3. BioSig for Octave and MAT-LAB

The BioSig project was started as a library for Matlab with an attempt to be compatible with Octave ( http://www.octave.org )

SubProject(rtsBCI, C/C++, SigViewer, etc)

=> Renamed as “Biosig for octave and MATLAB”

3-1. Data Formats and Storage

• Currently more than 40 different data formats.

• Common interface to various data formats

• Automated detection of the file format

• Do not need to care • which data format is actually used • how the proper scaling of the amplitude values is obtained • From where to obtain the sampling rate.

• Supports also simple preprocessing • selection of specific data channels, overflow detection

• However, none of these data formats contains all the features needed for BCI research.

• GDF(General Data Format):• Constructed with advantages of various data formats for biomedical signal pro-

cessing

3-1. Data Formats and Storage

GDF

◦ table20.3◦ Codes are grouped

According to different application areas

◦ EEG artifact code range:

0x0100 – 0x010f

◦ BCI experiment code range

: 0x0300 – 0x03ff

3-2. Preprocessing Intermediate state between data access and fea-

ture extraction.

Functions◦ Triggering the data : to extract data segments relative to a given

trigger

◦ Extracting the trigger time points from a continuous trigger channel

◦ Advanced algorithms for artifact processing quality control method(histogram analysis), reducing EOG artifacts algorithm(based on regression analysis), detecting muscle artifacts(based on inverse filtering)

Many signal processing methods are available Tool for handling of missing data is rare. Exception : TSA(time series analysis) toolbox, NaN toolbox,

part of OctaveForge, distributed with BioSig.

◦ NaN toolbox Represent missing values as not a number according to IEEE754 sumskipnan.m : skips all NaNs, and sums and counts all numerical val-

ues. Return value SUM, COUNT : calculate the mean of the data. expectation value of any statistic including covariance matrices and

more advanced statistics. more advanced ? Investigated.

◦ TSA(Time Series Analysis) toolbox Many functions for parametric modeling(autoregressive estimator) Burg and Levinson-Durbin algorithms for estimating autoregressive

parameter Adaptive autoregressive estimators like Kalman filtering RLS, LMS algorithms Multivariate autoregressive estimators like the Nuttal-strand method Multichannel Levinson algorithm

3-3. Signal Processing and Feature Extraction

3-4. Classification Various need for classifier

◦ Combines various features into a single output

◦ cross validation to prevent overfitting

◦ Time lag of classification should be as small as possible.

◦ Optimal time point for applying the classifier must be identified.

Wrapper function(findclassifier.m)◦ First version : LDA for two classes

◦ Later : Quadratic classifier, svm for more than 2 classes

◦ Selection of the Proper time segment for estimating a classifier. Data were triggered, seperability were calculated for each time segment. Length of each segment can be varied from 1 to N samples Typically, fixed segment length of a fraction of a second is used.

Cross validation : trial based LOOM(Leave-One-Out Method)◦ Samples from the chosen segment of all but one trial

◦ This classifier used to all samples of testing trials.

◦ Repeated until each trial has been used once as test trial.

◦ used in the BCI competition 2005 on data Set I(rank7),Iva(rank 8)

3-5. Evaluation Criteria Different evaluation criteria

◦ accuracy(error rate), information transfer, cohen’s kappa coef-ficient, others.

◦ Analyzing the time courses of the different criteria are provided.

◦ These functions were used for analyzing results of the BCI competition 2003(datasetIII), BCI competition 2005(datasetIIIa,IIIb),

Plota.m(Wrapper funtion)◦ for displaying various results.

◦ Generate a certain result depends on input data

◦ ex> outcome of findclassifer.m for two classes

: time course of the error rate and SNR

◦ ex > outcome of findclassifier.m for N>2 classes

: time course of the accuracy and the kappa coefficient.

◦

Sview ◦ Present the Raw EEG data

◦ in a compressed form on a single screen.

- Useful to get an overview of the available data

- Scoring of EEG data.

- Two packages : Sviewer, SigViewer.

3-6. Visualization

3-7. SViewer : a viewing and scoring software

Written in MatlabCommon interface

◦ Can read any data format supported by BioSig for Octave and Matlab

annotation, markers, event can be edited.

Not a stand-alone program

4. Real-time BCI System Based on MATLAB/Simulink-rtsBCI

rtsBCI : framework for the development and rapid prototyp-ing of real-time BCI system

To enable hard real-time computing on Windows platforms◦ Real-time Windows Target(RTWT)◦ Real-Time Workshop(RTW)◦ BioSig for Octave and Matlab(for data format handling)◦ TCP/UDP/IP toolbox(for network communication support)

Data Acquisition Card is required to sample the biosignals.

Licensed under GNU GPL Based on Matlab/Simulink, running on Microsoft Windows Used toolbox

◦ Real-time Windows Target(enable hard real-time computing on win-dows platforms)

◦ Real-Time Workshop(for the generation of stand-alone C code)◦ Tunable parameters: INI file

5. SigViewerViewing and scoring program for biosignals.Multichannel signal(EEG,ECT,EMG,EOG)Scoring : user can annotate to the signal written in C++

◦ Able to be run under Linux, windows, MacOS◦ designed as platform-independent

stand alone program Now only GDF file, but

can be extended to other file format

6. BioProFeed Data acquisition module Can be used with following 3 data acquisition hardware now

◦ G. mobilab from g.tec

◦ NI-60243 PCMCIA from National Instruments

◦ ModularEEG from OpenEEG

The name “BioProFeed”Stand for◦ Portable Biosignal acquisition

◦ Real-time processing

◦ Feedback generation framework

Part of BioSig project, it is a software framework for processing and storage unit.

Specifications◦ Platform independent

◦ Support different data acquisition hardware

◦ Real-time data processing should be possible

◦ Storage of the data in GDF format should be possible

Runs on : Linux, MacOs, Windows XP with Cygwin Autotools are used to build software framework

7. Summary Plan to improve Almost every part of BioSig

Artifact processing, quality control should be evaluated offline -> on-line.

Additional signal processing, feature extraction methods need to be developed including the ability to handle missing value.

Further data formats should be supported in SigViewer.

Biosig contains modules for data acquisition, BCI experiments with on-line feedback, viewing and scoring software for reviewing the raw data, functions for offline analysis of BCI data, obtaining a classifier.

Almost all components can be used with free software. GNU GPL

Chap 21.

BCI2000 A General-Purpose software Plat-form for BCI

1. Abstract BCI2000 is a flexible general-purpose platform for brain-com-

puter interface research Aim : Reducing the complexity and cost of implementing BCI sys-

tems. Joint Project : since 2000 Wadsworth center of the New York State Department of Health in Albany, New York

Institute of Medical Psychology and Behavioral Neurobiology at the University of Tub-ingen, Germany.

Used in more than 110 Laboratories around the world BCI2000 Supports

◦ Variety of data acquisition systems

◦ Brain signals

◦ Feedback modalities

By reducing labor and cost

=> different BCI system, psychophysiological experiment http://www.bci2000.org : freely available!

2. Overview Goals of the BCI2000 Project:

◦ 1. To create a system that can facilitate the implementation of any BCI system

◦ 2. to incorporate into this system support for the most commonly used BCI methods

◦ 3. to disseminate the system and associated documentation to other laboratories.

◦ Reduce time, effort, expense of testing new BCI methods, by providing a standard-ized data format, engage BCI research.

2007.02, 110 Labs Consist of 4 modules (independent and interchangeable modules)

◦ 1. Signal acquisition

◦ 2. Signal processing

◦ 3. User feedback

◦ 4. Operating protocol

To date,◦ Variety of data acquisition

Sensorimotor rhythms, cortical surface rhythms, slow cortical potentials, p300 potentials

◦ different outputs Several kinds of cursor, robotic arm control , sequential menu selection, selection from a matrix.

Written in C++

3. Dissemination of BCI2000

Available free of charge for research and educational pur-poses

http://www.bci2000.org 2006, 01 70 Labs

◦ Use the existing BCI2000 system without changing the software

◦ Implement new methods or system capabilities into BCI 2000

◦ Use BCI2000as a real-time signal acquisition platform to develop systems for research not related to brain-computer interfaces.

Example configurations, descriptive documentation, tutorial introductions.

Online bulletin board, question and answer

4. BCI Model and Modules BCI2000 is based on a model that can describe any BCI

system.◦ Source(data acquisition and storage)

Brain signals are processed synchronously in blocks containing a fixed number of samples that are acquired by the Source module.

◦ Signal processing Each time a new block of data is acquired, the source module sends it to signal

processing, which extracts signal features, translates those features into con-trol signals and sends them on to the User Application module.

◦ User application application module sends the resulting event markers back to the source mod-

ule where they and the raw signals are stored to disc.

◦ Operator interface

4-1. Module1. Source Module and File Format

Digitizes and stores brain signals◦ Passes them on without any further preprocessing to

Signal Processing◦ Consists of data acquisition component, data storage

component

BCI2000 file format Consist of ASCII header, binary signal sample, event marker

value. Accommodate any number of signal channels, system pa-

rameters, event markers.

4-2. Module2. Signal Processing Module

Converts signal from the brain into signals that control an output de-vice.

Realized using a chain of signal filters.

◦ Feature extraction (Three filters) 1. A linear calibration routine that converts signals from A/D units into uV 2. A linear spatial filtering operation (calculating a matrix multiplication of the input signals with a spatial filtering

matrix) 3. Temporal filter (6)

Slow wave filter : process slow cortical potentials Autoregressive spectral estimation Spectral estimation based on the FFT A general finite impulse response filter : process sensorimotor rhythms A peak detection routine : extracts firing rates from neuronal action potentials Filter that averages evoked responses(p300)

◦ Feature Translation Translates the extracted signal features into device independent control

signals. ( 2 filters) Linear classifier Filter normalizes the output signals such that they have zero mean and a specific value range

4-3. Module3. User Application Module

Receives control signals from signal processing

and uses them to drive an application

◦ 1. Sensorimotor rhythm amplitude◦ 2. Slow cortical potential amplitude

Visually on a computer screen Selection of targets, letters, or icons. 1~3 dimensional cursor movement. Two dimensional robotic arm control

◦ 3. Evoked potential classification Auditory and visual stimuli

◦ 4. p300 evoked potentials A matrix spelling application based on.

4-4. Module4. Operator Interface Module

Provides a graphical interface that displays ◦ Current system parameters ◦ Real-time analysis results communicated to it from

other modules.Allows investigator to start, stop, suspend,

resume, reconfigure system operation.

Monitor 1 : User feedback is displayed Monitor 2 : Operator module’s graphical interface

4-5. System Variables BCI2000 incorporates three types of system

variables : parameters, event markers, signals.◦ System Parameters : do not change throughout a data file

◦ Event markers: record events that occur during operation, can change from one data sample to the next.

◦ System signals: functions of the user’s brain signals that are re-ceived and modified by the modules.

Each module can request that the operator module create any number of system parameters or event markers.◦ EX> Source module : parameter for signal’s sampling rate.

◦ This parameter is constant during some defined period of online operation, and all other modules, and is recorded automatically in the data file.

◦ EX> Signal processing filter designed to detect artifacts : an event marker to mark artifacts in the signal

◦ EX> Application module : an event marker to record stimulus conditions.

4-6. System Requirements and Real-Time Processing

Microsoft Windows 2000 or a more recent windows-based operating system.

Any recent desktop or laptop computer, One of the data acquisition devices that BCI2000 sup-

ports Real-Time Processing

◦ windows is not a real-time operating system, thus does not guarantee specific timing of events

◦ System latency and latency jitter easily satisfied the real-time requirements of BCI operation.

(latencies are well below 20ms)◦ Processor load was sufficiently low to guarantee reliable

operation.

4-7. Offline Analysis Tools

We currently provide tools◦ To visualize signals in the time domain

◦ To get information on the configuration stored in a data file

◦ To convert data files to ASCII and Matlab

Mario◦ sensorimotor rhythm , p300evoked potential

Created Framework ◦ To compile and link BCI2000 signal processing components

◦ As command-line programs that read BCI2000 parameter

BCI2000 filters may also be compiled into dynamically loaded li-braries(DLLs)

4-8. Rapid Prototyping with MATLAB Matlab code can

Act on blocks of brain signal data in real-time Request its own parameters and event markers Read and modify parameter and marker values Report configuration and run-time errors

5. Future Development Hardware support

◦ Data acquisition systems tend to be the most expensive components.◦ Add support for additional devices.

Platform independence◦ BCI2000 depends on the Borland C++ compiler and the Borland VCL application

framework.◦ By moving toward platform independent libraries, and by replacing non-portable por-

tion from the code base,◦ More compatible with multiple compilers and operating systems

Clinical implementations◦ To make BCI setup, operation, and analyses available to users who are not experts on

BCI technology is critical.◦ Less complex and flexible version is needed.◦ Support simple menu selection using sensorimotor rhythms and p300-evoked poten-

tials.

Integration with other software◦ Interfacing BCI2000 with such external software is an important goal of its future de-

velopment.

6. Availability More information is available at Web Site

BCI2000 source code and executables is free for re-search and educational purposes

But Wadsworth requires a Material Transfer Agree-ment between a user’s institution and Wadsworth.(Prohibits commercial u se and limits liability.)

Compilation of the full BCI2000 source code requires ◦ version 5.4 of the Borland C++ ◦ Version 6.0 of the Borland VCL application framework is

sufficient to build the offline signal processing environ-ment.

<Four-Target Selection Using Tactile Feedback> -The subject controls movement of a cursor on the screen to select one of four targets. - The subject cannot see the screen, - Receives feedback on the target cur-rent cursor position through eight vi-brotactile actuators placed on his shoulders.