1

APCTP 8/07

Paul SajdaDepartment of Biomedical Engineering

Columbia University

Recipes for the Linear Analysis of EEG… and applications…

APCTP 8/07

Can we “read” the brain non-invasivelyand in real-time?

if YES then

decoder 1001110…

2

APCTP 8/07

Rehabilitation Cognitive Neuroscience

Performance Augmentation

Can we “read” the brain non-invasivelyand in real-time?

if YES then

requires single-trial analysis

decoder 1001110…

APCTP 8/07

Outline

Tutorial on the Linear Analysis of EEG

Matlab/EEGLab Demo

Real-time, On-line Applications: Image Triage and ErrorCorrection

3

APCTP 8/07

Estimating “Interesting” ComponentsThrough Projections

… what is w?

APCTP 8/07

Estimating “Interesting” ComponentsThrough Projections

Signal summation

choose

3dB improvement in SNR

4

APCTP 8/07

Estimating “Interesting” ComponentsThrough Projections

Signal subtraction

choose

APCTP 8/07

Estimating “Interesting” ComponentsThrough Projections

Linear Model for EEG forward model

Source Estimation by Linear Projectionbackward model

For Gaussian noise withknown correlation structurethis is an ML estimator

noise collinear with the source

5

APCTP 8/07

Estimating “Interesting” ComponentsThrough Projections

Minimizing Interference via Subtraction

Estimate interfering source(backward model)

Estimate contribution tomeasurements (forward model)

has no activity correlated with

however it has reduced rank--must deal with appropriately

APCTP 8/07

Estimating “Interesting” ComponentsThrough Projections

Forward Model Estimate

6

APCTP 8/07

Some Objectives for FindingInteresting Components… or how do we estimate w…

• Maximum Difference• Maximum Power• Statistical Independence

APCTP 8/07

Maximum Difference

Use all electrodes in estimation of interference

!

xEBR

(t) = (I" ˆ A eyeˆ A eye

#)x(t)

removal of eye-motion artifacts

7

APCTP 8/07

Maximum Difference

No blink

Blink

APCTP 8/07

Maximum DifferenceMaximum Magnitude Difference

8

APCTP 8/07

Maximum Power

Maximum Power-Ratio

APCTP 8/07

Maximum Power

max wge

min wge

9

APCTP 8/07

Maximum Power

ERD/ERS with generalized eigenvalues.

Subject responds to a visual stimulus with a button press.

Prior to the maximum-power ratio analysis, all EEG channels are bandpass filtered between 5-40Hz.

The covariance matrices R1 and R2 are computed in a window 200ms before (R1) and 200ms after (R2) the button press.

APCTP 8/07

Maximum PowerTop left: Scatter plot of the correspondingactivity for two of the 64 EEG sensors. Solid lineindicates the orientation, wge, along with the twodistributions having a maximum power(variance) ratio, estimated using generalizedeigenvalues.

Bottom left: Estimated forward modelcorresponding to wge. Clear is that the sourceactivity originates over motor areas (it ismaximal over C3 and CP4) and has oppositesign (180 phase delay) between thehemispheres

Right: Spectrogram computed for thecomponent y(t) (averaged over 300 button pressevents) Button press indicated with a verticalwhite line. Alpha band activity (maximal at 12Hzfor this subject) decreases (de-synchronizes) forabout 500ms after the button push.

10

APCTP 8/07

Statistical independence implies for all i≠j,t,τ,n,m:

For M sources and N sensors each t,τ,n,m gives M(M-1)/2conditions for NM unknowns in A.

E[sin(t) sj

m(t +τ)] = E[sin(t)]E[sj

m(t +τ)]

Sufficient conditions if we use multiple:

use sources assumed condition statistic algorithmt non-stationary W Rx(t) W

T = diag covariance decorrelationτ non-white W Rx(τ) W

T = diag cross-correlation SOBIn, m non-Gaussian W Cx(i,j) W

T = diag 4th cumulants JADE (ICA)

Statistical Independence

APCTP 8/07

Example: Non-stationaryIndependent Sources

Example: First 8 independentcomponents that explain 64observed EEG sensors x invisual discrimination task 250 msbefore and after stimuluspresentation

EEG sensor projections A=W-1

The independence assumptionestablishes that the covariance Rx(t) isdiagonalized by W for all times t:

Ry(t1) = W Rx(t1) WT = diag

Ry(t2) = W Rx(t2) WT = diag

Combining these we obtain thesolutions again with the GeneralizedEigen-vectors:

Rx(t2) -1Rx(t1) W = W λ

More robust if we use simultaneousdiagonalization of multiple covariances.

11

APCTP 8/07

ICA Components

APCTP 8/07

GEVD Components

12

APCTP 8/07

LR Components

APCTP 8/07

Demo

13

APCTP 8/07

Using Linear Multivariate Processing

artifact removal anddimensionality reduction

linear discriminationData P(T)

64-128 channels

1

s1

s2

ps1 (x)

ps2(x)

WXY =

observationstransformation

matrix

recoveredsources

APCTP 8/07

Single-trial Detection with Spatial Integration

Linear discriminants: Computespatial weighting w which maximallydiscriminates sensor array signalsx(t) for two different conditions.

Ex: Detect motor planningactivity Predict button pressfrom 122 MEG sensors withlinear discriminator w suchthat s(t) differs the most during100-30 ms window prior tobutton push.

E[y

(t)]

t false positive rate

true

posi

tive

rate

Conventional Event Related Potentials (ERP) averages over trials.We substitute trial averaging by spatial integration:

s(t) = wT x(t)

x(t)

14

APCTP 8/07

Localization of Discriminating Component... possible because we have a linear model ...

What is the electrical coupling a ofthe hypothetical source s thatexplains most of the activity X?

Least squares solution:

Strong coupling indicates low attenuation.Intensity on these “sensor projections” aindicates closeness of the source to thesensors.

a =XssTs

APCTP 8/07

Localization of Discriminating Componentpossible because we have a linear model

Strong coupling indicates low attenuation. Intensityon these “sensor projections” a indicates closenessof the component to the sensors.!

a =Xy

yTy

Linear discriminants: Compute spatialweighting w which maximally discriminatessensor array signals x(t) for two differentconditions.

!

" = tk#$

2% t

k+$

2

& ' (

) * +

Single-trial Discrimination

Stimulus locked time (ms)0 1000500250 750

Tri

als

TrainingWindow (τ)Stimulus Response

!

y(t) = w" ,# ,$

Tx(t)

vs.

Parra, Sajda et al. Neuroimage, 2002Parra, Spence, Gerson & Sajda, Neuroimage, 2005

15

APCTP 8/07

Single-trial Discrimination

Tri

als

Stimulus locked time (ms)

Az

0.5 1p = 0.01

250 ms

0.75

Tri

als

Stimulus locked time (ms)

Az

0.5 1p = 0.01

!

y(t) = w" ,# ,$

Tx(t)

!

a =Xy

yTy

Discriminationperformance

APCTP 8/07

Hypotheses:

• EEG can be used to detectedcognitive events related tovisual target detection,discrimination, and perceivederror.

• Such cognitive events can bedetected more quickly andreliably than overt (motor)responses.

Decision

Application: Cognitive User Interface

fastslow

Objective: Use EEG signatures of cognitive events to improve task performance

16

APCTP 8/07

Cortically-coupled computer vision (C3 Vision)

Image Sequence

Single-trial decoder

priority list

APCTP 8/07

Pre-triage Post-triage

Cortically-coupled computer vision (C3 Vision)

17

APCTP 8/07

On-line Real-timePortable Image Triage System

Display Laptop(EPrime & Python)

Analysis Laptop(Matlab DLL & C))

parallel port (display events)

serial COMS port (detection events)

BiosemiActive Two

64 channels

USB(EEG data streaming)

APCTP 8/07

Hierarchical Discriminating Components…online estimation of all parameters…

!"#

$%& '

+'

(=)22

kktt K

!

y(t) = w"

Tx(t)

time

trials

tstimulus 100ms 1000ms

!

z = cT

< y(t) >

!

c(1)

!

c(n)

18

APCTP 8/07

Triage results

Original Sequence EEG (no motor)

EEG (motor) Button EEG (motor) and Button

Triage performance

Imag

e nu

mbe

r

Gerson, Parra & Sajda, IEEE TNSRE, 2006Sajda et al., Trends in BCI, 2007

APCTP 8/07

Detection of Error Related NegativityDuring a Visual Discrimination Event

Error Related Negativity (ERN) occurs following perception of errors. It ishypothesized to originate in Anterior Cingulate and to represent responseconflict or subjective loss.

Example: Erikson Flanker task

Discrimination of error versus correct response (64 EEG sensors, 100ms)

19

APCTP 8/07

Real-Time On-Line Error Correction

Linear classifierfor ERNdetection

Adaptivethreshold forerror correction

Machine Corrected Errors Overall Human-Machine Performance

Linearfiltering &eye blinkremoval

64 EEGchannels

APCTP 8/07

Linearfiltering &eye blinkremoval

Machine Corrected Errors

Real-Time On-Line Error Correction

64 EEGchannels

200 ms latency

20

APCTP 8/07

Further Reading/Info

• Papers and code at http://liinc.bme.columbia.edu