cognitive ability assessment by brain–computer interface

Journal of Neuroscience Methods 201 (2011) 239 250

Contents lists available at ScienceDirect

Journal of Neuroscience Methods

jou rna l h om epa ge: www.elsev ier .com

Cogniti terValidat gn

P. Perego ttab

a Politecnico dib IRCCS Eugenic SXT Telemed s

a r t i c l

Article history:Received 17 FeReceived in reAccepted 25 Ju

Keywords:BCIBrainComputer InterfacePsychometric assessmentColor Progressive Matrices Raven Test

systmusct whehis add Proissta

phases: phase 1 was aimed at conguring the BCI system on the subjects features and train him/her touse it; during phase 2 the BCI system was recongured and the test performed. A step-by-step checkingprocedure was adopted to verify progressive inclusion/exclusion criteria and the underpinning variables.The protocol was validated on 19 healthy subjects and the BCI-based administration was comparedwith a paper-based administration. The results obtained by both methods were correlated as known fortraditional assessment of a similarly culture free and reasoning based test. Although our ndings need

1. Introdu

Computticipation, underlying sis, CP CeBrainCommunicationreal-time dcal signals a2002). The Bcephalograpwhich can devices (e.gtion and intpossibilitiesUp to now,

CorresponTel.: +39 341 4

E-mail add

0165-0270/$ doi:10.1016/j.to be validated on pathological participants, in our healthy population the BCI-based administration didnot affect performance and added a further control of the response due to the several variables includedand analyzed by the computerized task.

2011 Elsevier B.V. All rights reserved.

ction

er-based technology supports communication and par-even when interaction is severely limited by anpathology (e.g. ALS Amyotrophic Lateral Sclero-rebral Palsy). Among the most interesting solutions,

puter Interface (BCI) is a system providing a direct com- channel to the human brain and a computer due toetection and classication of neural electrophysiologi-ctivated during a specic mental activity (Wolpaw et al.,CI system detects changes in the patterns of electroen-hic signals and transforms them into an output signal

be used by the computer to control different kinds of. spelling device, home automation, etc.). Communica-eraction with the external world broaden participation, with a signicant improvement in the quality of life.

research in BCI has grown rapidly to demonstrate its

ding author at: Via Ghislanzoni, 24, 23900 Lecco, Italy.88897; fax: +39 341 488884.ress: [email protected] (P. Perego).

reliability, accuracy, and bit-rate (Allison et al., 2008; Cheng et al.,2002).

Only a few studies have explored the cognitive abilities neededto control the BCI system or, on the other hand, have used BCItechnology as a tool to assess cognitive competencies in patientswith severe motor impairment. Iversen et al. (2008a,b) used a BCIinterfaced and fully computerized method for presenting a vari-ety of experimental two-choice tasks to an ALS patient with theaim of assessing his cognitive abilities. The evaluation of cognitivecompetencies (when motor impairment is severe) is a challengingissue. Up to now, most psychometric tests have needed at least aminimal motor response (e.g. gaze control, nodding) (Ding et al.,2006). Moreover, experimental cognitive tasks, such as those usedby Iversen et al. (2008a,b), describe the performance of a single caseclearly and adequately, but lack the possibility of comparison.

Neurological diseases produce modications in the brain struc-ture (both at cortical and subcortical level) and related signals,thus affecting the users ability to achieve control of the BCI sys-tems. Furthermore, they may also signicantly affect the usersengagement in the task, both because of specic neuropsychologi-cal decits and general cognitive abilities. However, neurocognitivecompetencies are very difcult to detail in the presence of a

see front matter 2011 Elsevier B.V. All rights reserved.jneumeth.2011.06.025ve ability assessment by BrainCompuion of a new assessment method for coa,, A.C. Turconib, G. Andreonia, L. Maggic, E. Bere

Milano, Indaco Dept., Milan, Italyo Medea, Bosisio Parini (LC), Italyrl, Lecco, Italy

e i n f o

bruary 2011vised form 24 June 2011ne 2011

a b s t r a c t

BrainComputer Interfaces (BCIs) arecontrol to people with severe neuromethod for psychometric assessmento the subjects output impairment. Twidely used clinical test (Raven Colorein terms of cognitive resource with a m/ locate / jneumeth

Interfaceitive abilities

, S. Parini c, C. Gagliardib

ems which can provide communication and environmentalular diseases. The current study proposes a new BCI-basedn traditional or computerized testing cannot be used owingministration protocol was based on, and validated against, a

gressive Matrix) in order to verify whether BCI affects the braintement result. The operating protocol was structured into two

240 P. Perego et al. / Journal of Neuroscience Methods 201 (2011) 239 250

neuromuscular impairment which hampers the traditional admin-istration of assessment protocols. In fact, if we accept thatBCI performance depends on the interaction of two differentadaptive controllers (one being the human and the other the hard-ware/softwevaluate theful not onlyalgorithms experiencedmay thus bebrain structin pathologneuropsych

In EEG-b(Visual EvoSlow Corticcan be divid

Dependenpathwaysneeded toVEP, P300

Independnormal ououtput pamost seve

BCIs can

Synchronulus and synchronwindows

Asynchrowithout uation at fanalyzed

We aimeveried admis freely concognitive ab composewith severeby-step prosingle phaseobtained wbased and p

2. Materia

Since thperformancunusual setobtain resuthis study a

Do healthadministe

Can BCI bthe Raven

Does the the psych

ig. 1.

e psy

Raveens rm o

Spreng frs wised ence

intellectual performance (Zaini et al., 2005) and non-verbalence (Counter et al., 2005) or visuo-perceptual and visuo-

functions (Nagano-Saito et al., 2005).yo et al. (2008) have demonstrated, in a sample of partici-ith Cerebral Palsy, that the performance on Ravens Colored

ssive Matrices was associated with visuo-perceptual, lan- visual and verbal memory, but not with frontal functions.

RCPM is a fast, easy-to-administer test; it consists of 36 non-entational colored pictures incomplete in the bottom right, organized in three sets (A, AB and B) of 12 diagrammatics (Fig. 1); the subject is given 6 alternatives from which tothe one that best completes the given pattern. The items areted in a bold, accurately drawn and pleasant-looking for-

maintain concentration and minimize fatigue; the test doesolve manipulative activities, and verbal instructions are keptinimum. Although mainly used for children (up to 15 yearsd elderly people (>75 years old), norms for adults have beened, too (Basso et al., 1987; Measso et al., 1993).

ce we decided to develop a BCI-based protocol to assess cog-abilities in neuromuscular disorders, we chose the RCPMe it:

ndardized and validated;ublished normative data;on-verbal test and thus language and limited language abil-do not impact on the results;are system) (Wolpaw et al., 2002), in order to correctly global and specic system results, we have to be mind-

of the BCIs technical characteristics (e.g. translationand outputs), but we also have to explore the difculties

by users due to their own characteristics. Interaction affected both by signal alterations (due to alterations inure and stream) and by the subjects characteristics (e.g.ical situations: neurocognitive impairment and specicological decits).ased BCI the most used brain patterns are: P300, VEPked Potential), and rhythms (Motor Imagery) andal Potential (SCP) (Wolpaw et al., 2002). These protocolsed into two categories:

t BCI system: it does not use the brains normal output to carry the message, but activity in these pathways is

generate the brain activity (e.g. EEG) that carries it (e.g.).ent BCI: it does not depend in any way on the brainstput pathways and offers the brain a completely newthway. It is likely to be more useful for people with there neuromuscular disabilities.

also be (Mason et al. 2006):

ous: the system is externally paced by a cue stim-the EEG signals have to be analyzed and classied

ously to each cue stimulus in xed, predened time.nous (or self-paced): the system is internally pacedsing any cue stimulus and the user intends an oper-ree will. In this case the EEG signals are continuouslyand classied.

d at developing a new, easy-to-used and step-by-stepinistration tool for cognitive tests by a BCI system thattrolled by the user (self-paced BCI) and requires as littleility as possible to be controlled. The proposed method

d of a software and a protocol is meant for people impairments (e.g. ALS). This paper presents the step-tocol designed to check the variables underpinning thes both on the computer and human sides and the results

ith a group of healthy participants. A comparison of BCI-aper-based administration evidenced similar results.

ls and methods

e BCI could potentially interfere with the cognitive teste at different levels (mainly because of the relativelyting and the need to adopt new strategies in order tolts and for the effort required for sustaining attention),imed to answer to the following questions:

y participants perform similarly in a cognitive taskred by a BCI-based and a paper-based method?e used for the administration of a cognitive task such ass Colored Progressive Matrices Test (RCPM)?BCI system interfere with a users performance duringometric test?

F

2.1. Th

2.1.1. Rav

pler fo2004).sufferipersonbeen uintellig2004),intelligspatial

Puepants wProgreguage,

Thereprescornerpuzzleselect presenmat tonot invto a mold) anpublish

Sinnitive becaus

is sta has p is a n

ities Example of paper based Ravens Colored Progressive Matrices.

chometric test

ns Colored Progressive Matrices (RCPM)Colored Progressive Matrices (RCPM) is the shorter, sim-f the Ravens Progressive Matrices Test (Lezak et al.,en and Strauss (1998) recommended it for personsom physical disabilities, aphasias, CP or deafness, andth intellectual disability. The RCPM has traditionallyto measure global cognitive performance in terms of

(Nakamura et al., 2000), mental age (Alevriadou et al.,

P. Perego et al. / Journal of Neuroscience Methods 201 (2011) 239 250 241

is widely used; is easy-to-use and fast to administer in order to avoid boredom

or fatigue; can easily be administered by a computer.

2.1.2. CatteAs far as

BCI system tration. Wea more tradsame test band the newith the RC1949) is theintelligencetallized intof uid intepsychometrlar (the subof displayedtion index ret al., 2002)dealing witverbal answ

We choswe intendepetencies aanswers, limand uent rcated in oth

2.2. Brain

2.2.1. The BWe app

(Parini et alsubjects to

The softused with dImagery). Inis considere

2.2.2. ERP dThe EEG

and T6 electem (Fig. 2)cases it waremoving Tground elecmastoid. ThAU) with ated via Bluethey were rModule. WFIR) to remlter (elliptfrequency (

2.2.3. The BThe BCI s

(Perego et areal-time alation and usystem. Thehardware adifferent pe

e sube tre (Hrecoers irocetlabce prol fosed u

BCI et aluser

SSVrs vbasesual sistedthin am a

afteh sidensional direction (up, down, left and right) corresponding

four BCI commands. previously validated algorithm for real-time processing andntication of SSVEP patterns into EEG signals (see for detailst al., 2009) relied on the Matlab engine and is characterizedws:

rocessing and Data Windowing: the eight EEG channel data ltered using two different lters (50 Hz 60 dB stop-band

545 Hz 4th order FIR pass-band) and then windowed using-second length window (with n from 1 s to 4 s according toNR).ial ltering and channel combining: the 8-EEG channels pre-essing data were spatially ltered and combined using ae linear combiner with channel-specic weights obtained

an automatic method (CSPs). An in-depth explanation oflter can be found in Parini et al. (2009).res extraction: the algorithm used a stimulus-locked aver-

o extract an information index from pre-processed data. Thisx was used both for biofeedback and classication.lls culture fair test we know, the RCPM has never been administered by aand its normative data refer to a paper-based adminis-

thus needed to compare results obtained by BCI withitional assessment, since we could not administer theecause of both implicit and explicit learning processesed to use the test in its entirety to get a score. AlongPM, the Culture Fair Intelligence Test (CFIT) (Cattell,

most widely used test to measure the so-called uid (while other well-known tests measure mainly crys-elligence). Both have been shown to be good predictorslligence and typically load high on a general factor inic studies (Carroll, 1993). The two tests are quite simi-ject has to choose the target stimulus among a number

options), with a consistent, though moderate, correla-anging from 0.40 (Salmoni et al. 1984) to 0.57 (Conway. Both tests are (relatively) culture-free (like everythingh intelligence) and require neither verbal reasoning norers; the subject has to point to his/her choice.e two tasks characterized by a similar construct sinced to design a protocol exploring what cognitive com-re needed to control a BCI system: no need of verbalited verbal instructions, displayed visual alternatives

easoning. Moreover, our protocol could be easily repli-er countries since the two tests are culture-free.

Computer Interface

CI paradigmlied our already validated SSVEP-based BCI system., 2009), which is a self-paced dependent BCI allowingsend four different commands to the user interface.ware is independent of the BCI system so that it can beifferent BCI paradigms (e.g. SSVEP, P300, SCP and Motor

this experiment we used a SSVEP BCI, which at presentd the most efcient system (Vialatte et al., 2010).

ata recording recording set-up consisted of O1, Oz, O2, P3, Pz, P4, T5trodes placed according to the international 1020 sys-. Given the high reliability of the BCI system, in somes possible to reduce to 5 the number of electrodes by5, T6 and Pz. The forehead (Fpz) electrode served astrode. The reference electrode was placed on the righte EEG amplier used was gMobilab (g.tec GmbH, Graz

sampling rate of 256 Hz. The signals were transmit-tooth from the EEG device to the operators PC whereecorded and processed by the BCI++ Hardware Interfacee used an on-line band-pass lter (545 Hz, 4th-orderove artifacts due to eye movements or blinks. A notchic lter with about 60 dB stop-band attenuation) on AC50 Hz) was also activated during acquisitions.

CI systemystem was based on the Open Source BCI++ Frameworkl., 2009) as a technological platform for data acquisition,lgorithm management, protocol development, stimu-ser interface. Fig. 3 shows the structure of the entire

BCI system consisted of two modules one for thend the other for the users interface running on tworsonal computers which can be located separately so

that thmodulSoftwasition, the ustime pand Mainterfaible toBCI-ba

Our(Parinia new

Ourthe useSSVEP-fore vi7) consand a the befatigueon eaca 2-dimto the

Thethe ideParini eas follo

Pre-pwereand aan nthe S

SpatprocsinglwithCSP

Featuage tindeFig. 2. The adopted electrodes conguration.

ject can be tested in a quiet and silent room. The rsthe operators one supports the Hardware InterfaceIM) (Perego et al., 2009) and is used for signal acqui-

rding and visualization. It enables communication withnterface (i.e. the second module) and carries out real-ssing through algorithms developed using both C/C++ (MathWorksTM, USA). The second module the usersogram executes the AEnima software which is a ex-r the implementation of new operating protocols forser applications.

system used already validated algorithms (for HIM)., 2009; Perego et al., 2009) for SSVEP-based BCI, whileinterface (for AEnima) was developed for the RCPM.EP-based BCI system is a dependent BCI, since it relies onision and ability to control gaze direction. Moreover, ad BCI uses a stimulus-related potential, needing there-timulators. Our visual stimulation system (Fig. 3, point

of four 3 cm-side cubic boxes, built with matt materialsemi-transparent lm on the front in order to diffusend avoid direct light exposure which could cause eyer few minutes. Each box included a green LED and stoode of a 19 LCD monitor to associate each stimulator with


Fig. 3. Structurecord the sigcommand. It pLCD display. T

Classica(left, righRegularizthereby gin differeexternal tures obt(describeexplanati2009.)

2.2.4. The RTo avoid

fatigue, we faithfully rethe paper-bsubject fouhe/she coultion is highanswer (Figre of the SSVEP BCI system: (1) The brain generates the pattern needed to control the BCI system. (2) An EEG cap with 8 electrodes (3) and an EEG ampliernals and transmit them to the (4) operator PC that stores them in a database (5). The operator PC uses an online algorithm to process signals and extract aasses the commands via TCP/IP to the user PC on which an (6) user interface is running. The user pc also regulates the stimulator (7) placed on each side of ahe user interface can show the user a biofeedback in order to help him during use of the BCI.

tion: the features extracted were classied by a 5-classt, up, down and null) supervised classier based on theed Linear Discriminant Analysis (Wolpaw et al., 2002),iving a command which can be used by the computernt ways (displacement of a pointer, actions to controlperipherals). The classier was trained using the fea-ained from the acquisition during the training sessiond in the Operating Protocol paragraph). (An in-depthon of our LDA classier can be found in Parini et al.,

CPM graphics interface language interference and prevent user boredom andadapted a validated and widely used psychometric testplicating the structure, colors, shapes and textures ofased version. In the PC-based version of the test the

nd all the possible solution on two rows, among whichd move with the BCI commands. His/her current posi-lighted through a 1.25 magnication of the selected. 4). The PC-based interface showed the main picture

Fig. 4. Computer Graphics Interface of a Ravens CPM puzzle. In the current state,the answer 1 is highlighted by zooming in.


Fig. 5. Ravens as 5 s BCI system, th e answ

in the uppthe bottombut can alsowhich provfrom white(RGB = 190,

The selecan be sum

A diagram5 s no comhighlightethe screen

Afterwardthe BCI inanswer (Fcould useto 6) and was slighcorrect anbut the su

Once the it for 3 s uthat the U

The subjethe answdue to fahe/she cowith the l

This seleresponse, tha low bit-raminimizingaffect the sthe given aselected anslightly incrthe system answer. Thexplore therecorded almoves perf

2.2.5. BiofeThe brai

feedback ordard outpuoutputs sucing and suboversight ofSalmoni et amotor skillsskills deteri

lar oubackn ad

BCI l ove

movial a

f feed

ides res a

ovestionfeedb

().alse respo

sube BCIers be BCImma

to tdaptd to utualonglyo ober anach o

learamef dayp howore faptats disre ation uentcienmpoe Figtly s CPM test graphics interface during selection: (A) The puzzle is shown, the user he user selects the answer. (C) A black border highlights the selected answer. (D) Th

er half of the screen and the six possible options in half (the graphic interface was tested on the RCPM

be used with Standard and Advanced Raven Matrices,ide up to eight solutions). The background was changed

(as in the classic paper-based test) to a light grey color 190, 190) in order to avoid visual fatigue.ction method was based on a scanning modality andmarized in the following four steps:

matic puzzle and solutions were displayed. For the rstmand was accepted by the system. This phase was

d by a small animated picture in the top right corner of (Fig. 5A).s the software enabled the left and right commands ofput system, by which the user could select the desiredig. 5B), also shifting to all targets back and forth. The user

the right command to move upwards (from answer 1the left one to move downwards. The selected answertly zoomed in. There was no time limit to identify theswer (i.e. the system allowed moving after the rst 5 sbject could take all the time needed to start moving).subject had selected his/her answer, he/she had to holdntil a shadow border surrounding the selection signaledP command was enabled (Fig. 5C).ct conrmed his/her selection by the UP command; ifer was not the designated one (e.g. because of a pausetigue or in case the subject changed his/her of mind),uld change his/her answer again by moving the cursoreft/right commands (Fig. 5D).

ction method implements a double conrmation of theus avoiding an incorrect selection of the answer due tote (with consequent high error rate) and consequently

false selections. This reinforced scan could in principleubjects exploration strategy, letting the subject checknswer easily in the traditional assessment (where theswer is not evidenced). The administration time waseased since immediate responses were inhibited sincecompelled the subject to wait 5 s before selecting theis time could (but not necessarily had to) be used to

stimulus and identify the desired answer. The softwarel the answers, the time spent for each, the number oformed to give an answer and other user information.

edbackns normal neuromuscular output channels depend on

muscuon feeduser ca

In acontrocursorbeneccons o

Prov Ensu

(+). Impr

sica The

tasks The f

EEG

Thethat ththe usand thcic cosystemlevel areferreas a mare strorder tthe usfrom esystemer par(time oby-stein a mthe adbut wato ensucalibraconseqthe efwas cotor (securren biofeedback for their successful operation. Both stan-t such as speaking or walking and more specializedh as singing or dancing require, for their initial learn-sequent maintenance, continued adjustments based on

intermediate and nal outcomes (Krakauer et al., 2000;l., 1984). When feedback is absent from the beginning,

do not develop properly. When feedback is lost later on,orate. As a replacement for the brains normal neuro-

The width pattern, calthe signal (the period osignal. Thisand can thuscarcely relis independto identify the right answer. (B) Using left and right command of theer is denitively selected using the up command.

tput channels (Wolpaw et al., 2002), a BCI also depends. Thus, a BCI system must provide feedback so that theapt to the BCI system and vice versa.system feedback is essential for skill development andr the EEG response. However, the feedback from theement can also have other effects, some of which arend some harmful. Below are featured the most pros andback:

continual motivation (+).ttention to the task by maintaining the subjects interest

performance by allowing rapid reaction to wrong clas-s (+).ack stimulus might hamper concentration on other

classications can elicit frustration and thus affect thense ().

ject and the BCI are adapted to each other. This means depends on the interaction of two adaptive controllers:rain, which produces the signal measured by the BCI,

system itself, which translates these signals into spe-nds (Wolpaw et al., 2002). The arrangement of the BCI

he user was described by Wolpaw et al. (2002) as rst-ation, while the adjustment of the user to the system isas third-level adaptation. This process can be dened

learning process. The two systems (man and machine) dependent but have to be adapted independently intain a well-balanced mutual adaptation. Theoretically,d the machine learn progressively and continuouslyther (in a so-called second-level adaptation): the BCIns from the user through the reconguration of classi-ters in order to adapt to short- and long-term variability, hormonal levels, fatigue, illness); the user learns step-

to use the BCI system and perform the mental tasksunctional (i.e. better classied) way. In our BCI systemion was continuous for the user (through biofeedback)crete for the machine (system re-calibration). In order

pseudo-mutual man-machine adaptation, frequent re-(especially in the rst BCI sessions) was carried out and,ly, a fast recalibration method was designed to increasecy of the BCI system. In our system, the biofeedbacksed of a yellow bar placed close to the visual stimula-. 3(7)). The biofeedback signal was given only on theelected BCI command (the one with the best feature).

of the bar corresponded to the power of the SSVEPculated as the ratio between the standard deviation ofwindowed with a window whose length was equal tof stimulation) and the standard deviation of the entire

kind of biofeedback is independent of the calibrations be used during the calibration session and in case ofiable calibration. Indeed, the use of a parameter whichent of calibration such as biofeedback could help the


user have ddistinguish

2.3. Particip

Since wetests by meexclusion cto check thpossibility t

The subje The subje

able (Z < 2 The subje The SSVE The subje

tion comm

Only heahealthy subwithout anyexperienceswhole expeDuring acqutance of aboslightly dimSensibilab CBosisio Parifrom the suof Helsinki.the E. Med

2.4. Admini

The diagare unable based) cogassessed heBCI-based asupplemen

We dividat least a 2-the second

In the reral cognitiIntelligencethe 1st exclenvisaged a

parameters and then congure, train and validate the BCI classi-er. Before this checkup session, the protocol provided 15 min forplacing the electrodes excluding uncooperative subjects or high-impedance electrode placement (>20 k) (2nd exclusion criterion).

eckuni etlibraquenource

Hz sond

ed inrequrm) sultsasy

ing ed. Aere ssionalibrper (lpedchecst thvel

t dided aclud

endratiest t

diret wasrrowed tipute

was to thive cy adealthotoc

onlyeuroapacor) a

secosual ningant wr to Fig. 6. Structure of the administration protocol.

irect control over the parameters, thus increasing classability by enhancing the signal repeatability.

ants

aimed at developing a protocol to administer cognitiveans of a BCI, we established the following graduatedriteria (with each step being a pass/no pass criterione eligibility for both cognitive competencies and theo use the BCI system):

ct has at least average or normal intelligence (IQ 90).ct is collaborative and electrode impedance is accept-0 k).ct produces an SSVEP response.P signal is strong enough to allow classier calibration.ct can understand and use the BCI system and the selec-ands.

lthy subjects were included in this study, specically 19jects (11 males and 8 females) aged from 13 to 40 years,

neurologic decit (Table 2). They had no previous BCI. Every subject with refractive disorder performed therimental session wearing appropriate corrective lenses.isitions, the subject sat on a comfortable chair at a dis-ut 80 cm from a LCD monitor in a noise-controlled and

room. All the measurements were carried out at theampus Point in Lecco (Italy) and the E. Medea IRCCS inni (LC Italy). Written informed consent was obtainedbjects and their families according to the Declaration

The research was approved by the Ethics Committee ofea Institute.

stration protocol

nostic protocol/system was intended for patients who

The chby Pariand cation frelight swith 1correspwere fTime-FtransfoJTFA refor an euser.

Duracquirrates wous sesin the cous pawas heentire to adju(rst lesubjecclassifore exsessioncongudrive tin eachbe senby an arecordto comsessionregardcognittionallwith htion prfuture,with nnitive cbehavi

Thecic vias scanredundin ordeto undertake traditionally administered (paper- or PC-nitive assessment. In this rst validation phase, wealthy subjects and compared their performances on administration vs. a traditional administration, adding atary step (the CFIT test) only for this special population.ed the protocol into two phases (Fig. 6), separated by

h break (the rst part was administered in the morning,part in the afternoon).st phase (and only for this sample), each subjects gen-ve competencies were assessed by Cattells Culture Fair

Test (CFIT) (Cattell, 1949). This task enabled us to verifyusion criterion. After this rst assessment, the protocol

rst checkup session to identify the BCI conguration

phase consat verifyingreplicated twas unnecealready beeproposed inhis/her rstin bit-rate (

The protgame appliBCI system tions, the mof the specp session was based on previous experiences as reported al. (2009) and consisted of two parts (Table 1): screeningtion. During screening the four most effective stimula-cies for the user were identied by presenting only one

at different stimulation frequencies from 6 to 17 Hztep. Each frequency was shown for 8 s, followed by aing resting phase with no stimuli. The recorded datato a specic ofine processing tool performing a Jointency Analysis (JTFA) (by means of a short-time Fourierfor each electrode referenced to the linked mastoid. The

were shown as a color map-based graph which allowedidentication of the best stimulation frequencies for the

calibration data useful to train the classier werell the four light sources were activated, and their ashinget based on the four frequencies identied in the previ-. Calibration was performed ofine using data recordedation phase through a specic tool described in a previ-Parini et al., 2009). From the calibration phase, the user

by the activation of the biofeedback bars (optional). Thek-up session lasted about 10 min. It could be repeatede classier parameters so as to enable the BCI systemof adaptation) to achieve high performance rate. If the

not show or had a weak SSVEP pattern, he/she wass being unable to use the BCI system and was there-ed from the test (3rd/4th exclusion criteria). The rsts with the driven test phase in order to validate the

on parameters adopted with the checkup session. In thehe user was instructed to give eight commands (twoctions) in the shortest possible time, the command to

selected randomly by the software and was indicated on the LCD screen. During the drive-test the software

mes and number of correct/wrong commands in order the speed of the system (as described below). The rst

therefore aimed at verifying the exclusion criteria withe participants characteristics, both in terms of generalompetencies and interaction with the BCI. The tradi-ministered cognitive test was necessary in this studyy subjects and was used to validate the administra-ol in order to verify the assessment reliability. In the

the checkup session will be performed with patientsmuscular diseases, thus testing in a single step the cog-ity (to understand the situation and subsequently adapts well as the degree of interaction with the system.nd phase was aimed at verifying the presence of spe-perceptual competencies underpinning the task such

or matching abilities. This phase was, in some sense,ith healthy subjects, but it will be crucial with patients

obtain reliable results (exclusion criteria). The secondisted of a second but shorter checkup session aimed

the data acquired during the rst one. This sessionhe rst phase except for the screening session, whichssary because the four best stimulation frequencies hadn identied. However, a second Training Session was

order to customize the BCI system for the subject after, short experience, thus allowing a consequent increaserst level of adaptation).ocol then envisaged a testing session consisting of somecations to let the subjects familiarize with the use of theand the selection methods. Among these game applica-atching game allowed us to verify the presence/absenceic visual perceptual skills required to solve the task,

P. Perego

et al.

/ Journal

of N

euroscience M

ethods 201 (2011) 239 250

245Table 1Administration protocol structure with relative descriptions, purposes, durations and subjects.

Phase Step Aim Level of adaptation (Wolpaw et al.,2002)

Mode Time Subjectspassed

Subjectsrejected

Cattells Culture Fair Intelligence Test Form A Scale 2 15 19 0Subject preparation 15 18 1

Checkup session Screening Screening Checking BCI suitability Focus on a blinking light sourcefor 8 s

184 16 2

Ofine check for steady state VEPpattern presence and selection ofthe best four frequencies for SSVEPstimulator

Calibration Training Setting up BCI classier First level of adaptation: BCIadaptation to the subjecta

Collecting data for calibrationfocusing on four light sourcesblinking at difference frequencies

160 15 1

Ofine processingTotal timec 10 15 3

Learning session Testing Driven test Verify and validate the BCI systemperformance (Kronegg et al., 2005)

Third level of adaptation: useradaptation to the BCI systemb

Driven sequence completion task 564 15 0

Verbally guided Visual feedback Target enlargement

AstroBrain ght game Train the subject using the BCIsystem


Move a cursor to achieve targets 310 15 0

Verbally guided Multimodal feedbackFig. 7c

Matching game Train the subject using theselection method with BCI system


Matching target 13 15 0

3 questions Conguration similar to thecognitive test Verbally guided Visual feedback Fig. 7d

Total timed 517 15 0Testing session Cognitive test Raven 1947 A General non-verbal intelligence

testThird level of adaptation: useradaptation to the BCI systemb

12 items PCM test 38 15 0

Choose answers through scanselection Visual feedback Verbally guided Fig. 7e

Raven 1947 AB General non-verbal intelligencetest




Raven 1947 B General non-verbal intelligencetest




Total timed 924 15 0Total time for the entire protocol with pause 4080 15 4

a First level of adaptation: the BCI system was calibrated on the users features; this adaptation is not continuous and should be repeated to enhance BCI performance.b Second level of adaptation: this kind of adaptation is continuous, the subject learns to use and adapts him/herself to the BCI system for the entire duration of the test.c The duration of the screening and calibration phase was set by the software. The total time of the checkup session also depended on the ofine processing which usually incremented the whole session time by about 10 min.d The time for the learning and testing session depended directly on the BCI performance. This table shows the minimum and maximum values measured during the test.


Table 2Data for each participant. MV, MR, RD, LM did not complete the test because they did not meet the second (MV), the third (MR and RD) and the fourth (LM) exclusion criteria.

ID Age Sex Years ofschooling

Bit-rate(Kronegg

RavensRCPM

z-Points

ena

CattellsCFIT IQb

z-PointsCattellc

Totalmoves

Total time(s)

MV 2

MR 3

RD 3

LM 3

SC 1

EF 1

GL 2

SA 3

CT 2

FC 4

LP 1

NF 2

GA 4

MP 2

CR 2

FD 2

MR 2

MM 2

PP 2

a The RCPMb Cattell IQ ic Cattells z-

namely to voptions andof the protosimple gamthe subjectmatching thgame was o(fth exclus

Afterwatask (RCPM5 min. No feon the subj

The entiage less thRCPM admgested in thof the protin Fig. 7.

3. Results

Table 1 cessful/failiet al., 2005) result Rav

9 M 13

6 M 17

9 F 13

0 M 17

4 M 8 57.31 33 0.89

4 M 8 64.80 36 1.38

8 F 17 67.09 35 1.21

0 F 17 61.94 34 1.05

9 M 17 67.10 36 1.38

0 F 17 62.53 35 1.21

3 M 8 65.08 34 1.05

3 M 14 67.37 34 1.05

0 M 17 63.90 35 1.21

2 M 14 60.95 36 1.389 F 17 58.03 34 1.05

5 F 17 55.11 36 1.38

8 F 17 54.70 36 1.38

6 F 16 62.52 35 1.21

7 M 17 60.58 36 1.38

z-points are calculated according to the statistics reported by Measso et al. (1993).s calculated from the row result of the test by means of standardized tables (Cattell, 194points are calculated according to the statistics reported in the manual.

erify the different processes underlying a task with six with a row and column spatial conguration. This partcol displayed the same conguration as the RCPM as ae; using the same selection method used for the RCPM,

had to select, from six different options, the imagee image shown in the upper center of the screen. Thever when the user performed three correct selectionsion criteria).rds the subject was introduced to the proper cognitive). The three Ravens sets were presented after a pause ofedback was provided and no time limits were imposedect.re protocol (rst and second phases) lasted on aver-an an hour. Despite the break introduced by BCI, theinistration took about 20 min, the same time sug-e RCPM administration manual. The detailed structureocol, with session and exclusion criteria, is shown

and discussion

and Fig. 7 show the whole procedure, the timing, suc-ng participants and a summary of the features of each

phase. All rst excluswas higherperformed second exclbecause ththe populavisual interexcluded bto control awere able torectly compthe SSVEP suse, nor didand FD) exp

Table 2 and the RCmance wasfrom the dformula prcomputatio

BR = V R133 2.2

122 1.47

96 0.26

118 1.2

97 0.2 57 775

122 1.47 60 584

109 0.60 56 578

127 1.80 56 747

118 1.20 54 428

118 1.20 56 609

127 1.80 60 517

102 0.13 62 440

133 2.20 59 578

122 1.47 64 567122 1.47 71 855

127 1.80 92 844

118 1.20 63 731

139 2.60 58 463

139 2.60 54 397

9).

19 subjects completed the CFIT test and passed theion criteria. The electrode impedance of a subject (MV)

than 50 k and the signal acquisition could not becorrectly; he was therefore excluded according to theusion criteria. Two subjects (MR and RD) were excludedey belonged to the 1025% (Nijholt et al., 2008) oftion unable to produce a steady-state response to amittent continuous stimulus. Another subject (LM) wasecause he was able to produce SSVEP but not enough

four-command BCI. The remaining 15 volunteers (83%) control the BCI and use the selection method; they cor-leted the entire protocol. None of the subjects perceivedystem or the entire protocol as annoying or difcult to

they report signicant fatigue; only two subjects (SCerienced a mild eye discomfort.presents the subjects results, the BCI performance,PM raw scores and corrected results. The BCI perfor-

calculated, as in our previous work (Parini et al., 2009),ata recorded during the Driven-Test phase with theoposed by Wolpaw (Kronegg et al., 2005) for bit-raten:


Fig. 7. Detailed structure of the administration protocol. The rst part is aimed at validating the protocol with healthy people: the protocol designed for patients begins withthe subjects preparation and can be performed in a single session.


Fig. 8. Occurrences of stimulation frequencies.

where

R = log2 N + P log2 P + (1 P) log21 PN 1

and BR is the bit-rate in bit/min (reported in Table 2), V is theclassication speed in symbols/min drawn from the driven testphase and R represents the information carried by one symbol innumber of bits per symbol. The carrier R was calculated consider-ing N as thand P as therror rate mtion has bee2005) in ordmances. Eausing the sdetails in Paverage timrst correctransfer ratistration ofand MM), tincrease inwas selectefact, a decrtion from threquiring anresults.

The whothe time tament of eleRavens tesfavorably w(Cattell, 19based and Border to avo

Fig. 8 shstimulationdominancephase allow

tion frequencies, we offered the possibility of selecting frequencieswhich were considered less stressful for the subject. Thus, the pre-dominance of low frequencies was related to the users choice inselecting these frequencies that were considered less tiring. We alsocompared the stimulation frequencies with the BCI performanceand test duration without nding a signicant difference or correla-tion (R2 = 0.058, F2,14 = 0,89, p = 0.198). In their study comparing theeffect of different mental tasks on SSVEP stimulation frequencies,Zhenghua and Dezhong (2007) suggested that the same cognitivetask affects different frequencies in a similar way, and low frequen-cies are a better choice than high ones for some of the different tasks(e.g. working memory study).

With regard to demographic data, our healthy participants hadan average age of 25.8 years (1340) and almost all of them hada university degree; their general cognitive abilities, assessed bothby a traditionally administered test (CFIT) and by BCI (RCPM), werewithin the normal range, although quite high, supposedly biasedby recruitment criteria. Most CFIT scores were above norm (seeTable 2) and most subjects obtained a full score on the RCPM. A

r ANOVA was carried out on the bit-rate. Factors were sex,ars otisti

wertem pantf the

(200ypicater sed hithe c

for te ervem

ear co to c

sults (194robaed pRCPM

sloponi

tested a t datt leaducaRCPMent

Fig. 9. Distribthis gure legee number of symbol states (N = 4, the four directions)e probability of correct selection (as one minus theeasured during the calibration session). This deni-

n widely used by other research groups (Kronegg et al.,er to evaluate and quantify BCI communication perfor-

ch subject performed the whole experimental sessionystem congured on a 3-s analysis window (furtherarini et al., 2009). This causes a delay consisting of thee interval between the control task trigger and the

t classication. Despite this lag, results show a highe which matches the aim of an easy and fast admin-

the test. For some participants (EF, GL, CT, NF, GA,he analysis window could be reduced to 1.5 s with an

BCI performance (bit-rate). The same window lengthd in order to simplify the analysis of recorded data. Inease in the windows length requires great concentra-e subjects to avoid false classications and errors, thus

increased cognitive load potentially affecting the test

le protocol lasted at most 1 h and 15 min, with most ofken up by the subject preparation for the EEG (place-ctrodes 1520 min) and BCI system conguration; thet required approximately 1025 min, which comparesith the timing suggested in the Ravens test manual

49). Nonetheless, both versions of the RCPM (paper-CI-based) have no time limit but only a suggestion inid fatigue and stress.ows a histogram calculated on the occurrences of each

frequency in the explored range (617 Hz). A low pre- of low frequencies can be seen. When the screeninged the subject to choose from more than four stimula-

5-factoage, yewas stafactorsBCI sysparticimost oet al.swere tcompuobtain

On moveslow, th(i.e. mothe linneededtest reCattellright, peducatbased similar

SalmCattellreportz-poinfore, ahigh ebased assessmution of Raven CPM results (a) and IQ as by Cattell (b): Blue: control group from normatnd, the reader is referred to the web version of the article.)f schooling, corrected RCPM score and CFIT. No effectcally signicant; the same result was obtained when thee combined. Fig. 10a shows that the performance of themeasured as a bit-rate is age independent. Although thes were interacting with a BCI system for the rst time,m were familiar with computers; this supports Allison8, 2010) recent conclusion that younger male subjectslly more accustomed in the use of BCIs because of theirkills. They were less annoyed by the ickering light andgher bit-rates.ontrary, the bit-rate was related to the number of totalhe entire RCPM (R2 = 0.75); in fact, when the bit-rate isror rate increases and can cause wrong classicationsent of the cursor in a wrong direction). Fig. 10f displaysrrelation (R2 = 0.73) between bit-rate and the total timeomplete the RCPM. Fig. 9 shows the distribution of thein our sample and in studies by Measso et al. (1993) and9). Despite being characterized by a shift towards thebly due to our recruiting and enrolling mainly highlyarticipants, the distribution of the results of the BCI-

and the paper-based Cattel test are comparable, withes.

et al. (1984) reported a correlation of about 0.4 for the and other non-verbal tests, and Conway et al. (2002)correlation of 0.57 between the RCPM and the CFIT:a in Table 1 show a similar correlation (0.45). There-st in our sample of healthy subjects with relativelytion, the results obtained by subjects during a BCI-

are similar to those obtained during a paper-based.

ive; Red: our sample. (For interpretation of the references to color in


F

4. Conclus

Our pronitive skillsinvolved inThis protocdata regardmal, paper-this comparlying the uspsychometrof the BCI s

As depicresults withscore was inTherefore, band given ohigh educaBCI togetheto use, didto inuencthe scores tered testscomparableual, althougscore.ig. 10. Bit-rate across age (a), gender (b), Ravens CPM results (c) and IQ Cattell (d). Time

ion

tocol provides an operative ow chart to assess cog- by a BCI system, checking out the different variables

the interaction by means of a step-by step-procedure.ol helped us verify whether BCIs could provide reliableing general cognitive abilities in comparison with a nor-based test in a sample of healthy subjects. The aim ofison was to evaluate whether the cognitive tasks under-e of a BCI system interfere with the nal result of theic test by a putative overload due to the use and controlystem.ted in Fig. 10e and f, participants obtained similar

paper-based and BCI-based assessments; the RCPMdependent of time (R2 = 0.07) and bit-rate (R2 = 0.004).earing in mind the cognitive burden of the BCI systemur samples characteristics (healthy subjects, relativelytion, high cognitive competencies), our SSVEP-basedr with the protocol described here was quite easy

not tire participants and, most of all, did not seeme the test results. In fact, the correlation betweenobtained by the traditionally and the BCI adminis-

was similar. The duration of the session was also with the session length suggested in the RCPM man-h accuracy was more relevant than time for the test

Our protion: were Rnecessary tfor differenRCPM versigle targets acompare anobtained bypetencies fan additionperformed,that could bconditions.

Performwithin the

The protbration sessuser. The pcalibration gle step. As reliable wittion of cognbeen used in ALS patiif subjects should be a for the test across Ravens test result (e) and bit-rate (f).

tocol seems suitable for BCI-based RCPM administra-CPM routinely administered through a BCI, it would be

o provide a new standardization for the BCI assessmentt populations (or different contexts). The computerizedon we provided is in fact in some sense facilitated: sin-re evidenced for a while letting the subject more easilyd verify the correct/wrong answer. Nevertheless, scores

our healthy participants seemed to depict their com-airly well. Thanks to the computerized RCPM version,al qualitative analysis of the cognitive task could be

thus highlighting selection strategies and difcultiese helpful in the diagnosis and treatment of pathological

ance on the BCI system was independent of age (at least1040 years range) (see Fig. 10a).ocol consisted of two parts, including a redundant cali-ion used to improve the adaptation of the system to therotocol length can be reduced by removing this doublesession and administering the entire protocol in a sin-described above, our novel method seemed feasible andh healthy subject for the SSVEP-based BCI administra-itive tests (RCPM). A SCP-based BCI system has alreadyby Iversen et al. (2008a,b) to assess cognitive functionents through a non-validated test. They assumed that,can control a BCI system with a two-choice task, theyble to answer questions related to their cognitive skills.


Seemingly, Iversen et al. did not consider the possibility that the BCIsystem may need a proper cognitive skill, potentially affecting theresponse. With this protocol we strove to detect variables involvedboth in the interaction underpinning the BCI system and in thesubjects characteristics, thus leading to a step-by-step procedureand an easier interpretation of data. Exclusion criteria were clearlyidentied. The neurocognitive functions identied as ground com-petencies necessary to solve the task were checked in order to avoida difcult to interpret oor effect.

We tested our protocol on a dependent BCI in which the gazecontrol is mandatory (although some studies use attention to mod-ulate SSVEP) (Wang and Wade, 2011). This kind of BCI may be morereliable than an eye-tracker, above all in cases when a neuromotordisease affects gaze causing for example nystagmus which pre-vents eye-tracker use. On the contrary, in other cases, eye-trackersare faster and simpler systems. Future work is needed to conrmthe applicability of the protocol with pathological participants; thesame protocol could be tested with different BCI paradigms to eval-uate the cognitive effort needed to drive SCP, Motor Imagery orP300-based BCI.

Acknowledgments

This work has been partially supported by the Italian Institute ofTechnologystaff (Giuliasupport and

References

Alevriadou Aindependeupper/mid

Allison B, Luthics: how mNeural Sys

Allison BZ, Mcindependetials. Clin N

Basso A, Capitvalues on

Carroll JB. HumCambridge

Conway ARAvariable aity, proce16383.

Cattell R. Cultuof Persona

Cheng M, Gaowith high

Counter SA, Buchanan LH, Ortega F. Neurocognitive impairment m lead-exposedchildren of Andean leadglazing workers. J Occup Environ Med 2005;47:30612.

Ding J, Sperling G, Srinivasan R. Attentional modulation of SSVEP power dependson the network tagged by the icker frequency. Cereb Cortex 2006;16(7):101629.

Iversen IH, Ghanayim N, Kbler A, Neumann N, Birbaumer N, Kaiser J. Conditionalassociative learning examined in a paralyzed patient with amyotrophic lat-eral sclerosis using braincomputer interface technology. Behav Brain Funct2008a;14:114.

Iversen IH, Ghanayim N, Kbler A, Neumann N, Birbaumer N, Kaiser J. Abraincomputer interface tool to assess cognitive functions in completelyparalyzed patients with amyotrophic lateral sclerosis. Clin Neurophysiol2008b;119:221423.

Krakauer JW, Pine ZM, Ghilardi MF, Ghez C. Learning of visuomotor transforma-tions for vectorial planning of reaching trajectories. J Neurosci 2000;20(23):891624.

Kronegg J, Voloshynovskiy S, Pun T. Analysis of bit-rate denitions forbraincomputer interfaces. In: Proceedings at the 2005 int. conf. onhumancomputer interaction (HCI05); 2005.

Lezak MD, Howieson DB, Loring DW. Neuropsychological assessment. 4th ed.; 2004.Mason S, Kronegg J, Huggins J, Fatourechi M, Schloegl A. Evaluating the performance

of self-paced BCI technology. Technical Report, Neil Squire Society; 2006.Measso G, Zappal G, Cavarzeran F, Crook TH, Romani L, Pirozzolo FJ, et al. Ravens

colored progressive matrices: a normative study of a random sample of healthyadults. Acta Neurol Scand 1993;88(1):704.

Nagano-Saito A, Washimi Y, Arahata Y, Kachi T, Lerch JP, Evans AC. Cerebral atro-phy and its relation to cognitive impairment in Parkinson disease. Neurology2005;64:2249.

Nakamura H, Nakanishi M, Furukawa TA, Hamanaka T, Tokudome S. Validity of briefintelligence tests for patients with Alzheimers diseases. Psychiatr Clin Neurosci2000;54:4359.

, Tanrfacing, Mag

based-trans10.115, Magrface a, Junqus a m;52:4

AW, ew anO, Strs, an

F, Ma: focu;90(4

Wadelumin

JR, ncom;113(

Z, Limorman;17:8a W,cies 04. (IIT). The authors would also like to thank the clinical Livetti and Luigi Piccinini) for their invaluable help and

all the subjects for their collaboration and patience.

, Hatzinikolaou K, Tsakiridou H, Grouios G. Field dependence-nce of normally developing and mentally retarded boys of low anddle socioeconomic status. Percept Mot Skills 2004;99:91323.

T, Valbuena D, Teymourian A, Volosyak I, Graeser A. BCI demograph-any (and what kinds of) people can use an SSVEP BCI? IEEE Transt Rehabil Eng 2010;18(2):10716.Farland DJ, Schalk G, Zheng SD, Jackson MM, Wolpaw JR. Towards annt braincomputer interface using steady state visual evoked poten-europhysiol 2008;119(2):399408.

ani E, Laiacona M. Raven coloured progressive matrices: normative305 adult normal control. Funct Neurol 1987;2:18994.an cognitive abilities: a survey of factor analytic studies. New York:

University Press; 1993., Cowan N, Bunting MF, Therriault DJ. Minkoff SRB a latentnalysis of working memory capacity, short-term memory capac-ssing speed, and general uid intelligence. Intelligence 2002;30:

re free intelligence test, scale 2, handbook. Champaign, IL: Institutelity and Ability; 1949.

X, Gao S. Design and implementation of a braincomputer interfacetransfer rates. IEEE Trans Biomed Eng 2002;49(10):11816.

Nijholt Ainte

Parini Stemhighdoi:

Perego PInte

Pueyo Rces a2008

Salmonirevi

Spreen norm

Vialattetials2010

Wang J, and

WolpawBrai2002

Zaini Mperf2005

Zhenghuquen971 D, Pfurtscheller G, Brunner C, Mill J, Allison B, et al. Braincomputer for intelligent systems. IEEE Intell Syst 2008;23(3):729.gi L, Turconi AC, Andreoni G. A robust and self-paced BCI sys-

on a four class SSVEP paradigm: algorithms and protocols for afer-rate direct brain communication. Comput Intell Neurosci 2009.,5/2009/864564.

gi L, Parini S, Andreoni G. BCI ++: a new framework for Brain Computerpplication. In: Proceedings at SEDE ISCA; 2009. p. 3741. C, Vendrell P, Narberhaus A, Segarra D. Coloured progressive matri-

easure of cognitive functioning in cerebral palsy. J Intellect Disabil Res3745.Schmidt RA, Walter CB. Knowledge of results and motor learning: ad critical reappraisal. Psychol Bull 1984;95(3):35586.auss E. A compendium of neuropsycological tests, administration,d commentary. 2nd ed.; 1998.urice M, Dauwels J, Cichocki A. Steady-state visually evoked poten-s on essential paradigms and future perspectives. Prog Neurobiol):41838.

AR. Differential attentional modulation of cortical responses to S-coneance stimuli. J Vis 2011;11(6).Birbaumer N, McFarland DJ, Pfurtscheller G, Vaughan TM.puter interfaces for communication and control. Clin Neurophysiol6):76791.

CT, Low WY, Harun F. Effects of nutritional status on academicce of Malaysian primary school children. Asia Pac J Public Health17.

Dezhong Y. The inuence of cognitive tasks on different fre-steady-state visual evoked potentials. Brain Topogr 2007;20(2):

Cognitive ability assessment by BrainComputer Interface1 Introduction2 Materials and methods2.1 The psychometric test2.1.1 Raven's Colored Progressive Matrices (RCPM)2.1.2 Cattell's culture fair test

2.2 BrainComputer Interface2.2.1 The BCI paradigm2.2.2 ERP data recording2.2.3 The BCI system2.2.4 The RCPM graphics interface2.2.5 Biofeedback

2.3 Participants2.4 Administration protocol

3 Results and discussion4 ConclusionAcknowledgmentsReferences

cognitive ability assessment by brain–computer interface

Documents