January 2007 17P u b l i s h e d b y t h e I E E E C o m p u t e r S o c i e t y

T E C H N O L O G Y N E W S

F or a long time, researchershave been working on a mar-riage of human and machinethat sounds like somethingout of science fiction: a brain-

computer interface. BCIs read electrical signals or

other manifestations of brain activ-ity and translate them into a digitalform that computers can under-stand, process, and convert intoactions of some kind, such as mov-ing a cursor or turning on a TV.

Several academic and corporateresearchers are now working tocommercialize the technology, whileother projects are taking innovativeapproaches to BCIs that could cre-ate interesting products or servicesin the not-too-distant future.

The technology holds greatpromise for people who can’t usetheir arms or hands normally be-cause they have had spinal cordinjuries or suffer from conditionssuch as amyotrophic lateral sclero-sis (ALS) or cerebral palsy. BCIcould help them control computers,wheelchairs, televisions, or otherdevices with brain activity.

There is even a research effortunderway to use BCI as the basis forbrainwave-based biometric authen-tication. The reaction of users’brains to a stimulus of some sortwould determine whether they

could, for example, access a com-puter or enter a building.

Thus, BCI has generated interestas an approach that could yield rev-enue in the marketplace.

However, the technology has along way to go before it can be usedwidely, as it still faces problemssuch as user acceptance and signalaccuracy.

BCI BACKGROUNDERBCI research includes disciplines

such as nanotechnology, biotech-nology, information technology, cog-nitive science, computer science,biomedical engineering, neuro-science, and applied mathematics.

HistoryScientists have been actively

researching BCI since the early1970s. At that time, Jacques Vidal,now a University of California, LosAngeles, emeritus professor, led theuniversity’s federally sponsoredBrain-Computer Interface Project.

Over time, researchers have exper-imented with implanting simple BCIsensors within rats, mice, monkeys,and humans.

In the late 1990s, researchers at theGeorgia Institute of Technology andEmory University demonstrated BCI’smedical potential by implanting anelectrode in the motor cortex of apatient who was paralyzed below theneck and unable to speak. The tech-nique let the patient communicate bymoving a computer cursor.

In 1999, scientists at the MCPHahnemann School of Medicine andDuke University Medical Centertrained rats to use their brain signalsto move a robotic water-dispensingarm.

Invasive versus noninvasive approaches

There are two principal BCIapproaches: invasive techniques,which implant electrodes directlyonto a patient’s brain; and nonin-vasive techniques, in which medi-cal scanning devices or sensorsmounted on caps or headbands readbrain signals.

Both approaches have drawbacks,according to University of SouthernCalifornia professor Theodore Ber-ger, chair of the World TechnologyEvaluation Center’s Panel on BrainComputer Interfaces.

Noninvasive approaches are lessintrusive but can also read brain sig-nals less effectively because the elec-trodes cannot be located directly onthe desired part of the brain. Invasivetechniques, however, require surgeryand carry the risk of infection orbrain damage.

Moreover, noninvasive approaches’ability to read signals from manypoints in the brain could help identifya wider range of brain activity. Thiswould be helpful because the cells thataddress the multiple types of motionsand movement of various body partsare in different parts of the brain,noted Berger.

However, he added, processing thelarge amount of data that neuronsin multiple parts of the brain would

Brain-ComputerInterfaces:WhereHuman andMachine MeetSixto Ortiz Jr.

r1tech.qxp 20/12/06 11:01 AM Page 17

18 Computer

T E C H N O L O G Y N E W S

Eventually, Cyberkinetics hopes toaugment the system to wirelesslysend brain signals to amplifiers andthen a computer, which would elim-inate the need for connectors andcables, said company president andCEO Timothy Surgenor.

Wadsworth Center:A noninvasive approach

The New York State Public HealthDepartment’s Wadsworth Center, apublic health laboratory, is using anoninvasive electroencephalogram(EEG) cap to acquire brain signalsby recording neuronal electricalactivity.

The research team—led byJonathan Wolpaw, chief of Wads-worth’s Laboratory of NervousSystem Disorders and a University ofAlbany professor—has developedthe BCI2000 research system.

The BCI2000 uses an EEG cap,which includes up to 200 electrodesthat are placed on the scalp alongwith conductive paste to aid in thecapture of electrical signals emittedby neurons in the brain.

The system uses a 16-channelbiosignal amplifier to boost the cap-tured signals. Its DSPs then extractand measure signal features, which a digitizer prepares for computerprocessing.

To maximize the system’s effective-ness, Wolpaw said, the researcherstrain subjects how to exercise somecontrol over their brain’s electrical signals.

One Wadsworth implementationhelps people with speech problemscommunicate via the P300 brain-wave, a positive voltage that showsup in an EEG shortly after a subjectexperiences an unexpected or sig-nificant sensory stimulus. The brain-wave doesn’t contain informationbut instead enables Wadsworth’ssystem to detect the subject’s desireto communicate in some way.

Patients watch a random sequenceof letters or images, and when theysee the one they want to communi-cate, their P300 brainwave spikes.The BCI2000 detects the spikes and

generate would be difficult for BCIsystems.

NEW BCI APPROACHESSeveral companies and universities

are conducting important research invarious BCI-related areas. For exam-ple, scientists at the Laboratory ofBrain-Computer Interfaces at GrazUniversity of Technology’s Institutefor Knowledge Discovery are usingBCI to help patients control a pros-thesis, said postdoctoral researcherand lecturer Gernot Müller-Putz.

NeuroSky is working on BCI appli-cations for healthcare providers,including treatments for attentiondeficit disorder, and for the enter-tainment industry, including videogames and toys, said companyspokesperson Johnny Liu.

Cyberkinetics and Brown:An invasive approach

Cyberkinetics NeurotechnologySystems has developed the Brain-Gate Neural Interface System, amedical device designed to helppatients with spinal cord injuries orother types of motor impairmentcontrol a computer with theirthoughts.

BrainGate is based on researchconducted by Brown University pro-fessor John Donoghue, who isCyberkinetics’ founder and chief sci-entific officer.

His research focuses on decipher-ing the way the brain turns thoughtinto the signals that cause actionssuch as arm movement. For exam-ple, he explained, when a personwants to move an arm, millions ofneurons emit pulses in a complexpattern that the spinal cord readsand translates into a signal that con-trols the muscles.

Using multielectrode recordingarrays and functional magnetic res-onance imaging (fMRI) techniques,Donoghue’s research team has cap-tured subjects’ brain signals andthen used digital signal processors(DSPs) and algorithms to translatethem into a format that a computercan understand and process.

The subject must first enter anMRI scanner. The scanner creates amagnetic field and uses radio signalsto capture the brain’s hemodynamicresponses to the person’s thoughtsabout moving, for example, a cur-sor or prosthesis. The hemodynamicresponses include increased bloodflow and hemoglobin supply to theneurons that are actively working.

Cyberkinetics used the BrainGatesystem in a clinical trial that helped avolunteer patient paralyzed from theneck down perform activities such asplaying Pong or controlling a TV.

A surgeon implanted a small micro-electrode array in the patient’s motorcortex, an area located on the brain’sfrontal lobe that helps controls move-ment by sending signals through thespinal cord to the body’s limbs.

The 4 � 4 mm array is equippedwith 100 silicon microelectrodesthat simultaneously sense and trans-mit the electrical impulses from mul-tiple neurons. Focused thought bythe patient about moving somethingyields a specific pattern of strongneuron firing and electrical spikes.

The signals travel to a titaniumconnector attached to the skull andthen via fiber-optic cabling to ampli-fiers that augment the signals androute them to a computer, whichprocesses them and turns them intothe desired output.

BrainGate is undergoing two fur-ther clinical trials, one seeking torestore limb movement in patientswith spinal cord injuries, strokes, andmuscular dystrophy; and anotherlooking to enable communication inpatients suffering from motor neurondiseases such as ALS. The companyexpects to launch the system com-mercially in 2008 or 2009.

Several companies anduniversities are

conducting importantresearch in various BCI-related areas.

r1tech.qxp 20/12/06 11:01 AM Page 18

January 2007 19

Currently, though, he noted,portability is an issue because MRImachines weigh several tons.

Stanford University: Detectingmovement planning

Stanford University BCI re-searchers are looking for ways toidentify the signals the brain makeswhen it is planning to direct thebody to move a certain way. Theseoccur before the signals the brainmakes when it actually directs a spe-cific movement.

“Every time we move, we first planthat movement,” explained Stanfordassistant professor Krishna Shenoy,director of the university’s NeuralProsthetic Systems Laboratory. “Un-derstanding [neural] planning activ-ity is of central importance forguiding prosthetic limbs because itcan tell you where the arm should endup even before you start moving it.”

Knowledge of the brain’s planningactivities could dramatically im-prove mathematical estimations ofhow the arm should move and yieldsystems that are faster and moreaccurate, said Shenoy.

Therefore, his team is investigat-ing how neurons in the brain’s pari-etal reach region plan and guide armmovements.

registers the corresponding image or letter.

This process is currently slow—two to four words per minute—because the system requires time tocollect signals from multiple areas ofthe brain, segregate the signals thatcome from the relevant neurons, andtranslate them appropriately intoactions, noted Wolpaw.

He said Wadsworth researchersare working to make the systemfaster by designing better signal-analysis methods.

The group wants to establish anonprofit foundation to make thesystem available to people who needit, he noted.

Honda: MRI technologyJapan’s Honda Motor Corp. and

ATR Computational NeuroscienceLaboratories recently used brain sig-nals to control a robot’s simplemovements.

Participants positioned within anMRI scanner move their hand or fin-gers in a certain way, explained ATRresearcher Yukiyasu Kamitani.

As in Cyberkinetics’ system,Honda’s scanner detects the subject’sbrain signals. The system sends theMRI signals over Ethernet cables,via TCP/IP, to a computer, whichprocesses the information and thenuses software to issue commandsthat operate a robotic hand.

If the participants in the scannermake a fist or spread their fingers,the robotic hand does the same, asFigure 1 shows.

In the next 10 years, said Hondaspokesperson Sachi Ito, the com-pany could utilize this research toimprove the company’s popularAsimo walking robot, used exten-sively in public relations and adver-tising campaigns.

In the distant future, the technol-ogy might generate auto-safety inno-vations. For example, Kamitani said,the system could decode driverintentions, such as the direction inwhich they want to turn, and com-municate them to other drivers andpedestrians.

Shenoy’s group hopes to use itsresearch results to develop effectiveprostheses, as well as systems thathelp users communicate or controldevices.

Columbia University:Image search

Scientists at Columbia University’sLaboratory for Intelligent Imagingand Neural Computing (LIINC) arecreating a BCI system designed tosearch through images much fasterthan humans or computers can ontheir own.

The federally funded researchcould help law-enforcement officialssearch through video and quicklyidentify criminals, terrorists, or sus-picious activity.

The Cortically Coupled ComputerVision (C3Vision) system utilizes thebrain’s ability to recognize novel,unusual, interesting, or rare elementsin images more quickly than humanscan identify them, explained profes-sor and LIINC director Paul Sajda.

The C3Vision system would showimages very quickly to viewers cho-sen for their ability to recognize spe-cific types of activity. For example,police officers could use the systemto look for criminal activity, andradiologists could utilize it to detect

Figure 1. Honda Motor Corp. has developed a BCI system that can operate a robotic

hand. When a subject moves a hand or fingers, the system’s MRI scanner detects the

brain signals, processes them, and uses software to command a robotic hand to move

the same way.

r1tech.qxp 20/12/06 11:01 AM Page 19

20 Computer

T E C H N O L O G Y N E W S

abnormalities in medical imagessuch as mammograms.

A user wears an EEG cap thatmeasures the electrical signals of neu-ron groups located in different areasof the brain. According to Sajda, theC3Vision system uses this informa-tion to recognize images in which theviewer has seen something novel orrare. It then ranks these images inimportance, based on the strength ofthe viewers’ brain signals.

This capability would mark a vastimprovement over computer-visionsystems, which lack the humanbrain’s capacity to rapidly recognizefamiliar objects in various contexts,poses, or environments, Sajda said.

Helsinki University of Technology: MEG

Scientists at Finland’s HelsinkiUniversity of Technology Labora-tory of Computational Engineering’sCognitive Science and TechnologyResearch Group are using an EEGcap to capture brain signals fromusers and help them control a virtualkeyboard by simply thinking abouthand movement.

The research group has also ex-plored the use of magnetoencephalog-

raphy (MEG), which measures thetiny magnetic fields generated by elec-trical activity in the brain. To deter-mine what subjects want to do,MEG-based systems would have tointerpret these magnetic fields, muchlike the way EEG-based systems inter-pret electrical signals.

The Helsinki scientists say they areinterested in MEG because of itshigh sensitivity and good spatial andtemporal resolution.

However, portability would be anissue because MEG machines arelarge devices and require position-ing subjects inside them, as is thecase with MRI scanners.

Carleton University:Biometrics

Canada’s Carleton University isusing a BCI system as the basis of a biometric identity-authenticationdevice. This could replace traditionalbiometric methods, such as scans thatcompare a person’s fingerprints orretinal patterns with those of autho-rized users of protected systems.

The Carleton approach uses thebrain’s response to stimuli, such assounds or images, as the authentica-tion method. Experiments indicate

that EEG signals generated by thebrain in response to a stimulus areunique to each person and thus havebiometric potential, said Carletondoctoral student Julie Thorpe.

The researchers have not built asystem yet, but conceptually, userswearing a headpiece with electrodeswould press a button or key on akeyboard to start the biometric iden-tification process, think of a thoughtthat would act as their password(called a pass-thought), and thenpress another key to stop the proce-dure. The system would then recordthe brain signal emitted when theusers produce the pass-thought andextract its features for computer processing.

When users want to access a pro-tected system, they would producetheir pass-thought. The biometricsystem would extract its featuresand compare it to those recorded forauthorized users, as Figure 2 shows.

Unlike other biometric approaches,the Carleton system would let userschange their identifier by changingthe stimulus.

Neural Signals: Neurotrophicelectrodes

Neural Signals has released a BCI-based speech-restoration projectthat uses surgically implanted neu-rotrophic electrodes to capture elec-trical signals from the brain’s Broca’smotor area. This area, which con-trols speech, is in the lower part ofthe frontal lobe.

The neurotrophic device inducesneurites—which are projectionsfrom a neuron’s cell body—to growinto the tip of the electrode, whosewires then transmit brain signals forrecording and processing, accordingto Neural Signals chief scientist PhilKennedy. This attachment betweenthe neuron and the electrode createslong-term signal stability, heexplained.

The researchers have identifiedwhich brain signals represent eachof 39 English phonemes. The systemcan thus convert signals from thepatient’s brain into words.

Storefeature subset

Feature subsetapproximately

matches storedfeature subset

Change Login

Feature subsettaken from

features

Featuresextracted from

signalSignal

Signal

“Memorablethought”

Source: Carleton University

Figure 2. In Carleton University’s proposed BCI-based biometric system, subjects use

specific thoughts as passwords (called pass-thoughts). When someone tries to access

a protected computer system or building, they think of their pass-thought. A

headpiece with electrodes records the brain signals.The system extracts the signal’s

features for computer processing, which includes identification of the feature subset

that best and most consistently represents the pass-thought.The biometric system

then compares the subset to those recorded for authorized users.

r1tech.qxp 20/12/06 11:01 AM Page 20

The company charges $250,000for a patient to restore communica-tion via implanted neurotrophicelectrodes, said Kennedy.

INTERFACE INTERFERENCEResearch is improving BCI tech-

nology. However, the approach facesnumerous challenges to widespreaduse.

First, the technology is so new,researchers are still learning how toeffectively implement it and adapt itto different patients, who have dif-ferent needs and physical character-istics.

And until they are widely com-mercialized, BCI systems will tend tobe very expensive, due to the use of sophisticated technologies. Theyalso will be large, making themimpractical for general usage.

BCI systems will have to be moreautomated for the approach to bewidely adopted. The systems arecomplex to use and their readingscan be difficult to interpret, thusrequiring the involvement of experts.

In addition, said Stanford’s Shenoy,technicians must be present to per-form various tasks, such as pluggingin connectors to brain implants,entering parameters needed for sig-nal processing, and adjusting the pro-cessing to accommodate changingneural signals.

Moreover, users must learn how touse their thoughts to create the brainsignals that generate desired actionsin BCI systems, a process that canrequire weeks or months. This posesa considerable challenge to wide-spread adoption of BCI systemsbecause subjects get frustrated andlose patience when the process takestoo long, USC’s Berger explained.

Another challenge, said Cyber-kinetics’ Surgenor, is that BCI com-mercial development is so new,many companies are not investingthe time and money necessary foreffective product development.

Signal accuracyAn issue with BCI technology, par-

ticularly noninvasive approaches, is

the ability to accurately capture sig-nals from the brain that indicate, forexample, the direction in whichpatients want to move their arm.

Getting reliable readings of signalsfrom sensors not in contact with thebrain is difficult, according to Berger.He said attenuation occurs becausesignals must travel a distance fromthe neuron to the sensor.

In addition, he noted, a BCI sys-tem must read signals from a specificset of neurons related to a subject’sthoughts, but when the sensors areoutside the skull, it also picks up sig-nals from other neuron sets.

Electrode performanceInvasive BCI systems can experi-

ence problems with their electrodes.The brain recognizes and tries toremove foreign objects, noted Berger.

When encountering foreign objects,the brain’s glial cells reproducerapidly and move quickly from onepart of the brain to the affected area.For example, if a blood vessel breakswithin the brain, glial cells move toencapsulate the broken vessel.

Thus, Berger said, when a surgeonintroduces an electrode onto thebrain, glial cells try to encapsulate it,which can attenuate signals anddecrease system performance. Also,he added, the brain’s inflammatoryresponse could limit effectiveness.

The brain’s response to the intro-duction of electrodes is a hot area ofresearch, he noted. Current effortsfocus on synthesizing peptides thatform on the outside of neurons andcoating electrodes with them,thereby fooling the body into accept-ing the electrodes as normal cells, heexplained.

Applying drugs, such as rapamy-acin, to electrodes could counter thebrain’s inflammatory response, atechnique used with medical tech-nologies such as heart stents, Bergersaid.

I n the short term, BCI could helpdisabled people control comput-ers, prostheses, wheelchairs, or

other assistive systems.

Over time, researchers hope tomake BCI systems less intrusive,more accurate, faster, and able tointerpret and generate more com-plex actions.

If the systems become more com-pact, said ATR’s Kamitani, theycould even be used in everyday set-tings, for example, to operate key-boards and phones.

And in the distant future, BCI sys-tems could help control complexrobots or electrical probes implanteddirectly into muscles, assisting par-alyzed or injured patients in movingtheir own limbs.

The systems could also help pilotsor drivers by determining when theyare experiencing cognitive overloadand providing feedback that couldease the problem, said USC’s Berger.

Automakers could put a BCI system into a vehicle’s headrest tomonitor drivers’ brain activity andsound an alarm if they start dozingoff.

The gaming industry, particularlyin Europe, is also interested in BCI’spotential for helping users to manip-ulate systems, he noted. GrazUniversity of Technology researchersare experimenting with sophisti-cated, immersive virtual-reality sys-tems that help people use BCI tomove through complex environ-ments by thought processes alone.

“In the long run,” said UCLA’sVidal, “the increasingly intimatecoupling of external computingresources with brain function andbehavior will find applications, somein quite unexpected directions.” ■

Sixto Ortiz Jr. is a freelance technol-ogy writer based in Spring, Texas.Contact him at sortiz1965@gmail.com.

January 2007 21

Editor: Lee Garber, Computer,l.garber@computer.org

r1tech.qxp 20/12/06 11:01 AM Page 21