decoding a new neural–machine interface for control of ...€¦ · decoding a new...

Decoding a New Neural–Machine Interface for Control of Artificial Limbs

Ping Zhou,1,2 Madeleine M. Lowery,1,2,6 Kevin B. Englehart,5 He Huang,1 Guanglin Li,1,2 Levi Hargrove,5

Julius P. A. Dewald,2,3,4 and Todd A. Kuiken1,2,4

1Neural Engineering Center for Artificial Limbs, Rehabilitation Institute of Chicago; 2Department of Physical Medicine and Rehabilitation,3Department of Physical Therapy and Human Movement Sciences, and 4Department of Biomedical Engineering, Northwestern University,Chicago, Illinois; 5Institute of Biomedical Engineering, University of New Brunswick, Fredericton, New Brunswick, Canada; and 6Schoolof Electrical, Electronic and Mechanical Engineering, University College Dublin, Dublin, Ireland

Submitted 16 February 2007; accepted in final form 26 August 2007

Zhou P, Lowery MM, Englehart KB, Huang H, Li G, Hargrove L,Dewald JP, Kuiken TA. Decoding a new neural–machine interfacefor control of artificial limbs. J Neurophysiol 98: 2974–2982, 2007.First published August 29, 2007; doi:10.1152/jn.00178.2007. Ananalysis of the motor control information content made available witha neural–machine interface (NMI) in four subjects is presented in thisstudy. We have developed a novel NMI–called targeted musclereinnervation (TMR)—to improve the function of artificial arms foramputees. TMR involves transferring the residual amputated nerves tononfunctional muscles in amputees. The reinnervated muscles act asbiological amplifiers of motor commands in the amputated nerves andthe surface electromyogram (EMG) can be used to enhance control ofa robotic arm. Although initial clinical success with TMR has beenpromising, the number of degrees of freedom of the robotic arm thatcan be controlled has been limited by the number of reinnervatedmuscle sites. In this study we assess how much control informationcan be extracted from reinnervated muscles using high-density surfaceEMG electrode arrays to record surface EMG signals over the rein-nervated muscles. We then applied pattern classification techniques tothe surface EMG signals. High accuracy was achieved in the classi-fication of 16 intended arm, hand, and finger/thumb movements.Preliminary analyses of the required number of EMG channels andcomputational demands demonstrate clinical feasibility of these meth-ods. This study indicates that TMR combined with pattern-recognitiontechniques has the potential to further improve the function of pros-thetic limbs. In addition, the results demonstrate that the central motorcontrol system is capable of eliciting complex efferent commands fora missing limb, in the absence of peripheral feedback and withoutretraining of the pathways involved.

I N T R O D U C T I O N

Improving the control and function of artificial arms remainsa great challenge, especially in the case of proximal amputa-tions where the disability is greatest. Myoelectric control usinga residual pair of agonist/antagonist muscles is the most com-mon method for operating a motorized prosthesis (Parker andScott 1986). With proximal amputation, the patient can controlonly one joint at a time and must switch back and forth tocontrol multiple joints. This type of control is not intuitive,requires a great deal of conscious effort, and generallyproduces slow, clumsy movements. Attempts to retrainproximal muscles (e.g., shoulder or back muscles) or toextract control information from these remaining muscleswith advanced signal processing techniques have had verylimited success (Hudgins et al. 1993).

We have developed a new technique to improve the functionof upper limb prostheses, termed targeted muscle reinnervation(TMR) (Kuiken 2003; Kuiken et al. 2004). TMR transfersresidual nerves from an amputated limb onto alternative mus-cle groups that are not biomechanically functional due to theamputation. The target muscles are denervated before the nervetransfer. The reinnervated muscle then serves as a biologicalamplifier of the amputated nerve motor commands (Hofer andLoeb 1980). TMR thus provides physiologically appropriatesurface electromyogram (EMG) control signals that are relatedto functions in the lost arm. TMR with multiple nerve transfersprovides simultaneous, intuitive control of multiple degrees offreedom by the motoneuronal activity originally associatedwith the amputated muscles. Great success has been achievedin clinical practice for myoelectric prosthesis control. Usingsimple myoelectric control paradigms based on amplitudemeasurement of the EMG signal from each discrete targetmuscle region, our first four successful subjects have been ableto, for the first time, control two degrees of freedom simulta-neously using only EMG signals. Functional task performancehas been measured by means of a box and block test and aclothes pin test. The subjects showed a 2.5- to 7-fold increaseof speed. Subjectively, they reported significantly easier andmore natural control of their prostheses (Kuiken et al. 2004,2007; Lipschutz et al. 2005; to view video see www.ric.org/research/centers/necal/).

Targeted reinnervation presents a unique tool for neurosci-entific study. The motor cortex dedicated to a limb is known tochange after amputation (Pascual-Leone et al. 1996) and onemight hypothesize that motor control pathways are perma-nently attenuated after long disuse in amputation or at leastwould require considerable training to evoke complex motorcommands. TMR allows access to motor control outputs toassess the robustness of dormant central motor pathways.

If high-fidelity motor commands can be elicited, then it maybe possible for TMR to make a much greater improvement inthe control of artificial limbs. For example, we have usedmedian nerve transfers to control only hand closing and haveused distal radial nerve transfers to control only hand opening.However, in a normal body these nerves innervate dozens ofmuscles in the forearm and hand and control movement of allof the fingers, thumb, and wrist. We hypothesize that muchmore motor control information can be extracted to controlwrist rotation, wrist flexion/extension, and possibly ulnar/

Address for reprint requests and other correspondence: T. Kuiken, 345East Superior Street, Room 1309, Chicago, IL 60611 (E-mail: [email protected]).

The costs of publication of this article were defrayed in part by the paymentof page charges. The article must therefore be hereby marked “advertisement”in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

J Neurophysiol 98: 2974–2982, 2007.First published August 29, 2007; doi:10.1152/jn.00178.2007.

2974 0022-3077/07 $8.00 Copyright © 2007 The American Physiological Society www.jn.org

by 10.220.33.2 on May 21, 2017

http://jn.physiology.org/D

ownloaded from

http://jn.physiology.org/

radial deviation. More dexterous hand operation may also bepossible. For example, pattern recognition combined withTMR may allow a user to select different hand grasp patternssuch as a three-jaw chuck, fine pinch, lateral pinch, or a powergrasp. Although pattern-recognition control is still sequential,its intuitive nature would allow much easier and faster pro-gression in the sequential control of multiple joints. This couldgreatly enhance the performance of artificial arms.

In this study we used high-density surface EMG recordingsto investigate whether further control information can be ex-tracted from TMR using postexperiment pattern-recognitionand signal-processing techniques. The results of this study areexpressed in terms of pattern classification accuracy. Althoughno actual real-time control is demonstrated in this study, ourresults demonstrate that TMR can provide a rich source ofmotor control information and this information in turn prom-ises to dramatically improve artificial arm function for peoplewith proximal arm amputations.

M E T H O D S

Surgical descriptions

Targeted muscle reinnervation was performed with three differentsurgical methods based on the level of amputation, remaining muscle,

and gender of the subject (Table 1). The essence of the technique isthat nonfunctional residual muscles are denervated and the majorresidual nerves of the amputee are transferred to these target muscles.Four brachial plexus nerve transfers were performed on the left side ofa 54-yr-old man with bilateral shoulder disarticulation (BSD) (Hijjawiet al. 2006; Kuiken et al. 2004) (Fig. 1) and a 23-yr-old woman witha very short transhumeral amputation (STH) (Fig. 2A) (Kuiken et al.2007). TMR was performed on two men with long transhumeralamputations (LTH) ages 45 and 50 yr old; the median nerve wastransferred to the medial head of the biceps and the distal radial nervewas transferred to the brachialis muscle (Fig. 2B). The lateral bicepsand triceps remained intact for control of elbow flexion/extension.

EMG data collection

The high-density EMG experiments were performed from 7 to 52mo after the TMR surgery. A grid of monopolar surface EMGelectrodes was placed over the reinnervated target muscles and thebiceps and triceps muscles if present (Fig. 3). Each electrode had acircular recording surface with 5-mm diameter. The grids weresquare, consisting of 79–128 electrodes (depending on the subject),with a center-to-center distance between adjacent electrodes of15–25 mm. A reference electrode was located on the shoulder. Themonopolar EMG signals were collected using a BioSemi Active IIsystem (BioSemi, Amsterdam, The Netherlands). For each chan-nel, the surface EMG signals were sampled at 2 kHz, with a

TABLE 1. Demographics of TMR subjects and timeline of experiments

SubjectAge at Time of

Amputation GenderMechanisms of

Injury

Month After InjuryTMR Surgery

Performed

Month AfterAmputation EMG

Experiment Performed

BSD 54 Male Electric burn 9 46, 57, and 61STH 24 Female Motorcycle 15 22 and 24LTH1 45 Male Automobile 12 41LTH2 50 Male Industrial trauma 8 18

FIG. 1. Schematic description of targeted muscle reinnervation technique in first human subject, a shoulder disarticulation amputee. Three arm nerves (inyellow) were transferred to 3 segments of the pectoralis major muscle. This reinnervated muscle then served as a biological amplifier of the residual nerve motorcommand signals. Surface EMG signals from these targeted muscle segments were then used to control a robotic arm. In this patient, the ulnar nerve transferto the pectoralis minor muscle on the lateral chest wall (not shown in the figure) was unsuccessful, likely due to compromise of the pectoralis minor vascularsupply.

2975NEURAL–MACHINE INTERFACE FOR CONTROL OF ARTIFICIAL LIMBS

J Neurophysiol • VOL 98 • NOVEMBER 2007 • www.jn.org

by 10.220.33.2 on May 21, 2017


ownloaded from


hardware low-pass filter setting the �3-dB point at one fifth of thesampling rate (410 Hz).

The subjects were asked to imagine and actuate 16 differentmovements involving the amputated limb. These movements werechosen to involve all aspects of arm function that might be incorpo-rated in advanced upper limb prostheses. The subjects were instructedto follow a video demonstration of each movement displayed on amonitor and attempt the movement with a comfortable consistenteffort. The subjects did not practice the movements before or duringthe experiments. Each trial consisted of 11 repetitions of one type ofmovement. After an initial video demonstration of a movement, thesubjects were then asked to repeat the movement 10 times along withthe video demonstration. For each repetition of a movement, thesubjects were asked to exert a comfortable level of contraction at amedium force that was held for about 4–5 s. To avoid muscle fatigue,the subjects were allowed to rest for 5 s between each repetition andfor 3 min between each trial. All data were analyzed off-line, after theexperiments.

Signal processing

The surface EMG signals were first processed with a fifth-orderButterworth high-pass filter at 5 Hz to remove the movement artifactand then down-sampled to 1 kHz. The majority of the noise contam-

inating the EMG signal from the reinnervated muscles is the electro-cardiogram (ECG) artifact. We have investigated the effects of theECG artifact on the accuracy of a pattern-classification–based schemefor myoelectric control of powered prostheses. It was found that ECGinterference, at levels typically encountered in a clinical measurement,has little effect on classification accuracy. Therefore in this study, theECG artifact was not removed from the EMG signals before classi-fication.

A suitable segmentation of contraction/no contraction epochs wasdetermined manually for each movement. A channel with clear EMGactivity and quiescent baseline in between was chosen as shown inFig. 4. An EMG amplitude threshold detection scheme was performedon this channel to select the muscle contraction period and segmentthe data. This segmentation was applied to all channels for the trial.Ideally, an automatic, amplitude-based threshold algorithm would beused to segment the data; however, in some recording sessions,subjects produced antagonist muscle activity on returning from theactuated movement to the neutral position. When using an automaticthreshold algorithm, these data would be incorrectly labeled as be-longing to the agonist class and would not allow for proper pattern-recognition training. To avoid mislabeling antagonist muscle activityas the actuated class, we used the manual segmentation method and

FIG. 2. Diagram of targeted muscle reinnerva-tion (TMR) surgeries showing where the terminalbrachial plexus nerves were transferred to whichmuscle segments. A: female subject with a very shorttranshumeral amputation. B: subjects with long trans-humeral amputation.

FIG. 3. Example of high-density monopolar surface EMG recording setup.EMG data were recorded from 115 monopolar electrodes over the reinnervatedpectoralis muscle in the subject while he tried to move his missing arm insynchrony to a video display of the desired arm movement.

FIG. 4. Example of the segmentation process of the surface EMG signals.For each movement, a monopolar channel with clear EMG activity wasselected. For this channel, the onset and offset times for each repetition of themovement were performed. Active data from each repetition of the movementwere concatenated for further processing.

2976 ZHOU ET AL.


by 10.220.33.2 on May 21, 2017


ownloaded from


carefully checked the segmentation process using the video as areference.

Pattern recognition was then performed on analysis windows thatwere 256 ms in duration. This may be considered the maximumpractical record length for real-time operation; data buffering forlonger would introduce a noticeable delay (the processing delay isnegligible). For each analysis window, a feature set was computed andthen provided to the pattern classifier. Overlapping analysis windows(Englehart and Hudgins 2003) were used to maximally utilize thecontinuous stream of data and to produce a decision stream that wasas dense as possible, with regard to the available computing capacity(Englehart and Hudgins 2003). Here, 256-ms windows were advancedby 64 ms, producing an overlap of 192 ms. The overlapping window-ing scheme was applied to the training data (the first half of the activedata) to obtain more training examples. Thus on average about 400windows were obtained from the five repetitions lasting severalseconds each. Disjoint analysis windows (i.e., nonoverlapping) fromthe second half of the active data were used as a test set to evaluate theclassifier’s accuracy. The five test repetitions thus produced about 100disjoint windows for analysis. The performance accuracy for eachintended movement was the percentage of correctly classified win-dows over all the testing windows for the movement. An overallperformance was then calculated as the percentage of correctly clas-sified windows over the testing windows including all the movements.It is worth noting that in clinical practice overlapping windows couldbe used for real-time myoelectric prosthesis control. The classificationaccuracy results do not change, however, regardless of the degree ofoverlap. For overlapping window analysis, the operational delay inreal-time control due to data buffering would be the duration of theoverlapping (e.g., 64 ms) instead of the length of the window (256 ms).

Two feature sets were used to represent the EMG data for classi-fication of the intended movements; a time domain (TD) feature setand a combination of autoregressive features and the root mean square(AR � RMS) feature set.

The TD feature set, first proposed by Hudgins et al. (1993), hasbeen shown to be an effective signal representation for EMG signalclassification. The TD features consist of four time domain statisticsof the EMG signal: the mean absolute value, the number of zerocrossings, the waveform length, and the number of slope sign changes.

Mean absolute value is an estimate of the mean absolute value ofthe signal x in the segment i, which is L samples in length and isgiven by

xi �1

L�k �1

L

�xk� for i � 1, . . . , I (1)

where xk is the k th sample in the segment i and I is the total numberof segments in the record. In our application, a segment correspondsto an analysis window. Therefore the length of each segment is 256ms and I is the total number of windows.

The number of zero crossings is a simple frequency measureobtained by counting the number of times the EMG waveform crosseszero. A threshold (�) must be included in the zero-crossing calculationto avoid zero crossings produced by low-level noise. Given twoconsecutive samples xk and xk�1, the zero-crossing count is incre-mented if

xkxk �1 � 0 and �xk � xk �1� � � (2)

The number of slope sign changes is a feature that may provideanother measure of frequency content. Again, a suitable thresholdmust be chosen to reduce noise-induced slope sign changes. Giventhree consecutive samples, xk�1, xk, and xk�1, the slope sign changecount is incremented if

�xk � xk � 1 and xk � xk � 1� or �xk � xk � 1 and

xk � xk � 1� and �xk � xk�1� � � or �xk � xk �1� � �

(3)

In calculating the number of zero crossings and the number of slopesign changes, the threshold � was set to 2.5% of the full-scale range,after amplification.

Waveform length is a feature that provides information on thewaveform complexity in each segment. This is simply the cumulativelength of the waveform over the segment, defined as

l0 � �k � 1

L

��xk� (4)

where �xk � xk � xk�1. The resultant values indicate a measure ofwaveform amplitude, frequency, and duration.

Graupe et al. (1982) showed that for stationary Gaussian statistics,the EMG signal can be modeled as a linear autoregressive (AR) timeseries

xk � �i � 1

p

aixk � i � ek (5)

where ai represents AR coefficients, p is the model order, and ek is theresidual white noise. It has been shown that the EMG spectrumchanges with muscle contraction state, resulting in a change in ARcoefficients. Therefore by monitoring the AR coefficients, one canestimate the muscle contraction state. In this study, a six-order ARmodel and RMS amplitude of the segment were used to build thefeature set.

For each analysis window, a feature set was extracted on eachchannel, producing an m-dimensional feature vector (m � 4 for TDfeature sets; m � 7 for AR � RMS feature sets). After concatenatingthe feature sets of all the channels, the final feature set vector [(m �n)-dimensional, where n is the number of channels] was provided tothe classifier. A linear discriminant analysis (LDA) classifier (Tou andGonzalez 1974) was used for classification of different movements.More complex and potentially more powerful classifiers may beconstructed, but it has been shown in previous work (Hargrove et al.2007) that the LDA classifier does not compromise classificationaccuracy. Compared with other classifiers, the LDA classifier is alsomuch simpler to implement and much faster to train.

Bipolar electrode configurations have a more focal recording area,are spatially selective with respect to muscle fiber direction, and aremore clinically relevant than monopolar recordings. Therefore thepattern-recognition analyses were also performed using bipolar elec-trode configurations. In the transhumeral subjects the bipolar elec-trodes were aligned with the humerus and the dominant muscle fiberdirection. In the BSD and short transhumeral subjects the pectoralmuscle fibers run in different directions; thus an analysis of bipolarspatial orientation was performed with vertical, horizontal, and diag-onal directions.

Channel selection

The high-density surface EMG recording was used to evaluate howmuch control information one can extract with the maximum possiblenumber of EMG signals from the TMR sources. However, it isimpractical to use the high-density surface EMG as a source forreal-time control. Therefore a preliminary study seeking a practicalnumber of electrodes was conducted. In this study, a straightforwardsequential feedforward selection (SFS) algorithm was used (Somolet al. 1999), which iteratively added the most informative channels, asdetermined by empirical classification performance. In the first itera-tion of this method, each channel was used, independently, to trainand subsequently test classification performance. The channel produc-ing the highest classification accuracy was chosen as the first “opti-mal” channel. For the next iteration, the first optimal channel waspaired with each of the other channels to form a two-channel EMGdata set for classification. The pair of EMG channels generating thehighest classification accuracy was considered the “optimal” two-channelsubset. This procedure was repeated until the number of “optimal”electrodes cumulated to a desired number of EMG channels.



by 10.220.33.2 on May 21, 2017


ownloaded from


R E S U L T S

The spatial EMG activity for each movement was charac-terized by contour plots or color maps where the root meansquare (RMS) value of each channel’s EMG was representedby different colors, interpolating the EMG amplitude betweenelectrode sites. With each intended arm movement, the inten-sity of the surface EMG signal above the reinnervated muscleshad a distinct and repeatable pattern. Figure 5 shows anexample of the spatial EMG activity for six movements char-acterized with contour plots in the BSD subject. It is worthnoting that for several movements such as thumb adduction,wrist pronation, and elbow flexion, the maximum EMG am-plitude focuses on a similar location. This suggests that theconventional control (i.e., solely based on EMG amplitude) isnot suitable for control of these movements. Pattern-recogni-tion–based control is needed in a case such as this.

A series of pattern-recognition analyses were performed ondata windows using a time domain (TD) feature set, and thecombination of autoregressive (AR) coefficients and the RMSof the signal as a feature set. The analysis was conducted usinga monopolar electrode configuration and three bipolar elec-trode configurations in transversal, longitudinal, and diagonaldirections, respectively. Classification accuracy for the 16intended movements was high using all of the surface EMGrecordings from the reinnervated muscles. Table 2 showsclass-to-class results from a typical experiment; examination ofthe specific movements revealed only a few movements thathad accuracies �95%. Table 3 summarizes the average clas-sification performance for all 16 movements in the four sub-jects. The classification accuracy for the 16 intended distal armmovements using surface EMG recordings from the reinner-vated muscles was high in all subjects. With the monopolarchannels the average overall classification accuracy was90.5 � 6.3% for TD feature sets and 90.0 � 7.3% for AR �RMS feature sets. Using bipolar electrode configurations con-sistently improved the accuracy of classification to an averageof 96.0 � 3.9% with TD and 95.0 � 5.2% with AR � RMSfeatures. Across all subjects, there was no notable difference inthe accuracy of the TD versus the AR � RMS feature sets.

Analyses were performed using bipolar electrodes aligned intransversal, longitudinal, and diagonal directions to determinewhether there might be an optimal angle with respect to theunderlying muscle fiber direction. There was no appreciabledifference in classification accuracy with the three bipolarelectrode orientations (2.5% difference in accuracy in anygiven experiment). Consistent levels of accuracy were ob-served in the repeated experiments of two subjects. Highaccuracy was found when experiments were conducted as longas 5 yr after injury and 4 yr after targeted reinnervationsurgery.

There were no pronounced patterns of error between sub-jects. Errors occurred in all movement classes with the major-ity of error related to fine finger and thumb movements. Thebest performance was seen in a long transhumeral amputationsubject (LTH2). Examination of his averaged class-to-classresults revealed only one movement had accuracy 99%;classification of index finger extension was only 77%. It wasconfused exclusively with fingers 3–5 extension and accountedfor 88% of all errors in this experiment. The classificationaccuracy was notably lower in the female subject with the hightranshumeral amputation compared with the other three sub-jects. We suspect that the decreased performance is due tobreast tissue attenuating the surface EMG from the mediannerve to sternal pectoral muscle transfer.

The preliminary channel reduction analysis indicated that agreatly reduced number of EMG electrodes could be usedwhile maintaining high classification accuracy. As shown inFig. 6, five to nine bipolar electrodes still allowed classificationaccuracy within 5% of each subject’s maximum accuracy usingall possible electrode combinations in these four subjects. Onlyfour to seven optimally placed bipolar electrodes were requiredto maintain 90% of the maximum accuracy.

The pattern-recognition methods used in this investigationare computationally efficient; a speed benchmark was de-scribed previously (Englehart and Hudgins 2003) in which a1-GHz Pentium III–based workstation requires roughly 4 msto process each EMG channel. Improvements in coding effi-ciency and processor speed currently place this delay at 0.25

FIG. 5. Example of the contour plots of surfaceEMG amplitude for 6 different movements. Contourplots were built using monopolar high-density sur-face EMG recordings on the reinnervated majorpectoralis muscle. They represent the EMG rootmean square (RMS) amplitude averaged over theentire interval of a contraction. Distinct surfaceEMG distributions are evident for differentmovements.

2978 ZHOU ET AL.


by 10.220.33.2 on May 21, 2017


ownloaded from


ms per channel, resulting in a roughly 4-ms delay for 16 EMGchannels, which is adequate for real-time control.

D I S C U S S I O N

Targeted muscle reinnervation is a new neural–machineinterface that uses residual muscles as biological amplifiers ofamputated arm nerve motor commands. Initial clinical suc-cesses with TMR using simple signal processing methodsbased on EMG amplitude measurement (i.e., direct control)have been very promising. However, only a single independentEMG signal can generally be acquired from each nerve transferand the motor control information of nerve is mixed in thereinnervated target muscle. Two independent antagonist EMGsignals are needed to control each degree of freedom in theTMR prosthesis. This has limited the technique to providecontrol of two simultaneous degrees of freedoms with TMR:1) hand open/close and 2) elbow flexion/extension. Other armmovements, like wrist flexion/extension and rotation, havebeen controlled with shoulder switches—a fairly unintuitiveand cumbersome method of operation.

In the current study we have demonstrated that by applyingpattern-recognition techniques with TMR, substantially moremotor control commands can be extracted from the reinner-vated muscles. Using pattern-recognition techniques on high-density surface EMG recordings allowed very high accuracy inpredicting the intended 16 movements (i.e., 8 degrees offreedom) of the targeted reinnervation subjects. The medianand distal radial nerve transfers best highlight the differencebetween direct control and pattern-recognition control. Withdirect control the muscle segments reinnervated by the medianand distal radial nerve operated only hand opening and closing.Using pattern recognition, information was extracted about thesubject’s desire to perform wrist flexion, wrist extension, wristrotation, and separately move the thumb, index finger, or digits3–5. This demonstrates a great potential to provide intuitivecontrol of more articulate artificial arms and further improvefunction for people with amputations.

In similar experiments using the forearm of able-bodiedindividuals to simulate control of transradial amputees similarlevels of classification accuracy to those observed here werefound (95–97%) (Chu et al. 2006; Huang et al. 2005), allowingthe subject proportional, sequential control of a virtual handand wrist in real time. These studies in able-bodied subjectsdemonstrate that pattern-recognition algorithms can success-fully be used to provide intuitive control of artificial limbs andproportional control of movement. However, neither of thesestudies included articulations of the thumb, index finger, andfingers 3–5 as performed by the TMR users. Presumably, thiswould be a very difficult task using only signals from theextrinsic muscles; TMR users have a distinct advantage as theypossess sites containing activity corresponding to the intrinsicmuscles of the hand.

TMR provides a rich source of additional control data thatare physiologically related to the missing limb. The highclassification accuracy was consistent within subjects, demon-strating good repeatability. It was also high between subjectswho had had different surgical procedures and had differentremaining posttraumatic anatomy and geometry of their targetmuscle, demonstrating that the surgical concept can be appliedto a broad array of injury levels. The lowest accuracy wasT

AB

LE

2.A

vera

gecl

assi

ficat

ion

resu

lts

for

BSD

subj

ect

from

thre

eex

peri

men

tsus

ing

TD

and

AR

�R

MS

feat

ure

sets

and

bipo

lar

elec

trod

esp

atia

lco

nfigu

rati

ons

intr

ansv

ersa

l,lo

ngit

udin

al,

and

diag

onal

confi

gura

tion

s

Pred

icte

dC

lass

Tar

get

Cla

ssW

rist

Flex

ion

Wri

stE

xten

sion

Thu

mb

Add

uctio

nT

hum

bA

bduc

tion

Wri

stSu

pina

tion

Wri

stPr

onat

ion

Elb

owFl

exio

nE

lbow

Ext

ensi

onH

and

Ope

nH

and

Clo

seT

hum

bE

xten

sion

Thu

mb

Flex

ion

Inde

xFi

nger

Flex

ion

Inde

xFi

nger

Ext

ensi

on

Fing

ers

3–5

Flex

ion

Fing

ers

3–5

Ext

ensi

on

Wri

stfle

xion

990

00

00

00

00

00

00

00

Wri

stex

tens

ion

010

00

00

00

00

00

00

00

0T

hum

bad

duct

ion

00

981

00

0s0

00

01

00

00

Thu

mb

abdu

ctio

n0

00

950

10

01

03

00

00

0W

rist

supi

natio

n0

00

010

00

00

00

00

00

00

Wri

stpr

onat

ion

00

00

098

00

20

00

00

00

Elb

owfle

xion

00

00

00

100

00

00

00

00

0E

lbow

exte

nsio

n0

00

00

00

100

00

00

00

00

Han

dop

en0

00

10

20

095

02

00

10

0H

and

clos

e0

00

00

00

00

990

00

01

0T

hum

bex

tens

ion

00

01

01

00

00

970

01

00

Thu

mb

flexi

on0

00

00

00

00

00

100

00

00

Inde

xfin

ger

flexi

on0

00

00

00

00

00

010

00

00

Inde

xfin

ger

exte

nsio

n0

00

00

00

00

00

00

100

00

Fing

ers

3–5

flexi

on0

00

00

00

00

00

00

199

0Fi

nger

s3–

5ex

tens

ion

00

01

00

00

00

30

04

092

Res

ults

are

expr

esse

das

perc

enta

ges.

The

resu

ltsal

ong

the

mai

ndi

agon

al,s

hade

din

light

,are

corr

ectc

lass

ifica

tions

(acc

urac

y)an

dth

ose

offt

hem

ain

diag

onal

,sha

ded

inda

rk,a

rein

corr

ectc

lass

ifica

tions

(err

orra

te).



by 10.220.33.2 on May 21, 2017


ownloaded from


found in our female patient with a humeral neck amputation.We believe the reason for her lower accuracies was breasttissue. The sternal pectoralis major muscle was reinnervated bythe median nerve. This muscle was covered by 1–4 cm ofbreast tissue, which caused substantial spatial filtering, atten-uation of signal amplitude, and increased cross talk betweenEMG signals from the motor units of the muscle.

Using a large number of electrodes is not practical forclinical implementation of pattern-recognition control. A pre-liminary analysis was thus performed to determine approxi-mately how many electrodes would be required while main-taining high classification accuracy. It was possible to achievehigh levels of classification accuracy with four to nine bipolarelectrodes. Similarly, an analysis of computational demandestimated that the algorithms used are efficient enough for anembedded system of �16 channels based on current micropro-cessor technology. These results demonstrate clinical feasibil-ity and robustness in the concept of using pattern-recognitiontechniques to decode control information from target muscles.Further investigations are needed to determine acceptable clin-ical electrode numbers and methods for best placement becausehigh-density EMG analysis is not routinely available in atypical clinical setting.

The ability to proportionally control velocity or force ofprosthetic components significantly enhances function. In theseexperiments, subjects were asked to maintain a moderateconstant force, and feedback regarding the level of contractionwas not provided; thus the results do not specifically addresswhether proportional control is possible in TMR subjects usingpattern-recognition techniques. On inspection of the data ofthis study, it is evident that the contraction levels do indeedvary substantially, suggesting that proportional control is in-deed possible. Also, proportional control has been demon-strated by our subjects using amplitude-based EMG controlwith their prostheses (Kuiken et al. 2004, 2007). Furtherexperiments are necessary to determine the dynamic range ofcontraction levels that is possible, without significantly degrad-ing classifier performance.

Direct comparisons between different NMI approaches aredifficult. For amputees, direct peripheral nerve recording andstimulation have been investigated using nerve-cuff electrodes,sieve electrodes, and penetrating arrays (Branner et al. 2004;Crampon et al. 2002; Stieglitz et al. 2002), although no clini-cally viable systems have been developed. Targeted reinner-vation is clearly a practical approach in that no implanteddevices are required to record the motor control, as opposed tobrain–machine interfaces using cortical implants and periph-eral nerve recording systems. The fidelity of motor control dataextracted with TMR is high; clinically we can simultaneouslycontrol 2 degrees of freedom very well and this study indicatesthe potential to have much more refined control in the wrist,hand, and even fingers.

Neural plasticity

It is known that the motor cortex dedicated to an amputatedlimb changes after amputation and one might hypothesize thatthe unused motor control abilities are lost with time (Pascual-Leone et al. 1996). This work demonstrates that the centralmotor control system is capable of eliciting complex efferentneural commands for a missing limb without any training toawaken these pathways. The time of complete nonuse for thesecentral pathways was at 18 mo (time from amputation toreinnervation and fitting of a TMR prosthesis) and high rec-ognition rates of finger and thumb movements could be foundover 5 yr after amputation. This unique experimental modelindicates and adds evidence that motor command pathways arevery enduring.

It has long been known that, peripherally, regeneratingmotor axons can cross-reinnervate a foreign muscle (Kuikenet al. 1995; Weiss and Hoag 1946). TMR utilizes this principle

TABLE 3. Pattern recognition results in amputee subjects

Monopolar Bipolar

Subject TD AR � RMS TD AR � RMS

BSD* 94.1 � 0.2 92.3 � 2.7 98.4 � 0.7 97.8 � 1.1STH** 81.1 � 3.5 79.5 � 6.5 90.3 � 2.9 87.6 � 2.9LTH1 94.2 92.0 97.1 95.5LTH2 92.4 96.2 98.3 99.2Average 90.5 � 6.3 90.0 � 7.3 96.0 � 3.9 95.0 � 5.2

Values are means � SD, expressed as percentages. Time domain (TD) feature set and a combination of autoregressive features and the root mean square (AR �RMS) feature sets were used. *Average of three experiments. **Average of two experiments. For both, three different bipolar electrode configurations were usedfor each experiment.

FIG. 6. Normalized movement classification accuracy vs. number of bipo-lar surface electrodes from 4 TMR subjects. Monopolar recordings werecombined in bipolar pairs in 3 different directions (transversal, longitudinal,and diagonal) resulting in over 400 bipolar combinations in each experiment.Each subject’s classification accuracy was normalized to his/her maximumaccuracy using all available bipolar EMG channels. Time domain (TD)features were used for classification of 16 intended arm movements.

2980 ZHOU ET AL.


by 10.220.33.2 on May 21, 2017


ownloaded from


to an extreme. Very large nerves that normally innervatedozens of different muscles functionally reinnervated the targetmuscles. A broad spectrum of motor units had to be present inthese relatively small reinnervated muscle segments to allowidentification of different finger, thumb, wrist, and elbowmovements. These motor units were recruited relatively easilybecause the subjects were instructed to perform the specificmovements at moderate force levels in a relaxed manner, not avigorous contraction that would activate motor units with highrecruitment thresholds. Furthermore, there was evidence oforganization in the reinnervating process as seen in the contourmaps of Fig. 3. In our BSD subject, his thumb abductorsreinnervated a separate muscle area than the other mediannerve flexion muscles. This adds to the growing evidence thatthere is functional organization of the proximal brachial plexusnerves (Stewart 2003).

Future developments for improved control of multiarticularprosthetic devices

This work demonstrates the potential for further improvingthe control of more advanced, highly articulated prostheticarms. More research is needed to implement a practical system,including minimizing electrode numbers, determining accept-able locations, and dealing with the challenges of recordingEMG signals in a dynamic environment. There are many otherpaths that could lead to increased control with TMR. Refine-ments in surgical technique may allow for the creation of moreindependent EMG control signals. If there is somatotopicorganization to nerves, then different fascicles of a nerve mayhave different motor functions. Separating the fascicles ofnerves and transferring each to separate muscle regions mayprovide more spatial separation and an increased number offunctionally independent muscle regions. For example, if thefascicles that went to the triceps could be separated from therest of the radial nerve in a shoulder disarticulation amputee,then two independent reinnervated muscle regions could beformed with distinctly separate functions.

There are additional computational approaches that may alsoallow improved simultaneous control of prostheses. Althoughpattern classification as used in this analysis precludes simul-taneous control of multiple joints, a hybrid approach of usingpattern-recognition techniques in conjunction with traditionaldirect control is promising. For example, the amplitude of theEMG from muscle areas reinnervated by the musculocutaneousnerve and radial nerve may be used to directly control elbowfunction, whereas median and radial nerve areas may be usedconcurrently to operate the wrist and hand with pattern recog-nition. Using parallel pattern-recognition techniques on differ-ent muscle regions may also enable simultaneous control ofmore movements in an intuitive manner. Also, the LDA clas-sifier used in this study is but one of many possible tools; otherdecoding schemes may yield better performance. Other com-putational approaches include source separation techniques,such as blind source separation (Farina et al. 2004), or tradi-tional “inverse model” approaches as used in cardiac physiol-ogy (Li et al. 2003) and neurophysiology (Ilmoniemi et al. 1985).

Finally, accessing EMG signals under subcutaneous fat,breast tissue, or from deeper muscle regions remains a chal-lenge with surface EMG. An implanted myoelectric sensorsystem (IMES) (Loeb et al. 2001; Lowery et al. 2006) could

ameliorate many problems such as signal attenuation by sub-cutaneous fat, cross talk, movement artifact, and skin imped-ance variation. It would bypass the skin interface, provideaccess to deeper tissues, and allow recording from more focalareas of target muscle. This may allow for more stable EMGrecording and higher discrimination of signal content, makingthe motor control information more consistent and the classi-fication algorithms more robust.

A C K N O W L E D G M E N T S

We thank J. Yao and B. Lock for assistance with this project and P. Marascofor drawing Fig. 2B.

G R A N T S

This work was supported by National Institute of Child Health and HumanDevelopment Grants R01 HD-043137, R01 HD0-044798, and Contract NO1-HD-5-3402; U.S. Department of Education, National Institute on Disabilityand Rehabilitation Research Grant H133F060029; Natural Sciences and En-gineering Research Council of Canada Grant 217354; and the Defense Ad-vanced Research Projects Agency, Army Research Office, DEKA PrimeContract W911NF-06-C-001.

R E F E R E N C E S

Branner A, Stein RB, Fernandez E, Aoyagi Y, Normann RA. Long-termstimulation and recording with a penetrating microelectrode array in catsciatic nerve. IEEE Trans Biomed Eng 51: 146–157, 2004.

Chu JU, Moon I, Mun MS. A real-time EMG pattern recognition systembased on linear-nonlinear feature projection for a multifunction myoelectrichand. IEEE Trans Biomed Eng 53: 2232–2239, 2006.

Crampon MA, Brailovski V, Sawan M, Trochu F. Nerve cuff electrode withshape memory alloy armature: design and fabrication. Biomed Mater Eng12: 397–410, 2002.

Englehart K, Hudgins B. A robust, real-time control scheme for multifunc-tion myoelectric control. IEEE Trans Biomed Eng 50: 848–854, 2003.

Farina D, Fevotte C, Doncarli C, Merletti R. Blind separation of linearinstantaneous mixtures of non-stationary surface myoelectric signals. IEEETrans Biomed Eng 51: 1555–1567, 2004.

Graupe D, Salahi J, Kohn K. Multifunctional prosthesis and orthosis controlvia microcomputer identification of temporal pattern differences in single-site myoelectric signals. J Biomed Eng 4: 17–22, 1982.

Hargrove L, Englehart K, Hudgins B. A comparison of surface and intra-muscular myoelectric signal classification. IEEE Trans Biomed Eng 54:847–853, 2007.

Hijjawi JB, Kuiken TA, Lipschutz RD, Miller LA, Stubblefield KA,Dumanian GA. Improved myoelectric prosthesis control accomplishedusing multiple nerve transfers. Plastic Reconstr Surg 118: 1573–1578, 2006.

Hoffer JA, Loeb GE. Implantable electrical and mechanical interfaces withnerve and muscle. Ann Biomed Eng 8: 351–360, 1980.

Huang Y, Englehart K, Hudgins B, Chan AD. A Gaussian mixture modelbased classification scheme for myoelectric control of powered upper limbprostheses. IEEE Trans Biomed Eng 52: 1801–1811, 2005.

Hudgins B, Parker PA, Scott RN. A new strategy for multifunction myo-electric control. IEEE Trans Biomed Eng 40: 82–94, 1993.

Ilmoniemi RJ, Hamalainen MS, Knuutila J. The forward and inverseproblems in the spherical model. In: Biomagnetism: Applications andTheory, edited by Weinberg H, Stroink G, Katila T. New York: Pergamon,1985, p. 278–282.

Kuiken TA. Consideration of nerve-muscle grafts to improve the control ofartificial arms. J Tech Disabil 15: 105–111, 2003.

Kuiken TA, Childress DS, Rymer WZ. The hyper-reinnervation of ratskeletal muscle. Brain Res 676: 113–123, 1995.

Kuiken TA, Dumanian GA, Lipschutz RD, Miller LA, Stubblefield KA.The use of targeted muscle reinnervation for improved myoelectric pros-thesis control in a bilateral shoulder disarticulation amputee. Prosthet OrthotInt 28: 245–253, 2004.



by 10.220.33.2 on May 21, 2017


ownloaded from


Kuiken TA, Miller LA, Lipschutz RD, Lock B, Stubblefield K, Marasco P,Zhou P, Dumanian G. Targeted reinnervation for enhanced prosthetic armfunction in a woman with a proximal amputation. Lancet 369: 371–380,2007.

Li G, Zhang X, Lian J, He B. Noninvasive localization of the site of originof paced cardiac activation in human by means of a 3-D heart excitationmodel. IEEE Trans Biomed Eng 50: 1117–1120, 2003.

Lipschutz RD, Miller LA, Stubblefield KA, Dumanian G, Phillips ME,Kuiken TA. Transhumeral level fitting and outcomes following targetedhyper-reinnervation nerve transfer surgery. Proc Myoelectric Control Symp,Fredericton, Canada, 2005, p. 2–5.

Loeb GE, Peck RA, Moore WH, Hood K. BION™ system for distributedneural prosthetic interfaces. Med Eng Phys 23: 9–18, 2001.

Lowery MM, Weir RF, Kuiken TA. Simulation of intramuscular EMGsignals detected using implantable myoelectric sensors (IMES). IEEE TransBiomed Eng 53: 1926–1933, 2006.

Parker PA, Scott RN. Myoelectric control of prostheses. Crit Rev BiomedEng 13: 283–310, 1986.

Pascual-Leone A, Peris M, Tormos JM, Pascual JP, Catala MD. Reorga-nization of human cortical motor output maps following traumatic forearmamputation. Neuroreport 7: 2068–2070, 1996.

Somol P, Pudil P, Novovicova J, Paclik P. Adaptive floating search methodsin feature selection. Pattern Recognit Lett 20: 1157–1163, 1999.

Stewart JD. Peripheral nerve fascicles: anatomy and clinical relevance.Muscle Nerve 28: 525–541, 2003.

Stieglitz T, Ruf HH, Gross M, Schuettler M, Meyer JU. A biohybrid systemto interface peripheral nerves after traumatic lesions: design of a highchannel sieve electrode. Biosens Bioelectron 17: 685–696, 2002.

Tou JT, Gonzalez RC. Pattern Recognition Principles. Reading, MA: Add-ison–Wesley, 1974.

Weiss P, Hoag A. Competitive reinnervation of rat muscles by their own andforeign nerves. J Neurophysiol 9: 413–418, 1946.

2982 ZHOU ET AL.


by 10.220.33.2 on May 21, 2017


ownloaded from


decoding a new neural–machine interface for control of ...€¦ · decoding a new...

Documents