374 ...cspl.ee.pusan.ac.kr/sites/cspl/download/inter... · 374...

374 IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING, VOL. 23, NO. 3, MAY 2015

Neuron Selection Based on Deflection CoefficientMaximization for the Neural Decoding of

Dexterous Finger MovementsYong-Hee Kim, Nitish V. Thakor, Fellow, IEEE, Marc H. Schieber, and Hyoung-Nam Kim, Member, IEEE

Abstract—Future generations of brain-machine interface (BMI)will requiremore dexterousmotion control such as hand and fingermovements. Since a population of neurons in the primary motorcortex (M1) area is correlated with finger movements, neural ac-tivities recorded in M1 area are used to reconstruct an intendedfinger movement. In a BMI system, decoding discrete finger move-ments from a large number of input neurons does not guarantee ahigher decoding accuracy in spite of the increase in computationalburden. Hence, we hypothesize that selecting neurons importantfor coding dexterous flexion/extension of finger movements wouldimprove the BMI performance. In this paper, two metrics are pre-sented to quantitatively measure the importance of each neuronbased on Bayes risk minimization and deflection coefficient maxi-mization in a statistical decision problem. Sincemotor cortical neu-rons are active withmovements of several different fingers, the pro-posedmethod is more suitable for a discrete decoding of flexion-ex-tension finger movements than the previous methods for decodingreaching movements. In particular, the proposed metrics yieldedhigh decoding accuracies across all subjects and also in the case ofincluding six combined two-finger movements. While our data ac-quisition and analysis was done off-line and post processing, ourresults point to the significance of highly coding neurons in im-proving BMI performance.

Index Terms—Brain-machine interfaces (BMI), neural de-coding, neural prosthesis, neuron selection.

I. INTRODUCTION

A BRAIN-MACHINE interface (BMI) is a methodologyto form a communication pathway between a brain and

an external device. It provides an appropriate alternative forpeople with damaged cognitive or sensory-motor functions [1].The key point of BMI systems is a neural decoding procedure,

Manuscript received September 05, 2013; revised February 01, 2014, May01, 2014, June 17, 2014, August 29, 2014; accepted October 04, 2014. Date ofpublication October 22, 2014; date of current version May 06, 2015. This workwas supported by the Basic Science Research Program through the National Re-search Foundation of Korea (NRF) funded by the Ministry of Education, Sci-ence and Technology under Grant 2012R1A1A2008555.Y.-H. Kim and H.-N. Kim are with the Department of Electrical and Com-

puter Engineering, Pusan National, University, Busan 609-735, Korea (e-mail:[email protected]).N. V. Thakor is with the Department of Biomedical Engineering, Johns Hop-

kins University, Baltimore, MD 21218 USA.M. H. Schieber is with the Departments of Neurology, Neuro-Biology and

Anatomy, Brain and Cognitive Sciences and Physical Medicine and Rehabilita-tion, University of Rochester, Rochester, NY 14642 USA.Color versions of one or more of the figures in this paper are available online

at http://ieeexplore.ieee.org.Digital Object Identifier 10.1109/TNSRE.2014.2363193

which interprets the recorded spike signals from neural popula-tion and infers an intended movement. Up to now, many relatedstudies have been conducted, such as a 2-D target tracking task,a closed-loop control of a computer cursor, and a 3-D reachingtask [2]–[6]. Currently, to broaden the practical application, dex-terous motion control strategies have been proposed, such ashand and finger movements [7]–[16].To achieve neural decoding for finger movements, activity

of multiple neurons must be simultaneously decoded. However,all recorded neurons do not contribute equally to the all move-ments and several neurons are related weakly or not at all tothe specific movements. Hence, simply using as many neuronsas possible does not guarantee a higher decoding accuracy, andeven may degrade the decoding performance with the extremelysmall training data sets [17]. In addition, a large number of inputneurons put a computational burden on finding an optimal so-lution especially when the goal is to implement such decodingalgorithms in a prosthetic hardware. Hence, in the neural de-coding of arm reaching/grasping tasks, researchers have devel-oped many techniques to evaluate neurons' relative importance[19]–[22]. Some techniques have utilized the filter parametersof a trained decoding filter to ascertain the importance of neu-rons because training a decoding filter assigns weights or coef-ficients to the corresponding neurons hinging on their relativecontribution.On the basis of this principle, sensitivity analysis and single

neuron correlation analysis through a linear Wiener filter wereproposed to quantitatively rate the importance of neurons inneural-to-motor mapping [19]. Individual removal error (IRE),which is the change in decoding error following removal ofa given neuron, was presented to initially tune a populationvector-based system for neural control without arm movements[20]. In another approach, ensemble fractional sensitivity wasproposed by calculating the partial derivative of the outputwith respect to the input activity [21]. On the other hand,several approaches have attempted to evaluate the importanceof neurons by analyzing probabilistic distributions of recordedneuron spikes regardless of a decoding model. For a 2-Dtarget-reaching task to move the cursor on a computer screen, ameasure called tuning depth, defined as the difference betweenthe maximum and minimum values of the cellular tuning,was proposed [19]. The mutual information between the spikeand the delayed linear filter kinematics vector was appliedas a metric for evaluating instantaneously neural receptiveproperties [22].

1534-4320 © 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

KIM et al.: NEURON SELECTION BASED ON DEFLECTION COEFFICIENT MAXIMIZATION 375

These previous works were based on the decoding of contin-uous reaching movement, for which each motor cortical neuronwas found to be tuned with respect to the specific directionof reaching movement, and thus a “preferred direction” couldbe assigned for each neuron [23]. However, it was shown thatsingle motor cortical neurons do not fire specifically only fora particular flexion-extension finger movement but instead areactive with movements of different fingers [24]. Although theprevious work using the population vector analysis found that75% ofmotor cortical neurons related to finger movements weretuned to specific directions in a 3-D instructed hand movementspace and population vectors could be utilized as a predictorof instructed finger movement [24], the decoding method basedon the population vector yielded only 50%–60% decoding accu-racy [16]–[18]. Thus, several classification schemes have beenpresented for the decoding of discrete flexion-extension fingermovements based on the Skellam-based maximum likelihoodestimation [17] and the artificial neural network [18]. Theseworks have focused on the discrete decoding of finger move-ments and achieved the improved decoding accuracies. In thesame context, a neuron selection method considering the char-acteristics of neural population activity for discrete finger move-ments is required.In the sense of Bayesian inference, for the observed feature

parameter of a neuron we can make a decision minimizingthe probability of error, , based on the a priori probability,

, and the conditional probabilities, for(where and denote the movement types and the

total number of . Furthermore, by the feature space analysisin Fig. 1, the probability of error for each particular neuron canbe evaluated by analyzing the conditional PDFs. Intuitivelyeach PDF of an appropriate input neuron should be separatedto minimize the probability of error by reducing the overlappedregions between PDFs. The tuning depth provides a way to finda neuron highly modulated for a specific movement direction bymeasuring the difference between the maximum and minimumvalues of the tuning curve, which conveys the expected values ofeach PDF [19]. For a discrete decoding framework, the methodof relative importance in [26] gives a straightforward way tocheck the separation of the conditional PDFs by calculating onlythe variance of their means. However, since in the training phasethe conditional PDFs can be modeled by the observed neuronalspike data, we may try to utilize more information by analyzingthe characteristics of PDFs as well as the variance/min-max oftheir means.In this respect, we model the underlying PDFs for the neural

activity of each neuron and ascertain their Bayes risk. The Bayesrisk, a generalized version of the probability of error, is the costassigned to the all types of decision errors. Thus, neuron selec-tion in the sense of the minimum Bayes riskmay provide an op-timal way to select the appropriate neurons for neural decoding.However, to calculate the Bayes risk the exact analytic expres-sion of each PDF is required. Also, it entails a large computa-tional burden to compute the probability of all types of errors;with the increase in the number of finger movements , thetotal number of error types becomes the square of .Therefore, we need an alternative to cope with the neuron se-lection problem in neural decoding of the finger movements.

Fig. 1. Conceptual procedure of a neuron selection method consists of foursteps. Firstly, a feature parameter is extracted from the observed data setsand its probability density functions (PDFs) for each neuron are modeled in atraining phase. By analyzing the characteristics of these conditional PDFs of aneuron, called feature space analysis, the importance of a neuron can be evalu-ated, and then input neurons are selected for decoding of finger movements.

Hence, we propose a neuron selection method based on thedeflection coefficient in a binary hypothesis testing problemsuch as detecting the desired signal component (e.g., whetherit exists or not) [27]. The deflection coefficient is defined as

, or the ratio of the difference ofmeans and the variance of the PDFs when the two distributionsare independent and identically distributed ( ) with thesame variance [24]. Since the deflection coefficient isdirectly related to the area of overlapped region between twoPDFs, it totally characterizes the decision performance in abinary hypothesis testing [27]. Also, the deflection coefficientcan be obtained by only calculating the mean and variance fromobserved data set without modeling the exact PDFs. Therefore,to decode types of finger movements we derive a deflectioncoefficient for the -ary hypothesis testing problem and thenuse it to ascertain the importance of each neuron for the neuraldecoding.The remainder of the paper is organized as follows. In

Section II, the experimental setup and the neuron selectionmethods, including the Bayes risk minimization and the deflec-tion coefficient maximization, are described. Section III showsthe decoding results of off-line tests, and Section IV presentsthe conclusion.

II. CRITERION FOR NEURON SELECTION

A. Experimental Setup

Three male rhesus (Macaca mulatta) monkeys—C, G, andK—were trained to perform visually cued movements of the


individuated fingers, wrist, and combined, meaning a pair offingers at a time. The monkey placed its right hand in a pistol-grip manipulandum which separated each finger into a differentslot. A ventral or dorsal micro-switch at the end of each slotwas closed by flexing or extending each digit a few millimeters.In addition, the pistol grip manipulandum was mounted on anaxis allowing flexion and extension of the wrist. The monkeyperceived a display on which each digit was represented by arow of five light emitting diodes (LEDs). If the monkey closedthe instructed switch within the response time, 700 ms, and heldit closed for a hold time of 500 ms, then the trial was considereda success. When a micro-switch was closed, the LEDs providedthe feedback to the subject. There were 12 distinct individuatedmovements: flexion and extension of each finger (

through ) and the wrist and six combinedtwo-finger movements: .The monkeys were prepared for single-unit recording by sur-

gically implanting both a head-holding device and a rectangularLucite recording chamber that permitted access to the area en-compassing primary motor cortex (M1) [6]. Single-unit activi-ties were recorded from task-related M1 neurons. Independenttrials were repeated six times for each movement, respectively.A detailed description of the methods used to train the mon-keys and the actual experimental protocols can be found in [6]and [7].

B. Mathematical Model of the Neural Activity

To infer the instructed stimulus from recorded data, it isneeded to appropriately quantify the neural activity and modelthe underlying probability distribution of a neuron. The neuralactivity of the instructed finger movement consists of the twostates which are the period of rest and movement. Therefore,the change in firing rate before and after finger movements waspresented to quantify the neural activity [17]. Since all responsetimes between the time of cue onset and switch closure aredifferent, neuronal spike trains were aligned at the time of theswitch closure (the center of each trial data) as shown in Fig. 2.From the aligned spike trains, the neural activation was definedas the difference of firing rates between before and after the in-structed finger movement [17]. Each firing rate was calculatedby counting and averaging the number of spikes in the dividedtwo windows (from 0 to 800 ms, from 800 to 1100 ms) basedon the time of the switch closure as depicted in Fig. 2.Since each neuronal spike train can be assumed as a real-

ization of an underlying random variable, we termedto represent a random variable of firing rate of a neuron fora given movement type . In particular, denotes thebaseline activity of the neuron before the movement of .Then, to measure how much a neuron, is activated by the in-structed finger movement , the neural activation is defined asfollows [17]:

(1)

where is the number of total finger movement types. ThePDF of is represented as , and the mean of

Fig. 2. Neural activity induced by a movement is defined as the difference ofan averaged firing rate before and after the instructedfinger movement. All trials were aligned such that switch closure occurred at 1s. The baseline activity was obtained by averaging the number of spikes from0 and 800 ms. The neural activation after finger movements was obtained bycalculating the firing rate of spikes between 800 and 1100 ms.

can be estimated by the sample mean of the trial data,that is

(2)

where denotes the th trial data of the th neuron bythe instructed finger movement . is the number of totaltrials.

C. Bayes Risk Minimization

In the -ary hypothesis testing, we wish to distinguish be-tween hypotheses, where is larger than 2. Each hypothesisin neural decoding means a decision which finger movementevokes the observed neuronal firing spike among the flexion andextension of each finger and the wrist. If the conditional PDFs,

, for each finger movement are completely known, thediscrimination problem is relatively simple comparing to thecase of the PDFs with unknown parameters. The primary ap-proach to simple hypothesis testing is the Bayesian approachbased on the minimum (probability of error) criterion [28].A probability of error is given by

(3)

where is the conditional probability that indicates theprobability of deciding when is true. can be general-ized to the Bayes risk by assigning costs to each type of error[28]. The expected cost or the Bayes risk is defined by

(4)


where is the cost when we decide but is true. Usually,if no error is made ( is equal to ), a cost is zero, so that

. Otherwise, all costs, , are equal to one. So, theBayes risk is the same with the probability of error . However,for ease of explanation we will retain the more general form.In the neural decoding problem we assume that the cost of

each finger movement is identical and a priori probability ofany hypothesis is uniform, and thus the Bayes risk of the thneuron can be modified as

(5)

where denotes the hypothesis that the decision of neuraldecoding is determined by th finger movement . Since theBayes risk of a particular neuron indicates the proba-bility of error, of that neuron, it can be utilized to ascertainthe importance of each neuron and select the input neuron setfor the neural decoding.Although, in a practical neural decoding problem, it is hard

to obtain the complete knowledge about the probabilistic struc-ture between the feature parameter and the instructed move-ment, we can model the conditional PDFs of a neuron from theobserved spike trains in the training phase. However, since thetotal number of the decision error types is equal tofor the types of finger movements, a high computationalburden is entailed to calculate the summation of each area ofthe overlapped regions. To cope with this problem, an alterna-tive is needed to quantify and ascertain the importance of eachneuron with an acceptable computational burden.

D. Deflection Coefficient Maximization

Fig. 3 shows the relationship between each type of decisionerror and the overlapped region of the conditional PDFs in thecase of a neural decoding with only three movements. As shownin Fig. 3, the total area of overlapped regions is directly relatedto . So, a neuron whose conditional PDFs tend to be sepa-rated to each other may be an appropriate input for the neuraldecoding procedure. In this respect, to evaluate the importanceof a neuron without calculating the Bayes risk, we propose aneuron selection method based on the deflection coefficient [27].In the binary hypothesis testing problem such as signal

detection determining the presence or absence of a desiredsignal component, the deflection coefficient is a measure forevaluating the detection performance [27]. The two condi-tional PDFs, and , for two hypotheses(absence) and (presence) are assumed to be independentand identically distributed (i.i.d.) with the same variance,

. Then, the deflection coefficient, ,is defined by [27], [28]

(6)

Fig. 3. Relationship between four types of decision errors and the overlappedregion of the conditional PDFs in the case of a neural decoding with only threemovements ( , and ).

Since in (6) means the ratio between the difference of meansand their variance, it has a large value when each PDF is sep-arated with narrow width and thereby the area of overlappedregions decreases. In order to apply the deflection coefficient tothe neuron selection problem, we define the modified deflectioncoefficient, , in the -ary hypothesis testing for the neuraldecoding of finger movements, which is given by

(7)

where of a particular neuron is the ratio of and , andis the total number of finger movements. The variance of

each mean of PDFs, , can be estimated by

(8)

where the mean of the neural activities over the finger move-ments is given by

(9)

Since the variances of each PDF are generally different in theneural decoding, the mean of variances is used in the denom-inator in (7) unlike in that of (6). The estimate of can becalculated from the trial data as follows:

(10)

where is the sample variance of the th trialover trials as an estimate of the variance of and is ob-tained by

(11)

Since the overlapped region of PDFs is affected by not onlythe separation but also the width of each PDF, the deflectioncoefficient is a more appropriate criterion to ascertain the


importance of a neuron compared to the method using onlyin [26].In terms of the computational load, the neuron selection

based on the deflection coefficient maximization is more ef-fective than the Bayes risk minimization. To derive the Bayesrisk of a neuron, the exact analytic expression of each PDFneeds to be modeled, and then the area of over-lapped regions corresponding to each type of decision errorscalculated. This computation, the integral of each ,should be estimated by the summation of the sampled PDFs,thus the computational burden also relates with the numberof sample points . The total number of multiplications is

to derive the Bayes risk of a neuron. On theother hand, the deflection coefficient requires multiplica-tion to calculate and from the observed data. Hence, thedeflection coefficient can be a good alternative of the Bayesrisk to quantify the importance of neurons for the statisticalinference of finger movements.

E. Individual Removal Error

For comparison purposes, we briefly describe the conven-tional individual removal error (IRE) method. The IRE is de-fined as the change in error angle following removal of a givenneuron from the ensemble [20] and is given by

(12)

where is the average angular error which is the angle be-tween the population vector and a line connecting the cursorto the active target using all available units. is the av-erage angular error computed for all available units except theneuron indexed by “ ”. Originally, even though the IRE is pre-sented for rapid tuning of a population vector-based system forneural control without arm movement [20], it can be employedfor various decoding models with an appropriate modificationbecause its basic framework is based on the change of errormeasurement with and without a particular neuron. However,when the applied classifier achieves a high performance with asmall number of input neurons, the decoding accuracy or errormeasurements cannot be changed by removing any particularsingle neuron. In this case, the IREs of all neurons are zero, soan alternative way should be considered to quantify the IRE ofa neuron.

F. Maximum Likelihood Neural Decoding With the SelectedNeurons

In order to compare the effectiveness of the two criteria ondecoding a finger movement, we perform offline analysis withthe neurons selected by the proposed methods. Several differentmethods have been presented for decoding finger movementsfrom neural activity [6]–[10], [16]–[18]. Here we use the max-imum likelihood (ML) decoding method in [17] which is basedon Skellam distribution. In the statistical inference, the ML esti-mation is asymptotically optimal as the number of training dataincreases [27]–[29].

ML decoding estimates the instructed finger movement bychoosing the argument that maximizes the log-likelihood,that is

(13)

where is the number of input neurons used for ML decoding.To solve (13) the probability density function of the neuronactivity needs to be modeled. The Skellam distribution is pre-sented in [17] and it showed the decoding accuracy of 99% with25 randomly selected neurons.The PDF of the Skellam distribution is as follows:

(14)

where and are themeans of random vari-ables describing a firing rate of before and after the starting mo-ment, respectively. is the time interval for observation ofneuronal spikes and is the modified Bessel function ofthe first kind. is defined by

(15)

Assuming that and are independent [8], we get

(16)

for the Skellam distribution.

III. RESULTS AND DISCUSSION

The performance of the proposed neuron selection methodswas evaluated using leave-one-out cross validation, wherein thedata set was partitioned into six folds because the total numberof trials is six for each finger movement. From six independenttrials’ data, a single trial was used for the validation and the re-maining trials were used for the training data, i.e., . Thisprocedure was repeated such that each trial in the data set is usedonce as the validation data, so six independent tests were con-ducted to examine the performance of the proposed methods.The offline tests without ordering were performed with ran-

domly chosen neurons from total 115 neurons (312 and 125neurons for the monkey C and G, respectively). The randomselection was repeated 300 times. On the other hand, for therank-ordered neurons, if neurons are used for ML decoding,there is only one input set, consisting of the first-ranked neuronto the th ranked one. In order to increase the possible neuronsets, we added next-ranked neurons to highly ranked neu-rons, resulting in ranked neurons. The value of wasempirically set to in our testing, then a subset of neu-rons was randomly selected from neurons. So, the pos-sible number of selections becomes .


Fig. 4. Performance of the neural decoding of the individuated finger movements are plotted according to the different monkeys (C, G, and K) and the classificationalgorithms (the ML classifier and the optimal population vector method). First three subplots show the decoding results of the ML decoding for the three monkeyssuch as: (a) C, (b) G, and (c) K. Performance of the optimal population vector decoding for the monkey C is shown in (d) and the number of input neurons isexpanded to 80 populations.

A. Neural Decoding of Individual Finger Movements WithRank-Ordered Neurons

Two classification algorithms, the optimal PV and MLdecoding methods [16], [17], were applied to validate theeffectiveness of proposed methods and to compare their per-formances with those of the existing neuron selection methodssuch as the tuning depth [19] and the IRE [20]. First, wecarried out the ML-based neural decoding, which is the mostaccurate classifier for decoding discrete finger movements [17]of the individuated finger movements (12 flexion/extensionmovements listed in Section II-A) by using the neuronal spikesrecorded from the three monkeys (C, G, and K). Fig. 4(a), (b),and (c) shows the resultant decoding accuracies according tothe number of input neurons. For offline testing, we appliedfour different neuron selection methods to determine inputneuron sets such as the random selection [17], the tuning depth[19], and the individual removal error (IRE) [20], Bayes riskminimization, and deflection coefficient maximization. Sincethe ML decoding achieved the accuracy of 100% regardless ofwhich single input neuron is removed, the IRE of all neuronswere zero for the ML decoder. So, to derive the IRE of a neuronfor the decoding of discrete finger movements, we alternatively

employed an abstract movement space in [24] and it allowedthat each discrete finger movement could be matched to aparticular vector in 3-D space. Then, the IREs of each neuronwere calculated by the two neuronal activity models whichwere based on the regular population vector (PV) [24] and theoptimal population vector (OPV) [16]. In the regular PV, theweighting vectors are set a priori to the preferred directionswhile those of OPV are determined in the sense of minimizingthe average squared distance between the estimated-PV and theinstructed movement vector. In the decoding of discrete fingermovements, the OPV achieved a remarkably improved accu-racy compared to that of the PV [16]. Note that though the IREwas quantified by using the population vector analysis (OPV-or PV-based IRE), its decoding accuracies were evaluated bythe ML decoding in Fig. 4(a), (b), and (c).Three monkey subjects showed different performances ac-

cording to their inherent biological characteristics and achieve-ment of training, hence the effects of the proposed neuron selec-tion methods on decoding accuracy differ across three monkeysubjects. When we used the randomly selected 15 neurons, eachdecoding result differed: 89%, 83%, and 92% for the monkeysC, G, and K, respectively. With as few as 15 neurons selectedby the proposed two metrics, the decoding accuracies of greater


than 98% were achieved in the cases of all monkey subjects.To statistically analyze the performance differences between theproposed methods and the random selection, a statistical t-testwas performed on the null hypothesis at the significant level of0.01 and the number of test data set was 300. Under the null hy-pothesis, for the three monkeys with 15 neurons, all the -valueswere extremely small (almost zero, ) and less than thepredetermined significant level . These results indi-cate that the performances of the proposed methods are statisti-cally different from those of the random selection.In the case of the tuning depth, the accuracy of the monkey G

stayed around 92% while the performances of other two mon-keys, C and K, were enhanced over 96% by using 15 rank-ordered neurons. The tuning depth did not work well for themonkey G because a larger tuning depth does not guaranteewide separation between each conditional PDF in decoding dis-crete flexion/extension tasks of fingers. In both cases of the PVand OPV, the IRE method showed higher accuracies comparedto those of randomly selected neurons for the three subjects, butthe OPV-based IREmethod yielded more accurate results for allthe three monkeys. In particular, the OPV-based IRE methodyielded comparable accuracies to those of the tuning depth inthe cases of monkeys C and G, respectively, while the tuningdepth showed the more accurate performance for the monkeyK as shown in Fig. 4. These results imply that the impact ofthe IRE method would vary with the applied decoding modeland its effectiveness could be enhanced by using a more precisemodel instead of the PV or OPV analysis. On the other hand,the proposed methods achieved the best performance across allsubjects by only analyzing the recorded test data set without anyadditional consideration of a decoding model.Although we used an alternative way to utilize the IRE cri-

terion for the ML decoding framework, basically the IRE valueof a neuron is required to be quantified and evaluated under thesame decoding algorithm. So, additional off-line tests have beenconducted to compare the decoding accuracies of each neuronselection method by employing the OPV-based decoding algo-rithm instead of the ML classifier. These tests allowed a fair andobjective evaluation of performance difference between the pro-posed methods and the IRE criterion. As shown in Fig. 4(d), theoverall decoding performances were degraded due to the rel-atively low performance of the OPV-based decoding methodcompared to that of the ML decoder, but the proposed methodsstill achieved the highest accuracies.In Fig. 5, the conditional PDFs of first-ranked neurons by

the three different metrics (the tuning depth, Bayes risk, anddeflection coefficient) are presented for the selected sevenmove-ments (f1, f2, f5, e1, e3, e4, and ew) from 12 individual fingermovements ofmonkeyG. The neuronsG0283nwere ranked firstby the tuning depth and conditional PDFs are shown in Fig. 5(a).Those of two first-ranked neurons (G0242n and G0240n) bythe Bayes risk and the deflection coefficient are plotted inFig. 5(b) and (c), respectively. In Fig. 5(a), the conditional PDFsof neuron G0283n tend to be overlapped around neural activity

although it has the largest difference between themaximumandminimumvalues in neural activation.On the otherhand, in the cases of first-ranked neurons by the Bayes risk andthe deflection coefficient, G0242n andG0240n, their conditional

Fig. 5. Neurons of G0283n, G0283n, and G0283n are ranked first by three dif-ferent neuron selection methods for the monkey G. Their probability densityfunctions of the selected seven different finger movements are plotted (f1, f2, f5,e1, e3, e4, and ew). The 12 finger movements consist of flexion and exten-sion of each finger ( through ) and the wrist :(a) First-ranked neurons by the tuning depth, (b) Bayes risk, and (c) Deflectioncoefficient.

PDFs tend to be separated. This means that the proposed twometrics could be appropriate to quantify and analyze the distri-bution of conditional PDFs to evaluate the importance of eachneuron for decoding discrete finger movements.For a more robust BMI system the neural decoding should

yield a desired performance regardless of the condition of sub-jects. For this reason, we compared the decoding performanceacross the different neuron selection algorithms according to themonkey subjects in Table I, in which the results were reported


TABLE ISUMMARY OF DECODING RESULTS FOR INDIVIDUAL FINGER MOVEMENTS

TABLE IISUMMARY OF DECODING RESULTS FOR COMBINED FINGER MOVEMENTS

TABLE IIISUMMARY OF DECODING RESULTS FOR TOTAL FINGER MOVEMENTS

as accuracy and standard deviation. These results show that theproposed methods could achieve consistently the higher accura-cies for all monkey subjects over the OPV-based IRE and tuningdepth. Especially, even when the subject achieves relatively lowdecoding accuracy with randomly selected neurons (monkeyG), the level of performance improvement was higher than thoseof monkeys C and G by employing the proposed methods. Onthe other hand, the decoding accuracies of the tuning depth weredegraded in the case of monkey Gwhile those of monkeys C andK are relatively high, and this tendency of the performance vari-ation did not change even when the number of input increases to15 neurons. In other words, the proposed methods worked wellacross all subjects compared to the tuning depth and the IREand will be helpful in realistic environment wherein a reliableperformance should be achieved regardless of the condition ofsubjects.With only five neurons, the accuracy of 83%, 75%, and 83%

was achieved for the monkeys C, G, and K when the input neu-rons were selected based on the Bayes risk (82%, 72%, and83% by the deflection coefficient), and these decoding accura-

cies were approximately 9%, 13%, and 2% better than those ofthe tuning depth, respectively (7%, 10%, and 2% for the de-flection coefficient). In the cases of monkeys C and G, -valueswere extremely small (almost zero, ). However, forthe well-performed subject (monkey K), -values of the t-testwere larger than the predetermined significance level (0.01) inboth cases of Bayes risk and the deflection coef-ficient . These results mean that the performanceof the tuning depth and the proposed methods are statisticallysimilar in the case of monkey K. Compared to the OPV-basedIRE method, the decoding accuracy of Bayes risk was approxi-mately 10%, 7%, and 9%, respectively (8%, 4%, and 9% for thedeflection coefficient). In addition, for the three monkeys, all the-values were almost zero , so these results indicatethat the performance difference between the proposed methodsand the OPV-based IRE method are statistically significant.In these results, the decoding accuracy by using the input neu-

rons selected by the proposed two methods achieved the com-parable decoding accuracy. Therefore, since the deflection co-efficient is derived by only calculating the mean and variance


Fig. 6. Performance of the ML decoding for finger movements of the monkeyK: (a) Decoding accuracy of the combined two-finger movements and (b) de-coding accuracy of total finger movements including 12 individuated and sixcombined two-finger movements.

from the neuronal spike data, it can be an appropriate criterionfor the neural decoding problem when the goal is to implementsuch decoding algorithms in a prosthetic hardware.

B. Neural Decoding of Complex Two-Finger and Total FingerMovements With Rank-Ordered Neurons

Furthermore, we performed the neural decoding of othertypes of finger movements: the combined two-finger move-ments and the total finger movements. The decoding resultswere plotted in Fig. 6 and the averaged accuracies were sum-marized in Tables II and III, respectively. These neuronalspike data were recorded from the monkey K which showedthe highest accuracies in decoding the individuated fingermovements. The combined two-finger movement means thatthe monkey moves its two fingers at the same time, and thetotal finger movements includes the individuated and combinedtwo-finger movements. In the case of the combined two-fingermovements, the neurons selected by the proposed methodsachieved almost 96% accuracies with five neurons as depictedin Fig. 6(a), while the random selection and the tuning depth

Fig. 7. Confusionmatrices show the decoding results of total fingermovementsincluding 12 individuated and 6 combined two-finger movements when therank-ordered 20 neurons are selected among total 115 neurons of the MonkeyK. (a) Tuning depth and (b) the deflection coefficient.

resulted in the accuracies of 71% and 82%, respectively. Also,the corresponding -values were almost zero . Onthe other hand, the total finger movement including the indi-viduated and the combined-two finger movements

showed the degraded accuracy for all the selectionmethods because of the increase in uncertainty and ambiguitybetween the related tasks such as and [26]. In spite of thisproblem, Fig. 6(b) shows the enhanced accuracy of total fingermovements when we used the neurons selected by the proposedmethods. With 20 neurons selected by the proposed methods,average 96% accuracies of neural decoding were achieved,respectively, while the random selection and the tuning depthresulted in the accuracy of 89% and 92%, respectively. In bothcases, the -values were less than the significant


level , so the null hypothesis was rejected. These re-sults showed that the performance enhancement by employingthe proposed methods is statistically meaningful comparedto the random selection and the tuning depth in the case ofdecoding the total finger movements.To check all the types of decision errors, the confusion ma-

trices are presented in Fig. 7. Twenty input neuronswere selectedby the tuning depth and the deflection coefficient. In Fig. 7(a) and(b), most of the decision errors occurred between the individualfinger movements and the combined two-finger movements re-gardless of the neuron selectionmethod. In other words, a partic-ular single-finger movement tends to be classified into the two-finger movement which includes itself and vice versa. Althoughthese false decisions can be reduced by employing the proposedneuron selection method as shown in Fig. 7, it is a challengingproblem to implement realistic hand prosthetics since real handmotions consist of more dexterous movements such as flexion/extension of multiple fingers, simultaneously.As a final discussion, the relationship between the decoding

performance and the number of input neurons needs to be ad-dressed. The decoding results presented in our work tend to in-crease monotonically according to the number of input neurondata. This might be because the Skellam conditional PDFs couldbe estimated appropriatelywith five training sets as suggested by[17]. Referring to (13) and (16), the decoding performance wasevaluated based on the ML estimation, wherein the joint proba-bilities in (16) are derived by multiplying conditional PDFs of aneuron. If a nonstationary th input neuron is extremely domi-nant enough to cause a change in the existing joint probabilitiesdetermined by the previous neurons, the inclusion of thisneuron can result in the degradation of decoding accuracy. Theprevious study onML decoding of the fingermovements alreadyshowed that the additional inclusion of new input neurons doesnot guarantee the higher accuracy when the number of trainingdata sets is extremely small [17]. Also, in the neural decodingbased on a nonlinear decoder such as the artificial neural net-work (ANN), the decoding accuracies were not monotonicallyincreased according to the number of input neurons [18]. There-fore, although the decoding results presented in our work tend toincrease monotonically according to the number of input neurondata, the problemof performance degradation caused by the non-stationary neuron should be carefully considered. In this sense,the rejection of these undesired neurons as well as the selectionof highly contributing neurons can be helpful for improving theperformance of neural decoding. The related issuewas also dealtwith in [20], where the authors presented the change in decodingerror following the removal of a given neuron.

IV. CONCLUSIONFor the neural decoding of dexterous finger movements,

we proposed new metrics for the neuron selection: the Bayesrisk minimization and the deflection coefficient maximization.The proposed criteria, the deflection coefficient and the Bayesrisk, have as a trade-off a slight degradation in performance forreduced computational complexity. When the neurons were se-lectedby theBayes riskminimizationcriterion, themoreaccuratedecoding performance was achieved. However, this methodsuffers from the computational load because it needs to calculatethe probability of all decision errors based on the exact analyticexpressionof theprobabilistic distribution.On theotherhand, the

deflection coefficientminimization can be an appropriate alterna-tive because it shows the comparable performance to the Bayesriskmaximizationwithmuch smaller computational complexity.The use of the proposed metrics improved the decoding accu-

racy comparedwith the randomselection, the individual removalerror,andthe tuningdepth.Also, thedecodingperformanceswererobust across the different subjects (the monkeys C, G, and K),so the proposed methods will be helpful in achieving a desirableaccuracy by compensating the subject variability. In particular,the decoding accuracywasmore improved in the decoding of thetotal finger movements compared to the case of individual fingermovementof themonkeyK.Since the real handmotionsare com-posed of the individuated and combined finger movements, thisperformance enhancement will be of particular importance todevelop a BMI device for multi-fingered hand control.The results of these experiments and offline tests suggest that,

in a closed-loop system, an appropriate neuron selection for theinitial tuning can be expected to alleviate the computationalburden and raise the robustness across a variety of differentsubjects. Also, when a BMI system requires retraining becauseof unexpected and undesirable changing factors, such as move-ments of electrode arrays and noise induced by degradation atthe electrode tissue interface, the proposed neuron selectionmethods will be more helpful and effective. However, sinceour experimental model and study focus on decoding discretefingermovements when the task-related time schedule is alreadyknown on the basis of open loop and offline analysis framework,considerable works, such as a continuous decoding of varyingdegrees of flexion/extension of multiple fingers and continuousstate decoding of dexterous finger and hand motion, remain toeventually design and implement a real-time dexterous pros-thetic hand system operating in closed loop environments.

ACKNOWLEDGMENT

The authors would like to thank the anonymous reviewers fortheir valuable and constructive comments to improve the paper.

REFERENCES[1] P. R. Kennedy, R. A. Bakay, M. M. Moore, K. Adams, and J. Gold-

waithe, “Direct control of a computer from the human central nervoussystem,” IEEE Trans. Rehabil. Eng., vol. 8, no. 2, pp. 198–202, Apr.2000.

[2] J. Wessberg, C. R. Stambaugh, J. D. Kralik, P. D. Beck, M. Laubach,J. K. Cahpin, J. Kim, S. James Biggs, M. A. Srinivasan, and M. A.L. Nicolelis, “Real-time prediction of hand trajectory by ensembles ofcortical neurons in primates,” Nature, vol. 408, no. 6810, pp. 361–365,2000.

[3] D. W. Moran and A. B. Schwartz, “Motor cortical activity duringdrawing movements: Population representation during spiral tracing,”J. Neurophysiol., vol. 82, no. 5, pp. 2693–2704, Nov. 1999.

[4] D. M. Taylor, S. I. H. Tillery, and A. B. Schwartz, “Direct corticalcontrol of 3D neuroprosthetic devices,” Science, vol. 296, no. 5574,pp. 1829–1832, Jun. 2002.

[5] M. D. Serruya, N. G. Hatsopoulos, L. Paninski, M. R. Fellows, and J.P. Donoghue, “Instant neural control of a movement signal,” Nature,vol. 416, no. 6877, pp. 141–142, Mar. 2002.

[6] M. H. Shieber and L. S. Hibbard, “How somatotropic is the motorcortex hand area?,” Science, vol. 261, no. 5120, pp. 489–492, Jul. 1993.

[7] A. V. Poliakov and M. H. Schieber, “Limited functional grouping ofneurons in the motor cortex hand area during individuated finger move-ments: A cluster analysis,” J. Neurophysiol., vol. 82, pp. 3488–3505,Dec. 1999.

[8] M. H. Schieber, “Individuated finger movements of rhesus monkeys: Ameans of quantifying the independence of the digits,” J. Neurophysiol.,vol. 65, pp. 1381–1391, Jun. 1991.

[9] M. H. Schieber, “Motor cortex and the distributed anatomy of fingermovements,” Adv. Exp. Med. Biol., vol. 508, pp. 411–416, 2002.


[10] W. Wu, J. E. Kulkarni, N. G. Hatsopoulos, and L. Paninski, “Neuraldecoding of hand motion using a linear state-space model with hiddenstates,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 17, no. 4, pp.370–378, Aug. 2009.

[11] J. Zhuang, W. Truccolo, C. Vargas-Irwin, and J. P. Donoghue, “De-coding 3-D reach and grasp kinematics from high-frequency local fieldpotentials in primate motor cortex,” IEEE Trans. Biomed. Eng., vol. 57,no. 7, pp. 1774–1784, Jul. 2010.

[12] W. Wu and N. G. Hatsopoulos, “Real-time decoding of nonstationaryneural activity inmotor cotex,” IEEE Trans. Neural Syst. Rehabil. Eng.,vol. 16, no. 3, pp. 213–222, Jun. 2008.

[13] C. Kemere, K. V. Shenoy, and T. H. Meng, “Model-based neural de-coding of reaching movements: Maximum likelihood approach,” IEEETrans. Biomed. Eng., vol. 51, no. 6, pp. 925–932, Jun. 2004.

[14] J. M. Carmena, M. A. Lebedev, R. E. Crist, J. E. O'Doherty, D. M.Santucci, D. F. Dimitrov, P. G. Patil, C. S. Henriquez, and M. A.Nicolelis, “Learning to control a brain-machine interface for reachingand grasping by primates,” PLoS Biol., vol. 1, 2003, article E42.

[15] A. P. Georgopopoulous, J. Kalaska, R. Caminiti, and J. Massey, “Onthe relations between the direction of two-dimensional armmovementsand cell discharge in primate motor cortex,” J. Neurosci., vol. 2, no. 11,pp. 1527–1537, 1982.

[16] S. B. Hamed, M. H. Schieber, and A. Pouget, “Decoding M1 neuronsduring multiple finger movements,” J. Neurosci., vol. 98, no. 1, pp.327–333, 2007.

[17] H.-C. Shin, V. Aggarwal, S. Acharya, M. H. Schieber, and N. V.Thakor, “Neural decoding of finger movements using Skellam basedmaximum likelihood decoding,” IEEE Trans. Biomed. Eng., vol. 57,no. 3, pp. 754–760, Mar. 2010.

[18] V. Aggarwal, S. Acharya, F. Tenore, H.-C. Shin, R. Etienne-Cum-mings, M. H. Schieber, and N. V. Thakor, “Asynchronous decoding ofdexterous finger movements using M1 neurons,” IEEE Trans. NeuralSyst. Rehabil. Eng., vol. 16, no. 1, pp. 3–14, Feb. 2008.

[19] J. C. Sanchez, J. M. Carmerna, M. A. Lebedev, M. A. Nicolelis, J. G.Harris, and J. C. Principe, “Ascertaining the importance of neurons todevelop better brain-machine interfaces,” IEEE Trans. Biomed. Eng.,vol. 51, no. 6, pp. 943–953, Jun. 2004.

[20] R. Wahnoun, J. He, and S. I. Helms Tillery, “Selection and parameteri-zation of cortical neurons for neuroprosthetic control,” J. Neural Eng.,vol. 3, pp. 162–171, Jun. 2006.

[21] G. Singhal, V. Aggarwal, S. Acharya, J. Aguayo, J. He, and N. V.Thakor, “Ensemble fractional sensitivity: A quantitative approach toneuron selection for decoding motor tasks,” Comput. Intell. Neurosci.,Feb. 2010.

[22] Y. Wang, J. C. Principe, and J. C. Sanchez, “Ascertaining neuron im-portance by information theoretical analysis in motor brain-machineinterfaces,” Neural Netw., vol. 22, no. 5, pp. 781–790, Jul. 2009.

[23] A. P. Georgopoulos, A. B. Schwartz, and R. E. Kettner, “Neuronalpopulation coding of movement direction,” Science, vol. 233, pp.1416–1419, 1986.

[24] A. P. Georgopoulos, G. Pellizzer, A. V. Poliakov, and M. H. Shierber,“Neural decoding of finger and wrist movements,” J. Comput. Neu-rosci., vol. 6, pp. 279–288, 1999.

[25] D. H. Johnson, “Point process models of single-neuron discharges,” J.Comput. Neurosci., vol. 3, pp. 275–299, 1996.

[26] H.-N. Kim, Y.-H. Kim, H.-C. Shin, V. Aggarwal, M. H. Schieber, andN. V. Thakor, “Neuron selection by relative importance for neural de-coding of dexterous finger prosthesis control application,”Biomed. Sig.Process. Contr., vol. 7, no. 6, pp. 632–639, Nov. 2012.

[27] S. M. Kay, Fundamentals of Statistical Signal Processing: DetectionTheory. New York, NY, USA: Prentice-Hall, 1998.

[28] S. M. Kay, Fundamentals of Statistical Signal Processing: EstimationTheory. New York, NY, USA: Prentice Hall, 1993.

[29] R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification, 2nded. Hoboken, NJ, USA: Wiley, 2000.

Yong-Hee Kim received the B.S., M.S., and Ph.D.degrees in electrical and computer engineering fromPusan National University, Busan, Korea, in 2007,2009, and 2014, respectively.

He currently is a Postdoctoral Researcher at thesame university. His research interests are in the areasof neural signal processing, passive emitter localiza-tion, RADAR/SONAR signal processing, and esti-mation/detection theory.

Nitish V. Thakor (F’94) received the B.Tech. degreein electrical engineering from the Indian Institute ofTechnology, Bombay, India, in 1974, and the M.S.degree in biomedical engineering and Ph.D. degree inelectrical and computer engineering from the Univer-sity of Wisconsin, Madison, USA, in 1978 and 1981,respectively.He is a Professor of biomedical engineering,

electrical and computer engineering, and neurologyat Johns Hopkins University, Baltimore, MD, USA,and directs the Laboratory for Neuroengineering in

the same university. His technical expertise is in the areas of neural diagnosticinstrumentation, neural microsystem, neural signal processing, optical imagingof the nervous system, rehabilitation, neural control of prosthesis, and brainmachine interface. He is the Director of Neuroengineering Training programfunded by the National Institute of Health. He has published 225 referredjournal papers, generated 11 patents, cofunded four companies, and carriesout research funded mainly by the Nation Institutes of Health (NIH), NationalScience Foundation (NSF), and Defense Advanced Research Projects Agency.Dr. Thakor was the Editor-in-Chief of IEEE TRANSACTIONS ON NEURAL

SYSTEMS AND REHABILITATION ENGINEERING (2005–2011). He is the recip-ient of a Research Career Development Award from the NIH and a Presiden-tial Young Investigator Award from the NSF, and is a Fellow of the AmericanInstitute of Medical and Biological Engineering and Founding Fellow of theBiomedical Engineering Society, Technical Achievement Award from IEEE andDistinguished Alumnus award from Indian Institute of Technology and Univer-sity of Wisconsin.

Marc H. Schieber received the A.B. degree in 1974,and the M.D. and Ph.D. degrees in 1982, all fromWashington University, St. Louis, MO, USA.He currently is a Professor of neurology, neurobi-

ology, and anatomy in the University of Rochester,Rochester, NY, USA, and Attending Neurologiston the Rehabilitation and Neurology Unit at UnityHealth, Rochester. His research focuses on howthe nervous system controls muscles to performdexterous movements of the hand and finger.Dr. Schieber is a member of the Society for Neural

Control of Movement and the Society for Neuroscience. He has been a recipientof an NINDS Javits Investigator Merit Award and has served as Chair of theNational Institutes of Health Sensorimotor Integration Study Section.

Hyoung-Nam Kim (M’00) received the B.S., M.S.,and Ph.D. degrees in electronic and electrical engi-neering from PohangUniversity of Science and Tech-nology (POSTECH), Pohang, Korea, in 1993, 1995,and 2000, respectively.

From May 2000 to February 2003, he was withElectronics and Telecommunications Research Insti-tute (ETRI), Daejeon, Korea, developing advancedtransmission and reception technology for terrestrialdigital television. In 2003, he joined the faculty ofthe Department of Electronics and Electrical Engi-

neering, Pusan National University (PNU), Busan, Korea, where he is currentlya Professor. From February 2009 to February 2010, he was with the Departmentof Biomedical Engineering, Johns Hopkins University School of Medicine, asa Visiting Scholar. His research interests are in the area of digital signal pro-cessing, adaptive filtering, biomedical signal processing, and RADAR/SONARsignal processing, in particular, signal processing for brain-computer interface,digital multimedia broadcasting, and digital communications.Dr. Kim is a member of IEICE, IEEK, and KICS.

374 ...cspl.ee.pusan.ac.kr/sites/cspl/download/inter... · 374...

Documents