970 ieee transactions on systems, man, and...

970 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART C: APPLICATIONS AND REVIEWS, VOL. 42, NO. 6, NOVEMBER 2012

Classification of Upper Limb Motion TrajectoriesUsing Shape Features

Huiyu Zhou, Huosheng Hu, Senior Member, IEEE, Honghai Liu, Senior Member, IEEE,and Jinshan Tang, Senior Member, IEEE

Abstract—To understand and interpret human motion is a veryactive research area nowadays because of its importance in sportssciences, health care, and video surveillance. However, classifica-tion of human motion patterns is still a challenging topic becauseof the variations in kinetics and kinematics of human movements.In this paper, we present a novel algorithm for automatic classifi-cation of motion trajectories of human upper limbs. The proposedscheme starts from transforming 3-D positions and rotations ofthe shoulder/elbow/wrist joints into 2-D trajectories. Discrimina-tive features of these 2-D trajectories are, then, extracted using aprobabilistic shape-context method. Afterward, these features areclassified using a k-means clustering algorithm. Experimental re-sults demonstrate the superiority of the proposed method over thestate-of-the-art techniques.

Index Terms—Classification, expectation maximization, healthcare, motion trajectory, shape contexts (SCs).

I. INTRODUCTION

IN the past decade, ageing population in developed coun-tries have rapidly increased. It is necessary to provide peo-

ple immediate access to high-quality, affordable, and accessiblehealth-care services any place and anytime without increasingcosts. Solutions, such as telemedicine, can help reduce the du-ration of hospital stays and the need for frequent visits to ahospital. One of the active research areas in telemedicine is toprovide remote monitoring services, where human behaviorsare fully analyzed so that appropriate medical treatments can bedelivered.

To understand and interpret human motion, based on videosequences, is an active research area with popular applicationsin sports sciences, medicine, biomechanics, and surveillance(see, e.g., [13]). Human motion analysis is mainly driven by

Manuscript received March 19, 2011; revised August 6, 2011; acceptedOctober 23, 2011. Date of publication December 19, 2011; date of currentversion October 12, 2012. The authors would like to acknowledge the supportsfrom Grant No. EP/G041377/1, Engineering and Physical Science ResearchCouncil, U.K. This paper was recommended by Associate Editor L. Zhang.

H. Zhou is with the School of Electronics, Electrical Engineering and Com-puter Science, Queen’s University Belfast, Belfast BT7 1NN, U.K. (e-mail:[email protected]).

H. Hu is with the School of Computer Science and Electronic Engineer-ing, University of Essex, Wivenhoe Park, Colchester CO4 3SQ, U.K. (e-mail:[email protected]).

H. Liu is with the School of Creative Technologies, University of Portsmouth,Portsmouth, Hampshire PO1 2UP, U.K. (e-mail: [email protected]).

J. Tang is with the Department of Computer Network and System Adminis-tration, School of Technology, Michigan Technological University, Houghton,MI 49931-1295 USA (e-mail: [email protected]).

Color versions of one or more of the figures in this paper are available onlineat http://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/TSMCC.2011.2175380

the developments in computer graphics, computer vision, andbiomechanics [13]. In recent years, studies on kinematics anddynamics of human upper limbs have become overwhelming inmedical science (see, e.g., [34]), as upper extremity dysfunctionstrongly influences the capacity of a person to perform activitiesof daily life (ADL), such as self-care, dressing, eating, and objectmanipulations [9].

It has been reported that appropriate aids and treatments,which are carefully prescribed, can enhance the recovery of armmobility among the patients [5]. Therefore, a precise, objective,and sensitive quantification of dysfunction of upper limbs is aprerequisite before a care plan is generated [31]. This quantifica-tion also allows clinicians to better understand the nature of thedisease. Current strategies for assessing functional motion abil-ities of upper limbs are limited to subjective evaluations, whichare conducted by clinicians working in hospital, using the crite-ria, such as dexterity and speed of single-hand movements, anddexterity and speed of both hands [30].

These test criteria, in spite of their simplicity and efficiency,tend to be lack of robustness. For example, the impairment indexof Parkinson’s disease can vary up to 40% among differentobservers [39]. As a promising solution toward this problem,camera-based intelligent decision systems have been developedto recognize the movements of human upper limbs (see, e.g.,[27], [34]).

Among the established work for human motion analysis,shape-based classification approaches recently draw much at-tention of the research community [35]. Dynamic time warping(DTW) is a template-based dynamic programming algorithm forshape matching. This technique is of conceptual simplicity andstability [22]. However, it compromises efficiency and accuracyand requires string alignments in the starting point. Shape con-text (SC) is a feature-based scheme that utilizes the histogramof the relative polar coordinates of all other points [2]. SC alsoneeds high computational effort but can be used to describe thesimilarity of two deformed shapes. Other commonly used shapedescriptors are minimum description length, Fourier analysis,curvature scale space, Zernike moment, structural information,and patch- and skeleton-based computation [21].

In this paper, we demonstrate how motion trajectories of hu-man upper limbs are automatically classified, based on the mea-sures of shape similarity calculated beforehand. To our knowl-edge, most of the existing feature-based classifiers rarely takeinto account the association between two images in the stageof feature extraction. In many cases, this practice very likelyexperiences large uncertainty in the classification stage, as theextracted features may not fully describe the images and inher-ent characteristics.

1094-6977/$26.00 © 2011 IEEE

ZHOU et al.: CLASSIFICATION OF UPPER LIMB MOTION TRAJECTORIES USING SHAPE FEATURES 971

Our work is to explore the perceptual difference using aniterative parsing process, where SC is used to calculate shapedistance (SD) after two shapes have been somehow registered.SC is used here because of its robustness performance [2]. Themain contribution of our approach is the extension of SC incombination with a probabilistic scheme, which ensures twoshapes to be well registered. This “similarity maximization” ap-proach, in spite of its computational effort, is used to improvethe discrimination capability of the classical SC-based meth-ods in different motion circumstances. The applications of theproposed approach include human action analysis and under-standing in video surveillance.

The rest of the paper is organized as follows. In Section II,related work with the applications in biomedical engineeringis summarized. In Section III, a classical SC approach and itsproperties are introduced. A new shape-matching strategy basedon the expectation-maximization algorithm is reported in Sec-tion IV. Afterward, the evaluation of the proposed strategy isperformed in Section V. Finally, conclusions and future workwill be given in Section VI.

II. RELATED WORK

In this section, the state-of-the-art techniques that are de-veloped for classification and recognition of the upper limbmovements are summarized. Particularly, the focus is on theapplications of these classical approaches in biomedical fields.First, the feature- and evidence-based approaches for motionanalysis of the upper extremity are addressed. Afterward, theestablished classification approaches that are used in the mobil-ity measurements will be introduced.

A. Applications of Feature-Based Motion Analysis

In [15], the authors presented an upper-limb-movement clas-sification system for the measurement of the impairment degreesof cerebral palsy children using the momentum analysis. Visualtracking tests were used to obtain a quantified statistical descrip-tion of the involuntary movements of the arm around the elbowjoint in a group of patients suffering from athetoid cerebralpalsy [23], where frequency components were extracted.

In [28], the author quantified the variability in arm trajec-tories of seven children with dyskinetic cerebral palsy throughrepeated outward reaching movements. It was observed thatchildren with dyskinetic cerebral palsy had a significantly re-duced signal-to-noise ratio compared with healthy children ofsimilar ages. In [33], the authors studied upper extremity move-ments in individuals with neurological disorders, such as strokeand spinal cord injury (SCI), by the measurement of movementtrajectories and their derivatives. In [41], the authors presenteda study to quantitatively assess the effects of speed variationson the joint kinematics during multisegment-seated reachingmovements.

In [10], the authors performed quantitative measures so asto assess spasticity and dystonia. Spasticity was measured asthe slope of the force–velocity relationship from a test whenone measures the forces that is required to passively extend theelbow at different velocities.

B. Classification Techniques for Mobility Measurement

In [37], an approach for automatically segmenting sequencesof natural activities into atomic sections and, finally, clusteringthem was presented. The continuous observations were trans-formed to a discrete symbol sequence. Integration of spatial andtemporal contexts that require a discrete sequence of symbolvectors have been used for action recognition [29].

A view-independent approach was presented to the recogni-tion of human gestures in a low-resolution sequence from multi-ple calibrated cameras [4]. Hu moments were used in [3] for thematching of temporal templates. A visual hull was reconstructedby the computation of the polyhedral approximations of the sil-houettes [8]. In [38], the authors proposed a new framework,which modeled actions using 3-D occupancy grids as visualhulls, built from multiple viewpoints in an exemplar-based hid-den Markov model (HMM). In [11], the authors introduceda novel trajectory classification approach, depending on theLevenshtein distance on trajectories (LDT) as a measure of thesimilarity between two motion trajectories.

In [17], the authors present a two-threshold segmentation al-gorithm to automatically decompose a complex motion into asequence of simple linear dynamic models. In [14], the authorsproposed an action-recognition method by extending canonicalcorrelation analysis (CCA) of two sets of vectors into two videovolume tensors. Tensor-based approaches have also been inte-grated in, e.g., [36] for the action-recognition purpose. In [1], theauthors proposed a set of kinematic features, e.g., divergence,vorticity, symmetric, and antisymmetric flow fields, which arederived from the optical flow for human action recognition invideos.

Recently, SC has been used as reference points to offer a glob-ally discriminative characterization for object recognition [2].SC is related to the distribution of a set of vectors that originatefrom a point to the remaining points on a curve. The similar-ity/dissimilarity measurements between the two curves, namely“shape distance (SD),” are computed as a sum of matching er-rors between the corresponding points. The applications of SCcan be found in [16].

III. BACKGROUND

A. Preliminary

In Fig. 1, we illustrate an example in which motion trajectoriesof the right wrist with respect to the shoulder position havebeen obtained for different postures. In this example, the threepostures demonstrate certain similar patterns. The mean-squarespectrum may be used to separate these different trajectories incase they have different average spectrum values.

In time and frequency domains, other statistical features canalso be used to discriminate the measurements, e.g., autoregres-sive coefficients of orders 2 and 6 (AR2 and AR6), integratedabsolute value (IAV), mean absolute value (MAV), root meansquare (RMS), slope sign change (SSC), and waveform length(WL) [6], [24]. In Table I, we show individual results after thestatistical features have been generated from a set of pull, push,and punch patterns.


Fig. 1. Motion trajectories and feature extraction of the right wrist against the right shoulder for different postures: (a) pull, (b) push, and (c) punch. In Rows 1and 2, we represent the trajectories against time and the computed mean-square spectrum of individual motion patterns, respectively. The data were collected ata sample rate of 200 Hz using a VICON Motion System, Oxford. The mean-square spectrum can be considered as a distinctive feature to separate these motiontrajectories.

TABLE ISTATISTICAL RESULTS OF THE MOTION TRAJECTORIES IN TIME AND FREQUENCY DOMAINS

However, the effectiveness of the statistical features is in-consistent. For example, in Fig. 2, it is shown that the mean-square spectra of the overall motion trajectories look very simi-lar. Moreover, in Table II, we show that the difference betweenany two of the calculated statistical features is much smallerthan the corresponding items that are reported in Table I. Es-pecially, if both “pull” and “punch” occur at a similar speed,the statistical features (e.g., IAV, MAV, RMS, and SSC) thatare extracted from these motion patterns become less discrim-inative than expected. Therefore, it is necessary to develop anew strategy to maintain the discrimination capability in all thecases.

B. Shape Contexts

Classification of two motion trajectories is literally a pro-cedure, where the similarity degree of two trajectories is cal-culated. It is also a correspondence/registration problem: corre-spondence or registration errors can be used to measure the simi-larity between them. SC is such a special descriptor that it can beused for the evaluation of the shape similarity. In this paper, forconvenience, a motion trajectory is called a shape/curve, whichis formed as the motion distance of individual joints against afixed reference.

Consider a point ai on the first shape and a point bj on thesecond shape, where (i,j) indicate the indices of the points onthe two different shapes. The χ test statistic can be used todescribe the cost of matching these two points

Ci,j = C(ai, bj ) =12

K∑

k=1

[hi(k) − hj (k)]2

hi(k) + hj (k)(1)

where hi(k) and hj (k) denote the K-bin normalized histogramsat ai and bj , respectively. Given the overall corresponding pairsover the two groups, our intention is to minimize the total costof matching as follows:

H(γ) =∑

i

C(ai, bγ (j )) (2)

which is subject to the constraint of one-to-one matching. (γ isa permutation.)

This minimization procedure is a square assignment (orweighted bipartite matching) problem. To solve this problem,one can use the Hungarian method or its variants [26]. As in-dicated in [2], shape matching must be invariant under vari-ous scaling and translation conditions, while maintaining robustperformance in the presence of certain geometrical distortions,occlusions, and outliers.


Fig. 2. Motion trajectories and feature extraction of the right wrist against the right shoulder for different postures: (a) pull, (b) push, and (c) punch. In Rows 1and 2, we represent the trajectories against time and the computed mean-square spectrum of individual motion patterns, which are not distinctive, respectively.

TABLE IISTATISTICAL RESULTS OF THE MOTION TRAJECTORIES IN TIME AND FREQUENCY DOMAINS

Suppose that the two shapes can be corresponded subjectto a geometrical transformation. Therefore, the relationship ingeometry of these two shapes is now investigated. Assume thatthere is a transformation, i.e., T : R2 → R2 , that can be usedto map arbitrary points from one shape to another. An affinemodel is usually used for the transformation, i.e., T (x) = Ax +ς , where A is the transformation matrix, and ς is a translationaloffset vector. A least-square solution T (A, ς) can be obtainedusing the following form:

{ς = 1

n

∑ni=1(ai − bγ (j ))

A = (φ+ϕ)t(3)

where n is the number of the extracted feature points, and φ andϕ comprise the homogeneous coordinates of θ and ψ, respec-tively, where

φ =

⎛

⎜⎜⎜⎝

1 a11 a12· · ·· · ·· · ·1 an1 an2

⎞

⎟⎟⎟⎠ (4)

and φ+ is the pseudoinverse of φ. The thin plate spline (TPS)model that is established in [20], because of its stability, canbe used to interpret the coordinate transformation. If differentlocations (xi ,yi) are noncollinear, the TPS interpolant f(x, y)

is used to minimize the bending energy

If =∫ ∫

R2

(∂2f

∂x2

)2+ 2

( ∂2f

∂x∂y

)2+

(∂2f

∂y2

)2dxdy (5)

with f(x, y) = c1 + cxx + cy y +∑n

i=1 wiU(‖ (xi, yi) −(x, y) ‖), where the kernel function U(x) is defined byU(r) = r2 log r2 , and U(0) = 0. Let

{ ∑ni=1 wi = 0

∑ni=1 wixi =

∑ni=1 wiyi = 0

(6)

and f(xi, yi) = vi ; one can have(

K | φ|φT | 0

) (w

c

)=

(v

0

)(7)

where Kij = U(‖ (xi, yi − (xj , yj ) ‖); the ith row of φ is(1, xi , yi); w and v are column vectors from wi and vi ; andc is the column vector with elements c1 , cx , and cy .

It has been proved that If ≈ wT Kw [2]. As pointedout earlier, the SD, based on SC, will be used to mea-sure similarity degrees of two motion trajectories. Generallyspeaking, SD between the two trajectories is calculated us-ing the weighted sum of SC, image appearance distance,and bending energy. The SD between shapes φ and ϕ isdescribed as: DSC (θ, ψ) = 1

n

∑a∈θ arg minb∈ψ C(a, T (b)) +

1m

∑b∈ψ arg mina∈θC(a, T (b)), where T is the estimated TPS


Fig. 3. Illustration of SC. (Left) Deformed model according to “pull” and“push” patterns. (Right) Deformed model according to “pull” and “punch”trajectories.

TABLE IIISTATISTICAL RESULTS OF SHAPE MATCHING USING CLASSICAL SCS

shape transformation. Assuming that the appearance informa-tion (e.g., texture and color) of an object is available, the SD canbe defined as the sum of squared brightness differences withina Gaussian window around the corresponding image points

DSC (θ, ψ) =1n

n∑

i=1

∑

Δ∈Z 2

G(Δ)[Iθ (ai + Δ)

− Iψ (T (bγ (i)) + Δ)]2 (8)

where Iθ and Iψ are the gray-level images that correspond toshapes θ and ψ, respectively. Δ is a differential vector offset,and G is a windowing function (Gaussian). A sum over squareddifferences within a window around the corresponding points isthen used to score the intensity similarity. This score is computedif and only if the TPS transformation T has been applied to alignthe two images.

In Fig. 3, we depict the calculated SC of different pairs ofmotion patterns. It shows that the pair of pull/punch has a sig-nificantly different representation from the pair of pull/push. Letus take a look at the shape matching procedure. In Fig. 4, weillustrate an example of shape matching using the classical SC.In this example, three motion trajectories (i.e., “pull”, “push”and “punch”) that are shown in Fig. 2 are used for the compu-tation of shape similarity. The resultant statistics are tabulatedin Table III, which demonstrate that SC can be used to separatethese three patterns.

IV. PROPOSED APPROACH

SC has demonstrated its capability of discriminating similarshapes/motion trajectories. Our main interest is to maintain theperformance of SC-based classifiers in different circumstances.For this purpose, the system performance is further investigatedafter one has applied SC for similarity measurements. It is evi-dent that in the first graph in the second row of Fig. 4, part of theconcave areas on the curves lacks necessary correspondences.

Ideally, the first two peaks of these two curves are expectedlymatched. However, this does not occur in the final outcome.Similar observations are also obtained in the third graph of thesecond row. This visual discrepancy can be optimally handledif a new shape transformation approach is properly designed. Inother words, it is intended to find a better registration strategy.

A. Minimizing Shape Distances

Let f(θt , ψt , Tt) be a dynamic representation for the shapematching, where θt and ψt are the image observations of shapesθ and ψ, and Tt is the estimated shape transformation at time t.If Tt is optimal, one will be able to match the points of the twoshapes using a maximum likelihood estimation, i.e., finding amaximized p(ψt |Tt).

Let image points be conditionally independent of each other(as each point indicates an independent event), the followingjoint probability stands: p(ψt |Tt) =

∏i p(ψti

|Tt), where i isthe index of one of the image points on the shape. By the use ofBayes’ rule, one obtains: p(ψti

|Tt) =∑

j p(ψti|θtj

, Tt)p(θtj),

where j is a different image point on the shape. The maximumlog-likelihood estimate of Tt is defined as

L(Tt) ≡ log p(ψti|Tt). (9)

Correct correspondence between two sets of image points can-not be known a priori. Therefore, the conditional logarithm ofthe posterior probability Q(Tt |Tt−1) can be computed over theprevious estimates of Tt . Using the Markovian properties, it isexpected to maximize the conditional log-likelihood as follows:

Q(Tt |Tt−1) = E(log p(ψt |Tt)|ψt , T0 , . . . , Tt−1)

=∑

i

∑

j

p(θtj|ψti

, Tt−1)

· log p(ψti|θtj

, Tt). (10)

B. Expectation-Minimization Algorithm

To estimate the transformation over t, which is the target inthis stage, one can use a maximum likelihood estimation (MLE)scheme. Correspondence outliers usually are inevitable and anexpectation-maximization (EM) strategy can be used in orderto reach the minimization of SD. The EM algorithm starts froman initial estimate of shape transformation, which is performedusing random samples from the image points on the curves.An optimal correspondence is sought over two shapes using aniterative scheme.

1) E-step: The a posteriori probability of the incompletedataset p(θtj

|ψti, Tt−1) is used in (10). The maximization of

the conditional probability p(θtj|ψti

, Tt−1) now changes to thatof p(θtj

|Tt−1) [or p(ψti|θtj

, Tt−1)]:

p(θtj|ψti

, Tt−1) =p(θtj

|Tt−1)p(ψti|θtj

, Tt−1)∑k [p(θtk

|Tt−1)p(ψti|θtk

, Tt−1)]. (11)

First, the probability p(θtj|Tt−1) is described using

p(θtj|Tt−1) = 1

N

∑i∈N p(θtj

|ψti, Tt−1), where N is the num-

ber of the involved image points. It indicates that the posterior


Fig. 4. Illustration of the shape matching using SCs. (Top Row) Samples of the motion trajectory images. (Bottom Row) Outcomes of the matching process,where column 1 shows the estimated correspondences relative to the untransformed model (“pull” versus “push”), column 2 shows the result of transforming themodel based on the correspondence (“pull” versus “push”), column 3 shows the estimated correspondences relative to the untransformed model (“pull” versus“punch”), and column 4 shows the result of transforming the model based on the correspondence (“pull” versus “punch”). x-coordinate: samples and y-coordinate:amplitudes (mm). (Better view in color.)

probability p(θtj|Tt−1) can be determined by the mean of the in-

dividual joint densities p(θtj|ψti

, Tt−1) over the selected imagepoints.

Second, the densities p(ψti|θtj

, Tt−1) are assumed to be amember of the exponential family of densities [19]. In thecase of the n-dimensional Gaussian g, the total probabilityis the mixture of multinomial probabilities: p(ψti

|θtj, Tt−1) =∑

m gm (θtj, um )

∑k gk |m (θtj

, um ,k )pm,k (ψti), where u is a

gating parameter that determines the importance of imagepoints in the shape-matching procedure. By the assump-tion that the model is Gaussian, one then has the followingform: p(ψti

|θtj, Tt−1) = 1

(2π )n2 σn

∑m gm (θtj

, um )∑

k gk |m

(θtj, um ,k ) exp(− 1

2σ 2 (ψti− μ)T (ψti

− μ)), where gm andgk |m are two probabilities, depending on the inputs/parameters.μ is the mean of ψti

. Therefore, (10) can be rewritten as follows:

Q(Tt |Tt−1) =1

2σ2

∑

i

∑

j

p(θtj|ψti

, Tt−1)

(ξ + β(ψti− μ)T (ψti

− μ)) (12)

where ξ is the total summation of other terms after usinglogarithms.

2) M-step: In this step, (12) is iteratively deployed till itsmaximum value has been found. Alternatively, an optimal so-lution, i.e., Tt = M(Tt−1), is pursued so that Q(Tt |Tt−1) ≥Q(Tt−1 |Tt−1). To obtain an optimal solution to the aforemen-tioned equation, one can differentiate Q(Tt |Tt−1) in terms ofthe SD DSC and, then, set this to be zero. Therefore, an optimalshape transformation can be obtained by solving the followingnumerical equations:

∂Q(Tt |Tt−1)∂DSC

= 0. (13)

The following approximation is valid:

p(θtj|ψti

, Tt−1) ≈ K exp(−

(θtj− ψt)T Σ−1(θtj

− ψt)2

)

(14)where K = 1/(2π)d/2 |Σ|−1 , d is the number of the used imagepoints, and ψt and Σ are the mean and covariance of componentψti

within a small neighborhood (e.g., 5×5). These parametersallow the point matching to be executed in a finer level. Ac-cording to the definition of the SD/contexts, such a relationshipis valid: (θtj

− ψt)|transformation → DSC . Hence, the left-handside of (13) can be decomposed as follows:

∂Q(Tt |Tt−1)∂DSC

≈ DSC βΣ−1K2σ2

∑

i

∑

j

[p(θtj

|ψti, Tt−1)

(ξ + β(ψti− μ)T (ψti

− μ)) × exp(− 1

2

(θtj− ψt)T Σ−1(θtj

− ψt))]

. (15)

The aforementioned equation indicates that the shape transfor-mation fully depends on the SD DSC and the weighted summa-tion (within the brackets). This iterative search will stop if oneof the two components experiences minor changes. This alsoallows the similarity between the two shapes to be quantified.

C. Algorithmic Flowchart

The proposed algorithm works in this way: motion trajec-tories are formed by the calculation of the Euclidean distancebetween the positions of the wrist (or elbow) and shoulder jointsover time. The angles (flexions) between 1) the upper and lowerarms and 2) the upper arm and the human trunk are also used.Then, a number of segments of a predefined length, which areextracted from the motion trajectories, is converted from a 2-D matrix (amplitudes versus time) to BMP images. To form


Fig. 5. Illustration of the shape matching using SCs and a random samplingalgorithm. Left: the obtained correspondences relative to the untransformedmodel (“pull” versus “push”); right: the obtained correspondences relative tothe untransformed model (“pull” versus “punch”). x-coordinate: samples, y-coordinate: amplitudes (mm). Better view in color.

TABLE IVSTATISTICAL RESULTS OF SHAPE MATCHING USING

THE PROPOSED ALGORITHM

continuous curves from these measurements, a spline interpola-tion technique can be applied.

Afterward, the SD between each segment of a motion trajec-tory and the reference segment (a segment of “push” patterns)is calculated. To maximize the similarity measurement betweenthe two shapes, an EM algorithm is applied to find the bestregistration. See Algorithm 1 for a summary of the proposedalgorithm (note that this does not include a classification step).In Fig. 5, we illustrate the results of combining SC and the EMalgorithm. It demonstrates that the correspondences that weremissed in Fig. 4 have been found in Fig. 5. In Table IV, weshow the significant difference in If values using the proposedscheme, compared with those of Table III.

Fig. 6. Experimental setup with a Vicon system.

V. EXPERIMENTAL WORK

In this section, the proposed algorithm will be compared withseveral state-of-the-art techniques for motion trajectory classifi-cation. Data acquisition is started in the first instance, followedby feature extraction. The classification procedure and a fullevaluation will be introduced subsequently. Right arms havebeen commonly used to manipulate objects. Without loss ofgenerality, the classification of motion trajectories of the rightarms is mainly discussed herein.

A. Experimental Setup

1) Data Acquisition: The motion datasets were collectedfrom the upper limbs of five healthy male adults of 20–40 yearsold using a reflective-marker-based motion tracking system (Vi-con, Oxford, U.K.). The Vicon system has a sample rate of 200Hz. Five Vicon markers were placed on the following posi-tions for the representation of the right arms’ movements (seeFig. 6): 1) humeral great tubercle, 2) humeral lateral epicondyle,3) styloid process, 4) forehead, and 5) ulna. Sensors placed atforehead and ulna ensure the measurements’ stability and con-sistency. After appropriate calibrations, the Vicon system wasused to measure the 3-D positions of the markers that indicatethe actual motion trajectories of the arms.

The subjects started performing arm movements after beinggiven simple instructions of where and how to move their arms.During the data sampling, the subjects could comfortably movetheir arms in all sessions. The motion patterns to be used arebased on the daily activities, which include: reaching, waving,punching, clapping, handshaking, slapping, pulling, and push-ing. In Fig. 7, we depict exemplar trajectories of the eight actionsthat are used in this paper. To make the performance of the ac-tions more realistic, a demonstrator has been asked to help thesubjects to perform handshaking, pushing, and pulling sessions,while for the slapping and punching actions necessary assistiveequipment, such as punching pads and standing bags has beenused [32].

2) Data Segmentation: Extraction of a sample set (or seg-ment) is very important in the analysis as it strongly affects thecomputed parameters, e.g., mean and variance. To determine anoptimal segment, which is related to a small data block in a lim-ited time slot, one needs to make a balance between efficiencyand effectiveness. A short segment results in a fast process butlacks sufficient contextual information. On the contrary, a long


Fig. 7. Exemplar trajectories of the wrist and elbow joints in terms of the shoulder joint, where row 1 represents “reach,” row 2 represents “slap,” and row 3represents “hand shake.” (a) Wrist. (b) Elbow. (c) Flexion angle.

segment provides enough contextual information, while sacri-ficing processing time.

To our knowledge, no prior study is available for the de-termination of appropriate segments of motion trajectories. Asegment of 2.5 s is used in all the experiments that are re-ported in this paper. Such a segment may contain motion noiseor instable trembles, and hence, a zero-crossing method [25] isused to adaptively separate the meaningful waveforms from thebackgrounds.

In this paper, the impact of segment-length variations on theclassification performance is investigated. Particularly, the caseswith the segment lengths of 1, 2.5, and 3 s are examined. Tocope with the “baseline drift” problem, where the human armscannot return to the starting positions, one can use a wavelet-based approach [18] to remove the drifts by zeroing the scalingcoefficients of the discrete wavelet transform.

3) Feature Extraction: Following the recommendations thatare made in [24], the used datasets are categorized into timeand frequency domains, respectively. The features in the timedomain include the MAV, RMS, WL, and SSC. In the frequencydomain, autoregressive coefficients of orders 2 and 6 (AR2 andAR6) are used. The features of the signals are extracted from thesegments with the length of 2.5 s and then concatenated togetherbefore they go to the classifier. The other features consist ofSDs using the classical SCs and the proposed scheme in thispaper. Note that the amplitudes of different waveforms musthave the same unit before they are picturized or stored in twodimensions.

Fig. 8. Histograms of extracted AR6 features from the eight motion patterns.

In total, there are eight motion patterns to be classified inthis section, i.e., “clapping,” “hand shaking,” “pulling,” “punch-ing,” “pushing,” “reaching,” “slapping,” and “waving.” Forsimplicity, hereafter, these patterns are denoted as classes 1,2, 3, 4, 5, 6, 7, and 8. The entire data sampling takes 1 h. Aftera motion exercise of 5 min, there is 10-min rest before the nextpractice. This interval is designed for maintaining the concen-tration and attention of the subjects. As an example, in Fig. 8, we


TABLE VCONFUSION MATRIX OF MOTION PATTERN CLASSIFICATION USING THE

EXTRACTED WL FEATURES

TABLE VICONFUSION MATRIX OF MOTION PATTERN CLASSIFICATION USING THE

EXTRACTED SSC FEATURES

show the histograms of the AR6 features that are extracted fromthe eight motion patterns. (The features have been averaged overthe entire session.)

B. Evaluation

In the evaluation, the classification results of the eight motionpatterns are investigated, using the extracted features (MAV,RMS, WL, SSC, AR2, AR6, SC, and the proposed scheme)with a classical k-means clustering method. Furthermore, theproposed algorithm is compared with the state-of-the-art tech-niques, i.e., DTW [22], LDT [11], and affine registration (AR)[12]. The first experiment is conducted comparing the perfor-mance of single-feature-based k-means clustering over those ofdifferent measurements (i.e., positions of the wrist and elbowjoints and flexion angles). Second, it will be investigated how thecombination of different measurements affects the classificationperformance. The third experiment is about the fusion of twoor more different features and its application in the classifica-tion. Fourth, the consequence of applying motion trajectories ofvarious segment lengths to clustering approaches is examined.Afterward, the classification performance of the proposed sys-tem is studied, when the upper limbs move at different speeds.Finally, efficiency of these algorithms is studied.

1) Single Feature Based: The classification results of wristmovements using individual features, i.e., MAV, RMS, WL,SSC, AR2, AR6, and SC, are tabulated in Tables V–XII. Ingeneral, single-feature-based methods only favor some of themotion patterns in terms of classification accuracy. More detailsare as follows.

Table V: Using the extracted WL features, “clapping” and“waving” patterns can be perfectly detected (see Fig. 9). How-ever, “slapping” has the lowest classification rate because ofits strong similarity to “pulling” in the feature space. In themeantime, “hand shaking,” “pulling,” “punching,” “pushing,”

TABLE VIICONFUSION MATRIX OF MOTION PATTERN CLASSIFICATION USING THE

EXTRACTED RMS FEATURES

TABLE VIIICONFUSION MATRIX OF MOTION PATTERN CLASSIFICATION USING THE

EXTRACTED MAV FEATURES

TABLE IXCONFUSION MATRIX OF MOTION PATTERN CLASSIFICATION USING THE

EXTRACTED AR2 FEATURES

TABLE XCONFUSION MATRIX OF MOTION PATTERN CLASSIFICATION USING THE

EXTRACTED AR6 FEATURES

TABLE XICONFUSION MATRIX OF MOTION PATTERN CLASSIFICATION USING THE

EXTRACTED SC FEATURES

and “reaching” patterns have the classification rates of approx-imately 70% (some variations are observed as well).

Table VI: “Slapping” and “waving” patterns only obtain theaccuracy of 50% or so. This is because of the fact that their SSCs


TABLE XIICONFUSION MATRIX OF MOTION PATTERN CLASSIFICATION USING THE

PROPOSED APPROACH

Fig. 9. Bar chart histograms of a set of WL features that are extracted from“clapping” and “waving” patterns, respectively.

are less distinctive than “clapping,” “pushing,” and “reaching,”respectively, and the classifier cannot correctly differ thesechanges from other motion patterns. “Hand shaking” and“reaching” patterns have been successfully recognized. Thissuccess attributes to the regular SSC measurements.

Table VII: “Pushing” and “reaching” patterns have relativelylower classification rates than the remainder. This weakness isbecause of the fact that these two patterns share similar featureswith “clapping,” “reaching,” and “slapping,” resulting in certainambiguity in the classification stage.

Table VIII: “Reaching,” “slapping,” and “waving” patternscannot be effectively distinguished, while the others have beensuccessfully separated. It is because these three motion trajec-tories have similar features to those of “waving,” “punching,”and “reaching,” respectively.

In Tables IX and X, we show some interesting results. Whilethese two tables demonstrate the individual roles of AR2 andAR6 features, AR2+AR6 features can help improve the classi-fication rates if properly combined. For example, the values thatare shown in the last three columns of these two figures showthat one feature causes low classification rates, while the otherfeature augments the classification performance.

In Tables XI and XII, we show that, in general, the classicalSC and the proposed features lead to better classification out-comes than the other features introduced earlier. Additionally,the classification rate of the proposed approach is 3% higherthan that of the classical SC in average. To find out whether ornot other measurements help obtain better classification perfor-mance than the case of only using the wrist trajectories, it isrecommended to consider the motion trajectories of the elbowjoint and flexion angles. Instead of showing individual results,the statistical performance of each measurement against indi-vidual features is summarized in Table XIII.

In Table XIII, DTW denotes the well-established algorithm“dynamic time warping.” LDT is used to calculate the LD be-tween two 3-D motion trajectories (i.e., wrist/elbow positionsthat are measured over time), and AR can be used to calcu-late the difference between two 2-D trajectories after the shaperegistration is accomplished. In Fig. 10, we show the extractedDTW features; it is witnessed that the unnormalized distanceof two different “clapping” segments (distance is between 10and 20) is significantly less than that of “clapping” versus“waving” patterns. In this case, “clapping” can be separatedfrom “waving.”

In Table XIII, we indicate that the proposed scheme leadsto the best classification accuracy. To further compare the per-formance of the classical SC and the proposed approaches, thecomputed classification rates are studied using the Student’s t-test. Using a 0.95 confidence, we calculate the p-values for threepairs of classification rates, i.e., wrist(SC)/wrist(proposed), el-bow(SC)/elbow(proposed), and flexion(SC)/flexion(proposed).Three p-values (3.92 × 10−4 ; 4.24 × 10−11 ; 1.60 × 10−10) areavailable, which suggest that the proposed algorithm is statisti-cally better than the classical SC approach. In Fig. 11, we showthat the performance of the proposed algorithm attributes to lessmean SD (approximately 7%) than the classical one. It alsoshows that the proposed scheme leads to more correspondancesthan the classical SC.

C. Combined Measurement Based

For a better demonstration purpose, the case, where two out ofthe three available measurements are combined, is shown here.As an example, the combination of wrist and flexion measure-ments is used with individually extracted features for clustering.In Table XIV, we illustrate the classification results (diagonalelements of the confusion matrix), taking into account the com-bination of wrist/flexion trajectories. By the comparison of theresults that are shown in Tables XIII and XIV, one can observethat the combinatorial measurements leads to approximately0–10% increments in the classification rates. This observationholds for three different combinations of the wrist/elbow/flexionmeasurements.

D. Combined Feature Based

In this section, tests are performed using different combina-tions of features for the motion classification purpose. First ofall, let us take a look at the combination of two features, e.g.,RMS and MAV. Second, it is worthy to investigate: How thecombination of three features (e.g., WL, SSC and SC) affectsthe classification performance? The experimental details are asfollows.

First, features, i.e., RMS and MAV, are extracted from one ofthe three measurements (wrist, elbow and flexion) for classifi-cation. In Table XV, we denote the classification performanceagainst the three measurements. One observes that the combi-nation of the two features actually helps enhance the classifi-cation rates of the three motion patterns, compared with theresults that are shown in Table XIII. Second, one can concate-nate three features WL, SSC, and the proposed scheme. To


TABLE XIIISTATISTICS OF CLASSIFICATION PERFORMANCE USING VARIOUS MEASUREMENTS AND FEATURES: (MEAN±STANDARD DEVIATION)%

Fig. 10. DTW features: unnormalisd distance between two motion patterns.

Fig. 11. Image pairs from “push” and “reach,” and SD calculated by theclassical SC (lower left) and the proposed approaches (lower right).

TABLE XIVCLASSIFICATION RESULTS OF COMBINING THE WRIST/FLEXION TRAJECTORIES

WITH INDIVIDUAL FEATURES (MEAN WITH STANDARD DEVIATION)

choose appropriate features for the classification purpose, the“fast correlation-based filler (FCBF)” algorithm that is reportedin [40] is applied for feature selection, where features are sortedusing a relevance score. In Table XVI, we show significantlyaugmented classification rates. This is because of the fact thatFCBF is capable of identifying the discriminative features in thefeature pool.

TABLE XVCLASSIFICATION RESULTS OF COMBINED FEATURES RMS AND MAV FOR THE

WRIST, ELBOW, AND FLEXION MEASUREMENTS (MEAN WITH

STANDARD DEVIATION)

TABLE XVICLASSIFICATION RESULTS OF COMBINING FEATURES WL, SSC, AND THE

PROPOSED ALGORITHM FOR THE WRIST, ELBOW, AND FLEXION

MEASUREMENTS (MEAN WITH STANDARD DEVIATION)

TABLE XVIICLASSIFICATION RESULTS OF COMBINING FEATURES WL, SSC, AND THE

PROPOSED ALGORITHM FOR THE WRIST, ELBOW, AND FLEXION

MEASUREMENTS USING THE SEGMENT LENGTHS OF 1, 2.5, AND 3 S (UNIT %)

E. Changes of Segment Length

This is an important aspect as changed segment lengths maycause degraded classification performance. As an example, ourexperimental results show how the varied segment lengths af-fect the final classification results. In Table XVII, we denotethe outcomes of motion pattern classification only using theproposed algorithm with segment lengths of 1, 2.5, and 3 s, in-dividually. Note that this example utilizes the wrist, elbow, andflexion measurements, independently. No significant changes inthe classification results have been observed because of the var-ied segment lengths. Similar results have also been found usingother features.


Fig. 12. Percentage difference of classification rates at varied motion speeds:0.1, 0.15, 0.2, and 0.25 ms−1 . Better view in color.

TABLE XVIIICOMPUTATIONAL COSTS OF DIFFERENT ALGORITHMS IN FEATURE EXTRACTION

F. Changes of Motion Speeds

Speed variations of human upper limbs are inevitable in dailylife. These variations may also affect the accuracy of a classifier.To find out what will happen in different motion speeds, armmotion is performed at four different speeds (approximately):0.1, 0.15, 0.2, and 0.25 ms−1 . As an example, the evaluationof the proposed algorithm is here introduced, according to theexperimental setup that has been used to generate Table XIII.The percentage difference between the newly obtained classi-fication rates and those reported in the corresponding columnsof Table XIII is denoted in Fig. 12. Even though motion speedssignificantly change, the percentage difference appears to beminor (<5%).

G. Computational Costs

Efficiency is a critical aspect in real-time processing. In thisstudy, the computational requirements of different algorithms inthe feature extraction step is investigated. The algorithms thatinvolve in this evaluation consist of the classical SC, WL, DTW,and the proposed scheme in this paper. The computational costsare calculated and, then, averaged over 20 iterations, whichare performed in the MATLAB environment (The MathWorks,Natick, MA), running on a PC with Intel Xeron Quad-Core CPU2.4-GHz and 3-GB RAM. In Table XVIII, we show that the WL-based algorithm is the fastest, while the proposed approach usesthe majority of the computational effort to search for the bestregistration.

VI. CONCLUSION AND FUTURE WORK

A new shape feature extraction scheme for motion patternrecognition has been presented in this paper. An expectation-maximization algorithm has been used with the classical SCsscheme to pursue the maximum similarity between two shapes.The proposed strategy has been compared with the state-of-the-art techniques, and the experimental results favor the proposedapproach in the classification of eight motion patterns of theupper limbs.

Future work will be directed toward investigating the com-bination of multiple features in the classification of motion tra-jectories. The current study has revealed that the combinationof different features can result in better classification results. Itwill be very interesting to find out in the future study that whichphysical feature dominates the classification in the combinato-rial form, and how the best feature for reliable classification canbe used consistently. On the other hand, reducing computationalcomplexity and system requirements of the current design is alsoone of our objectives.

REFERENCES

[1] S. Ali and M. Shah, “Human action recognition in videos using kinematicfeatures and multiple instance learning,” IEEE Trans. Pattern Anal. Mach.Intell., vol. 32, no. 2, pp. 288–303, Feb. 2010.

[2] S. Belongie, J. Malik, and J. Puzicha, “Shape matching and object recog-nition using shape contexts,” IEEE Trans. Pattern Anal. Mach. Intell.,vol. 24, no. 4, pp. 509–522, Apr. 2002.

[3] A. F. Bobick and J. W. Davis, “The recognition of human movement usingtemporal templates,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 23,no. 3, pp. 257–267, Mar. 2001.

[4] C. Canton-Ferrer, J. Casas, and M. Pardas, “Human model and motionbased 3d action recognition in multiple view scenarios,” in Proc. Eur.Signal Process. Conf., 2006, pp. 1–21.

[5] J. Chae, M. Harley, T. Hisel, C. Corrigan, J. Demchak, Y.-T. Wong, andZ.-P. Fang, “Intramuscular electrical stimulation for upper limb recoveryin chronic hemiparesis: An exploratory randomized clinical trial,” Neu-rorehabil. Neural Repair, vol. 23, no. 6, pp. 569–578, 2009.

[6] A. Chan and G. Green, “Myoelectric control development toolbox,” inProc. 30th Conf. Can. Med. Biol. Eng. Soc., 2007, pp. 1–4.

[7] X. Chen, C. Zhang, S.-C. Chen, and S. Rubin, “A human-centered mul-tiple instance learning framework for semantic video retrieval,” IEEETrans. Sys. Man Cyber., C, Appl. Rev., vol. 39, no. 2, pp. 228–233,Mar. 2009.

[8] C. Chu and I. Cohen, “Posture and gesture recognition using 3-D bodyshapes decomposition,” in Proc. IEEE Workshop Vis. Human–Comput.Interact., 2005, p. 69.

[9] S. Costi, M. D. Bari, P. Pillastrini, R. D’Amico, E. Crisafulli, C. Arletti,L. Fabbri, and E. Clini, “Short-term efficacy of upper-extremity exercisetraining in patients with chronic airway obstruction: A systematic review,”Phys. Therapy, vol. 89, no. 5, pp. 443–455, 2009.

[10] L. Gordon, J. Keller, E. Stashinko, A. Hoon, and A. Bastian, “Can spastic-ity and dystonia be independently measured in cerebral palsy?,” Pediatr.Neurol., vol. 35, no. 6, pp. 375–381, 2006.

[11] M. Hahn, L. Kruger, and C. Wohler, “3d action recognition and long-termprediction of human motion,” in Proc. Int. Conf. Comput. Vis. Syst., 2008,pp. 23–32.

[12] J. Ho, A. Peter, A. Rangarajan, and M.-H. Yang, “An algebraic approachto affine registration of point sets,” in Proc. IEEE Int. Conf. Comput. Vis.,2009, pp. 1335–1340.

[13] W. Hu, T. Tan, L. Wang, and S. Maybank, “A survey on visual surveillanceof object motion and behaviors,” IEEE Trans. Syst., Man, Cybern., C,Appl. Rev., vol. 34, no. 3, pp. 334–352, Aug. 2004.

[14] T.-K. Kim and R. Cipolla, “Canonical correlation analysis of video volumetensors for action categorization and detection,” IEEE Trans. Pattern Anal.Mach. Intell., vol. 31, no. 8, pp. 1415–1428, Aug. 2009.

[15] J.-D. Lee, K.-W. Wang, L.-C. Liu, and C.-Y. Wu, “An upper-limb-movement classification system of cerebral palsy children based on arm


motion detection,” in Proc. IEEE Conf. Eng. Med. Biol., 2005, pp.6878–6881.

[16] H. Ling and D. Jacobs, “Using the inner-distance for classification of ar-ticulated shapes,” in Proc. IEEE Comput. Soc. Conf. Comput. Vis. PatternRecognit., 2005, pp. 719–726.

[17] C. Lu and N. Ferrier, “Repetitive motion analysis: Segmentation and eventclassification,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, no. 2,pp. 258–263, Feb. 2004.

[18] J. Martinez, R. Almeida, S. Olmos, A. Rocha, and P. Laguna, “A wavelet-based ECG delineator: Evaluation on standard databases,” IEEE Trans.Biomed. Eng., vol. 51, no. 4, pp. 570–581, Apr. 2004.

[19] P. McCullagh and J. Nelder, Generalised Linear Models. London, U.K.:Chapman & Hall, 1983.

[20] J. Meinguet, “Multivariate interpolation at arbitrary points made simple,”J. Appl. Math. Phys. (ZAMP), vol. 5, pp. 439–468, 1979.

[21] B. T. Morris and M. M. Trivedi, “A survey of vision-based trajectorylearning and analysis for surveillance,” IEEE Trans. Circuits Syst. VideoTechnol., vol. 18, no. 8, pp. 1114–1127, Aug. 2008.

[22] C. Myers and L. Rabiner, “A comparative study of several dynamic time-warping algorithms for connected word recognition,” Bell Syst. Tech. J.,vol. 60, no. 7, pp. 1389–1409, 1981.

[23] P. Neilson, “Measurement of involuntary arm movement in athetotic pa-tients,” J. Neurol. Neurosurg. Psychiatry, vol. 37, no. 2, pp. 171–177,1974.

[24] M. Oskoei and H. Hu, “Support vector machine-based classificationscheme for myoelectric control applied to upper limb,” IEEE Trans.Biomed. Eng., vol. 55, no. 8, pp. 1956–1965, Aug. 2008.

[25] C. Panagiotakis and G. Tziritas, “A speech/music discriminator basedon rms and zero-crossings,” IEEE Trans. Multimedia, vol. 7, no. 1, pp.155–166, Feb. 2005.

[26] C. Papadimitriou and K. Stieglitz, Combinatorial Optimisation: Algo-rithms and Complexity. Upper Saddle River, NJ: Prentice-Hall, 1982.

[27] K. Petuskey, A. Bagley, E. Abdala, M. James, and G. Rab, “Upper extrem-ity kinematics during functional activities: Three-dimensional studies in anormal pediatric population,” Gait Posture, vol. 25, no. 4, pp. 573–579,2006.

[28] T. Sanger, “Arm trajectories in dyskinetic cerebral palsy have increasedrandom variability,” J. Child Neurol., vol. 21, no. 7, pp. 551–557, 2006.

[29] M. Shimozaki and Y. Kuniyoshi, “Integration of spatial and temporalcontexts for action recognition by self organizing neural networks,” inProc. IEEE Conf. Intell. Robots Syst., 2003, pp. 2385–2391.

[30] H. Smith, “Smith hand function evaluation,” Amer. J. Occup. Ther., vol. 27,no. 5, pp. 24–51, 1973.

[31] E. Taub, G. Uswatte, D. King, D. Morris, J. Crago, and A. Chatterjee,“A placebo-controlled trial of constraint-induced movement therapy forupper extremity after stroke,” Stroke, vol. 37, no. 4, pp. 1045–1049, 2006.

[32] T. Theodoridis, A. Agapitos, H. Hu, and S. Lucas, “Ubiquitous roboticsin physical human action recognition: A comparison between dynamicANNs and GP,” in Proc. Int. Conf. Robot. Autom., 2008, pp. 3064–3069.

[33] C.-C. Tsao and M. Mirbagheri, “Upper limb impairments associated withspasticity in neurological disorders,” J. Neuroeng. Rehabil., vol. 4, no. 45,pp. 1–15, 2007.

[34] A. Vega-Gonzalez, B. Bain, P. Dall, and M. Granat, “Continuous mon-itoring of upper-limb activity in a free-living environment: A validationstudy,” Med. Biol. Eng. Comput., vol. 45, no. 10, pp. 947–956, 2007.

[35] L. Wang, W. Hu, and T. Tan, “Recent developments in human motionanalysis,” Pattern Recognit., vol. 36, no. 3, pp. 585–601, 2003.

[36] L. Wang, X. Wang, C. Leckie, and K. Ramamohanarao, “Characteristic-based descriptors for motion sequence recognition,” in Proc. Pacific-AsiaConf. Knowl. Discov. Data Mining, 2008, pp. 369–380.

[37] T.-S. Wang, H.-Y. Shum, Y.-Q. Xu, and N.-N. Zheng, “Unsupervised anal-ysis of human gestures,” in Proc. of the Second IEEE Pacific Rim Confer-ence on Multimedia, London, U.K., Springer-Verlag, 2001, pp. 174–181.

[38] D. Weinland, E. Boyer, and R. Ronfard, “Action recognition from arbitraryviews using 3d exemplars,” in Proc. IEEE Conf. Comput. Vis., 2007, pp. 1–7.

[39] P. Wilson, N. Foreman, and D. Stanton, “Virtual reality, disability andrehabilitation,” Disabil. Rehabil., vol. 19, no. 6, pp. 213–220, 1997.

[40] L. Yu and H. Liu, “Feature selection for high-dimensional data: A fastcorrelation-based filter solution,” in Proc. Int. Conf. Mach. Learn., 2003,pp. 856–863.

[41] X. Zhang and D. Chaffin, “The effects of speed variation on joint kinemat-ics during multisegment reaching movements,” Human Movement Sci.,vol. 18, no. 6, pp. 741–757, 1999.

Huiyu Zhou received the B.Eng. degree in radiotechnology from the Huazhong University of Sci-ence and Technology, China, in 1990, the M.Sc.degree in biomedical engineering from the Univer-sity of Dundee, U.K., in 2002, and the Ph.D. degreein computer vision from the Heriot-Watt University,Edinburgh, U.K., in 2006

Currently, he holds a permanent post with theSchool of Electronics, Electrical Engineering andComputer Science, Queen’s University Belfast, U.K.His research interests include computer vision, hu-

man motion analysis, intelligent systems, and human–computer interface.

Huosheng Hu (M’94–SM’01) received the M.Sc.degree in industrial automation from Central SouthUniversity, China in 1982 and the Ph.D. degree inrobotics from the University of Oxford, U.K., in 1993.

He is currently a Professor with the School ofComputer Science and Electronic Engineering, Uni-versity of Essex, U.K., where he is leading the Hu-man Centred Robotics Group. His current researchinterests include autonomous mobile robots, human–robot interaction, evolutionary robotics, multirobotcollaboration, embedded systems, pervasive comput-

ing, sensor integration, intelligent control, and networked robotics. He is theauthor or coauthor of more than 300 papers published in various journals,books, and conferences.

Prof. Hu is the Editor-in-Chief of International Journal of Automation andComputing, and an Executive Editor of International Journal of Mechatronicsand Automation. He is a Fellow of the Institution of Engineering and Technol-ogy (IET) and a Fellow of the Institute of Measurement and Control.

Honghai Liu (M’02–SM’06) received the Ph.D. de-gree in intelligent robotics from King’s College, Uni-versity of London, U.K., in 2003.

He is currently a Professor in intelligent systemswith the School of Creative Technologies, Universityof Portsmouth, Portsmouth, U.K. He previously heldresearch appointments at the University of Londonand University of Aberdeen, and project leader ap-pointments in large-scale industrial control and sys-tem integration industry. He has authored and coau-thored numerous papers published in peer-reviewed

international journals and conferences. His research interests include approx-imate computation, pattern recognition, intelligent video analytics, cognitiverobotics, and their practical applications with an emphasis on approaches, whichcould make contribution to the intelligent connection of perception to action us-ing contextual information.

Dr. Liu received four best paper awards. He is a Fellow of the Institution ofEngineering and Technology (IET).

Jinshan Tang (M’00–SM’03) received the Ph.D. de-gree from Beijing University of Posts and Telecom-munications, Beijing, China, in 1998.

He joined ATR Media Integration and Communi-cation Research Laboratories, Japan, as an InvitedResearcher in 1998. In 2000, he was a Postdoc-toral Researcher with the Harvard Medical School. In2001, he joined the University of Virginia for aboutthree years. He was a Visiting Fellow with the Na-tional Cancer Institute, National Institutes of Health,Bethesda, MD, and was a Senior Engineer with Intel,

Hudson, MA. From 2006 to 2010, he was an Assistant Professor with Al-corn State University, Alcorn, MS. He is currently an Associate Professor withthe Department of Computer Network and System Administration, School ofTechnology, Michigan Technological University, USA. He has authored andcoauthored more than 80 published journal and conference papers on imageprocessing and medical imaging.

Dr. Tang is a member of the Technical Committee on Information Assuranceand Intelligent Multimedia-Mobile Communications, IEEE Systems, Man, andCybernetics Society.

970 ieee transactions on systems, man, and...

Documents