upper-limb emg-based robot motion governing using ... · using empirical mode decomposition and...

J Intell Robot Syst (2012) 68:275–291DOI 10.1007/s10846-012-9677-6

Upper-Limb EMG-Based Robot Motion GoverningUsing Empirical Mode Decomposition and AdaptiveNeural Fuzzy Inference System

Hsiu-Jen Liu · Kuu-Young Young

Received: 31 March 2011 / Accepted: 30 April 2012 / Published online: 22 May 2012© Springer Science+Business Media B.V. 2012

Abstract To improve the quality of life for the dis-abled and elderly, this paper develops an upper-limb, EMG-based robot control system to pro-vide natural, intuitive manipulation for robot armmotions. Considering the non-stationary and non-linear characteristics of the Electromyography(EMG) signals, especially when multi-DOF move-ments are involved, an empirical mode decom-position method is introduced to break down theEMG signals into a set of intrinsic mode functions,each of which represents different physical charac-teristics of muscular movement. We then integratethis new system with an initial point detectionmethod previously proposed to establish the map-ping between the EMG signals and correspondingrobot arm movements in real-time. Meanwhile,as the selection of critical values in the initialpoint detection method is user-dependent, we em-ploy the adaptive neuro-fuzzy inference systemto find proper parameters that are better suited

K.-Y. Young (B)Department of Electrical Engineering,National Chiao Tung University,1001 University Road, Hsinchu 300, Taiwane-mail: [email protected]

H.-J. LiuNational Space Organization,8F, 9 Prosperity 1st Road, Hsinchu Science Park,Hsinchu 30078, Taiwane-mail: [email protected]

for individual users. Experiments are performedto demonstrate the effectiveness of the proposedupper-limb EMG-based robot control system.

Keywords Electromyography (EMG) ·Human-assisting robot · Upper-limb motionclassification · Empirical mode decomposition(EMD) · Adaptive neuro-fuzzy inference system(ANFIS)

1 Introduction

Electromyography (EMG) signals, generated dur-ing muscle contraction, in some sense reflect hu-man intentions. Therefore, some research hasfocused on the study of using EMG signals tocontrol rehabilitation devices [1–4] and human-assisting manipulators [5–10], so that the physi-cally handicapped as well as the elderly may usethem to improve their mobility and quality of life.However, since nonlinear, non-stationary charac-teristics and high variations are inherently presentin EMG signals, these disturbances make it hardto analyze and discriminate among EMG signals.Consequently, figuring out how to achieve a highdiscrimination rate for EMG pattern recognitionstill remains a challenge. Several methods forrecognizing the intended movements from EMGsignals have been proposed. In the time domain,there are mean absolute value, variance, bias

276 J Intell Robot Syst (2012) 68:275–291

zero-crossing, Willison amplitude, AR-model [1,2, 11], Euclidean distance and standard deviation[12], and hidden Markov model [13], etc. In thefrequency domain, there are Fourier transform[14] and wavelet analysis [15, 16], etc. Unfortu-nately, the time-domain approaches usually in-clude high computational complexity [2]. As forapproaches in frequency domain, the data must belinear and strictly stationary for Fourier transform[17] to work; however, EMG signals are non-linear and non-stationary signals, especially forcontraction levels higher than 50% of maximumvoluntary contraction [18]. Meanwhile, a motherwavelet has to be defined a priori for waveletanalysis [19]. Unsuitable mother wavelets maylead to unsatisfactory results.

In contrast, Hilbert–Huang transform (HHT) isa time-frequency method. Based on the local timescale of the data, HHT breaks down a signal intoseveral intrinsic mode functions (IMFs) via empir-ical mode decomposition (EMD), and then calcu-lates the instantaneous frequency of each IMF atany point in time via Hilbert transform. Hence,it is suitable for nonlinear and non-stationarydata analysis. HHT has been broadly applied innumerous scientific disciplines and investigations,e.g., analysis on the bioelectricity signal, failuretesting, and earthquake signals, etc [20]. Severalresearchers have applied HHT to EMG relatedstudies. Xie et al. [18] and Peng et al. [21] adoptedHHT to find the features of muscular fatigue. Maand Luo [20] proposed using HHT and AR-modelto extract the surface EMG feature to recog-nize hand-motions. Wang et al. [22] presented afeature extraction technique based on EMD toclassify the walking activities from accelerometrydata. Chen et al. [23] employed HHT to extractthe frequency features of the stump to controltransfemoral prosthesis. Zong and Chetouani [17]presented a feature extraction technique basedon HHT for emotion recognition from physio-logical signals. In our previous research [24], weemployed the EMD method to extract the upper-limb EMG signals for governing 1-DOF robotarms in real time.

This paper proposes developing an upper-limbEMG-based robot control system that can governhuman assisting robot manipulators in natural andintuitive manner. The developed system is not

intended to serve as a classifier that accuratelyidentifies human intention from EMG signals.The development of such a classifier needs towell consider the influence from the muscle type,strength of muscle contraction, fatigue level, strat-egy in performing the task, and others. Instead,we attempt to develop a system for effective ro-bot motion governing in real-time. At the currentstage, we aim to govern the robot manipulatorof 2 DOFs via the EMG signals measured fromfour surface electrodes placed on Biceps Brachii,Triceps Brachii, Pectoralis Major, and Teres Mi-nor of the human operator. To tackle the non-stationary and nonlinear EMG signals generatedduring multi-joint movements, the EMD methodis employed to break down the EMG signalsinto a set of IMFs, representing different physicalcharacteristics for muscular movement recogni-tion. We then utilize the initial point detectionmethod previously proposed [10] to establish therelationship between the upper-limb EMG signalsand corresponding robot arm movements in realtime. As the selection of critical values in theinitial point detection method is user-dependent,we need to find proper system parameters thatare better suited for individual users. For such sys-tem parameter search, the adaptive neuro-fuzzyinference system (ANFIS) [10, 25] is employed torealize the fuzzy system, due to high complexityexhibited by the 2-DOF movements. A series ofexperiments are performed to demonstrate theeffectiveness of the proposed system. Some otherlearning schemes have been proposed for relatedapplications. Chan et al. [26] proposed a fuzzyapproach to classify single-site EMG signals forprosthesis control based on the time-segmentedfeatures, which uses the Basic Isodata algorithmto cluster the features without supervision andthe back-propagation algorithm to train the fuzzyrules. Vachkov and Fukuda [27] proposed struc-tured learning and decomposition of fuzzy modelsfor robotic control. The random walk algorithmwith variable step size was used to tune the an-tecedent parameters of the membership functionsand a local learning algorithm to tune the conse-quent parameters of the singletons.

The rest of the paper is organized as follows. InSection 2, the upper-limb EMG-based robot con-trol system is described, including the modules for

J Intell Robot Syst (2012) 68:275–291 277

EMG signal measurement and processing, EMDfeature extraction, and motion classification withadaptation. Section 3 presents the experimentalresults and discussions. The conclusion is given inSection 4.

2 Proposed Upper-Limb EMG-Based RobotControl System

Figure 1 shows the system diagram of the pro-posed upper-limb, EMG-based robot control sys-tem for governing a 2-DOF robot manipulator,which consists of three main modules: signal mea-surement and processing, EMD feature extrac-tion, and motion classification with adaptation.The signal measurement and processing modulemeasures the raw EMG signals and also filtersout noise. The filtered EMG signals are then sentto the EMD feature extraction module to breakdown the signals into a set of IMFs. With theIMFs, the motion classification with adaptationmodule then determines the arm movement of theoperator and generates the commands to drive thehuman-assisting robot. From the resultant robotmotion, the operator evaluates the performanceand determines the next movement. These threemodules are described below.

2.1 Signal Measurement and Processing

For this upper-limb, EMG-based robot controlsystem, we extract the EMG signals of BicepsBrachii, Triceps Brachii, Pectoralis Major, andTeres Minor. To obtain more precise EMG sig-

nals, the electrodes are placed on the belly ofthe muscle. The recommended inter-electrodedistance (from one differential electrode to theother) is about 1∼2 cm [28, 29]. Several typesof noises may affect the measurement of theEMG signals, such as ECG crosstalk, electro-magnetic induction from power lines, and armand cable movements. The ECG crosstalk canbe suppressed by measuring signals from thosemuscles away from the heart. The frequency ofthe electromagnetic noise is around 60 Hz. Whilea notch filter at that frequency can be an option, itshould be avoided, because EMG generates manyextra signals at these and neighboring frequen-cies (the primary frequency of the EMG signal is50∼150 Hz). We thus let the proposed approachtackle its influence as the disturbance. Meanwhile,the frequency distribution for the arm and cablemovements is around 0–20 Hz, which can be han-dled using a high-pass filter.

2.2 Empirical Mode Decomposition FeatureExtraction

Hilbert–Huang transform (HHT) [21, 23, 30] is anadaptive signal processing technique based on thelocal characteristic time scale of the data. The keypart of the HHT is the EMD process, which usesthe sifting process to break down a complicatedsignal without a basis function, such as sine orwavelet functions, into several IMFs that are em-bedded in the complicated signal [31]. Each IMF,linear or nonlinear, represents a simple oscillation,which has the same number of extremes and zero-crossings. Figure 2 shows the flow chart of the

Fig. 1 Proposedupper-arm EMG-basedrobot control system

Robot motion

Filtered EMG

Upper-limb motion Signal

Measurement & Processing

IMFs

Motion command

EMD Feature

Extraction

Motion Classification

with Adaptation

Human- Assisting

Robot

Operator


Fig. 2 Flow chart forempirical modedecomposition

No

FilteredEMG

Signals

SiftingProcess

IntrinsicMode

Functions

Trendor

Constant

Yes

Empirical Mode Decomposition

x(t) h (t)i r (t)i

Stop

EMD process, which breaks down the completesignal into a set of IMFs in eight steps:

Step 1 Obtain the upper envelop U(t) of thefiltered EMG data x(t) by using a cubicspline curve to interpolate between thosemaximum points.

Step 2 Apply the same actions in Step 1 to obtainthe lower envelope L(t).

Step 3 Compute the mean value of the upperand lower envelope m1(t):

m1(t) = U(t) − L(t)2

(1)

Step 4 Subtract the running mean value m1(t)from the original data x(t) to obtain thefirst component h1(t):

h1(t) = x(t) − m1(t) (2)

Step 5 Iterate Steps 1–4 on h1(t) for k times

h11(t) = h1(t) − m11(t)••

h1k(t) = h1(k−1)(t) − m1(k−1)(t)

(3)

until the resulting component h1k(t) sat-isfies two conditions: (a) the differencebetween the number of local extremesand that of zero-crossings is zero or oneand (b) the running mean value is zero. InEq. 3, m1(k−1)(t) is the mean value of theupper and lower envelope of h1(k−1)(t).

Step 6 Designate c1(t) = h1k(t) if h1k meets thetwo requirements mentioned above.

Step 7 Subtract the first IMF c1(t) from the orig-inal data to obtain the residual r1(t):

r1(t) = x(t) − c1(t) (4)

Step 8 Treat r1(t) as the new data and repeatSteps 1–7 on r1(t) to obtain all thesubsequent i.e., r2 (t) = r1 (t) − c2 (t) , . . .,rn (t) = rn−1 (t) − cn (t) until the finalresidual rn(t) meets the predefinedstopping criteria as a monotonic function,considered as the trend.

Based on the procedure above, the original datax(t) can be exactly reconstructed by a linear su-perposition:

x (t) =n∑

i=1

ci(t) + rn(t) (5)

where n is the number of IMFs.The number of IMFs depends on the charac-

teristics of the data. Complex data can be de-composed into more IMFs, which increases thecomputational load in EMD. As EMG signalsare nonlinear, non-stationary, and varying, usinga high data sampling window would lead to asituation where the mapping between the upper-limb EMG signals and corresponding robot armmovements cannot be established within the ex-pected time interval. After several experiments,it was found that combining a sixth-order band-pass Butterworth filter (a type of filter designedto have as flat a frequency response as possible inthe passband) with the cut-off frequencies at 20and 400 Hz with a window containing 20 samples


per second could reduce the computational loadand noises, and this was used afterward. Figure 3shows a typical example of the EMD processwhen a muscle relaxes and then flexes. Each ofexample includes the empirical EMG signal x(t),three IMFs (c1(t), c2(t), c3(t)), and residue (r3(t)).

Judging from Fig. 2a and b, c1(t) should be thebackground noise induced by skin impedance andtemperature, cable movement, etc., since its mag-nitude did not vary with arm movement evidently.In contrast, those of c2(t) and c3(t) did vary. Wefound that the variation of the magnitude of c2(t)

Fig. 3 A typical exampleof an empirical EMGsignal and correspondingempirical modedecompositioncomponents, including3 IMFs and 1 residue(trend): muscle ina relaxation and b flexion

0 2 4 6 8 10 12 14 16 18 20-20

0

20x(

t)

0 2 4 6 8 10 12 14 16 18 20-10

0

10

0 2 4 6 8 10 12 14 16 18 20-5

0

5

0 2 4 6 8 10 12 14 16 18 20-5

0

5

0 2 4 6 8 10 12 14 16 18 20-5

0

5

Time (Samples)

(a) Muscle in relaxation

0 2 4 6 8 10 12 14 16 18 20-20

020

0 2 4 6 8 10 12 14 16 18 20000

0 2 4 6 8 10 12 14 16 18 20000

0 2 4 6 8 10 12 14 16 18 20.505

x 10-3

0 2 4 6 8 10 12 14 16 18 20.505

Time (Samples)

(b) Muscle in flexion

c 1(t

) c 2

(t)

c 3(t

) c 1

(t)

c 2(t

) c 3

(t)

r 3(t

) r 3

(t)

x(t)


is incremental, which basically followed that of themuscle activities, while that of c3(t) is decremen-tal. Therefore, c2(t) was identified to represent thelimb movement.

2.3 Motion Classification with Adaptation

To achieve real-time motion classification, weadopt the initial critical value detection methodpreviously proposed. Note that this method is notintended to provide a classifier that accuratelyidentifies human intention from EMG signals, butto efficiently establish the relationship betweenthe upper-limb EMG signal and correspondingmovements. Its feasibility and effectiveness havebeen verified via experiments based on one-jointupper-limb movement [10, 24]. Details of themethod can be referred to [10, 24], and are brieflydescribed below.

The initial point detection method determinesthe onset of the upper limb motion by detectingthe instant when the magnitude of the extractedfeature reaches the critical value, as illustratedin Fig. 4. In Fig. 4, the state of the muscle, MS,is determined to be active when the value forthe extracted feature Fk, calculated by the rootmean square (RMS) method, is larger than thepredefined upper critical value CVu, and MS isinactive when Fk is smaller than the predefinedlower critical value CVl. Furthermore, an activeMS corresponds to an “ON” robot command and

Fk

uCV

State

ON

OFF

State

Robot Command

OFF

lCV

Fig. 4 Conceptual diagram for initial value detection

an inactive one to “OFF.” This relationship can beformulated as:

MS ={

1, if Fk > CVu

0, if Fk < CVl(6)

where Fk = RMS(c2(t)), 0 ≤ t ≤ 20 samples, andc2(t) is the 2nd IMF.

The proper selection of CVu and CVl is criti-cal for the performance of the motion classifier.And, their selection is user-dependent. Findingthe proper CVu and CVl for individual usersis very challenging, especially when limb move-ments of multi-DOF are involved. In the searchof proper CVu and CVl, we employ the adaptiveneuro-fuzzy inference system (ANFIS) [25] to uti-lize the fuzzy system due to its excellence at adap-tation. Figure 5 shows the conceptual diagram ofthe proposed ANFIS for CVu and CVl determina-tion, which consists of fuzzy rule, fuzzifier, fuzzyinference engine, and defuzzifier. We utilize theempirical knowledge to generate the fuzzy rules,listed in Table 1, where INa, INb, INc, and INd

stand for the EMG signals of BB, TB, PM, andTM, respectively, OUT for CVu, and W, M, S, VL,L, H, and VH for weak, middle, strong, very low,low, high, and very high. Fuzzifier transforms themeasured EMG signals of Biceps Brachii (BB),Triceps Brachii (TB), Pectoralis Major (PM), andTeres Minor (TM) into linguistic variables. Inaddition, a one-degree Sugeno-type inference sys-tem is employed to depict the fuzzy rules in thefuzzy inference engine. The fuzzy rules are formu-lated as:

Ri : IF BB is Ai and TB is Bi and PM is Ci

and TM is Di THEN CVu (CVl)

= pi × BB + qi × TB + ri × PM + si

× TM, i ∈ {1, 2, . . . , 54} (7)

where BB, TB, PM, and TM are the inputvariables, A, B, C, D = {W, M, S} the linguis-tic variables, CVu(CVl) the output variable, and[pi qi ri si] the consequent parameter set, whichcan be determined by the least-squares method.Defuzzifier transforms the fuzzy results of theinference into a real CVu(CVl) using the weightedaveraged method.


Fig. 5 Conceptualdiagram of the proposedANFIS to determine CVuand CVl

Fuzzy

Inference

Engine

Fuzzy Rule

Base

ANFIS

Biceps Brachii

Fuzzifier Defuzzifier

Training

Data

Fuzzy System

Triceps Brachii

Teres Minor

CVl,BB,TB,PM,TM

(CVu,BB,TB,PM,TM)

Pectoralis Major

Via extensive experiments, the values of theW, M, and S in INa ∼INd and those of VL, L,M, H and VH in OUT are empirically set, asshown in Table 2, and CVl is set to be 0.7 timesthe value of CVu. Since some conditions, suchas skin impedance and temperature, etc. may bedifferent from those for training, the values afore-mentioned can be adjusted via a compensating

factor λ, ranging from 0.8 to 1.3. The procedure fordetermining CVu andCVl is described as follows:

Step 1 Set the compensating parameter λ to be0.8 in the proposed ANFIS to obtain thefirst set of CVu and CVl.

Step 2 Ask the operator to perform the motionof flexion, extension, internal rotation,

Table 1 Fuzzy rule base Rule No. INa INb INc INd OUT Rule No. INa INb INc INd OUT

1 W W W W VL 28 M W W W L2 W W W M VL 29 M W W M L3 W W W S L 30 M W W S M4 W W M W VL 31 M W M W L5 W W M M L 32 M W M M M6 W W M S L 33 M W M S M7 W W S W VL 34 M W S W L8 W W S M L 35 M W S M M9 W W S S L 36 M W S S M10 W M W W L 37 M M W W M11 W M W M L 38 M M W M M12 W M W S L 39 M M W S M13 W M M W M 40 M M M W H14 W M M M M 41 M M M M H15 W M M S M 42 M M M S H16 W M S W M 43 M M S W H17 W M S M M 44 M M S M H18 W M S S M 45 M M S S H19 W S W W H 46 M S W W VH20 W S W M H 47 M S W M VH21 W S W S H 48 M S W S VH22 W S M W H 49 M S M W VH23 W S M M H 50 M S M M VH24 W S M S H 51 M S M S VH25 W S S W VH 52 M S S W VH26 W S S M VH 53 M S S M VH27 W S S S VH 54 M S S S VH


Table 2 Values of the linguistic variables in the fuzzy rulebase

INa ∼ INd OUT

W M S VL L M H VH1/3 2/3 1 1/3 2/3 1 4/3 5/3

and external rotation two timesconsecutively.

Step 3 If the successful discrimination rate(SDR), defined in Eq. 8 below, is lower

than 80%, add 0.1 to λ, and iterate Steps2 and 3 until the SDR is equal to or morethan 80%.

SDR = Number of successful motions followingTotal number of classif ications

× 100% (8)

The proposed ANFIS consists of five layers, asshown in Fig. 6. Layer 1 is the input layer. Each

Fig. 6 Structure of theANFIS for the proposedsystem

A1

A2

B1

B2

Π

A3

B3

Π

Π

Π

Π

Π

N

N

N

N

N

N

Σ

BB

TB

Layer1 Layer2 Layer3 Layer4 Layer5

)( lu CVCV

1w

C1

C2

D1

D2

C3

D3

TM

PM

1w CVu1

(CVl1 )

2w CVu2

(CVl2 )

3w CVu3

(CVl3)

4w CVu4

(CVl4)

5w CVu5

(CVl5 )

6w CVu6

(CVl6)

Π

Π

Π

Π

Π

N

N

N

N

N

Π N

49w CVu49

(CV l49)

50w CVu50

(CVl50)

51w CVu51

(CVl51)

52w CVu52

(CVl52 )

53w CVu53

(CVl53)

54w CVu54

(CVl54 )

•

)( 545454 lu CVCVw

)( 111 lu CVCVw

8w

18w

28w

38w

48w

• •

• •

• •

• •

• •

• •

• •

• •

• •

54w

•

•

•

•

•

•

•

•

•

•

•

•

•

•


node in this layer represents an input variable ofthe model with the membership function:

O1i = μAi (BB) , O1

i+3 = μBi (T B) ,

O1I+6 = μCi (PM) , O1

i+9 = μDi (T M) , i = 1, 2, 3

(9)

The bell-shaped membership function is em-ployed, shown in Fig. 7, and is expressed as:

μXi (Y) = 1

1 +{[

(Y − ci) /ai]2

}bi, i = 1, 2, 3 (10)

where [X, Y] ∈ {[A, BB], [B, T B], [C, PM], [D,T M]}, [ai b i ci] represent the premise parameterset, which can be determined by the backpropa-gation gradient descent method. Layer 2 is the in-ference layer. Each node in this layer is multipliedby the input signal to become wi:

O2i = wi = μAi (BB) × μBi (T B) × μCi (PM)

×μDi (T M) , i = 1, 2, · · ·, 54 (11)

wi stands for the firing strength of the rule. Layer3 is the normalization layer that normalizes thefiring strength by calculating the ratio of ith firingstrength to their sum:

O3i = wi = wi∑54

j=1 w j

, i = 1, 2, · · ·, 54 (12)

Layer 4 is the output layer. Each node multipliesthe normalized firing strength by the consequentfunction to generate the qualified consequent ofeach rule. The output of the node is computed as:

O4i = wiCVu (CVl)i

= wi (pi BB + qiT B + ri PM + siT M) ,

i = 1, 2, · · ·, 54 (13)

Layer 5 is the defuzzification layer, which com-putes the weighted average of the output signalsfrom the output layer:

O5i =

∑54

i=1wiCVu (CVl)i

=∑54

i=1 wiCVu (CVl)i∑54i=1 wi

(14)

3 Experiments

We performed a series of experiments to evaluatethe performance of the proposed system. Figure 8shows the implementation of the proposed systemfor experiments. In Fig. 8, the measured EMGsignals are first amplified using the ETH-256 phys-iological signal amplifier (manufactured by iWorxSystems, USA), and the amplified analog signals(voltages) then transformed into digital signals

Fig. 7 Bell-shapedmembership functions forinput variables

0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.70

0.2

0.4

0.6

0.8

1

0

0.2

0.4

0.6

0.8

1

WMS

0.4 0.5 0.6 0.7 0.8 0.9 1

WMS

Input variable: BB (PM)

Input variable: TB(TM)

Mem

bers

hip

Func

tion

(µ)


Fig. 8 Systemimplementation of theproposed scheme

LabView DevelopmentSystem

EMG signal

Analog signal

Digital signal EMG

Amplifier Data

Acquisition Device

Robot motion

Motion command

Butterworth Filter

EMD Feature Extractor

Motion Classifier

(Initial Point Detection &

ANFIS)

via a National Instrument USB-6009 A/D dataacquisition device with a 1 KHz sampling rate.The digital signals are further forwarded to theLabVIEW development system, which includes asixth-order band-pass Butterworth filter with thecut-off frequencies at 20 and 400 Hz, respectively,a 20-sampling-data-window EMD feature extrac-tor; and a motion classifier with the initial pointdetector and ANFIS. By using this processingmethod, robot motion commands can be deter-mined, and then sent to a 6-DOF Mitsubishi RV-2A robot manipulator for execution (with onlyJ1 and J2 manipulated). The entire experimentalsetup is shown in Fig. 9.

Four sets of electrodes were placed on BB,TB, PM, and TM, as shown in Fig. 10. The clas-sifier is designed to let the feature extracted fromBB, TB, PM, and TM correspond to upper limb

ETH-256

RV-2A

LabVIEW Develop System

USB-6009 A/D

Fig. 9 Experimental setup

flexion, extension, internal rotation, and externalrotation, respectively, and those for both BB andPM and both TB and TM together for flexion-internal rotation and extension-external rotation,respectively. Their muscle states will determinewhether it is an up, down, turn-left, turn-right, up-left, or down-right movement. Due to some mus-cle crosstalk or imprecise feature identification,there may be some conflict movement decisionsbetween the two muscles. Under such circum-stances, the classifier will send out an error signal.Totally, there are eight outputs for the classifier:STOP, UP, DOWN, LEFT, RIGHT, UP-LEFT,DOWN-RIGHT, and ERROR. Table 3 summa-rizes the mapping from EMG to robot movement,and Fig. 11 illustrates the classification outputs of1∼6 corresponding to the robot arm movements.

Two male and two female subjects (with theirphysical data listed in Table 4) were asked toperform the following two experiments: (1) themotions of flexion, extension, internal rotation,and external rotation for four times consecutively,and (2) the motions of flexion plus internal ro-tation and extension plus external rotation forthree times consecutively. The first experimentwas used to evaluate the capability of the pro-posed system to recognize those motions relatedto basically one set of muscles (BB and TB or PMand TM), while the second experiment was usedto recognize the motions involving the interactionbetween muscles, which incurred larger mutualinterferences.

Figure 12 shows the results for the first exper-iment, which includes the subjects’ filtered raw


Fig. 10 Electrodelocations: a BicepsBrachii and PectoralisMajor and b TricepsBrachii and Teres Minor

(a) (b)

Biceps Brachii (CH1)

Pectoralis Major (CH3)

Triceps Brachii(CH2)

Teres Minor (CH4)

Table 3 Mapping from EMG to robot movement

EMG Upper limb status Classifier Robot arm

BB TB PM TM output(CH1) (CH2) (CH3) (CH4)

OFF OFF OFF OFF Relaxation 0 STOPON OFF OFF OFF Flexion 1 J2 axis UPOFF ON OFF OFF Extension 2 J2 axis DOWNOFF OFF ON OFF Internal Rotation 3 J1 axis TURN LEFTOFF OFF OFF ON External Rotation 4 J1 axis TURN RIGHTON OFF ON OFF Flexion-Internal Rotation 5 J1 axis TURN LEFT and J2 axis UPOFF ON OFF ON Extension-External Rotation 6 J1 axis TURN RIGHT and J2 axis DOWNAll others Error 7 STOP

Fig. 11 Illustrations ofclassification outputscorresponding to therobot arm movements

6 5 4 3 2 1 0

Cla

ssif

ier

Out

put

Table 4 Physical data of the subjects

Subject Height Weight Gender Body(cm) (kg) type

A 166 60 Male SlenderB 164 82 Male OverweightC 155 52 Female NormalD 150 50 Female Normal

EMG signals, muscle states, and classification out-puts. The muscle states reveal that subjects C andD exhibited certain muscle mutual interferences

during movements, especially for subject D. TheSDR for the subjects is 97%, 99%, 87.9%, and81.8%, respectively. For these four kinds of mo-tions dominated by basically one set of muscles,the proposed system achieved quite high a suc-cessful discrimination rate, even for subject D.

Figure 13 shows the results of the second ex-periment. Different from the motions executedin the first experiment, these motions involvedthe onsets of two muscles simultaneously, leadingto larger mutual interferences and couplings be-tween muscles. In Fig. 13, some muscle states of


0 200 400 600 800 1000 1200 1400 1600 1800 2000 2200-20

0

20

CH

1 R

aw E

MG

0 10 20 30 40 50 60 70 80 90 100 1100

2

4

CH

1 M

S

0 200 400 600 800 1000 1200 1400 1600 1800 2000 2200-20

0

20

CH

2 R

aw E

MG

0 10 20 30 40 50 60 70 80 90 100 1100

5

10

CH

2 M

S

0 200 400 600 800 1000 1200 1400 1600 1800 2000 2200-20

0

20

CH

3 R

aw E

MG

0 10 20 30 40 50 60 70 80 90 100 1100

5

10

CH

3 M

S

0 200 400 600 800 1000 1200 1400 1600 1800 2000 2200-20

0

20

CH

4 R

aw E

MG

0 10 20 30 40 50 60 70 80 90 100 1100

2

4

CH

4 M

S

0 10 20 30 40 50 60 70 80 90 100 1100246

Cla

ssifi

catio

n O

utpu

t

Time (Samples)

(a) Subject A

0 500 1000 1500 2000 2500 3000-20

0

20

CH

1 R

aw E

MG

0 50 100 1500

2

4

CH

1 M

S

0 500 1000 1500 2000 2500 3000-20

0

20

CH

2 R

aw E

MG

0 50 100 1500

5

CH

2 M

S

0 500 1000 1500 2000 2500 3000-20

0

20

CH

3 R

aw E

MG

0 50 100 1500

2

4

CH

3 M

S

0 500 1000 1500 2000 2500 3000-20

0

20

CH

4 R

aw E

MG

0 50 100 1500

2

4

CH

4 M

S

0 50 100 1500246

Cla

ssifi

catio

n O

utpu

t

Time (Samples)

(b) Subject B

Fig. 12 Experimental results for the motions of flexion, extension, internal rotation, and external rotation for four timesconsecutively


0 500 1000 1500 2000 2500-20

0

20

CH

1 R

aw E

MG

0 20 40 60 80 100 120 1400

5

CH

1 M

S

0 500 1000 1500 2000 2500-20

0

20

CH

2 R

aw E

MG

0 20 40 60 80 100 120 1400

5

10

CH

2 M

S

0 500 1000 1500 2000 2500-20

0

20

CH

3 R

aw E

MG

0 20 40 60 80 100 120 1400

5

10

CH

3 M

S

0 500 1000 1500 2000 2500-20

0

20

CH

4 R

aw E

MG

0 20 40 60 80 100 120 1400

5

CH

4 M

S

0 20 40 60 80 100 120 14002468

Cla

ssifi

catio

n O

utpu

t

Time (Samples)

(c) Subject C

(d) Subject D

0 500 1000 1500 2000 2500 3000 3500 4000 4500-20

0

20

CH

1 R

aw E

MG

0 50 100 150 2000

2

4

CH

1 M

S

0 500 1000 1500 2000 2500 3000 3500 4000 4500-20

0

20

CH

2 R

aw E

MG

0 50 100 150 2000

5

10

CH

2 M

S

0 500 1000 1500 2000 2500 3000 3500 4000 4500-20

0

20

CH

3 R

aw E

MG

0 50 100 150 2000

2

4

CH

3 M

S

0 500 1000 1500 2000 2500 3000 3500 4000 4500-20

0

20

CH

4 R

aw E

MG

0 50 100 150 2000

5

10

CH

4 M

S

0 50 100 150 20002468

Cla

ssifi

catio

n O

utpu

t

Time (Samples)

Fig. 12 (continued).


0 100 200 300 400 500 600 700 800 900 1000-20

0

20

CH

1 R

awE

MG

0 5 10 15 20 25 30 35 40 45 500

5

CH

1 M

S

0 100 200 300 400 500 600 700 800 900 1000-20

0

20

CH

2 R

awE

MG

0 5 10 15 20 25 30 35 40 45 500

5

10

CH

2 M

S

0 100 200 300 400 500 600 700 800 900 1000-20

0

20

CH

3 R

aw E

MG

0 5 10 15 20 25 30 35 40 45 500

2

4

CH

3 M

S

0 100 200 300 400 500 600 700 800 900 1000-20

0

20

CH

4 R

awE

MG

0 5 10 15 20 25 30 35 40 45 500

5

CH

4 M

S

0 5 10 15 20 25 30 35 40 45 5002468

Cla

ssifi

catio

n O

utpu

t

Time (Samples)

(a) Subject A

0 200 400 600 800 1000 1200-20

0

20

CH

1 R

aw E

MG

0 10 20 30 40 50 600

5

10

CH

1 M

S

0 200 400 600 800 1000 1200-20

0

20

CH

2 R

aw E

MG

0 10 20 30 40 50 600

5

CH

2 M

S

0 200 400 600 800 1000 1200-20

0

20

CH

3 R

aw E

MG

0 10 20 30 40 50 600

5

CH

3 M

S

0 200 400 600 800 1000 1200-20

0

20

CH

4 R

aw E

MG

0 10 20 30 40 50 600

5

10

CH

4 M

S

0 10 20 30 40 50 6002468

Cla

ssifi

catio

n O

utpu

t

Time (Samples)

(b) Subject B

Fig. 13 Experimental results for the motions of flexion plus internal rotation and extension plus external rotation for threetimes consecutively


0 200 400 600 800 1000-20

0

20

CH

1 R

aw E

MG

0 10 20 30 40 500

2

4

CH

1 M

S

0 200 400 600 800 1000-20

0

20

CH

2 R

aw E

MG

0 10 20 30 40 500

5

CH

2 M

S

0 200 400 600 800 1000-20

0

20

CH

3 R

aw E

MG

0 10 20 30 40 500

5

10

CH

3 M

S

0 200 400 600 800 1000-20

0

20

CH

4 R

aw E

MG

0 10 20 30 40 500

5

10

CH

4 M

S

0 10 20 30 40 5002468

Cla

ssifi

catio

n O

utpu

t

Time (Samples)

(c) Subject C

0 100 200 300 400 500 600 700 800 900 1000-20

0

20

CH

1 R

aw E

MG

0 5 10 15 20 25 30 35 40 45 500

5

10

CH

1 M

S

0 100 200 300 400 500 600 700 800 900 1000-20

0

20

CH

2 R

aw E

MG

0 5 10 15 20 25 30 35 40 45 500

5

10

CH

2 M

S

0 100 200 300 400 500 600 700 800 900 1000-20

0

20

CH

3 R

aw E

MG

0 5 10 15 20 25 30 35 40 45 500

5

10

CH

3 M

S

0 100 200 300 400 500 600 700 800 900 1000-20

0

20

CH

4 R

aw E

MG

0 5 10 15 20 25 30 35 40 45 500

5

10

CH

4 M

S

0 5 10 15 20 25 30 35 40 45 5002468

Cla

ssifi

catio

n O

utpu

t

Time (Samples)

(d) Subject D

Fig. 13 (continued).


BB and PM (also TB and TM) did not simulta-neously reach the individual critical upper value.However, the SDR for the subjects was found tobe 84.6%, 92.3%, 76.9%, and 99%, respectively,indicating that the proposed system still managedthese complex motions well. Videos for these ex-periments can be located via the link connected toour laboratory Web page (http://140.113.149.114).

4 Conclusion

This paper presents an upper-limb EMG-basedrobot control system for achieving natural and in-tuitive robot manipulation. By integrating the ini-tial point detection method previously proposedwith the EMD approach and the ANFIS, the map-ping between the upper limb EMG signals andcorresponding robot arm movements has beenestablished in real-time. Experiments have beenperformed to demonstrate its effectiveness. Forthe potential user to apply the proposed systemfor movements involving upper or other limbs, orinvolving more DOFs, they can follow the pro-posed procedure, possibly at the expense of moresophisticated learning schemes to deal with theincurring complexity. In future works, we plan toinvestigate the possibility of using varying CVs,because fixed CVs lead to consistent classificationfor about 5∼10 min, depending on the status ofmuscle fatigue. Our intention is to let the systemmaintain a high successful discrimination rate fora longer time. We also plan to apply the proposedsystem for full limb movement governing.

Acknowledgements This work was supported in part bythe National Science Council under grant NSC 99-2221-E-009-157, and also Department of Industrial Technologyunder grants 97-EC-17-A-02-S1-032.

References

1. Kermani, M.Z., Wheeler, B.C., Badie, K., Hashemi,R.M.: EMG feature evaluation for movement controlof upper extremity prostheses. IEEE Trans. Rehabil.Eng. 3(4), 324–333 (1995)

2. Huang, H.P., Chen, C.Y.: Development of a myoelec-tric discrimination system for a multi-degree prosthetichand. In: IEEE International Conference on Roboticsand Automation (1999)

3. Chen, L., Yang, P., Zu, L., Guo, X.: Movement recog-nition by electromyography signal for transfemoralprosthesis control. In: IEEE Conference on IndustrialElectronics and Applications (2009)

4. Harada, A., Nakakuki, T., Hikita, M., Ishii, C.: Robotfinger design for myoelectric hand and recognition offinger motions via surface EMG. In: IEEE Conferenceon Automation and Logistics (2010)

5. Fukuda, O., Tsuji, T., Kaneko, M., Otsuka, A.: Ahuman-assisting manipulator teleoperated by EMGsignals and arm motions. EEE Trans. Robot. Autom.19(2), 210–222 (2003)

6. Artemiadis, P.K., Kyriakopoulos, K.J.: EMG-based po-sition and force control of a robot arm: application toteleoperation and orthosis. In: IEEE/ASME Interna-tional Conference on Advanced Intelligent Mechatron-ics (2007)

7. Ferreira, A., Celeste, W.C., Cheein, F.A., Bastos-Filho,T.F., Sarcinelli-Filho, M., Carelli, R.: Human-machineinterfaces based on EMG and EEG applied to roboticsystems. J. Neuroeng. Rehabil. 5(10), 1–15 (2008)

8. Gopura, R.A.R.C., Kiguchi, K.: EMG-based controlof an exoskeleton robot for human forearm and wristmotion assist. In: IEEE International Conference onRobotics and Automation (2008)

9. Bu, N., Okamoto, M., Tsuji, T.: A hybrid motion clas-sification approach for EMG-based human–robot in-terfaces using Bayesian and neural networks. EEETrans. Robot. 25(3), 502–511 (2009)

10. Liu, H.J., Young, K.Y.: An adaptive upper-arm EMG-based robot control system. Int. J. Fuzzy Syst. 12(3),181–189 (2010)

11. Li, Y., Tian, Y., Chen, W.: Multi-pattern recognitionof sEMG based on improved BP neural network algo-rithm. In: Chinese Control Conference (2010)

12. Phinyomark, A., Hirunviriya, S., Limsakul, C.,Phukpattaranont, P.: Evaluation of EMG featureextraction for hand movement recognition basedon Euclidean distance and standard deviation. In:International Conference on Electrical Engineer-ing/Electronics Computer Telecommunications andInformation Technology (2010)

13. Meng, M., She, Q., Gao, Y., Luo, Z.: EMG signalsbased gait phases recognition using hidden Markovmodels. In: IEEE International Conference on Infor-mation and Automation (2010)

14. Gora, J., Szecowka, P.M., Wolczowski, A.R.: Controlof dexterous hand—algorithm implementation issues.In: International Conference on Information Technol-ogy and Applications in Biomedicine (2009)

15. Karlsson, S., Yu, J., Akay, M.: Enhancement of spec-tral analysis of myoelectric signals during static con-tractions using wavelet methods. IEEE Trans. Biomed.Eng. 46, 670–684 (1999)

16. Mahaphonchaikul, K., Sueaseenak, D., Pintavirooj,C., Sangworasil, M., Tungjitkusolmun, S.: EMG sig-nal feature extraction based on wavelet transform.In: International Conference on Electrical Engineer-ing/Electronics Computer Telecommunications and In-formation Technology (2010)

http://140.113.149.114


17. Zong, C., Chetouani, M.: Hilbert–Huang transformbased physiological signals analysis for emotion recog-nition. In: IEEE International Symposium on SignalProcessing and Information Technology (2009)

18. Xie, H., Wang, Z.: Mean frequency derived viaHilbert–Huang transform with application to fatigueEMG signal analysis. Comput. Methods Programs Bio-med. 82, 114–120 (2006)

19. Andrade, A.O., Nasuto, S., Kyberd, P., Sweeney-Reed,C.M., Van Kanijn, F.R.: EMG signal filtering basedon empirical mode decomposition. Biomed. SignalProcess Contr. 1, 44–45 (2006)

20. Ma, W., Luo, Z.: Hand-motion pattern recognition ofsEMG based on Hilbert–Huang transformation andAR-model. In: IEEE International Conference onMechatronics and Automation (2007)

21. Peng, B., Jin, X., Min, Y., Su, X.: The study onthe sEMG signal characteristics of muscular fatiguebased on the Hilbert–Huang transform. In: Interna-tional Conference on Computational Science (2006)

22. Wang, N., Ambikairajah, E., Celler, B.G., Lovell, N.H.:Accelerometry based classification of gait patterns us-ing empirical mode decomposition. In: IEEE Interna-tional Conference on Acoustics, Speech, and SignalProcessing (2008)

23. Chen, L., Yang, P., Zu, L., Guo, X.: Movement recog-nition by electromyography signal for transfemoralprosthesis control. In: IEEE Conference on IndustrialElectronics and Applications (2009)

24. Liu, H.J., Young, K.Y.: Robot motion governing us-ing upper limb EMG signal based on empirical modedecomposition. In: IEEE International Conference onSystems, Man, and Cybernetics (2010)

25. Jang, J.S.R.: ANFIS: adaptive network based fuzzy in-ference system. IEEE Trans. Syst. Man Cybern. 23(3),665–683 (1993)

26. Chan, Francis H.Y., Yang, Y.S., Lam, F.K., Zhang,Y.T., Parker, P.A.: Fuzzy EMG classification for pros-thesis control. In: IEEE Trans. Rehabil. Eng. 8(3), 305–311 (2000)

27. Vachkov, G., Fukuda, T.: Structured learning and de-composition of fuzzy models for robotic control appli-cations. J. Intell. Robot. Syst. 32(1), 1–21 (2001)

28. Luca, C.J.D.: The use of surface electromyography inbiomechanics. J. Appl. Biomech. 13(2), 135–163 (1997)

29. Henryk, K., Grzegorz, S., Anton, N.: Effect of elec-trode position on EMG recording in pectoralis major.J. Hum. Kinet. 17, 105–112 (2007)

30. Huang, N.E., Zhen, S., Long, S.R., Wu, M.C., Shin,H.H., Zheng, Q., Yen, N.C., Tung, C.C., Liu, H.H:The empirical mode decomposition and the Hilbertspectrum for nonlinear and non-stationary time se-ries analysis. Proc. R. Soc. Lond. 454(A), 903–995(1998)

31. Li, X., Li, D., Liang, Z., Voss, Z.J., Sleigh, J.W.: Analy-sis of depth of anesthesia with Hilbert–Huang spec-tral entropy. Clin. Neurophysiol. 119(11), 2465–2475(2008)

upper-limb emg-based robot motion governing using ... · using empirical mode decomposition and...

Documents