optimal learning gain selection in model reference
TRANSCRIPT
Optimal Learning Gain Selection in Model Reference IterativeLearning Control Algorithms for Computational Human Motor Systems
Shou-Han Zhou, Denny Oetomo, Ying Tan, Etienne Burdet, and Iven Mareels
Abstract— The role of learning gains in the ability of a com-putational framework to better capture the behaviour of humanmotor control in learning and executing a task is the subjectofdiscussion in this paper. In our previous work, a computationalmodel for human motor learning of a task through repetitionwas established and its convergence analysed. In this paper, theperformance of the model is investigated through the additionof degrees of freedom in selecting learning gains, specifically theability to independently select the learning gain for the dampingterm. A particle swarm optimisation (PSO) algorithm is util isedto obtain a set of gains optimised to reduce the discrepancybetween the experimental data and the simulated trajectories.It is found that it is possible to improve the accuracy of thecomputational model through the appropriate choice of learninggains. The results and interesting findings are presented anddiscussed in this paper.
I. I NTRODUCTION
In the literature of human motor control and human motorlearning, computational models play an ever increasing role.It has been proposed in the literature that two of the mainmechanisms which contribute to human motor learning arethe formation of an internal model [1][2] and the adjustmentof body impedance [3]. Both mechanisms seek to improvethe execution of a given tasks over repetitive practice throughobservations of past attempts. Consequently, in order to learn,it is necessary for the human to behave as a system whichperforms discontinuous operations repetitively. Controlofsuch systems so as to improve the transient responses andtracking performance has been widely studied in the field ofiterative learning control (ILC).
Although various computational models using ILC havebeen successful in generating results comparable to thosemeasured in the experiment [4], [5], [6], the controllersinvolve an adjustment of the system inputs states. In compu-tational models of human motor learning, the input statesare usually the desired joint acceleration or the desiredjoint/muscle torques. The adjustment of these states are incontrast to one of the most widely used theories in themotor control literature, which argues that the central nervoussystem controls the human body through the adjustmentof parameters of the motor system such as impedance ordamping [7].
Recent models have been proposed to explain how taskexecution is given effect and improved over time [8], [9],
S. Zhou, D. Oetomo, Y. Tan and I. Mareels are with the Mel-bourne School of Engineering, The University of Melbourne,Parkville,VIC, Australia. Emails: [email protected],{doetomo, yingt,i.mareels}@unimelb.edu.au
E. Burdet is with Department of Bioengineering, Imperial College Lon-don, UK. Email:[email protected]
[10], i.e. how human learns to execute a particular task.The proposed computational human motor model identi-fies the role of ILC in the human motor system. In thecomputational model developed in [9], a model referenceadaptive control strategy combined with iterative learningcontrol (ILC) algorithms was utilised to represent the motorcontrol and adaptation of a person learning to performa new task through repetitions. The proposed frameworkwas then tuned and validated against a set of experimentaldata of human subjects performing the task of reachingunder dynamic disturbances [3]. The results showed thatthe proposed computational model based on model referenceiterative learning control can well present the human being’sability to learn unknown environment. The simulation resultswell match the experimental results. For example, the well-known stages in human learning such as before learningbehaviors, after learning behaviors can be clearly observedin the simulation results. This shows the effectiveness of theproposed computational framework.
In the model reference iterative learning algorithms, alearning gain has been selected for each component (un-known time-varying parameter). Learning with respect toposition and velocity is treated equally. The convergence ofthe proposed learning algorithms has been shown in [8], [10].However, experiments/simulation results show that humanbeings learn faster than our computational human modeldoes. This suggests that the learning gain selected by trialand error may not be the best.
In addition to convergence (indicating that the task islearned), the rate at which human can learn a new taskis quite crucial. As pointed out in [11], stroke patientsdid not have a learning deficit, but they have weakness-related slowness to develop the required force to implementanticipatory control. The convergence of learning algorithmsis dependent on the learning gains. Therefore, the learninggain can potentially be used as a qualitative measure todetermine the human being’s ability to learn motor skills.
This paper explores the learning gains of the modelreference learning algorithms in the computational model.Inother words, it is desired to to find the best possible learninggains that produce simulation results closest to that obtainedin the experiment. This is done by allowing a set of gainsto be determined through an optimization technique andanalyzing the resulting behavior. The resulting informationmay provide insights into the learning and motor controlmechanisms that the body / central nervous system (CNS)relies on in carrying out a manipulation task.
Moreover, the computational framework in this work also
investigates the effects of the additional degrees of freedomcreated when the learning gain weights for position errorsand velocity errors are different. By introducing this addi-tional freedom, the search region for the optimization algo-rithm increases, allowing a better performance in matchingthe results of the computational model to the experimentalresults. A Particle Swarm Optimisation (PSO) algorithm [12]was used to provide a quick view on the possibilities ofoptimising learning gains for a particular task. The discussionin this paper highlights the roles of the learning gainsin model reference iterative learning control. The obtainedcomputational model can thus better characterize the abilityof human to learn.
II. CONSTRUCTION OF THEMODEL
This section briefly summarizes the computational modeldeveloped in [9], [10].
The structure of the proposed framework established in[8][9][10] is shown in Figure 1, based on the schemesof iterative learning control and model reference adaptivecontrol (MRAC).
ryd
Ideal
Trajectory
Model
(GR)Human/Plant Model
(Gp)u
e
+
-
y
Parameter/Gain Adjustment
(K)
TASK
Reference Model (GM)
Controller
(ILC)
Fig. 1: Model Reference Adaptive Controller Framework
Under the scheme, control and learning are achievedthrough the adjustment of the controller gains, which is incontrast to adjustment of control inputs as used by traditionaliterative learning algorithms. The framework consists of threemain components:
1) The reference model2) The plant model3) The iterative learning controller
The reference model GM is used to determine the desiredoutputyd ∈ ℜm from a given reference inputr ∈ ℜm. In thiscase, our desired output is the ideal motion of our body (e.g.hand) while performing the task set by the reference input.The ideal model can be thought of as the motion of a personperforming the task in the absence of any disturbances.Through the process of system identification performed onthe experimental data, the reference model was found to bea second order system.
Theplant model GP represents the dynamics of the humanbody and the unknown, finite environmental disturbances. Inthe framework, the plant model is formed by constructinga model of the body, such as the upper limb, while theunknown environmental disturbanced(t), is represented bya time-varying, but iteration-invariant disturbances. Under
the formulation, the model assumes that the descendingmotor commands from the CNS regulate the human motorsystem at the acceleration level intask space. The model ofthe dynamics of the human body is feedback linearised, toproduce an overall plant model expressed as:
y(t) = Ay(t) + Bu(t) + d(t) ∀t ∈ [0, T ] (1)
where T is the time required to achieve the task. More-over, the disturbance is assumed to satisfy the conditionmax
t∈[0,T ]‖d(t)‖ ≤ bd, wherebd is an unknown positive con-
stant. More details on the reference model and the plantmodel can be found in [8][9][10].
The controller is an iterative learning controller whichadjusts the controller gainK across each iterationi. This isthe focus of the study reported in this paper and hence it ispresented in a more detailed manner. During each iteration,the controller generates the plant inputui using the statefeedback control law
ui(t) = K1i(t)yi(t) + K2i(t)r(t) − K3i(t)
= Ki(t)φi(t), ∀t ∈ [0, T ] (2)
where Ki =[
K1,i K2,i −K3,i
]
∈ ℜ2×6 and φi =
φ1,i
φ2,i
φ3,i
∈ ℜ6×1 with φ1,i = yi, φ2,i = r and φ3,i =
[
1 1]T
.By denoting the error between the desired and actual
outputs at any time instant within one iteration ase(t) =yd(t)− yi(t) and assuming that the plant and the referencemodels satisfy the following conditions:
• The reference model is minimum phase with a knownrelative degree;
• The plant model is approximated as a stable linearsystem of ordern;
• The relative degree of the plant model is finite andequals that of the reference model;
• The order of the reference model (nM ) is known andsatisfiesnM ≤ n;
• At each iteration, the initial value satisfiesyd(0) =yi(0),
then the learning algorithm can be expressed as:
Kk,i(t) = Kk,i−1(t) − βkeiφTk,i, ∀t ∈ [0, T ] (3)
where Kk,0(t) = {0}2×2, k is the index for the chosenstates andT denoted the transpose.βk > 0 is the learningrate for each controller gain. The convergence properties ofthe proposed learning model reference adaptive controller(3)is shown in [8]. The convergence analysis is based on thecomposite energy functions proposed in [13]. The learninggain βk, k = 1, 2, 3 is selected by trial and errors.
Although convergence is guaranteed when a selectedlearning gainβ =
[
β1 β2 β2
]Tis used, the perfor-
mance may not be optimal. Experiments show that humanbeings learn faster than our computational human model.This suggests that the learning gain selected by trial and error
may not be the best. On the other hand, the error vector in(3) is composed of the error in positionep and the error
in velocity ev. e =
[
ep
ev
]
. In the sequel, Equation (3) is
rewritten as
Kk,i(t) = Kk,i−1(t) −
[
βk,1epi(t)
(
φk,i(t))T
βk,2evi(t)(
φk,i(t))T
]
.(4)
It should be noted that the updating law (3) is a specialcase of (4) whenβk,1 = βk,2 = β. It is intuitively clearthat human beings have different sensitivity to position andvelocity. Introducing these two gains can well-present thiskind of sensitivity difference. On the other hand, optimizationalgorithms are used to find the optimal gains. The newfreedom introduced by two gains will enlarge the searchdomain and may lead to the better performance.
Indeed, there exists evidence in the literature suggestingthat humans adjust parameters of their body system throughobservation of both position and velocity errors [14]. TheILC (4) adjusts the gain matrices using the errors betweenthe desired and actual position and velocity. By slightly mod-ifying the convergence proof in [9], the error is convergenteven in the presence of finite external disturbance. Otherthan convergence, other performance indices are also neededto evaluate the performance of the proposed computationalhuman motor models. In particular, the work in this papertries to find the best set of learning gains to represent thelearning ability of human beings.
In modelling human motor control, the performance of thealgorithm is determined by the error between the simulatedtrajectory and the trajectory observed in the experiment. Theperformance of the ideal ILC will result in a simulationoutcome identical to the experiment trajectory. To achievethis performance, it is necessary to identify and optimise theparameters of the controllers which affect the convergenceproperties of the ILC.
The simulation of this learning behavior is conductedusing two different sets of learning gains. The first set ofgains are constructed with a constraint: learning rate for theposition and velocity error are set to be equal (“constrained”).The second set of gains allow the exploration of the different(separate) learning rates for position and velocity motionprofile of the human body. By introducing this additionaldesign freedom, it is evident that the unconstrained gainspossess the potential to obtain a simulation result that canbetter match the experimental results.
This is demonstrated in Figure 2 where a human subject isrequired to reach for a target in front him through a viscousdisturbance field that demonstrate a general trend of pushingthe subject hand to the left when it is aiming to go straight inthe forward direction. The performance of the human subjectin learning to deal with the disturbance field is recordedexperimentally in Figure 2(a), which shows an improvementover iterations. The simulation using the fixed one learninggain and optimal two learning gains are given in Figure 2(b)and 2(c), respectively. In the three cases in Figure 2(a-c),the first iteration in the learning process is the path with the
largest deviation from the straight line path. The performanceimproves over iterations and the path taken of the humansubject hand gets closer to the intended straight line path.Qualitatively, the simulation results coming from optimaltwo learning gains(Figure 2(c)) provide a closer match tothe experimentally recorded response.
−0.05 0.050.3
0.4
0.5
0.6
x (m)
y (m
)
(a)
−0.05 0.050.3
0.4
0.5
0.6
x (m)
y (m
)(b)
−0.05 0.050.3
0.4
0.5
0.6
x (m)
y (m
)
(c)
Fig. 2: Human subject hand motion during the learning process ofa task of reaching within a viscous disturbance field that pushes thehand to the left :(a) experimentally obtained results (b) simulatedwith a set of user selected (fixed) learning gain (c) simulated witha β1 andβ2 of learning gain obtained through optimisation.
More precisely, in order to compare the learning perfor-mance, the following performance measure is proposed,
ei = maxt∈[0,T ]
|ei,sim − ei,exp| ∀i ∈ [0, N ] (5)
where N is the maximum number of trials used in theexperiment. Here the discrepancies is the difference betweenthe error of the simulation from the ideal trajectory inthe x and y direction during each iterationei,sim and theerror of the experiment from the ideal trajectory in thex and y direction during each attemptei,exp. The errorei is plotted against the number of iteration as shown inFigure 3. From the figure, it is evident that the errorscommitted with a additional tuning parameter (β1, β2) issignificantly lower than the case when onlyβ is used. Thissuggests that there exists appropriate learning gain for humanbeing. Moreover, the learning speed for position informationand velocity information is different. One question arisesnaturally, is it possible to find tuning parameters such thattheobtained computational model of human learning system bestmatches experimental results? The answer to this questionis important to qualitatively determine the human being’sability to learn. Consequently, the learning gains might beused as performance indices to indicate discrepancies (suchas impairment) in the motor learning functionalities of ahuman subject, for example in stroke patients. In this work,particle swarm optimisation algorithms (PSO) are used tofind the optimal learning gains that best fit the experimental
1 11 21 31 410
0.05
0.1
0.15
0.2
Iteration
ei=
max
t∈[0
,T]|esim
−eex
p|
ConstrainedOptimised
Fig. 3: Comparison of the performance of the simulation for thecase of constrained learning gainsβ and the unconstrained case
data (Section III). This would specifically address the roles ofthe learning gains within the computational model proposedin this paper. The implications of these roles within humanmotor learning systems is to be further analysed in futurework.
III. O PTIMISING LEARNING GAINS
In the computational model, each state considered in thecontroller (2) is associated with a gain matrixK. Theiterative learning algorithm (4) adjusts the gain matricesusing the two errorsep andev.
Early work has assumed equal contributions of these errorsin the convergence of the controller gainsK towards asteady-state value in the iteration domain. In this paper, thisassumption is removed andβ1 and β2, which govern thecontribution of each error to the convergence ofK, areallowed to vary independently. As such, each contributionin (4) is given the freedom to converge to its respectiveoptimum values at different rates, resulting in a separation oftime scales for the respective error dynamics. Following thisconcept, there exists the possibility of an ideal set of learninggainsβ∗ to be used by the computational model such that thetrajectory simulated follows that of the trajectories observedin the experiment.
From the definition of the controller as shown in (2), itis established that the parameter space, in which the idealset of learning gains exists, is six dimensional. Particleswarm optimisation (PSO) algorithm [12] is used to obtain anideal set of learning gains. PSO is a computational methodwhich optimises a given measure of quality by attemptingto improve candidate solutions of the computation model.This does not guarantee a global optimal solution, however,it serves the purpose of the study in this paper and is a highlypractical method given the size of the problem.
The measure of quality is represented in this paper by acost functionJ which is constructed as the total discrep-ancies between the simulated trajectory and the experimenttrajectory at each time instant of one iteration and across allthe trials performed. The ideal trajectory is the optimum tra-jectory made for the given task. The cost function thereforeis given as:
J =
∫ N
0
∫ T
0
(ξi(t))T
(ξi(t)) dt di, ∀t ∈ [0, T ] (6)
whereξi(t) = ei,sim(t) − ei,exp(t) represents the error be-tween the simulation and experiment for the two directions.The errors are calculated using the data sets obtained fromthe experiment and generated from the simulation, both ofwhich are described in the following sections.
IV. EXPERIMENTAL DATA SET
In the experiment, a human subject was to perform the taskof reaching for a target location while holding the robot end-effector. The task is planar and the target is located directly infront of the human subject. Disturbances are generated by therobot attached to the human arm and the objective is for thehuman subject to learn to perform the task of reaching for thetarget in the face of the injected disturbances. Two studieswere conducted in which the robot generates disturbancesdependent on the hand end-effector velocity (Velocity Field)and end-effector position (Divergent Field). Details of theexperimental setups are given in [4] [5] [10].
Velocity Field or VF The robot generated a disturbanceforce field dependent on the subject’s end-effector velocityin the cartesian coordinates, as described by:
(
Dvx
Dvy
)
= −
(
13 −1818 13
) (
Px
Py
)
(7)
where Dv =[
Dvx Dvy
]T∈ ℜ2 was the disturbance
forces applied to the hand/robot end-effector in the Cartesiancoordinates andP =
[
Px Py
]T∈ ℜ2 was the position
of the hand end-effector.Divergent Field or DF. In this case, the robot generated
a Divergent Field (DF) environment, which was dependenton the subject’s end-effector position in the generalizedcartesian coordinates, as described by the following equation:
(
Ddx
Ddy
)
= −
(
450 00 0
) (
Px
Py
)
(8)
whereDd ∈ ℜ2 was the disturbance forces applied to thehand/robot end-effector in cartesian coordinates. The purposeof this artificial environment was to observe evidences ofimpedance adjustment within the human motor system dur-ing learning. The study in the DF environment is conductedin the same manner as that within the VF environmentdescribed in Case 2.
Throughout this paper, the hand end-effector position datafor the two experiments are presented for one subject forthe purpose of clear illustration. The set of data presentedis representative of the overall behaviour observed in this
objective, and hence can be used to describe the character-istics of the human motor system without significant loss ofgenerality.
V. SIMULATING HUMAN MOTOR LEARNING ON THE
PROPOSEDCOMPUTATIONAL FRAMEWORK
The optimisation algorithm from Section III is incorpo-rated into the computational framework described in SectionII. The framework is used to simulate the behaviour of humanmotor learning in a task where human subject is required toreach for a target location while subject to the environmentaldisturbances used in the experiment. The optimisation tech-nique is carried out by adjusting the learning gains,β1 andβ2 in order to obtain the simulated results that best mimicthe learning behaviour as demonstrated by the experimentaldata set:
β∗ =
(
β∗
1
β∗
2
)
= argminβ
(J) (9)
whereJ is defined is (6).The following assumptions are made in the simulation:
1) Independent control systems are used to control motionin the generalisedx andy directions in task space. Thisfollows from the experimental evidence suggestingthat for the task of planar reaching, there exist twosets of control such that the human motion in taskspace is decoupled along their respective generalisedcoordinates [15].
2) The inertia of the robot is assumed to be negligible,hence its dynamics is not considered in the simulationmodel. This is justified since the robot used for the ex-periment was designed to produce very little apparentinertia through good balancing and also because thesubjects were allowed to familiarise themselves withthe dynamic of the robot before the actual experimenttrials.
3) Noise is modelled as a band limited Gaussian whitenoise to the reference inputr. This implies that fora given movement, only a crude ideal trajectory isrequired to be planned by the CNS for a given task[16].
In order to demonstrate the changes in the performance ofthe ILC algorithm, the computational model is simulated forfollowing settings:
β1 =[
β1,1 β2,1 β3,1
]T
β2 =[
β1,2 β2,2 β3,2
]T
• Case 1: The set of learning gainsβ1 = β2 = β =
[
300 300 10]T
is used. These values are obtained from previous work[8].
• Case 2: The learning gains are unconstrained andoptimised through the particle swarm optimisation tech-nique. While the technique does not guarantee globaloptimality, the resulting behaviour has been through asystematic search in the solution space and provide a
set of learning gains that minimises the cost function(6).
These two cases are simulated for the Velocity Field (VF) andDivergent Field (DF) environments. The results are presentedand discussed in the following section.
VI. RESULTS AND DISCUSSION
Recently, human motor learning has been modeled usinga computational model based on model reference adaptivecontrol and iterative learning control. Although stability andconvergence of the controller has been rigorously demon-strated [9], less attention is focused on the performance ofthealgorithm. In this paper, the performance of the controller,which is governed by the learning gains of the algorithmβ, is analysed. In previous models, the learning gains wereassumed to be constrained. This assumption is relaxed in thispaper in order to use the learning gains as a design variable tooptimise the performance of the iterative learning controller.
The results of both cases (Case 1 and Case 2) along withthe experiment results are given in Figure 4 and Figure 5for the velocity field and the divergent field environments,respectively.
The results are decomposed into five phases, representingthe stages in which learning occurs. These phases have beenreferred previously in the literature [3][17]. In analysingeach phase, it is evident that for both environments, bothcases were able to generate similar learning trends observedin the experiment. Nevertheless, it is also observed thatthe trajectories simulated for each case are significantlydifferent for each phases of learning. Moreover, it seems thatunconstrained learning gains has the potential to significantlyimprove the simulation performance for specific phases.
Taking the the velocity field as an example, it is evi-dent that during the before learning phase, in which thesubjects are first exposed to the new environment, Case1 simulation is able to capture the experiment observationthat the subject’s end-effector only approaches the targetand is unable to reach it within the specified time. Case2generated trajectories where initial maximum deviation fromthe straight line are more similar to those of the experiment.In observing the after learning phase of the velocity field,it is evident that the trajectories of Case2 are significantlycloser to that of the experiment compared to that of Case1.
The optimisedβ1 is identified to be significantly largerin magnitude than thatβ2. The values of the learning gainsfor Case 2 after optimisation is given as:
β∗
1 =[
105 123 86]T
β∗
2 =[
65 53 6]T
Considering thatβ1 and β2 represent the contributionsof the position and velocity errors respectively, this impliesthat the CNS prioritiseep over ev during learning. As abetter fit to the experimental data is obtained through theuse of different learning gain forβ1 andβ2, it can also be
Velocity Field
Experiment Results
−0.05 0.050.3
0.4
0.5
0.6
x (m)
y (m
)
(a) Natural Motion
−0.05 0.050.3
0.4
0.5
0.6
x (m)
y (m
)
(b) Before Learning
−0.05 0.050.3
0.4
0.5
0.6
x (m)
y (m
)
(c) During Learning
−0.05 0.050.3
0.4
0.5
0.6
x (m)
y (m
)
(d) Post Learning
−0.05 0.050.3
0.4
0.5
0.6
x (m)
y (m
)
(e) After Effect
Simulation: Case 1
−0.05 0.050.3
0.4
0.5
0.6
x (m)
y (m
)
(f) Natural Motion
−0.05 0.050.3
0.4
0.5
0.6
x (m)
y (m
)
(g) Before Learning
−0.05 0.050.3
0.4
0.5
0.6
x (m)y
(m)
(h) During Learning
−0.05 0.050.3
0.4
0.5
0.6
x (m)
y (m
)
(i) Post Learning
−0.05 0.050.3
0.4
0.5
0.6
x (m)
y (m
)
(j) After Effect
Simulation: Case 2
−0.05 0.050.3
0.4
0.5
0.6
x (m)
y (m
)
(k) Natural Motion
−0.05 0.050.3
0.4
0.5
0.6
x (m)
y (m
)
(l) Before Learning
−0.05 0.050.3
0.4
0.5
0.6
x (m)
y (m
)
(m) During Learning
−0.05 0.050.3
0.4
0.5
0.6
x (m)y
(m)
(n) Post Learning
−0.05 0.050.3
0.4
0.5
0.6
x (m)
y (m
)
(o) After Effect
Fig. 4: Comparison of the experimentally measured responseand the simulation results produced using the proposed computational modelin Velocity Field (VF) environment
implied that humans utilise the degree of freedom to isolateand learn the damping component of its hand motion duringthe execution of a new task. Much of the current researchhas been focused on the roles of impedance and the use ofinternal model. These implications will be subject to futureresearch on real subjects.
Another interesting observation is that in the velocity field,during the before learning phase, the simulation in Case1 is closer to that of the experiment than that of Case2while during the after learning phase, the simulation of case2 is closer. Therefore, the controller may require differentsets of update rates before and after learning in order tofully capture the experiment behaviour. This means that theCNS prioritises different states to adjust the body parametersduring learning, implying that the learning gainsβ1 andβ2
may be required to be adjusted. This hypotheses agrees withthe findings of [18] and may be subject to future work.
VII. C ONCLUSION
In this paper, the role of learning rates/gains is investigatedfor a recently proposed computational model of humanmotor learning. From the iterative learning algorithm, thelearning gains are identified to govern the convergence ofthe controller and therefore can affect the performance ofthe simulation in matching the experiment behaviours. In
this paper, the gains are optimised in order to seek for thebest gains so as to minimise the discrepancies between thesimulation and the experiment.
The results of this paper suggests that although, differentlearning gains are superior in capturing a specific feature ofthe experiment, there exist no optimal learning gains existwhich can fully capture all the behaviour. The implicationsof this result is that the CNS may prioritise the states ituses to learn and, in addition, the priorities of the states areadjusted during learning.
REFERENCES
[1] R. Shadmehr and F. A. Mussa-Ivaldi, “Adaptive representation of dy-namics during learning of a motor task,”The Journal of Neuroscience,vol. 14, no. 5, pp. 3208–3224, 1994.
[2] Z. Ghahramani, D. Wolpert, and M. Jordan, “An internal model forsensorimotor integration,”Science, vol. 269, pp. 1880–1882, 1995.
[3] E. Burdet, R. Osu, D. W. Franklin, T. E. Milner, and M. Kawato,“The central nervous system stabilizes unstable dynamics by learningoptimal impedance,”Nature, vol. 414, no. 6862, pp. 446–449, 2001.
[4] E. Burdet, K. P. Tee, I. Mareels, T. E. Milner, C. M. Chew, D. W.Franklin, R. Osu, and M. Kawato, “Stability and motor adaptation inhuman arm movements,”Biological Cybernetics, vol. 94, no. 1, pp.20–32, 2006.
[5] D. W. Franklin et al., “Endpoint stiffness of the arm is directionallytuned to instability in the environment,”The Journal of Neuroscience,vol. 27, no. 29, pp. 7705–7716, 2007.
[6] P. L. Gribble and D. J. Ostry, “Compensation for loads duringarm movements using equilibrium-point control,”Experimental BrainResearch, vol. 135, no. 4, pp. 474–482, 2000.
Divergent Field
Experiment Results
−0.05 0.050.3
0.4
0.5
0.6
x (m)
y (m
)
(a) Natural Motion
−0.05 0.050.3
0.4
0.5
0.6
x (m)
y (m
)
(b) Before Learning
−0.05 0.050.3
0.4
0.5
0.6
x (m)
y (m
)
(c) During Learning
−0.05 0.050.3
0.4
0.5
0.6
x (m)
y (m
)
(d) Post Learning
−0.05 0.050.3
0.4
0.5
0.6
x (m)
y (m
)
(e) After Effect
Simulation: Case 1
−0.05 0.050.3
0.4
0.5
0.6
x (m)
y (m
)
(f) Natural Motion
−0.05 0.050.3
0.4
0.5
0.6
x (m)
y (m
)
(g) Before Learning
−0.05 0.050.3
0.4
0.5
0.6
x (m)y
(m)
(h) During Learning
−0.05 0.050.3
0.4
0.5
0.6
x (m)
y (m
)
(i) Post Learning
−0.05 0.050.3
0.4
0.5
0.6
x (m)
y (m
)
(j) After Effect
Simulation: Case 2
−0.05 0.050.3
0.4
0.5
0.6
x (m)
y (m
)
(k) Natural Motion
−0.05 0.050.3
0.4
0.5
0.6
x (m)
y (m
)
(l) Before Learning
−0.05 0.050.3
0.4
0.5
0.6
x (m)
y (m
)
(m) During Learning
−0.05 0.050.3
0.4
0.5
0.6
x (m)y
(m)
(n) Post Learning
−0.05 0.050.3
0.4
0.5
0.6
x (m)
y (m
)
(o) After Effect
Fig. 5: Comparison of the experimentally measured responseand the simulation results produced using the proposed computational modelin Divergent Field (DF) environment
[7] A. G. Feldman and M. L. Latash, “Testing hypotheses and theadvancement of science: recent attempts to falsify the equilibriumpoint hypothesis,”Experimental Brain Research, vol. 161, no. 1, pp.91–103, 2005.
[8] S.-H. Zhou, D. Oetomo, I. Mareels, and E. Burdet, “Modelling ofhuman motor control in an unstable task through operationalspaceformulation,” in Control Automation Robotics and Vision (ICARCV),2010 11th International Conference on, 2010, pp. 2030–2035.
[9] S.-H. Zhou, “Human motor control: simulation and systemidentifica-tion,” Masters Research, The University of Melbourne, 2010.
[10] S.-H. Zhou, D. Oetomo, Y. Tan, I. Mareels, and E. Burdet,“Humanmotor learning through iterative model reference adaptivecontrol,” inIFAC World Congress, 2011.
[11] J. W. Krakauer, “Motor learning: its relevance to stroke recovery andneurorehabilitation,” p. 8490, 2006.
[12] M. Clerc, Particle Swarm Optimization. Wiley-ISTE, 2006.[13] J. X. Xu and Y. Tan,Linear and Nonlinear Iterative Learning Control,
1st ed. Springer-Verlag, Berlin, 2003.[14] M. L. Latash, M. F. Levin, J. P. Scholz, and G. Schoner, “Motor control
theories and their applications,”Medicina (Kaunas), vol. 46, no. 6, pp.382–92, 2010.
[15] D. N. Peter and D. N. Megan, “An overview of adaptive model theory:solving the problems of redundancy, resources, and nonlinear interac-tions in human movement control,”Journal of Neural Engineering,vol. 2, no. 3, p. S279, 2005.
[16] E. Bizzi, N. Accornero, W. Chapple, and N. Hogan, “Posture controland trajectory formation during arm movement,”The Journal ofNeuroscience, vol. 4, no. 11, pp. 2738–2744, 1984.
[17] N. Bhushan and R. Shadmehr, “Evidence for a forward dynamicsmodel in human adaptive motor control,” inAdvances in NeuralInformation Processing Systems, M. S. Kearns, S. A. Solla, and D. A.Cohn, Eds. Cambridge: MIT Press, 1999, vol. 11, pp. 3–9.
[18] K. Wei and K. Kording, “Relevance of error: What drives motoradaptation?”J Neurophysiol, vol. 101, no. 2, pp. 655–664, 2009.