a neural network model for arm trajectory formation using … · 2020-05-24 · namics model (fdm)....
TRANSCRIPT
Neural Networks, Vo!.6, pp, 919-932, 1993Printed in the USA. All rights reserved.
ORIGINAL CONTRIBUTION
0893-6080/93 $6.00 + .00Copyright © 1993 Pergamon Press Ltd.
A Neural Network Model for Arm Trajectory Formation UsingForward and Inverse Dynamics Models
YASUHIRO WADA AND MITSUO KAWATO
ATR Human Information Processing Research Laboratories
(Received 25 May 1992; revised and accepted 14 January 1993)
Abstract-The minimum torque-change model predicts and reproduces human multi-joint movement data quitewell. However, there are three criticisms ofthe current neural network models for trajectory formation based on theminimum torque-change criteria: ( I ) their spatial representation oftime, (2) back propagation is essential, and (3)they require too many iterations. Accordingly, wepropose a new neural network modelfor trajectory formation basedon the minimum torque-change criterion. Our neural network model basically uses a forward dynamics model, aninverse dynamics model, and a trajectoryformation mechanism, whichgenerates an approximate minimum torquechange trajectory It does not require spatial representation oftime or back propagation. Furthermore, there are lessiterations required to obtain an approximate optimal solution. Finally, our neural network model can be broadlyapplied to the engineeringfield becauseit is a new methodfor solving optimization problems with boundary conditions.
Keywords-Minimum torque-change model, Forward dynamics model, Inverse dynamics model, Optimizationproblem, Trajectory formation.
1. INTRODUCTION
To control voluntary movements, one must solve thefollowing three computational ill-posed problems: ( 1)desired trajectory determination, (2) transformationof the task-space coordinates of the desired trajectoryto intrinsic body coordinates, and (3) motor commandgeneration to achieve the desired trajectory. One wayto solve these problems is to introduce a smoothnessperformance index. Twoexperimentally confirmed objective functions for voluntary movement were proposed. Flash and Hogan (1985) proposed a mathematical model, the minimum-jerk model. They proposed that the trajectory followed by the subject's armstended to minimize the integral of the square of thejerk (rate of changeof acceleration)of the hand positionin Cartesiancoordinate space, integrated overthe entiremovement. The unique trajectory that yields the bestperformance is easily computed by applyingthe Euler-
Acknowledgements: Part of this work was done at ATR Auditoryand Visual Perception Research Laboratories. We would like to thankDr. Y. Uno for discussing this work. We wish to express our thanksto thank Dr. Y. Tohkura of ATR Human Information ProcessingResearch Laboratories for his encouragement. This work was supported by a Human Frontier Science Project Grant to Mitsuo Kawato.
Requests for reprints should be sent to Yasuhiro Wada, ATR Human Information Processing Research Laboratories, 2-2 Hikaridai,Seika-cho, Soraku-gun, Kyoto 619-02, Japan.
919
Lagrange equation because their model is based solelyon the kinematics of movement, independent of thedynamics of the musculoskeletal system. Several hardware models that can compute minimum-jerk trajectories have been proposed using recurrent neural networks (Jordan, 1989; Massone & Bizzi, 1989; Hoff &Arbib, 1994)'
Based on the idea that the objective function mustbe related to dynamics, Uno, Kawato, and Suzuki( 1989) proposed a minimum torque-change modelthataccounts for desired trajectory determination. Themodel is based on the theory that the trajectory of thehuman arm is determined so as to minimize the timeintegral ofthe square of the rate of torque change. Because the dynamics of the human arm or a roboticmanipulator are nonlinear, finding the unique trajectory based on the minimum torque-change model is anonlinear optimization problem. This is a rather difficult optimization problem because the smoothnesscriterion is represented in the motor command spaceon the one hand. On the other hand, movement conditions such as target point locations, via-points, andobstaclesare represented in task-oriented coordinates.Thus, the optimization problem is computationallyvery intensive to be solved using the Euler-Lagrangeequation. Hitherto, two kinds of hardware models havebeen proposed. To generate a trajectory based on theminimum torque-change model, Kawato, Maeda, Uno,
920 Y. Wada and M. Kawato
2. A NEURAL NETWORK FOR OPTIMALARM TRAJECfORY FORMATION
Here, (X, Y) are Cartesian coordinates of the hand,and [fis the movement time. Flash and Hogan (1985)showed that the unique trajectory yielding the best performance agreed with the experimental data on movement within the region just in front of the body. Theiranalysis was based solely on the kinematics of movement, independent of the dynamics of the musculoskeletal system, and was successful only when formulated in terms of hand motion in extracorporeal space.
Uno, Kawato, and Suzuki (1989) proposed the following alternative quadratic measure of performance.The objective function of the model is related to armdynamics:
and Suzuki (1990) proposed the cascade neural network, which is a cascade structure of the forward dynamics model (FDM). Conversely, a neural networkmodel for the minimum torque-change criterion thatuses the inverse dynamics model (IDM) was proposedby Nakamura, Uno, Suzuki, and Kawato ( 1990). Thereare three criticisms of these neural networks: (1) theyuse a spatial representation of time, (2) back propagation is essential, and (3) they require too many iterations to obtain an optimal trajectory. In this paper,we propose a new model for trajectory formation thatuses both the FOM and 10M. This model solves thethree shortcomings above, and can be implemented asa biologically plausible neural network. The proposednetwork model can be widely applied to the engineeringfield because it is a new method for solving generaloptimization problems with boundary conditions.
2.1. Minimum Torque-Change Criterion
This section briefly explains the minimum torquechange model. Trajectory formation is an ill-posedproblem because there is an infinite number of possibletrajectories by which the hand can move from the startto the target point. Therefore, a unique trajectory cannot be determined. However, humans can move thearm between two targets, selecting one trajectory fromamong an infinite number of trajectories. Therefore,the brain should be able to compute a unique solutionby attaching an appropriate criterion to the ill-posedproblem.
Flash and Hogan (1985) proposed the minimumjerk modelthat is based on the kinematics of movement,independent of the dynamics of the musculoskeletalsystem. Their proposed performance index is the following quadratic measure:
2.2. A Neural Network for Optimal Arm TrajectoryFormation Using Forward and Inverse Models
In this section, a new neural network model for trajectory formation is proposed. A performance index forthe movement between two targets is defined as thesum ofthe smoothness constraint energy multiplied by
where r! is the torque generated by the jth actuator ofM actuators, and {Jis the movement time. The objectivefunction is the sum of the square of the rate of changeof the torque, integrated over the entire movement. Theminimum torque-change model can predict and produce human arm trajectories quite well. The optimization problem is to find the torque that minimizes thecriterion CT' However, it is difficult to get an optimaltrajectory based on minimum torque-change becausetorque should be determined using complex nonlineardynamics. That is, a nonlinear optimization problemwith boundary conditions must be solved.
For movements between a pair oftargetsjust in frontof the body, predictions of both the models were closeto the experimental data. However, the trajectories predicted by the minimum torque-change model werequite different from the minimum-jerk model in fourbehavioral situations. It was found that the minimumtorque-change model predicted the actual data betterthan the minimum-jerk model (Uno et al., 1989). Thefour situations were as follows: (1) discrete point-topoint movement: The starting point is an outstretchedarm to the side and the end point is in front of thebody, (2) movements between two points whileresistinga spring, one end of which is attached to the hand whilethe other is fixed, and (3 ) vertical movements affectedby gravity. In these three cases, the minimum-jerkmodel always predicts a straight path regardless of external forces or gravity. On the other hand, the minimum torque-change model predicts a curved path andthese predictions are close to the experimental data.Finally, the most compelling evidence was examined:(4) a pair of via-point movements with identical startand end points, but with dictated mirror-image viapoints. Because the objective function CJ of the minimum-jerk model does not vary under translation, rotation and roll, the minimum-jerk model predicts anidentical path for roll as well as identical speed profilesfor the two subcases. On the other hand, the minimumtorque-change model predicts two different trajectories.For a concave path, the speed profile has two peaks.However, for a convex path, the speed profile has onlyone peak. These predictions are close to the humandata (Uno et al., 1989).
However, the two objective functions, CJ and CT ,
are closely related because the rate of torque change islocally proportional to the jerk. If the arm dynamicsare approximated by a point mass system, the two performance indexes are identical.
(1)
(2)l lf M(dr J )2Cr = L - di,o J=I dt
= itf {( d3X )2 (d
3y )2}CJ d 3 + d 3 di.ott
Arm Trajectory Formation
the regularization parameter A and three hard constraints:
1 N M . . 1 M . .
E ="2 A L: L: (r1 - r1_d2 + '2 :L: (8~ - 8~)2;-1 )-1 j=1
+i ~ (8~-O{,)2+i ~ ((j~_ij{,)2, (3))-1 j=1
where r{ is the torque generated by thejth actuator ofM actuators at the time i, The performance index isformulated here using discrete time i, and N shows thefinal time. Let 0, il, and 0denote the position, velocity,and acceleration of the joint angle respectively; 8~,
il~, and O~ represent the desired position, desired velocity, and desired acceleration of the jth joint anglerespectively. 8~, 8~, and O~ are the position, velocity,and acceleration ofthejth joint angle at the end timeas predicted by the neural network model. The firstterm of eqn (3) is simply a discrete version of the minimum torque-change criterion [eq n (2)]. The second,third, and fourth terms ofeqn (3) are hard constraintsregarding the movement conditions, that is, desirableposition, velocity, and acceleration at the end of themovement.
Here, the gradient descent of energy E [eqn (3)] iscalculated as follows:
dr{ _ 8E _ \) ) ) )d - - 8J - 1\(r 1+1 + r 1-1 - 2r1sri
921
Equation (5) shows the operation used to smooththe torque. The trajectory (0, il, 0) generated using thenew torque updated by eqn (5), does not usually satisfythe desired boundary conditions. Thus, an incremental,compensatory trajectory .6.0 is generated to cancel theerror in the terminal conditions. That is, .6.0 is generatedaccording to the following method, and the trajectoryo + .6.0 is obtained, that satisfies the terminal conditions. The trajectory .6.0 is computed by addressing alinear optimization problem instead of a nonlinear optimization problem. When generating .6.0, the controlobject dynamics are approximated by a simple linearsystem. Furthermore, the trajectory is determined bysolvingthe linear optimization that minimizes criterionCr, whose terminal conditions are the terminal condition errors induced by eqn (5). The trajectory 0 +.6.0 then satisfies the terminal conditions. However, thetorque 7 and trajectory 0 + .6.0 do not satisfy the original nonlinear arm dynamics (neither FDM nor IDM).Here, the FDM and IDM are defined as the followingequations:
FDM: 0;+1 = F(0;, r,.)
IDM: r; = 1(0;)
• _ I 2 U "'2 'M "Iwhere 0; - «(); , 8;, '" , () i , () i , 0; , ... , () i , 0 i ,"2 "u _ 1 2 U0i, .. ·,()j ),Ti-(Tj,Ti, ... ,ri)'
Accordingly, the torque that satisfies both terminalconditions and nonlinear arm dynamics is obtained byusing the IDM:
wheres represents a relaxation time that is independentof movement time. The torque 71 that minimizes theperformance index E can be searched according to eqn(4). The second, third, and fourth terms of eqn (4)show that a time-backward calculation (error backpropagation) (Rumelhart et al., 1986) is needed to satisfythe desired conditions. In optimal control, this calculation is equivalent to solving the adjoint equationof the Euler-Lagrange equation. However, if the terminal conditions are satisfied, the time-backward calculation (error back propagation) is not needed, because the second, third, and fourth term all vanish.Thus, if the terminal-condition errors can be madeequal to 0, the trajectory can be generated by justsmoothing the torque [the first term of eqn (4 )]. Assume, for simplicity, that a trajectory at some momentsatisfies the terminal conditions. In this case, the relaxation rule of eqn (4), that is, the gradient descentof energyE, is expressed as follows:
M OOk M ~k
+ L: (8~-Ot)----!!+ L (iJ~-iJ~)-2.k=1 Qr1 k=1 ar1
M '"k+ "\' (iF _ ijk)aO N
L, d Naj'k=1 r i
(4)
(5)
(6)
In this framework, the gradient descent of energy E iseasy to compute as it is simply expressed by the torquesmoothing term (eqn ( S) and error back propagationand spatial representation of time can be avoided.
The algorithm described above is shown in Figure1. Step 1: The torque and joint angle that satisfy theterminal condition and dynamics are calculated usingthe IDM, where 0 + .6.8 satisfies the terminal conditions. Step 2: The torque is smoothed. Step 3: The jointangle trajectory is generated from the torque smoothedin Step 2 through FDM. The trajectory does not satisfythe terminal conditions. The terminal-condition errorsare calculated. Step 4: Byfinding a solution to the linearoptimization problem, the compensatory trajectory ~0
which cancels the terminal-condition errors is obtained.The optimal trajectory based on minimum torquechange is obtained by repeating Steps 1-4.
Any initial trajectory can be chosen as the startingpoint of the calculation. However, if it is a good approximation of the minimum torque-change trajectory,faster convergence is expected. Therefore, two kinds ofreasonable initial trajectories are used; one is a trajectory based on the minimum-jerk criterion (Flash et al.,1985). The other is O. When 0 is used as the initialtrajectory, the IDM in the first iteration ofthe proposedschema outputs the torque 0 over time. Therefore, the
922 Y. Wada and M. Kawato
Step 3
FDM(Forward Dynamics Model)
HS(T)
Step 4 Step 2
Torque smoothing
Step 1
10M(Inverse Dynamics Model)
Approximated minimum torquechange for constraint error
S : smoothing operator
FIGURE 1. Neural network schema for arm trajectory formation using the forward dynamics model (FDM) and inverse dynamicsmodel (10M). Step 1 shows the 10Mwhose input is a trajectory that satisfies the boundary conditions, and output is a torque seriesthat satisfies the nonlinear arm dynamics. Torque is smoothed in Step 2. The terminal-condition errors are computed using theFOM (Step 3). In Step 4, the minimum torque-change trajectory for a linear apprOXimated dynamics model of the arm is derived.The output of Step 4 is the trajectory that satisfies the boundary conditions of the original nonlinear optimization problem.
hand position computed by the FDM in this iterationremains the initial position during movement time.Then, in the box "approximated minimum-torquechangefor constraint error" in Figure 1, a trajectory iscomputed by solving a linear optimization problemwith exactly the same boundary conditions as those ofthe nonlinear problem.
The smoothed torque is computed according to thenext equation, which is a discrete version of eqn (5).
. . dT~T~(S + 1) = 7~(S) + ds't..s
= des) + A(-r{+I(S) + .,.L(s) - 2·d(s», (7)
where lis is a time step of discrete time and assumedequal to 1.
In the following section, iteration of eqn (7) is usedas an example of the smoothing operation. The reasonfor this choice will be clarified in Section 5. Let k bean iterative computation index, the smoothed torqueis represented as follows:
des, k + 1)
= 71(s, k) + A(71+1(S, k) + 7~_I(S, k) - 2,d(s, k»
,d (s + 1) = 7~ (s, n + I) (8)
where k :=: 1,2, ... , n.Thus, as the number of iterative computations n in
creases, the torque becomes smoother. If that numberis quite large, the torque approaches 0 over time. Although the above smoothing operation was used in thecomputer simulation, we note that a variety of smoothing methods can be applied to the proposed algorithm.
3. COMPUTER SIMULATION OF DISCRETEPOINT-TO-POINT MOVEMENT
This section presents the results of applying the proposed method to 2-joint arm trajectory formation. Inthis simulation, the following mathematical model ofFDM and IDM was used. It has already been demonstrated that both the FDM and IDM can be achievedusing neural networks (Kawato et al., 1987, 1990; Kawato, 1990; Jordan et al., 1992).
71 = (II + h + 2M2L\S2COS ()2 + M 2(L I )2)0,
+ (/2 + M2L1S2COS ()2)82
- M 2LIS2(Z81 + 82)82sin ()2 + biOI (9)
72 = (I2 + M2 L IS2COS 82 )0\
+M2L,S2(O\)2sin02+b202' (10)
M], L, , S, and I, represent the mass, length, distancefrom the mass center to the joint, and the rotary inertiaof the link i around the joint, respectively. Here, thesame physical parameter values as those in Uno's paperare used (Uno et a1., 1989). b, and 7j represent thecoefficients of viscosity and the actuated torque of thejoint i. Joints 1 and 2 correspond to the shoulder andthe elbow. Joint 1 is located at the origin of the X- Ycoordinates.
Two kinds of approximated dynamic arm modelsarm are used to calculate the compensatory trajectory.l8 in the computer simulation. The first model is alinear approximated model along the previous iterationtrajectory. Thus, the approximated dynamics is described by a linear differential equation with timevarying coefficients. In this case, the optimal trajectory
Arm Trajectory Formation 923
is found by applying the Riccati equation (Bryson &Ho, 1975). The second model is a simple point-massmodel with time-invariant parameters and no interaction between the joints. The minimum torque-changetrajectory for this model is equivalent to the minimumjerk trajectory in the joint-angle-coordinate space. Thesecond model is a much poorer approximation thanthe first.
~I(O) = 0 Mtf) = M.
~I(O) = 0 ~I(tf) = t::.iJ 1
~2(0) = 0 h(tf) = M 2
b(o) = 0 ~2(tf) = t::.o2
1)1(0) = 0 1]1(tf) = t::.7 1
1)2(0) = 0 7/2 (tf) = t::.7 2 (13)
00000
a~(A) a~(A) a~(9) a~(A) a~(A) a~(A)
~ ao;- ae:- ao;- --a;:- a;;-ah(A) ali(A) ali(A) ali(A) ali(A) ali(A)ao:- fie;- ae:- ao;-~ BTZ
The linear approximated equation around the trajectory 0(t) = (81, 82 , 81,82 , 7 1, 72) , which is generatedby the smoothed torque 7 + S(7) = (71) 72), is described by eqn (12).
ddiX(t) = A(t)X(t) + BCt)U(t) (12)
XU) = (~ICt) ~2(t) ~ICt) ~2(t) TJl(t) TJ2(t))T
U(t) = (~I(t) ~2(t))T
ACt)
3.1. A Numerical Experiment Using the LinearApproximated Model
In this section, the optimality and convergence of thenew method are examined using the linear approximated model. First, the analytical computation methodof the compensatory trajectory based on minimumtorque-change, is described using the linear approximated model. Equations (9) and (10) are generallyrepresented by eqn ( 11).
d. . . .di O; =Ji(OI> O2, 01, O2, 7 1> T2) (I = 1,2). (11)
TABLE 1X-Y Coordinates of Initial
Target and Intermediate Points
A() I, 6.()2 represent the position errors at the end point,AO I , 6.02 represent the velocity errors at the end point,and A71' AT2 are the torque errors at the end point.These terminal condition errors are induced by thesmoothing operator S.
The minimum torque-change model is formulatedas the following optimization problem:
J = ~ f (UTQU) dt-« Min Q = (~ ~) (14)
The optimization problem of the linear system can besolved by applying the Riccati equation. As a result, acompensatory trajectory, that is an approximated minimum torque-change trajectory, is generated.
The simulation conditions were as follows: (1 )Movement time: 0.75 s; (2) sample time: 0.01 s; (3)trajectory: from T2 to T6 (the x-y Cartesian coordinates of initial and target points are shown in Table 1);and (4) the number of smoothing operation iterations:n = 100. The convergence of the minimum torquechange criterion is shown in Figure 2. The x-axis is thenumber of iterative calculations, the y-axis is the criterion function value [eqn (2)]. Two initial values wereused in this simulation. One was a trajectory based onminimum-jerk (Hash and Hogan, 1985). The otherwas equal to O. The optimal value of this problem canbe calculated using a Newton-like method, that is, aniterative scheme to solve the two-point boundary-valueproblem (Uno et al., 1989). It is assumed that a nearlyoptimal solution is obtained by the Newton-likemethod. The criterion function value for the proposedneural network model converged near the optimalvalue, and the converged value ofthe performance index
o
oo
o
o
o
o
oo
o 1
o 0
oo
o 0
o 0
o
o
o
o 0
BCt) = (~
o
Here, let ~l, b denote positions, ~1' ~2 denote velocities,and 1]1,1]2represent torque respectively, The subscriptdenotes joint number. The position ()j + ~i' the velocityOi +L and the torque 7j + 1]1, (i = 1,2) satisfy theboundary condition of discrete point-to-point movement.
Therefore, X (t) satisfies the following boundaryconditions:
T1T2T3T4T5T6PiP2
X (em)
-0.92-24.33-19.94
0.1521.0921.24
0.151.31
Y (ern)
30.3630.8947.1158.9249.3332.6358.9336.96
924 Y. Wada and M. Kawato
was obtained in less than 10 iterations. The convergedvalues for the two initial values were almost the same.Thus, the proposed method can produce a trajectoryclose to the minimum torque-change trajectory for theoriginal nonlinear arm dynamics in quite a small number of iterations when the minimum torque-change solution of the linear approximated model is used as thecompensatory trajectory.
FIGURE 2. Convergence of the value of the minimum torquechange criterion when the new proposed schema is applied toa T2-T6 movement in front of the body using a 2-joint arm. Inthis simulation, the linear approximated model with time-variantparameters is used as a model to compute the compensatorytrajectory for satisfying the boundary conditions. Two kinds ofinitial trajectory are examined. One is the minimum-jerk traleetory. The other corresponds to 0 (no movement). Both resultsare almost the same. Satisfactory solutions are obtained thatapproximate the optimal solution computed using the Newtonlike method.
a3 = {lO(~j(tJ) - ~;<O)) - 4~itJ)' tJ+ ~~)(tJ)' t}}lt}
a4 = {-15(~j(tJ) - ~;<O)) + 7k;<lj)' tr: VtJ)' t}}lt}
. r 2} 5as = {6(~itJ) - ~j(O)) - 3~;<tJ)' (J+ 2~)(tJ)' tJ Itf
where 'YJj, ~j> ~j, and ~j represent the torque, position,velocity and acceleration at the jth joint, respectivelyand I is the inertia of the link, a, (i = 0, ... , 5) areparameters that are determined by movement time andboundary conditions (position, velocity, and acceleration at the start and end point). The boundary conditions are the same as eqn ( 13); however, the boundarycondition for acceleration is given instead of the torqueconditions. In this simulation, mathematical equationsare used to obtain the compensatory trajectory. However, it is known that a recurrent neural network canlearn approximate optimal trajectories (Massone et aI.,1989; Jordan, 1989; Hoff et aI., 1992). Furthermore,Hoff and Arbib ( 1992) have shown that the rigorousminimum-jerk trajectory can be generated using a recurrent neural network.
The simulation conditions were as follows: (1)Movement time: 0.75 s (2) Sample time: 0.01 s (3)Trajectory: fivekinds ofmovement in front of the body(The start and end points are shown in Table I). Theparameters used in this simulation, and the number ofiterations to obtain a minimum for the objective function, are shown in Table 2. The trajectories of fivemovements, the speed profile of the T4-T1 movement,and the torque profile of the T4-T1 movement areshown in Figures 3,4, and 5, respectively. Each trajectory was obtained in less than 5 iterations. The valueof the criterion function attained at the first minimumpoint during the iterative computation is shown in Table 3. Each objective function value of the trajectorywas close to the optimal value calculated using theNewton-likemethod. The number of iteration to obtainthe minimum objective function value is almost thesame in the case using the linear approximated model,
-
-
+30
I
I
25
III
I I I
10 15 20
ITERATIONS
I
5
I
- initial state =minimum jerk trajectory
'.".' initial state = 0 (no movement)1.5
1. 6 tt"
1.3 t- \\",":-"':'.:::••~••~-------------,Iresult of Newton-like method I----
1.4
1 .2 c:1+:....-_..L..._---.l__...I:-_~:__-='='-___::':
o
>
"a:wzw
3.2. A Numerical Experiment Using aPoint-Mass Model
In this section, a simple point-mass model is used. Thismodel is expressed as eqn ( 15). Here, the minimumtorque-change trajectory of a point-mass model [eqn( 15)] is represented as the 5th order polynomial intime [eqn ( 16)] , which is easily derived using the EulerLagrange equation. Because, in this case, the change intorque is exactly proportional to the jerk of the jointangle, eqn ( 16) is equivalent to the optimal solution ofthe minimum-jerk criterion in the joint-angle-coordinate space.
1]) = J)~) ( 15)
~j(t) = ao + a.t + azt2 + a3t3 + a4t4 + astS (16)
ao = ~JCO) at = 0, a2 = 0
TABLE 2Computer Simulation Parameters and Number of IterationsRequired to Calculate a Minimum for the Objective Function
of Each Movement
Number of Number ofMovement A Smoothings Iterations
T2-T6 0.3 30 5T3-T6 0.3 60 1T1-T3 0.3 30 2T4-T1 0.3 30 3T4-T6 0.3 30 1
Ais the smoothingoperator parameter. Numberof smoothingsshows the number of iterative computations in the smoothing operation. Number of iterations shows the number of iterative computations required to obtain a minimum for the objective functionpoint in the proposed model for trajectory formation.
925
........... jerk method
- new method
... ,- Newton-like method
2
3
·2 Itorque 11 - new method
...... Newton-like method
-3 .......... jerk method
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
~
EzW:::J 0tF------.:"-"'=~~~~-----~oa:oI- -1
0.40.2-0.20.0�ooo_ 1..-__--"'-__--1 ---1
-0.4
Arm Trajectory Formation
0.8...... Newton-llke method
[ill0.6
[!I]
:[ 0.4>-
lID[IT) l!IJ
0.2
FIGURE 3. Trajectory of discrete point-to-point movements. Fivekinds of trajectory in front of the body using a 2-joint arm areshown. The origin of the X- Y coordinates represents the location of joint 1 (shoulder). The trajectory generated by theproposed method based on the minimum torque-change criterion, and that of the Newton-like method are compared.Unlikeminimum-jerk trajectories, neither trajectory is straight, butslightly convex.
TIME(s)
FIGURE 5. Torque profile of a T4-T1 movement in front of thebody. The torque profiles of the proposed model, the Newtonlike method which is an optimal profile with respect to the minimum torque-change model, and the minimum-jerk model arecompared. Torque 1 and torque 2 represent shoulder and elbowtorque, respectively. The profiles of the proposed model andthe Newton-like method are smoother than that of the minimumjerk model.
TIME(s)
FIGURE 4. Speed profile of a T4-T1 movement in front of thebody. The speed profiles using the proposed model, the Newton-like method which is an optimal profile with respect to theminimum torque-change model, and the minimum-jerk model,show a single-peaked, bell-shaped speed profile that agreeswith the speed profile observed in human arm movement. Theprofile of the proposed method is most similar to that of theNewton-like method. Thus, this method can generate a trajectory based on minimum torque-change.
4. AN EXTENSION TOVIA-POINT MOVEMENT
and the objective function value calculated using thelinear approximated model was closer to the optimalvalue. However, the values calculated using the pointmass model and the linear approximated model werenot so different. Each trajectory generated by the proposed method agreed with the hand paths calculatedusing the Newton-like method. For horizontal movement between two targets located approximately infront ofthe body, the minimum torque-change criterionpredicts hand paths that are not completely straight,that is, slightly convex. On the other hand, the handpaths based on the minimum-jerk model are completelystraight. Both models can also predict a single-peaked,bell-shaped speed profile of the hand. Accordingly, thehand paths generated by the proposed method wereslightlyconvex (Figure 3). This method also produceda single-peaked, bell-shaped speed profile (Figure 4).The torque produced by the proposed method wassmoother than that of the minimum-jerk model (Figure 5).
4.1. A Point-Mass Model for Via-Point Movement
The proposed method is extended to generate a trajectory with via-points. In this case, the position con-
- new method
........... Newton-llke method
..... jerk melhod1.0
1.2 IT4-T11
Ul 0.8.....E>:!:: ,o 0.6 .0 .....I .W ..>
0.4 ,./,
0.2 ...
0.0 "0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
926 Y. Wada and M. Kawato
TABLE 3Value of the Minimum Torque-Change Criterion
MovementProposedMethod Newton-Like
MinimumJerk
T2-T6T3-T671-T3T4-T1T4·T6
1.3741.183
3.164 x 10-1
1.920 X 10- 1
7.515 X 10-1
1.2291.131
3.051 x 10-1
1.589 X 10- 1
7.156 X 10-1
1.5731.184
3.227 X 10-1
2.968 X 10- 1
7.814 X 10- 1
(t8 )
where
I ... 2= -32 (IOLl8via - IOLlOo - IOLlOO/via - 2A8ot via) ' (24)lvi.
(22)
(21 )
aJ- ..-=0.aLlOvia
r (d~)2J( AOvia, Aita) = [2 0 dt dt
= [' r (6b3 +24b4t + 60bst2
) 2 dt. (20)
For LiOviaand D-.Ovia to minimize eqn (20), the folIowingconditions are necessary:
Therefore, the velocity and acceleration at via-time areobtained as follows:
. _ I ... ,A8via - - (lOAOvia - IOA8o - 6A8otvia - AOotvia) (23)
4tvia
~(O) = ABo ~(tvia) = M via
~(O) = AOo ~(tvia) = AOvia
~(O) = AOo ~(tvia) = AOvia.
D-.0o, b.80 , and D-.80 represent the position, velocity, andacceleration errors for the boundary conditions, respectively, at time O. D-.{;Ivia, D-.Ovia, and D-.Ovia representthose errors at time V. However, the values of b.Ovia andLi8viaare not yet specified at this point and will begivenbelow. A solution to eqn ( 18) is given by eqn ( 19) similar to eqn (16).
W) = bo + bit + b2t' + b3t3 + b4t
4 + bst S (19)
where hi (i = 0, ... , 5) could be determined if theabove six constraints were to be given. Here, the torquechange criterion function of the point-mass model isexpressed as eqn (20). Since the values b.Ovia andD-.8viaare not given, eqn (20) is a function of the velocityand acceleration at via-time.
That is, the via-point time can be chosen and the veloeity and acceleration computed, so as to minimizethe torque-change criterion function for the approxi-
straints at the via-points are in addition to the problemof discrete point-to-point movement. The performanceindex of a via-point movement is defined as follows,instead of eqn (3). Trajectory formation using onlyone via-point is explained, however, it is easy to extendthis to a movement with several via-points.
l N M 1 M
E = 2. ALL (T~ - T~_1)2 + - L (8~ - 0-:V)2/-1 j-1 2 j_1
+ ~ ~ (O~ - 0-:V)2 + ~ ~ (O~ - (j~)2j=1 j~1
1 ~ . j 2+"2 LJ (O~ia - 0 v), (t7)j~1
where 8{ia represents the desired position at a via-pointof the jth joint angle. Oiv represents the position at Vtime (1 :s V s; N - 1) ofthejthjoint angle. There aretwo differences between discrete point-to-point movement and via-point movement. First, the movementtime between the start and the via-point is not given;however, the movement time is given for discrete pointto-point movement. Second, the velocity and acceleration constraints at the time passing through the viapoint are not given. Thus, conversely, if the time passingthrough the via-point, the velocity, and the accelerationat the via-point can be determined, a trajectory withthe via-point can easily be generated using the samemethod for discrete point-to-point movements.
Here, the algorithm shown in Figure I is extendedfor via-point movements. The via-point time V for thecompensatory trajectory is chosen so that Lf!,l (O{ia {;IJV) 2 in eqn (17) is minimized after Step 3 (FDM) inFigure 1. The point-mass model is used as the approximate dynamics model for generating the compensatorytrajectory. The minimum torque-change trajectory ofthe point-mass model is expressed as eqn ( 16). Thus,the trajectory from the starting point to a via-pointcould be calculated if the movement time, position,velocity, and acceleration at the start and the Via-pointwere to be given. The torque-change criterion valuefrom time 0 to tvia when passing through the via-pointis expressed as follows:
rrvJ. (d~)2J = [2 Jo dt dt - Min
Arm Trajectory Formation 927
0.8
0.2-new method- - - Newton-like method.......... ierk method
0.40.20.0X (m)
-0.2-0.4
0.0
of the via-point. In this case, the two via-points PI andP2 were located symmetrically with respect to the lineconnecting the common start and end points. Theminimum-jerk model predicted identical speed profilesfor both cases; however, the minimum torque-changemodel predicted two different profiles: that for via-pointPI had only one peak; however, that for P2 had twopeaks (Uno et aI., 1989). The speed profile for viapoint P2 predicted by the proposed method had twopeaks, as shown in Figure 7. These simulation resultsshow that the proposed method for via-point movementcan generate approximately a trajectory passingthrough via-points based on minimum torque-changecriterion in only several iterations. We emphasize thatthe objective functions obtained by the proposedmethod are much smaller than those of the minimumjerk model (Table 5). If one compares Table 3 andTable 5, it is suggested that the new method is moreefficient for complex movements than it is for simplemovements.
0.6
.s 0.4>-
FIGURE 6. Trajectory of via-point movement in front of the bOdy(T3-P1-T5 and T3-P2-T5). The trajectories of the proposedmethod and the Newton-like method, which are based on theminimum torque-change criterion, are compared to that of theminimum-jerk model. The three trajectories are almost identical.
4.2. Numerical Experiments
The simulation conditions were as follows: ( 1) Movement time: 1.0 s; (2) sample time: 0.01 s; and (3) startpoint T3, end point T5, and via point PI or P2 (Thex-y Cartesian coordinates of each point are shown inTable I). The iteration number for smoothings, thesmoothing parameter A, and the number of iterationsneeded to reach the first minimum for the objectivefunction, are shown in Table 4. The minimum-jerktrajectory in the Cartesian coordinates was chosen asthe initial trajectory. The trajectories of the two movements, and the speed profiles of T3-P2-T5 are shownin Figures 6 and 7 respectively. Minimal values of thecriterion function for the three schemes (proposedmethod, Newton-like method, and minimum-jerkmodel) are shown in Table 5. Each trajectory was obtained in less than five iterations, and the minimumvalue of the criterion function was close to the optimalvalue obtained using the Newton-like method. Thehand paths generated using the proposed method werealmost the same as the hand paths of the Newton-likemethod (minimum torque-change model), and theminimum-jerk model. These two models can predicta curved hand path with a single-peaked or doublepeaked speed profile, and this depends on the location
TABLE 4Computer Simulation Parameters and Number of IterationsRequired to Calculate a Minimum for the Objective Function
of Each Movement
mate model. Furthermore, the algorithm describedabove can be applied to trajectory formation with morethan one via-point. Because the algorithm can determine the velocity and acceleration errors at the firstvia-point using only the error of the position, velocity,and acceleration at the start point, the velocity and acceleration errors at the second via-point can be similarlydetermined from errors at the first via-point. Equationssuch as (23) and (24) can be derived straightforwardly.The reason for this straightforward extension to multiple via-point cases is that only the objective functionfrom the start point to the first via-point is consideredin eqn (20) and the latter half of the integral is nottaken into account.
Ais thesmoothing operatorparameter. Number of smoothingsshowsthenumberof iterativecomputations in the smoothing operation. Number of Iterations shows the numberof iterative computations required to obtaina minimum for the objective functionin the proposed model for trajectory formation.
Movement
T3-P1-T5T3-P2-T5
0.30.3
Number ofSmoothings
3030
Number ofIterations
33
5. MATHEMATICAL CONSIDERATIONS OFTHE PROPOSED METHOD FOR
NONLINEAR OPTIMIZATION PROBLEMS
In this section, the proposed network is formulated asa general optimal algorithm, and the theoretical framework for this method is described. The optimality andconvergence of the new method applied to a nonlinear
928 Y. Wada andM. Kawato
1.2 _--.,..-----r----.,...---,---.....
TIME(s)
FIGURE 7. Speed profile of a T3-P2-T5 movement. The speedprofile based on the minimum-jerk criterion has only one peak,and in a T3-Pi-T5 movement it also has only one peak. Thespeed profile based on the minimum torque-change criterionhas one peak in a T3-Pi-T5 movement, however, in a T3-P2T5 movement, it has two peaks that correspond to the handtrajectory observed in human arm movement. Thus, in a T3P1-T5 movement, the speed profiles based on the two kindsof criteria are not particularly different, but in a T3-P2·T5 theyare quite different, as shown in this figure.
where x and u represent a state variable and a controlvariable, respectively. xdrand trrepresent a desired terminal value and an end time, respectively. It is assumedthat the nonlinear function fis differentiable with respect to x and u.
Let us first illustrate the iteration rule of the controlvariable by generalizing the neural network model proposed in Section 2 as follows. We define an inverse dynamics model G, that is an inverse function of the forward dynamics model F.
G(X) = u,
where (x, x) = X = F(u).The control variable at the j + 1th iteration, as shown
in Figure 1, is output by IDM, whose input is computedby adding the trajectory generated by the smoothedtorque at the jth iteration to the compensatory trajectory generated by an approximate linear dynamicsmodel. Thus, the control variable at the j + 1th iterationis derived as follows:
u j +1 = G(F(u j + S(u j » + Z), (29)
where u! + S( u j ) is the smoothed torque, and E = (~,
~) is the trajectory that compensates for the terminalcondition errors induced by u! + S( u j
) and is the solution for the linear optimization problem, defined below. The following equation is obtained using Taylor'sexpansion of function Fin eqn (29) around u',
u j +1 = G( F(u j ) + a~~tl) S(u j ) + o(S(uj » + Z) .Taking Taylor's expansion of G around X! = F(u i ) :
+1 . aG(Xj) (aF(U j) . . )
u! = ul + --ax-~ S(u J) + o(S(uJ»
aG(Xj) .....+----ax-~
. . aG(Xj) . aG(xJ)= u! + S( uJ
) +--ax- o( S( uJ» + ----ax E,
where a higher order term than the second order termis ignored because the trajectory change between thejth and thej + lth iteration is usually quite small. Thefourth term in the above equation is expressed as -'Y/j.1]) is obtained as the optimal solution to the followinglinear problem. If S( u j
) is small, the following equationis derived because o(S( u j ) ) becomes negligibly small.
u j +1 = u! + S(u j ) - 7/j •
(25)
(26)
(27)
(28)
1.00.80.60.4
dxdt = lex, u),
x(O) = 0, X(lf) = Xd!,
u(O) = 0, u(tf) = 0,
0.2
- new method. - - Newton-like method......... Jerkmethod1.0
0.4
0.0-----'----'----'----......--...0.0
0.8
0.6
0.2
Subject to
optimization problem is discussed mathematically inSections 5.1 and 5.2. The following discussion is forthe scalar case, but it can easily be extended to multivariable cases.
First, we define the following nonlinear optimizationproblem which minimizes the criterion function J under boundary conditions:{Nonlinear Optimization Problem: N]
J(dU)2J= dt dt >» Min
In
E>!::oo..JW>
TABLE 5Value of the Minimum Torque·Change Criterion
MovementProposedMethod Newton-Like
MinimumJerk
T3-P1-T5T3-P2-T5
3.850 X 10-1
4.348 X 10-13.168 X 10-1
3.322 X 10-16.709 X 10- 1
6.323 X 10-1
Arm TrajectoryFormation 929
However, it is not assumed that S( uJ) is small in thecomputer simulation and mathematical considerationdescribed below. Therefore, we define the iteration ruleof the control variable as follows:
where,
HO) = 0, WJ) = t1x;
1/(0) = 0, 17(f:r) = S(U*)(tf)' (36)
(30)
where S(u i) is siur, + [oG(Xi)/aX]o(S(u i».Note that the two following trajectories satisfy ex
actly the same boundary conditions.
aF(u j ) . .-- S(u}) + o(S(u}»
au,..,
-><,.....
The linear optimization problem is defined to compute a trajectory that compensates for boundary conditions.[Linear Optimization Problem: L]
J~ = J(~;r dt ... Min, (31)
Subject to
d~ aj(x j , u j ) alex}, uj )-;Ii = ax ~ + au 1/ (32)
~(O) = 0, WJ) = ~xj(tf) (33)
11(0) = 0, 1/(tJ) = S(Uj)(tf)' (34)
Here, xi, u! are the trajectory and the control variableat the jth iteration, respectively. ~xj(tJ) shows the terminal-condition errors when the input u! + S( ui ) isfed to the nonlinear dynamics eqn (26). Let (u*, x*)be an isolated optimal solution that minimizes the criterion function J, and let (1/*i, ~*J) be an isolated optimal solution that minimizes the linear optimal problem L. Thus, ~*i is the compensatory trajectory.
5.1. Optimality of the Converged Solution
In this section we will show that the convergence of theproposed algorithm is equivalent to the optimality ofthe solution. First we discuss the necessary condition:if u! is equal to u* (u l =: u*), then 'IJ*lbecomes equalto S(u"') (1/*i = S(ui)), thus the iteration converges.That is, if the control variable is equal to the optimalvalue, the iterative algorithm eqn (30) converges. Conversely, we then discuss the sufficient condition: if S( ul )is equal to 'IJ*i ('IJ*i = S( ui)), then u! becomes equalto u* (u l =: u*). That is, if the control variable converges, the control variable u! becomes the optimal solution.
A linear dynamic equation around the optimal solution is represented by the following equation:
d~ _ al(x*, u*) + al(x*, u*)-;Ii - ax ~ au '11, (35)
It is evident that the above necessary condition isequivalent to the following Lemma 1.
LEMMA 1. The optimal solution 1/* lor the linear optimization problem [L] at xi = x* is equal to S(u*).Proof We use reductio ad absurdum. Assume that theoptimal solution 1/* for [L] is not equal to S( u*)( 1/*+ S( u* )). The following shows that this assumptioncontradicts the assumption that u* is the optimal solution for [N]. The following new control variable isconstructed for [N].
tl=u*+e(S(u*)-71*) (Iel~l). (37)
If e is small, a- u" is approximated by the solutionto the linear eqn (35) with the input f(S(U*) - 1/*).Furthermore, the boundary conditions for S(u*) and1/* are exactly the same with that of [L]: according toeqn (36). Therefore, i1 satisfies the boundary conditionseqns (27) and (28), x(O) = 0, x(tJ) = Xd!, u(O) = 0,and U(lJ) = O. If the following inequality is shown, thisproof is completed, as the inequality contradicts theassumption that u* is the optimal solution for [N].
J(d£i)2 J(dU*)~dt dt < dt dt.
The next equation is obtained by ignoring the term E2•
f(~~Y dt= fl~{u* +E(S(U*)-71*nY dt
;; f(d~t* Ydi
+2 fdU*(dS(U*)_dll*)d (38)€ dt dt dt t.
Let I be an integral of the second term on the righthand side of eqn (38). e's sign can be determined according to I's sign. That is,
if I > 0 then € < 0
if I < 0 then e > O.
Thus the required inequality can be derived.Here, if I=: 0, because [dS( u*)/dt] - (d1/*/ dt) is notalways equal to °over time ('.: 1/* +S( u*)), i1 is alsooptimal to the first approximation. Therefore, U* doesnot become an isolated extreme value in problem [N],because the first variation ofthe criterion function corresponds to 0 in this case. (Q.E.D.)
The following Lemma 2 is equivalent to the sufficientcondition.
LEMMA 2. IIS( u i) is equal 1/*i (7/*i = S( ui)), then u!becomes equal to u* (u l =: u*).
(40)
930
Proof We use reductio ad absurdum again. Let us assume that the control variable u! is not the optimalsolution for [N] (u l =1= u*), even when the iterationconverges.
The following new control variable is constructedfor [L].
~ = S(u l ) + €(u" - u'), (39)
First, it will be shown that 7j satisfies the boundary conditions of [L]. The next equation shows the dynamicequation at the optimal point.
dx"----;j[=j(x*, u*).
Equation (41 ) is the linear approximated equation ofeqn (40) around u l where u* - u ' ,
d~ = aj(xl , til) ~ + aj(x}, ul) (u* _ ul) . (4\)m ax au
Then, both u* and u! satisfy the final conditions x(tj)=xdjof the dynamic eqn (26) , so ~, that is, the difference between the trajectory generated by u" and thatby u', satisfies the condition ~(tf) = 0. Furthermore,~ satisfies the boundary conditions ofeqn (33) becauseS( ul ) satisfies the boundary conditions of eqn (33).Moreover, 7j satisfies the boundary conditions of eqn(34) according to the following conditions:
1l*(0) = u*(tf} = lll(O) = Ul(tf) = O.
Equation (43) can be derived by using eqn (42) andignoring the term e2 •
d~ = dS(lIl) + €(dU* _ dUl) (42)
dt dt dt dt
f (~;r dt ~ f (dS~tUl)r dt
f dS(ul) (dU* dllJ ) d+2€ -d-t- &-dt t . (43)
Let I be an integral of the second term on the righthand side of eqn (43) . E'S sign can be determined according to I's sign in the same manner as the previousproof.
if J > 0 then e < 0
if J < 0 then E > o.Therefore , the required inequality is obtained:
f (~;r dt < f (dS~~J)r dt .This contradicts the basic assumption that r;* l =S(u l)is optimal for [L].If! = 0,
dll* du l---
dt dt'
is not always equal to 0 over time. Therefore, 'rj....J
Y. Wada andM. Kawato
does not become an isolated extreme value for[L]. (Q .E.D.)
5.2. Convergence of the Solution
This section discusses the monotone convergence ofthe criterion function . We ask whether the proposedalgorithm decreases the criterion function for every iteration (J (u l +1
) ~ J(ul ».The criterion function at the j + 1th iteration is
calculated using eqn (30 ).
J( u l + l) - J( u l )
= 2 J(duJ + dS(ul))(dS(U
J) _ d7l l) dt
dt dt dt dt
- J(dS(Ul) + d7l l)(dS(UJ
) _ d7l l) dt. (44)
dt dt dt dt
Also, the smoothing operator S in Sections 2, 3, and 4is represented in continuous time as follows:
. d2u{k)U{k+l ) = uhl + A----;R2 (k = 1,2, . .. , n A < I) ,
where n is the number of smoothings. For very largen, we assume that u! + S( ul ) is quite small. This isthe most important condition for convergence. We notethat this is not a reasonable assumption when the optimal solutions to [N] and [L] are very different. Onthe other hand when u! has not converged yet, then
S(U J} - 71 l ", O [·:eqn(30)].
Furthermore, S( u l ) + 7]j = u! + S( ul ) - (u! - 7]1) isnot small since u! - 7]1 is not smalL Because the firstterm of the right-hand side of eqn (44) becomes negligibly small compared to the second term, it Can beignored.
J( u l+! ) - .l(ul ) =J(~7 dt> J(dSd:1i)y dt ,
The following inequality holds because r;1 is optimalfor [L]
f (~lr dt - f (dS~tUl)r dt ~ O.
Finally, the warranted inequality that shows the monotone convergence is derived.
J(U)+I ) ~ J(uJ) .
Because J(u) is lower bounded, it converges to theminimum value.
6. DISCUSSION
A new neural network model for trajectory formationwas developed which basicallyuses a forward dynamicsmodel, an inverse dynamics model, and an approximate
Arm Trajectory Formation
TargetMovement I-ill-[ End
velocityTime (start)Forward f+ positionacceleration Dynamics velocityModel I. accelera tion
Target (final) + i(Time?~l4rlacceleration olJ
velocity .::l-... -'- 6-~¢-'
via-position "'"Via-points SpositionI~~ 1a.timevia-position Time Torque
•r , Search Smoothing
Approximated MinimumTorque Change Model Cl.l
'& =r E'
acceleration ~+~ Inverse S
velocityDynamics ~
,..:f- + Modelposition 1-
~----_._-----_._.__._-_..._....-...._.__..__..._-_..._-_......._..._..-.._-_.
931
FIGURE 8. Neutral network structure of arm trajectory formation. The five main parts, that is, the forward dynamics model, inversedynamics model, approximated minimum torque change model, torque smoother and via-points Time Search, can all be implementedas neural networks. In this tigure, the tlow of the signals is illustrated in detail. The inputs of the proposed neural network arepositions, velocities and accelerations at the start and the end times, movement time, and via-point positions. The position, velocity,and acceleration time courses that satisfy the boundary conditions are fed Into the Inverse dynamics model and it outputs thetorque. The smoothed torque is ted into the forward dynamics model. It outputs the time series of the velocity and acceleration,and then the errors of the position, velocity and acceleration at the end point are computed when compared with the desired values.The position and velocity at the end point are calculated using integral operations from the start time to the end time. The via-pointpositions and via..point passing times are solved in the vla·polnts time search. All the boundary condition errors are fed intoapproximated minimum torque change model. It outputs the compensatory trajectory. Finally, the position, velocity, and accelerationthat satisfy the boundary conditions are computed by adding the compensatory trajectory to the output trajectory of the forwarddynamics model. t1Tshows the time step in computation.
minimum torque-change trajectory generation mechanism. This model can solve the three difficult criticisms relating to the cascade, and other neural networks. These are: ( I ) they use a spatial representationof time, (2) back propagation is essential, and (3) toomany iterations are required to obtain the optimal trajectory. However, this model does not use a spatial representation of time. It does not require back propagation in iterative computation. It needs only an approximate linear model and an IDM that is an inversefunction of the FDM to satisfythe boundary conditions.Furthermore, this model can reach an optimal solutionin a few iterations as shown in the computer simulation.Accordingly, the new neural network model solves thethree criticisms of the previous models.
In the mathematical proofand computer simulation,a smoothing operator that needs many iterative calculations was used. Such an iterative smoothing operator is not necessary, however, and one-shot-typesmoothing filters can easily be designed.
We emphasize that the entire model can be implemented using neural networks. The model consists offive main parts; an FDM, an IDM, an approximatedlinear model, a smoother, and a via-point seeker. Asalready mentioned, an FDM can be obtained usingJordan's recurrent neural network (Jordan et al., 1992).Kawato et al. (Kawato et al., 1987;Kawato, 1990) have
already reported that an IDM can be obtained by neuralnetwork learning. The torque smoother and via-pointseeker are simple enough to be implemented as neuralnetworks. In the case of the point-mass model, theminimum torque-change trajectory can be expressedas the minimum-jerk trajectory in the joint angle space.Hoff and Arbib ( 1992) have already pointed out thata minimum-jerk trajectory can be obtained using a recurrent neural network when the final velocity and acceleration are equal to O. It is easy to extend their modelto general conditions, so that the velocity and acceleration at the start and end points are not necessarilyO. These conditions are needed in via-point trajectoryformation. That is, the minimum-jerk trajectory underthe above conditions can be defined using the followingdynamics:
ddt 0 = A0 + Be v . (45)
A=[ ~ 6 ~]-60/D3 -36/D 2 -9/D
B=[ ~ ~ ~]60/D 3 -24/D2 3/D
0== (e e ~)T
e, == (eo eo 80 ) T,
932
where v shows the end point and Bv , O,H and ev definethe given position, velocity, and acceleration at the endpoint. D represents the remaining movement time. Accordingly, when Ov = 0 and 8v= 0, the above equationcorresponds to the dynamic equation proposed by Hoffand Arbib ( 1992). Thus, the point-mass model can beobtained using a recurrent network in the same manneras Hoff and Arbib. The detailed model for trajectoryformation and its five main parts obtained usingneuralnetworks, are shown in Figure 8.
The new method is a general method for nonlinearoptimization problems with boundary conditions. Ithas several advantages in engineering applications. Forexample, it does not require an inversion of matricesin iteration, it reaches an optimal solution in a shorttime, and the output after passing through the IDMalwayssatisfies the boundary conditions and dynamicsequation. Therefore, it can be extended to many otherengineering problems.
In the future, the potential applications of this proposed model to a minimum muscle-tension-changemodel and a minimum motor-command-change modelwill be studied. The efficiency of the proposed modelwill be checked when the IDM is not a perfect inversefunction of the FDM. Furthermore, it is expected thatthis trajectory formation model can be used as a patternrecognition network because a kind of duality existsbetween pattern formation and recognition in thisframework (Kawato, 1989).
REFERENCESBryson, A. E., & Ho, Y. C. (1975). Applied optimal control. New
York: Wiley.
Y. Wada andM. Kawato
Flash, T., & Hogan, N. (1985). The coordination of arm movements:An experimentally confirmed mathematical model, The Journalof Neuroscience, 5(7), 1688-1703.
Hoff, B., & Arbib, M. A. ( 1992). Models oftrajectory formation andtemporal interaction of reach and grasp. Manuscript submittedfor publication.
Jordan, M. 1., & Rumelhart, D. E. (1992). Forward models: Supervised learning with a distal teacher. Cognitive Science, 16(3), 307354.
Jordan, M. 1. (1989). Indeterminate motor skill learning problems.In M. Jeannerod (Ed.), Attention and performance, XIII. Cambridge, MA: MIT Press.
Kawato, M., Furukawa, K., & Suzuki, R. (1987). A hierarchicalneural-network model for control and learning of voluntarymovement, Biological Cybernetics, 57, 169-185.
Kawato, M. ( 1989). Motor theory of speech perception revisited fromminimum torque-change neural network model. Proceedings ofthe8th Symposium on Future Electron Devices. 141-150.
Kawato, M., Maeda, Y., Uno, Y, & Suzuki, R. (1990). Trajectoryformation of arm movement by cascade neural network modelbased on minimum torque-change criterion, Biological Cybernetics, 62, 275-288.
Kawato, M. (1990). Computational schemes and neural networkmodels for formation and control of multijoint trajectory. In T.Miller, R. S. Sutton, & P. J. Werbos (Eds.), Neural networks forcontrol (pp. 197-228), Cambridge, MA: MIT Press.
Massone, L., & Bizzi, E. ( 1989). A neural network model for limbtrajectory formation. Biological Cybernetics, 61,417-425.
Nakamura, M., Uno, Y, Suzuki, R., & Kawato, M. (1990). Formationof optimaltrajectory in arm movement using inverse dynamicsmodel(Technical Report NC89-63) Japan: IEICE (in Japanese).
Rumeihart, D. E., Hinton, G. E., & Williams, R. J. ( 1986). Learningrepresentations by back propagating errors. Nature, 323 (9), 533536.
Uno, Y., Kawato, M., & Suzuki, R. (1989). Formation and controlof optimal trajectory in human arm movement-minimum torquechange model. Biological Cybernetics, 61, 89-101.