Movement Imitation with Nonlinear Dynamical Systems

in Humanoid Robots

Auke Jan Ijspeert & Jun Nakanishi & Stefan SchaalComputational Learning and Motor Control Laboratory

University of Southern California, Los Angeles, CA 90089-2520, USAATR Human Information Systems Laboratories, Kyoto 619-0288, Japan

Email: ijspeert@usc.edu, nakanisi@rubens.usc.edu, sschaal@usc.edu

Abstract

This article presents a new approach to movement plan-ning, on-line trajectory modification, and imitation learn-ing by representing movement plans based on a set ofnonlinear differential equations with well-defined attrac-tor dynamics. In contrast to non-autonomous movementrepresentations like splines, the resultant movement planremains an autonomous set of nonlinear differential equa-tions that forms a control policy (CP) which is robust tostrong external perturbations and that can be modifiedon-line by additional perceptual variables. The attractorlandscape of the control policy can be learned rapidly witha locally weighted regression technique with guaranteedconvergence of the learning algorithm and convergence tothe movement target. This property makes the systemsuitable for movement imitation and also for classifyingdemonstrated movement according to the parameters ofthe learning system.

We evaluate the system with a humanoid robot simu-lation and an actual humanoid robot. Experiments arepresented for the imitation of three types of movements:reaching movements with one arm, drawing movements of2-D patterns, and tennis swings. Our results demonstrate(a) that multi-joint human movements can be encodedsuccessfully by the CPs, (b) that a learned movementpolicy can readily be reused to produce robust trajecto-ries towards different targets, (c) that a policy fitted forone particular target provides a good predictor of humanreaching movements towards neighboring targets, and (d)that the parameter space which encodes a policy is suit-able for measuring to which extent two trajectories arequalitatively similar.

1 Introduction

This article is a continuation to our quest for robust move-ment encoding systems to be used with real and simulatedhumanoid robots. In [1, 2], we identified five desirableproperties that such systems should present: 1) the easeof representing and learning a desired trajectory, 2) com-pactness of the representation, 3) robustness against per-turbations and changes in a dynamic environment, 4) easeof re-use for related tasks and easy modification for newtasks, and 5) ease of categorization for movement recog-nition.

Our approach to fullfil these properties is to encode de-sired trajectories, or more precisely complete control poli-cies (CPs), in terms of an adjustable pattern generatorbuilt from simple nonlinear autonomous dynamical sys-tems. The desired trajectory becomes encoded into theattractor landscape of the dynamical system as a trajec-tory from an initial state to an end state, in a way whichdoes not require time-indexing and which is robust againstperturbations. Locally weighted regression is used to learnthe trajectory in a single shot. In contrast to other ap-proaches in the literature (see a review in [1]), the dynam-ical systems encode desired trajectories, not motor com-mands, such that an additional movement execution stageby means of a standard tracking controller (e.g., inversedynamics controller in our work) is needed. This strategy,however, does not prevent us from modifying the trajec-tory plans on-line through external variables, as will bedemonstrated later.

We apply the CPs to a task involving the imitation ofhuman movements by a humanoid simulation and a hu-manoid robot. This experiment is part of a project inrehabilitation robotics —the Virtual Trainer project —which aims at using humanoid rigid body simulations andhumanoid robots for supervising rehabilitation exercisesin stroke-patients. This article presents three experimentsdemonstrating how trajectories recorded from human sub-jects can be reproduced using the CPs. The purpose ofthe experiments is three-fold. First, we quantify how wellthese different trajectories can be fitted with our CPs. Sec-ond, we estimate to which extent a CP, which is fitted toa particular reaching movement, is a good predictor ofthe movements performed by the human subject towardsneighboring targets. And third, we compare the parame-ters of the dynamical system fitted to different trajectoriesin order to evaluate how similar the trajectories are in pa-rameter space. This last point aims at demonstrating thatthe CPs are not only useful for encoding trajectories, butalso for classifying them.

2 Dynamical Systems As ControlPolicies

For the purpose of learning how to imitate human-likemovement, we choose to represent movement in kinematiccoordinates, e.g., joint angles of a robot or Cartesian coor-

1

0 0.5 1 1.5 20

0.5

1

y

0 0.5 1 1.5 2−2

0

2

4

dy/d

t

0 0.5 1 1.5 20

0.5

1

1.5

v

Time [s]0 0.5 1 1.5 2

0

0.5

1

Ψi /

Σ i Ψi

0 0.5 1 1.5 20

0.5

1

1.5

dx/d

t

Time [s]0 0.5 1 1.5 2

0

0.5

1

x

Time [s]

Figure 1: Time evolution of the dynamical systems, withN=25, αz=8 [1/s], αv=8 [1/s], βz=2 [1/s], and βv=2 [1/s].The ci are spread out between c1=0.0 and cN=0.99 such as tobe equally spaced in time. The same parameters will be usedthroughout the article. In this particular example, g=1 andthe parameters wi were set by hand to arbitrary (positive andnegative) values.

dinates of an endeffector —indeed only kinematic variablesare observable in imitation learning. Thus, CPs representkinematic movement plans, i.e., a change of position asa function of state, and we assume that an appropriatecontroller exist to convert the kinematic policy into motorcommands. A control policy is defined by the following(z, y) dynamics which specify the attractor landscape ofthe policy for a trajectory y towards a goal g:

z = αz(βz(g − y) − z) (1)

y = z +

∑N

i=1 Ψiwi∑N

i=1 Ψi

v (2)

This is essentially a simple second-order system with theexception that its velocity is modified by a nonlinear term(the second term in equation 2) which depends on internalstates. These two internal states, (v, x) have the followingsecond-order linear dynamics

v = αv(βv(g − x) − v) (3)

x = v (4)

The system is further determined by the positive con-stants αv, αz, βv , and βz, and by a set of N Gaussian ker-nel functions Ψi

Ψi = exp

(

−1

2σ2i

(x − ci)2

)

(5)

where x = (x − x0)/(g − x0) and x0 is the value of x atthe beginning of the trajectory. The value x0 is set eachtime a new goal is fed into the system, and we assume thatg 6= x0, i.e. that the total displacement between the be-ginning and the end of a movement is never exactly zero.The attractor landscape of the policy can be adjusted bylearning the parameters wi using locally weighted regres-sion [3], see Section 3. Note that this dynamical systemhas a unique equilibrium point at (z, y, v, x) = (0, g, 0, g).

0 0.5 1 1.5 20

0.5

1

y

yy

plannedy

actual 0 0.5 1 1.5 2−2

0

2

4

dy/d

t

dy/dtdy/dt

planned

0 0.5 1 1.5 20

0.5

1

1.5

Time [s]

v

vv

planned

0 0.5 1 1.5 20

0.5

1

Ψi /

Σ i Ψi

0 0.5 1 1.5 20

0.5

1

1.5

Time [s]

dx/d

t

dx/dtdx/dt

planned

0 0.5 1 1.5 20

0.5

1

Time [s]

x

xx

planned

Figure 2: Time evolution of the dynamical systems under aperturbation. The actual position y is frozen between 0.5 and1.0s. For this example, αpy = 20 and αpx = 200 are used in(6) and (7) respectively.

The system is spatially invariant in the sense that a scalingof the goal g does not affect the topology of the attractorlandscape. In other words, the position and velocity of amovement will be scaled by a factor c, if a new goal cg isgiven to the system. This property will be used for goalmodulation in the experiment presented in Section 4.2.

Figure 1 demonstrates an exemplary time evolution ofthe equations. We identify the state y as the desired po-sition that the policy outputs to the controller, and y asthe desired velocity. The internal states (v, x) are usedas follows: x implements a timing signal that is used tolocalize the Gaussians (5), and v is a scaling signal whichensures that the non-linearity introduced by the Gaus-sians in Eq. 2 remains transient (v starts and returns tozero at the beginning and end of the discrete movement).v therefore guarantees the unique point attractor of (z, y)at (0, g). As is indicated by the reversal of movement di-rection in Figure 1, the internal states and the Gaussiansallow generating much more complex policies than a pol-icy that has no internal state. As will be demonstratedlater, the choice of representing a timing signal in formof autonomous differential equations will allow us to berobust towards external perturbations of the movement–essentially, we can slow down, halt, or even reverse “time”by means of coupling terms in the (v, x) dynamics (seeSection 2.2).

2.1 Stability

Stability can be analyzed separately for the (x, v) andthe (y, z) dynamics. It is clear that the (x, v) dynam-ics is globally asymtotically stable with the choice ofpositive constants. Formally, by following the input-to-state stability analysis in [4], we can conservatively con-clude that the equilibrium state of the entire system isasymptotically stable. Intuitively, when t → ∞, we have

z = αz(βz(g−y)−z) and y = z since∑N

i=1 Ψiwi/∑N

i=1 Ψi

is bounded and v → 0 as t → ∞, Thus, it can be seen that(z, y) → (0, g) as t → ∞. We are currently analyzing thedomain of attraction.

2

2.2 Robustness against Perturbationsduring Movement Execution

In the current form, the CP would create a desired move-ment irrespective of the ability of the controller to trackthe desired movement. When considering applications ofour approach to physical systems, e.g., robots and hu-manoids, interactions with the environment may requirean on-line modification of the policy. As an example, weshow how a simple feedback coupling term can implementa stop in the time evolution of the policy.

Let y denote the actual position of the movement sys-tem. We introduce the error between the planned trajec-tory y and y to the differential equations (2) and (4) inorder to modify the planned trajectory according to ex-ternal perturbations:

y = αy

(

∑N

i=1 Ψiwi∑N

i=1 Ψi

v + z

)

+ αpy(y − y) (6)

x = v(

1 + αpx(y − y)2)−1

(7)

Figure 2 illustrates the effect of a perturbation wherethe actual position is artificially fixed during a short pe-riod of time. During the perturbation, the time evolu-tion of the states of the policy is gradually halted. Thedesired position y is modified to remain close to the ac-tual position y, and, as soon as the perturbation stops,rapidily resumes performing the (time-delayed) plannedtrajectory. Note that other ways to cope with perturba-tions can be designed. Such on-line modifications are oneof the most interesting properties of using autonomousdifferential equations for control policies.

3 Learning From Imitation AndClassification

Assume we extracted a desired trajectory ydemo from thedemonstration of a teacher. Adjusting the CP to embedthis trajectory in the attractor landscape is a nonlinearfunction approximation problem. The demonstrated tra-jectory is shifted to a zero start position, and the timeconstants of the dynamical system are scaled such thatthe time dynamics (v, x) reaches (0, g) at approximatelythe same time as the goal state is reached in the targettrajectory. Given that the goal state is known, it is guar-anteed that the policy will converge to the goal; only thetime course to the goals needs to be adjusted.

At every moment of time, we have v as input to thelearning system, and udes = ydemo − z as desired out-put (cf Eq. 2). Locally weighted regression correspondsto finding the parameter wi which minimizes the locallyweighted error criterion Ji =

∑

t Ψti(u

tdes − ut

i)2, for each

local model uti = wiv

t (i.e. for each kernel function Ψti),

where t is an index corresponding to discrete time steps.1

Assuming that the goal g is known, this minimum canbe computed recursively using locally weighted recursiveleast squares [3]. Note that locally weighted regression is

1In our experiments, the time step size corresponds to the integra-tion step (1ms). The recorded movements are linearly intrapolatedto the same step size.

Figure 3: Left: Sensuit for recording joint-angle data. Right:Rigid body simulation.

different from learning with radial basis function networks.One of the significant differences is that in locally weightedlearning the parameter wi for each local model is learnedtotally independently of all other local models while inlearning with RBFs the parameters are learned coopera-tively. From a statistical point of view, unlike RBF sys-tems, locally weighted learning methods are robust againstoverfitting.

4 Experimental evaluations

4.1 Humanoid robotics for rehabilitation

One of the motivations to develop the CPs comes from ourVirtual Trainer (VT) project. The aim of this project is tosupervise rehabilitation exercises in stroke-patients by (a)computerized learning of the exercises from demonstra-tions by a professional therapist, (b), demonstrating theexercise to the patient with a humanoid simulation, (c)video-based monitoring the patient when performing theexercise, and (d) evaluating the patient’s performance, andsuggesting and demonstrating corrections with the simu-lation when necessary.

The essential ingredients of VT are (a) a humanoidrobot (either simulated, see Figure 3 right, or real Fig-ure 9), (b) the CPs for imitation, and (c) a movement ex-ecution system. The robot is used to demonstrate the ex-ercise to the patient, as well as to provide visual feedbackby replicating the movements performed by the patient.The CPs are responsible for determining the kinematicsof the desired trajectory, while the movement executionsystem (an inverse dynamics algorithm) is responsible forthe correct production of the trajectory by the dynamicsimulation.

As part of the VT project, the requirements for the CPsare the following. First, they must be able to fit the trajec-tories of the demonstrated movements with high precision.This implies fitting all degrees of freedom (DOFs) (i.e. notonly end-points such as the hands), and all points of thetrajectories (not only the final positions). Second, theymust be robust against perturbations as some exercisesrequire interaction with external objects, e.g., tables ortools. Finally, they must provide a good basis for com-

3

0 0.5 1 1.50

0.5

1

1.5

SF

E

0 0.5 1 1.5

−0.5

0

0.5

1

SA

A

0 0.5 1 1.5−1

−0.5

0

0.5

HR

0 0.5 1 1.5

0

0.5

1

1.5

EB

T [sec]

0 0.5 1 1.5

0

0.5

1

1.5

SF

E

0 0.5 1 1.5

−1

−0.5

0

0.5

SA

A

0 0.5 1 1.5

0

0.5

1

1.5

HR

0 0.5 1 1.5

0

0.5

1

1.5

EB

T [sec]

0 0.5 1 1.5−0.5

0

0.5

1

SF

E

0 0.5 1 1.5−1

−0.5

0

0.5

SA

A

0 0.5 1 1.5

−0.5

0

0.5

1

HR

0 0.5 1 1.5

0

0.5

1

1.5

EB

T [sec]

0 0.5 1 1.5

−0.5

0

0.5

1S

FE

0 0.5 1 1.5

−1

−0.5

0

0.5

SA

A

0 0.5 1 1.5−0.5

0

0.5

1

HR

0 0.5 1 1.5−0.5

0

0.5

1

EB

T [sec]

0 0.5 1 1.5−0.5

0

0.5

1

SF

E

0 0.5 1 1.5

−0.5

0

0.5

1

SA

A

0 0.5 1 1.5

−1

−0.5

0

0.5

HR

0 0.5 1 1.5−0.5

0

0.5

1

EB

T [sec]

TOP

BOTTOM

40cm-40 -20 0 20

40cm

20

0

-20

-40

2.0rad

Figure 4: Grid containing the 25 reaching targets with 5 typ-ical reaching trajectories. Each trajectory is represented bythe joint angles of four degrees of freedom drawn from top tobottom: shoulder flexion-extension (SFE), shoulder adduction-abduction (SAA), humerus rotation (HR), and elbow flexion-extension (EB). The five points surrounded by a square wereused as test sets for the experiment described in Section 4.2.2.

paring movements. One important aspect of VT will beto assess how well the patient performs an exercise, whichtherefore requires a metric for measuring differences be-tween movements.

4.2 Experiment 1: imitation of reachingmovements

The first experiment aimed at systematically testing (1)the fitting of human arm movements, and (2) the modu-lation of trajectories towards new goals. We recorded tra-jectories performed by a human subject using a joint-anglerecording system, the Sarcos Sensuit (Figure 3 right). TheSensuit directly records the joint angles of 35 DOFS of thehuman body at 100Hz using hall effect sensors with 12 bitA/D conversion.

We recorded reaching trajectories towards 25 points lo-cated on a vertical plane placed 65cm in front of the sub-ject. The points were spaced by 20cm on a 80 by 80cm grid(Figure 4). The middle point was centered on the shoul-der, i.e. it corresponded to the (perpendicular) projectionof the shoulder onto the plane. The reaching trajecto-ries to each target were repeated three times. Figure 4shows five typical reaching trajectories in joint space. Ascould logically be expected, the trajectories present quali-tative differences between movements to the left and right,with the shoulder adduction-abduction (SAA) angle in-creasing and decreasing respectively. The differences inshoulder flexion-extension (SFE) angles and elbow flexion-extension (EB) are essentially quantitative rather thanqualitative.

0 0.5 1 1.5

0

0.5

1

trajectory 13_1

SF

E [r

ad]

0 0.5 1 1.5

0

2

4

trajectory 13_1

dSF

E/d

t [ra

d/s]

0 0.5 1 1.5

−0.5

0

0.5

SA

A [r

ad]

0 0.5 1 1.5

−2

0

2

dSA

A/d

t [ra

d/s]

0 0.5 1 1.5

−0.5

0

0.5

1

HR

[rad

]

0 0.5 1 1.5

−2

0

2

dHR

/dt [

rad/

s]

0 0.5 1 1.5

0

0.5

1

1.5

T [sec]

EB

[rad

]

0 0.5 1 1.5

0

2

4

T [sec]

dEB

/dt [

rad/

s]

Figure 5: Fitting of a reaching movement towards a targetlocated at <0,0>cm. The recorded and fitted angles of theDOFs are drawn in dotted and continuous lines, respectively.

4.2.1 Fitting each reaching movement

A reaching movement is fitted by the CPs as four inde-pendent trajectories, one for each degree of freedom (SFE,SAA, HR and EB). Each of these trajectories is fitted witha system of 25 kernel functions. We fitted the total of 75reaching movements performed. Figure 5 compares therecorded and fitted trajectories for one particular example.It can be observed that 25 kernel functions are sufficientto fit the human data with little residual error.

4.2.2 Modulation of the goal of the reachingmovement

We tested how well the CP fitted to one particular tra-jectory could be used as a predictor of the trajectoriesperformed by the human subject towards neighboring tar-gets. Five control policies —those fitted to the five targetssurrounded by a a square in Figure 4— were tested.

The test is performed as follows. Once a CP is fittedto one particular reaching movement, its parameters arefixed and only the goals g, i.e. the four angles (one perDOF) to which the CP will converge, are modulated. Thedifferent goals that we feed into the CP were extractedfrom the final postures of the human subject when reach-ing towards different target points. This strategy thereforeallows us to directly compare the angular trajectories pre-dicted by the CPs with those performed by the subject.The accuracy of the prediction is measured by comput-ing the square error between the recorded and predictedvelocities normalized by the difference between the max-imum and minimum velocities of the recorded movement∑T

t=0(vrec

t−v

pred

t

max(vrect

)−min(vrect

) )2, summed for the four degrees

of freedom, and averaged over the three instantiations ofeach recorded movement (T is the number of time steps).

Figure 6 shows the resulting square error measurementsbetween predictions and recorded movements for the fivechosen CPs. Two observations can be made. First, aCP tends to be a better predictor of recorded movementswhich were made towards a target close to the target forwhich it was fitted. This is logical, as it simply meansthat human reaching movements towards close by goals

4

−40 −20 0 20 40

−40

−20

0

20

40

−40 −20 0 20 40

−40

−20

0

20

40

−40 −20 0 20 40

−40

−20

0

20

40

−40 −20 0 20 40

−40

−20

0

20

40

−40 −20 0 20 40

−40

−20

0

20

40

Figure 6: Error between predictions of the CPs and recordedmovements. The targets of the recorded movements which wereused for fitting each of the five CPs are shown with circles. Thewidths of the squares are proportional to the squared errormeasurements as described in the text.

tend to follow similar trajectories.2 Second, a CP is a sig-nificantly better predictor of recorded movements whichare made towards targets located above or below the orig-inal target, compared to targets located left or right tothat target. This is in agreement with our observationthat human trajectories in our recordings present qualita-tive differences between left and right —which thereforerequire different CPs— and only quantitative differencesbetween top and bottom —which can be represented by asingle CP with different goals.

Note that the purpose of our CPs is not to explain hu-man motor control, and there is no reason to expect thata single CP could predict all these human reaching move-ments. As mentioned above, the movements exhibit qual-itative differences in joint space. One would need somecartesian space criterion (e.g straight line of the end pointof the arm) and some optimization criterion (e.g. min-imum torque change) to explain human motor control.Our purpose is rather to develop a system to be used inhumanoid robotics which can accurately encode specifictrajectories, replay them robustly, and modulate them todifferent goals easily. Here we observe that, for the spe-cific trajectories that we recorded from human subjects, aCP fitted towards a particular target has the additionalproperty of being a good predictor for human movementsto neighbor targets.

Experiments implementing the trajectories in a realrobot can be found in [2]. The movements per-

2Because of the variability of human movements, the predictionerror for the recorded trajectories towards the same targets as thoseused for the fitting are not zero, as the two other instantiations mightvary slightly from the trajectory used for the fitting.

−1.2 −1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4

−1.2

−1

−0.8

−0.6

−0.4

−0.2

0

X

Y

−0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1−1.4

−1.2

−1

−0.8

−0.6

−0.4

−0.2

0

0.2

X

Y

−0.5 0 0.5 1 1.5

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

X

Y

−0.4 −0.2 0 0.2 0.4 0.6 0.8 1 1.2 1.4

−0.2

0

0.2

0.4

0.6

0.8

1

1.2

X

Y

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

−1.2

−1

−0.8

−0.6

−0.4

−0.2

0

Y

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−1

−0.8

−0.6

−0.4

−0.2

0

0.2

X

Time [s]

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

−1.5

−1

−0.5

0

Y

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

−0.5

0

0.5

1

X

Time [s]

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0

0.5

1

1.5

Y

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0

0.5

1

1.5

X

Time [s]

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

−0.2

0

0.2

0.4

0.6

0.8

1

1.2

Y

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

−0.2

0

0.2

0.4

0.6

0.8

1

1.2

X

Time [s]

Figure 7: Examples of two-dimensional trajectories fitted bythe CPs. The demonstrated and fitted trajectories are shownwith dotted and continuous lines, respectively.

0 2 4 6 8 10 12 14 16 18 200

2

4

6

8

10

12

14

16

18

20

I C R A

I

C

R

A

Figure 8: Correlation between the weight vectors of the 20characters (5 of each letter) fitted by the system. The gray scaleis proportional to the correlation, with black corresponding to acorrelation of +1 (max. correlation) and white to a correlationof 0 or smaller.

formed by the robot appear strinkingly human-like.Movies of the recorded trajectories and those per-formed by the robot can be found at http://www-clmc.usc.edu/∼ijspeert/humanoid.html.

4.3 Experiment 2: Comparison of trajec-tories in parameter space

An interesting aspect of the CPs is that trajectories withsimilar velocity profiles tend to have similar wi parame-ters, and the CPs can therefore be used to classify move-ments. To illustrate this, we carried out a simple task offitting two-dimensional trajectories performed by a humanuser when drawing two-dimensional single-stroke patterns.Four characters — I, C, R and A — from the Graffiti al-phabet used in hand-held computers were chosen. Thesecharacters are drawn in a single stroke, and are fed as twotrajectories x(t) and y(t) to be fitted by our system. Fiveexamples of each character were presented. Using two setsof 25 parameters wi (one per dimension), each characterwas fitted in a single iteration with very little residual er-ror (Figure 7).

Given the temporal and spatial invariance of our policyrepresentation, trajectories that are topologically similartend to be fit by similar parameters wi. In particular,

computing the correlationw

T

awb

|wa||wb|between the parameter

5

Figure 9: Humanoid robot learning a forehand swing from ahuman demonstration.

vectors wa and wb of character a and b can be used to clas-sify movements with similar velocity profiles (Figure 8). Inthis case, for instance, each of the 20 examples, the correla-tion is systematically higher with the four other examplesof the same character. These similarities in weight spacecan therefore serve as basis for recognizing demonstratedmovements by fitting them and comparing the fitted pa-rameters wi with those of previously learned policies inmemory. In this example, a simple one-nearest-neighborclassifier in weight space would serve the purpose.

4.4 Experiment 3: Learning tennis swingswith a humanoid robot

In a last set of experiments, we implemented the systemin a humanoid robot with 30 DOFs [5]. The robot is a1.9-meter tall hydraulic anthropomorphic robot with legs,arms, a jointed torso, and a head. The robot is fixed atthe hip, such that it does not require balancing.

The task for the robot was to learn forehand and back-hand swings towards a ball demonstrated by a humanwearing the joint-angle recording system. The fitted tra-jectories were fed into an inverse dynamics controller foraccurate reproduction by the robot (Figure 9).

Afterwards, the robot was able to repeat the swingmotion to different cartesian targets. Using a system oftwo-cameras, the position of the ball was given to an in-verse kinematic algorithm which computes targets in jointspace. Similarly to the results of experiment 1, the modi-

fied trajectories reached the new ball positions with swingmotions very similar to those used for the demonstration.Movies of the experiments can be viewed at http://www-clmc.usc.edu/∼ijspeert/humanoid.html.

5 Discussion

We presented a new system for encoding trajectories withcontrol policies (CPs) based on a set of nonlinear dy-namical systems. Two key characteristics of the CPs are(1) that movement trajectories are not indexed by timebut rather develop out of integrating autonomous nonlin-ear differential equations, and (2) that the CPs do notencode one single specific desired trajectory but rather awhole attractor landscape in which the desired trajectoryis produced during the evolution from the initial states toa unique point attractor, the goal state. This provides theCPs the ability to smoothly recover from perturbations,which is one of the desired properties enumerated in theintroduction.

The other desired properties are equally well fulfilledwith our system. In particular, the CPs allow fast learningof a trajectory with a relatively low number of parameters.The number of kernel functions (hence of parameters) canbe adjusted depending on the desired accuracy of the fit,with more functions allowing the representation of finerdetails of movement (see [3] for an algorithm to do thisautomatically). The ease of modification is also ensured byhaving the goal state of the movement explicitly encodedinto the system. The spatial invariance property is usedto modulate the trajectory towards the new goal, whilekeeping the same velocity profile.

We are currently working on extending this work intwo directions. First, we are investigating different typesof metrics in parameter space for comparing qualitativeand/or quantitative features of movements. Second, weare exploring ways to extend the system to encode oscil-latory movements in addition to discrete movements.

AcknowledgementsThis work was made possible by support from the US Na-tional Science Foundation (Awards 9710312 and 0082995),the ERATO Kawato Dynamic Brain Project funded bythe Japanese Science and Technology Cooperation, andthe ATR Human Information Systems Laboratories.

References[1] A.J. Ijspeert, J. Nakanishi, and S. Schaal. Trajectory for-

mation for imitation with nonlinear dynamical systems. InProceedings of the IEEE/RSJ Int. Conference on IntelligentRobots and Systems (IROS2001), pages 752–757. 2001.

[2] A.J. Ijspeert, J. Nakanishi, T. Shibata, and S. Schaal.Nonlinear dynamical systems for imitation with humanoidrobots. In Proceedings of the IEEE International Confer-ence on Humanoid Robots, pages 219–226. 2001.

[3] S. Schaal and C.G. Atkeson. Constructive incrementallearning from only local information. Neural Computation,10(8):2047–2084, 1998.

[4] H.K. Khalil. Nonlinear System. Prentice Hall, 1996.

[5] C. G. Atkeson, J. Hale, M. Kawato, S. Kotosaka, F. Pollick,M. Riley, S. Schaal, S. Shibata, G. Tevatia, and A. Ude.Using humanoid robots to study human behaviour. IEEEIntelligent Systems, 15:46–56, 2000.

6