Learning Collaborative Impedance-based Robot Behaviors∗

Leonel Rozo and Sylvain Calinon and Darwin CaldwellDepartment of Advanced Robotics - Istituto Italiano di Tecnologia (Via Morego, 30. Genoa, Italy)

Pablo Jimenez and Carme TorrasInstitut de Robotica i Informatica Industrial CSIC-UPC (Carrer Llorens i Artigas, 4-6. Barcelona, Spain)

Abstract

Research in learning from demonstration has focusedon transferring movements from humans to robots.However, a need is arising for robots that do not justreplicate the task on their own, but that also interactwith humans in a safe and natural way to accomplishtasks cooperatively. Robots with variable impedance ca-pabilities opens the door to new challenging applica-tions, where the learning algorithms must be extendedby encapsulating force and vision information. In thispaper we propose a framework to transfer impedance-based behaviors to a torque-controlled robot by kines-thetic teaching. The proposed model encodes the exam-ples as a task-parameterized statistical dynamical sys-tem, where the robot impedance is shaped by estimatingvirtual stiffness matrices from the set of demonstrations.A collaborative assembly task is used as testbed. Theresults show that the model can be used to modify therobot impedance along task execution to facilitate thecollaboration, by triggering stiff and compliant behav-iors in an on-line manner to adapt to the user’s actions.

1 IntroductionOver the last decade Robotics is addressing new challeng-ing problems to bring robots closer to humans, as comput-ers have now become part of our everyday life. It is envis-aged that robots should collaborate with humans to performa large variety of tasks more easily, faster and in a safer way.To accomplish this goal, robots must be endowed with learn-ing capabilities allowing them to acquire new knowledgefrom examples given by a human or through their own expe-rience. Learning from demonstration (LfD ) is a natural wayto transfer knowledge to robots from human examples (Bil-lard et al. 2008). Most works have focused on developinglearning algorithms to encode trajectories using vision orop-tical systems to capture the teacher demonstrations, see e.g.,(Calinon, Sardellitti, and Caldwell 2010). Nevertheless,thenew variable impedance capabilities of recent robotic arms

∗This work was partially supported by the STIFF-FLOP Euro-pean project (FP7-ICT-287728), IntellAct European project (FP7-269959) and the Spanish project PAU+ (DPI2011-27510). L. Rozowas supported by the CSIC under a JAE-PREDOC scholarship.Copyright c© 2013, Association for the Advancement of ArtificialIntelligence (www.aaai.org). All rights reserved.

Figure 1: Top: Two humans assembling a wooden table.Bottom: demonstration (left) and reproduction (right) of theimpedance-based behavior.

(Rooks 2006; Albu-Schaffer et al. 2007) demand to refor-mulate these methods in order to exploit their new controlschemes in performing more complex tasks.

In this line, physical human-robot interaction (pHRI ) hasarisen a lot of interest recently, with the two challengingaspects of impedance control and haptic communication.On the one hand, an increasing effort has been devotedto exploiting the advantages provided by the impedance-based control of robots (Hogan 1985). On the other hand,impedance in humans has also been studied with the aimof gaining in-depth knowledge of the roles of the muscles,tendons, brain and spinal cord in modulating impedancewhen we interact with the environment (Burdet et al. 2001;Gomi and Kawato 1996). Also, efforts have been devoted tomimic human impedance with robots (Ganesh et al. 2010).In this context, it is desirable to transfer impedance-basedbehaviors from humans to robots (Kalakrishnan et al. 2011;Kronander and Billard 2012). Evrard and Kheddar (2009)tackled the problem of setting the robot’s role in a collab-orative lifting task by modifying the impedance controllerparameters. The approach was then extended to let the robotlearn this behavior autonomously from human demonstra-tions through probabilistic approaches (Calinon et al. 2009;Gribovskaya, Kheddar, and Billard 2011).

The second challenge in pHRI is the haptic communica-

Figure 2: Each assembly task is characterized by differentsequences, positions and orientations of components, withhaptic and movement patterns that are specific to each item.

tion between the partners. When physical interaction takesplace, forces-torques sensed by the partners constitute a veryrich and valuable communication channel which is used torecognize and/or predict intentions as well as to determinethe role of each participant in the task (Evrard and Khed-dar 2009; Groten et al. 2013; Reed and Peshkin 2008). Thechallenges from the robot side are to distinguish the hapticsignals related to the task from those communicating inten-tions, and to anticipate the human actions using informationabout the dynamics of the task and the force-based percep-tions generated along it (Thobbi, Gu, and Sheng 2011).

In this paper, we are concerned with learning collabora-tive impedance-based skills1 using visual and haptic infor-mation, where the robot behavior is conditioned by task vari-ables as well as by its haptic perceptions. We thus require amodel that can learn impedance behaviors from a set demon-strations, where the model is modulated by haptic and visioninformation. The proposed approach is tested in a collabora-tive assembly of a wooden IKEA table. In the learning phasetwo humans perform the task, where one is kinesthetically2

guiding the robot to demonstrate the robot’s role. The com-pliance behavior of the person holding the table changes toallow his/her partner to perform his/her corresponding sub-task more easily (see Figure 1). During reproduction, therobot replaces the user holding the table by automatically es-timating the levels and shape of stiffness ellipsoid requiredduring the interaction.

The remainder of the paper is organized as follows: Sec-tion 2 explains the learning algorithm and the compliancelevel estimation method. Section 3 describes the collabora-tive task and the experimental setting. Results are shown andanalyzed in Section 4. Finally, conclusions and future workare presented in Section 5.

2 Proposed approachWe extend the task-parameterized movement learning ap-proach recently proposed in Calinon et al. (2012) to a task-parameterized impedance learning problem. This approachrelies on a statistical representation of dynamical systemsthat can be modulated with respect to task variables repre-sented as candidate frames of reference. The model is here

1Here, impedance-based skill refers to different stiffnesslevelsthat a robot needs to accomplish a given task.

2The term refers to the procedure where the user is holding therobot, which is gravity compensated at its links, and moves it alongthe trajectories that need to be followed to accomplish the task.

extended to force-based impedance behaviors requiring toadapt the stiffness of virtual springs in Cartesian space driv-ing the robot’s behavior.

2.1 Task-parametrized Gaussian mixture modelWhen robots manipulate objects, their movements maylargely depend on the given goals and object poses, whichcan be defined through reference frames. Namely, the robotmotion is conditioned by a set oftask variables represent-ing the coordinate systems of relevant frames of reference.For generalization purposes, it is desirable to have a modelenclosing different movements as a function of the thesevariables, instead of representing each one with a differentmodel. The proposed approach relies on Gaussian productproperties to modulate the centers and covariance matricesof a Gaussian mixture model (GMM ). The advantages ofthis approach compared to other task-parameterized mod-els such as the parametric Hidden Markov Model (PHMM )(Wilson and Bobick 1999) are discussed in Calinon et al.(2012).

Formally, each demonstrationm ∈ {1, . . . ,M} containsTm datapoints forming a dataset ofN datapoints{ξn}

Nn=1

with N =∑M

m Tm. Eachξn ∈ RD is associated with

task variables{An,j , bn,j}NP

j=1representingNP candidate

frames of reference, with transformation matricesAn,j , andoffset position vectorsbn,j . D is the datapoint dimensional-ity, and the indexesn andj represent the time step and thecandidate frame, respectively.

The parameters of the modelare{πi,Zµ

i,j ,ZΣ

i,j}, rep-resenting respectively the mixing coefficients, centers andcovariances matrices for each framej and mixture compo-nenti. With this model, for an observation of frames at itera-tion n, the resulting centerµn,i and covariance matrixΣn,i

of each componenti are computed as products of linearlytransformed Gaussians

N (µn,i,Σn,i)=

NP∏

j=1

N(

An,jZµ

i,j+bn,j , An,jZΣ

i,jA⊤

n,j

)

.

By using the product property of normal distributions, theabove equation is computed as

µn,i = Σn,i

NP∑

j=1

(An,jZΣ

i,jA⊤

n,j)−1

(An,jZµ

i,j+bn,j),

Σn,i =(

NP∑

j=1

(An,jZΣ

i,jA⊤

n,j)−1

)

−1

. (1)

The parameters of the model are iteratively estimated withthe following EM procedure. In the E-step, (1) are used astemporary Gaussian parameters to compute the likelihood.

E-step:

hn,i =πiN (ξn|µn,i,Σn,i)

∑NK

k πkN (ξn|µn,k,Σn,k). (2)

M-step:

πi =

∑

n

hn,i

N, Zµ

i,j=

∑

n

hn,i A−1

n,j [ξn − bn,j ]

∑

n

hn,i

,

ZΣ

i,j=

∑

n

hn,i A−1

n,j [ξn−µn,i,j ][ξn−µn,i,j ]⊤A−⊤

n,j

∑

n

hn,i

,

with µn,i,j=An,jZµ

i,j+bn,j . (3)

Note that for force-based tasks, the datapoints, centers andcovariances can be decomposed into their position, force andCartesian torque components

ξn=

ξx

n

ξF

n

ξT

n

, µn,i=

µx

n,i

µF

n,i

µT

n,i

, Σn,i=

Σx

n,i ΣxF

n,i ΣxT

n,i

ΣFx

n,i ΣF

n,i ΣFT

n,i

ΣTx

n,i ΣTF

n,i ΣT

n,i

. (4)

The model parameters are initialized with ak-means pro-cedure, modified by following a similar task-parametrizedstructure. Model selection is compatible with the techniquesemployed in standard GMM (Bayesian information crite-rion, Dirichlet process, etc.).

One novelty with respect to (Calinon et al. 2012) is thatwe augment the model with virtual stiffness matricesKP

i

associated to each componenti, which will be estimated asexplained in Section 2.2. Thus, the complete set of parame-ters of the model is{πi, {Z

Σ

i,j ,Zµ

i,j}Np

j=1,KP

i }NK

i=1. Such ex-

tension allows us to apply the learning model to impedance-based behaviors transfer. Note that the variables of the taskare obtained from the position and orientation of a set of can-didate frames to learn the task. In our experimental setup, thetable legs and robot frames define variables in the collabora-tive assembly task (described later in Section 3).

Fig. 3 illustrates the approach with a simple example.(a)shows the demonstrations where the robot behaves compli-antly when another object (the green triangle) is far from itsend-effector, and becomes stiff when the object approachesit with a specific orientation.(b) displays the two phases ofthe task, where the robot motion is driven by a set of virtualsprings connected to the center of the model’s Gaussians.The mean and covariance vary according to the task vari-ables (i.e., the object and robot frames), and the influence ofeach model component (see Eq. (2)) determines how com-pliantly the robot behave.

2.2 Stiffness estimationSeveral approaches have been proposed to estimate fromcollected data the stiffness and damping parameters to con-trol robots. Erickson, Weber, and Sharf (2003) comparedfour different methods to estimate the robot impedancebased on signal processing, adaptive control and recursiveleast squares. Flacco and Luca (2011) estimated the non-linear stiffness of robot joints with flexible transmissionsby using dynamic residual signals along with least-squaresand regressor-based techniques. From a different perspec-tive, a LfD approach was proposed in Calinon, Sardellitti,and Caldwell (2010) to find a stiffness matrix using variabil-ity information extracted from training data in the form ofa GMM, where the stiffness matrix is estimated from theinverse of the observed covariance in the position space.Similarly, Lee and Ott (2011) used variability encoded inthe components of an HMM to define a motion refinementtube that permits a deviation from nominal trajectories for

(a) Demonstrations

(b.1) Compliant phase (b.2) Stiff phase

Figure 3: Simplified impedance behavior learning.(a) 3 dif-ferentdemonstrations showing compliant and stiff phases.The black line is the robot’s trajectory.(b) Reproduction ofthe task, where the robot’s behavior is governed by a 2-statesmodel with virtual springs connected to the Gaussians cen-ters. Dark ellipses and thick-line springs represent an acti-vated Gaussian. The candidate frames are displayed in redcolor.

kinesthetic corrections by controlling the stiffness value atthe robot joints level.

Here, we obtain an approximation through an algebraicclosed-form solution to find the closest symmetric positivesemi-definite stiffness matrix of a weighted least-squares(WLS) estimation. A stiffness matrixKP

i is estimated foreach componenti, by assuming that the robot behavior isdriven by a set of virtual springs (similar to Fig. 3)

F n =

NK∑

i=1

hn,i

[

KPi

(

µx

n,i − xn

)

]

, (5)

whereF n, µx

n,i andxn are respectively the sensed force,the positional part of the Gaussians’ centers in the model(see Eq. (4)), and the robot’s end-effector position at timestepn.

WLS is used to compute a first estimateKP

i =[

(X⊤

iW iXi)−1X⊤

iW iF]

of the stiffness matrices byconcatenating all theN datapoints in matricesXi =[

(µx

1,i − x1), . . . , (µx

N,i − xN )]

⊤

andF , with a weight-ing matrixW i = diag([h1,i, h2,i, . . . , hN,i]) (see Eq. (2)).Such estimate does not necessarily comply with the sym-metric positive semi-definite constraints of a stiffness ma-trix. Therefore, we resort to the formulation presented in(Higham 1988), to computeKP

i as the nearest symmetric

positive semi-definite (SPSD) matrix toKP

i according tothe Frobenius norm, computed as

KPi =

B+H

2, B=

KP

i +(KP

i )⊤

2, H=V ΣV⊤. (6)

Table 1: Learning and reproduction phases.

1. Task demonstrations- DetermineNP (number of frames or task variables)- ∀n ∈ {1, . . . , N}, collectξn and{An,j , bn,j}

NP

j=1

2. Model fitting- DetermineNK (number of components of the model)- Use Eq. (3) to learn{πi, {Z

µ

i,j ,ZΣ

i,j}NP

j=1}NK

i=1

3. Stiffness estimation- FindKP

i for each virtual spring by using Eq. (6)4. Reproduction(for each time stepn)

- Collectξn and{An,j , bn,j}NP

j=1

- Estimate{πi,µn,i,Σn,i}NK

i=1through Eq. (1)

- Compute activation weightshn,i using Eq. (2)- Apply the force command computed from Eq. (5)

H is the symmetric polar factor which can be found from thesingular value decomposition ofB, namely,B = UΣV⊤.Table 1 summarizes the learning and estimation processes.

3 Collaborative assembly taskWe consider a human-robot collaborative task where therobot’s role is to hold a wooden table while the user’s roleis to screw the four legs to it. Fig. 2 presents an example ofassembly instructions that can be found in “do it yourself”furniture catalog. Here, two small tables require specific se-quences of force and movement to get assembled. Learningsuch specificities is required for an efficient collaborative as-sembly. Instead of manually programming those specificitiesfor each item, we would like the robot to extract those au-tomatically from a set of demonstrations provided by twousers collaborating together to assemble the different partsof the table (see Fig. 1). After learning, the task can be re-produced by a single user, with the robot partner interactingappropriately with respect to the preferences of the user andthe specificities of the item being assembled. We thus do notneed to provide the robot with information about the pointsof assembly, the different options, orientation of table legs,etc. The robot instead learns these specificities from demon-strations.

3.1 Experimental setupWe use a KUKA lightweight 7-DoF robot (LWR) (Albu-Schaffer et al. 2007), with theFast-Research Interface(Schreiber, Stemmer, and Bischoff 2010), by using a Carte-sian impedance controller defined by

τ d = J⊤F d + V (κVd ) + f(q, q, q),

whereJ is the Jacobian of the robot,τ d is the desired torquevector ,F d the desired force computed from the resulting setof virtual springs (Eq. 5),V is a damping function with de-sired damping valuesκV

d andf(q, q, q) the dynamic modelof the robot.3

3Note that we only control the Cartesian position of the robotwhile the rotational DOF are set to be fixed during the reproductionphase.

User reorienting table User screwing Leg 2

Figure 4: Reproduction results at different phases of theinteraction. For each graph, the projection of the model’sGaussians in the tool’s frame (as ellipses) is shown on theleft, while theright part shows the trace of the resulting stiff-ness matrix. The black dotted line represents the leg’s trajec-tory, the table is shown in yellow with its 4 threads and thebrown cross corresponds to the current position of the leg.

The position and orientation of the table legs are trackedwith a marker-based NaturalPoint OptiTrack motion capturesystem, composed of 12 cameras working at a rate of 30fps. A transformation matrix is computed to represent theleg configuration in the fixed robot frameOR, from whichbleg

n andAleg

n define the Cartesian position and the orien-tation of the leg as a rotation matrix, respectively. Duringboth demonstration and reproduction phases,{Aleg

n , bleg

n }are recorded at each time stepn to determine the task vari-ables. Lastly, the other candidate frame{AR

n, bR

n} define therobot’s fixed frame of reference.4

The robot is equipped with a six-axis force-torque sensor(ATI Mini45) attached between its wrist and the wooden ta-ble, measuring the interaction forces generated while mov-ing the table and screwing the legs. In order to extract theforces generated by the interaction between the partnersthrough the table, it is necessary to remove other signalsfrom the sensor readings. During the demonstration and re-production stages, we consider that the sensor readings arecomposed of the noise, the load mass effects and the exter-nal applied forces, the latter corresponding to the interactionbetween the human and the table. Moreover, we assume thatthe interaction does not generate high linear/angular veloci-ties or accelerations, so that the dynamical components canbe neglected, similarly to Rozo, Jimenez, and Torras (2010).

3.2 Task descriptionTwo candidate frames (NP = 2) describe the task variablesin the experiment:OR and the leg frameOL. We assume thatone leg is used and tracked at a time. Let us defineDL andDT as the human user and the robot, respectively (see Fig.1). The collaborative scenario consists of screwing the legsat the four corresponding positions on the table.DT is firstcompliant to allowDL to move the table freely (compliantphase) until comfortable position and orientation are foundfor the work to be performed next. WhenDL grasps a legand starts inserting it into the screw thread in the table,DT

adopts a stiff posture, holding the table to facilitateDL’s part

4A 3D coordinate frame is replicated for the variablesx,F andT , the offset is only set tox.

1 2 3 4 50

70

140

210

280

350

State

Sti

ffnes

s

Figure 5: Stiffness level estimation for the 5-states model.The bars display the trace of the mean estimated stiffnessmatrices for each Gaussian.Dark blue bars represent toKP

Frb(our approach) whilelight red bars show theKP

Invval-

ues. The first state corresponds to the compliant behavior.

of the task (stiff phase).Note that the combination of vision and haptic informa-

tion is fundamental for this task. If only vision is used, therobot cannot distinguish the phase during which the useraligns the screw with the thread. Here,DT should insteadregulate its stiffness in accordance with the sensed force pat-tern. IfDT ’s behavior was based only on forces, the collab-oration could fail becauseDT could not distinguish whichforces correspond to interaction withDL and which are pro-duced by the screwing actions. This can be problematic be-cause these patterns might be similar in some situations.Both perception channels are thus needed to learn how theimpedance behavior should be shaped.

4 ResultsA model of five components (NK = 5) was trained withsixteen demonstrations (i.e., each leg is assembled fourtimes to its corresponding thread with specific vision andforce patterns). The resulting model automatically discov-ered four stiff components corresponding to the four screw-ing phases, with the remaining component representing thecompliant behavior. Each “stiff component” is characterizedby the force-torque pattern and the relative position of theleg with respect to the robot tool frame, which are differentfor each leg. The “compliant component” encodes the re-maining points in the data space, i.e., the interaction forces-torques as well as the varying robot end-effector and leg po-sitions. Fig. 4 shows that the Gaussian corresponding to thecompliant phase is already spatially distinguishable fromtheGaussians encoding the stiff behaviors during the screwingprocesses (four in this case). Note that the Gaussians in themodel representing the stiff phases show an elongated shapechanging its orientation during the task. Such type of time-varying information encapsulated in the covariance matricescannot be encoded by using the classic PHMM (no covari-ance parameterization).

Once the model is learned, the stiffness estimation is car-ried out as described in Section 2.2. In this experiment astiffness matrix is locally associated with each componentinthe model, describing a virtual spring connected to the cen-ter of the Gaussian. During reproduction, a force commandis estimated as a combination of the virtual springs (see Eq.5).

050

100150

|KP|

1 3 6 9 12 15 18 21 240

0.5

1

t

h(ξ

)

(a) Leg 1

050

100150

|KP|

1 3 6 9 12 15 18 21 240

0.5

1

t

h(ξ

)

(b) Leg 2

Figure 6: Resulting stiffness matrix trace described byKP =

∑NK

i hn,iKPi (top), and components influence on

the stiffness (bottom). Each plot displays how the weight be-longing to each component changes over time – between 0and 1 – showing its influence on the WLS-based estimation.Theactivated component at the stiff phase relies on the legto be assembled (dashed lines), while the compliant com-ponent isactivated regardless of the leg (dotted line). Thedeactivated components – whose weights are zero – are notshown in the graphs to emphasize the components partici-pating in the current demonstration.

The proposed approach was compared to the stiffness es-timation procedure based on the inverse of the observed co-variance (Calinon, Sardellitti, and Caldwell 2010) (see Fig.5), by computing the inverse of the sub-matrixΣ

x

n,i for eachGaussiani at each time stepn. A weighted average stiffnessKP

Invis then calculated. This is compared toKP

Frbobtained

as described in Section 2.2. With our training set, both ap-proaches estimate the different stiffness levels appropriately.However, the estimate of Calinon, Sardellitti, and Caldwell(2010) has the disadvantage that it takes only into accountthe positional information from the data, whose variabilitycan sometimes be too weak if only a few number of demon-strations are considered. In the experiment, the users coveredvarious portions of the workspace. In a more realistic sce-nario, the users might not be aware of this scaffolding teach-ing procedure, and a smaller number of datapoints mightbe acquired. In such situation, variability information maynot always be sufficient to estimate stiffness information.Incontrast, the approach that we adopt in this paper does takeinto consideration the haptic inputs in the estimation pro-cess. Fig. 5 displays the trace of the estimated stiffness ma-trices for each Gaussian, comparing the results obtained byboth approaches. The ratio between the stiff and compliantvalues (computed from the matrix traces) is higher using theproposed approach (10.12 as average) than those obtainedfrom the approach based on position variability (5.19 as av-erage), which allows a better clamping of the robot stiffnessconsidering the obtained maximum and minimum values.This indicates that the difference between the compliant andstiff levels is more pronounced when the estimation processis based on the haptic data.

Previous experiments with force and position data haveshown that it was often more challenging to exploit forcerecordings than positions in practice, mostly due to thelower accuracy of the sensors and disturbing forces (Rozo,Jimenez, and Torras 2013). However, a small variability hasbeen observed for force signals in the second phase of the

500 1000 15000.4

0.6

0.8X

x

500 1000 1500−20

0

20

Fx

500 1000 1500−4

−2

0

2

4

t

Tx

500 1000 1500−1

0

1

Xy

500 1000 1500−20

0

20

Fy

500 1000 1500−4

−2

0

2

4

t

Ty

500 1000 15000

0.2

0.4

Xz

500 1000 1500−20

0

20

Fz

500 1000 1500−4

−2

0

2

4

t

Tz

Figure 7: Estimated stiffness along the Cartesian axes of therobot (represented as an envelope surrounding the repro-duced trajectory) and the corresponding force-torque pro-files. The envelopes (light color area) in the first row showthe robot compliance (the wider the zone is, the more com-pliant the robot is). When the envelope is wide, the robot canbe moved easily byDL, while narrow envelope representsthe table holding phase with high stiffness.

task where the leg is screwed to the table. We attribute thislow variability to two main factors: 1) the force signals pre-processing and 2) the fact that our experiment protocol fa-cilitated the collaborative task to the users, which resultedin high consistency in the demonstrations. The two usersdemonstrated the collaborative assembly without talking,but by using their hands and their eyes. During reproduction,the robot however only had access to a limited subset of hap-tic and vision cues. For example, the users could observe thechanges of posture and even the gaze of each other (Strabalaet al. 2012; Mutlu, Terrell, and Huang 2013), which likelyprovided additional cues to jointly agree about the changeof stiffness behaviors.

We tested the reproduction and generalization capabilitiesof the system by carrying out the assembly process for all thelegs. In Fig. 6, we can observe how the compliant component(purple dotted line) is influential during the first half of thereproduction, dominating the other components. After this,the robot becomes stiff, with specific patterns depending onwhich leg is being screwed. This means that not all the com-ponents influence the robot impedance at the stiff phase, butmostly the Gaussian encoding the stiff behavior for the cor-responding leg (as observed from the different colors repre-senting the different stiff components), while the remainingactivation weights stay close to zero.

The proposed approach does not only learn when tochange the compliance in an on/off fashion, but also themanner to switch between the two behaviors. The sharpstiff/compliant difference is a characteristic of the collab-orative task presented here (mostly binary, but with smoothtransitions between the two compliance levels), which is cor-rectly learned by the proposed approach. Fig. 7 shows theresulting stiffness matrix for the demonstration correspond-ing to leg 1, where the Cartesian robot position is shownalong with the corresponding stiffness for each axis. We cansee how these values vary along the different Cartesian axes,

(a) Leg away from the threads (b) Leg turned upside-down

Figure 8: Situations not shown in the demonstration phase.Both correctly result in the robot behaving compliantly.

which is useful when the robot behavior demands to be stiffin a specific direction and compliant in the others.

In order to show the relevance of combining visual andhaptic data for generalizing the learned task, two situa-tions that did not appear in the demonstrations were pre-sented to the robot (Fig. 8). First,DL tried to screwthe leg at the center of the table, which means thatthe leg was placed at an incorrect position. In the sec-ond situation,DL positioned the leg in one of the tablethreads but the leg was tilted, making the screwing pro-cess unfeasible. In both cases, the robot behaved com-pliantly as expected, because neither corresponded to acorrect screwing phase. A video of the experiment andthe task-parameterized GMM sourcecode are available athttp://programming-by-demonstration.org/AAAI2013/

5 Conclusions and Future WorkWe presented a learning framework to encode and repro-duce impedance behaviors using a task-parameterized sta-tistical dynamical system. Our method allows to encode be-haviors that rely on task variables, yielding only one modelto encode the whole task. In contrast to previous approacheswhere robot impedance is learned from position variabil-ity, our framework extracts the impedance behavior from themanner in which the teacher behaves during the task, relyingon recorded force patterns and visual information. We useforces not only to encode the skill, but also to estimate thestiffness of virtual springs governing the collaborative be-havior, thus emphasizing that interaction forces-torquesvaryduring different phases of the task.

The proposed approach is used to learn a reactive behav-ior, where the model automatically provides soft clustersof the different impedance behaviors that the robot mightadopt. They are estimated as impedance controllers whoseparameters are learned from demonstrations and associatedwith each model component. We plan in future work to pro-vide the robot with a more active role. Initially, the roleswould be considered as time independent. Then, more com-plex task preferences between the partners would be ac-quired, where the robot could adopt a proactive role in thecollaboration. We devise to exploit the weighting mecha-nism in the model to influence the switching from reactiveto proactive behaviors, by anticipating the next part of thetask depending on user preferences. For example, if the userholding the leg is too far from the robot, it could take theinitiatives to anticipate the movement and move to the leg.

ReferencesAlbu-Schaffer, A.; Haddadin, S.; Ott, C.; Stemmer, A.;Wimbck, T.; and Hirzinger, G. 2007. The DLR lightweightrobot - design and control concepts for robots in humanenvironments.Industrial Robot: An International Journal34(5):376–385.Billard, A.; Calinon, S.; Dillmann, R.; and Schaal, S. 2008.Robot programming by demonstration. In Siciliano, B., andKhatib, O., eds.,Springer Handbook of Robotics. Secaucus,NJ, USA: Springer. 1371–1394.Burdet, E.; Osu, R.; Franklin, D.; Milner, T.; and Kawato, M.2001. The central nervous system stabilizes unstable dynam-ics by learning optimal impedance.Nature 414(6862):446–449.Calinon, S.; Evrard, P.; Gribovskaya, E.; Billard, A.; andKheddar, A. 2009. Learning collaborative manipulationtasks by demonstration using a haptic interface. InIntl. Conf.on Advanced Robotics, 1–6.Calinon, S.; Li, Z.; Alizadeh, T.; Tsagarakis, N.; and Cald-well, D. 2012. Statistical dynamical systems for skills ac-quisition in humanoids. InIntl. Conf. on Humanoid Robots,323–329.Calinon, S.; Sardellitti, I.; and Caldwell, D. 2010. Learning-based control strategy for safe human-robot interaction ex-ploiting task and robot redundancies. InIntl. Conf. on Intel-ligent Robots and Systems, 249–254.Erickson, D.; Weber, M.; and Sharf, I. 2003. Contact stiff-ness and damping estimation for robotic systems.Intl. Jour-nal of Robotics Research 22(1):41–57.Evrard, P., and Kheddar, A. 2009. Homotopy switchingmodel for dyad haptic interaction in physical collaborativetasks. InEuroHaptics, 45–50.Flacco, F., and Luca, A. D. 2011. Residual-based stiff-ness estimation in robots with flexible transmissions. InIntl.Conf. on Robotics and Automation, 5541–5547.Ganesh, G.; Albu-Schaffer, A.; Haruno, M.; Kawato, M.;and Burdet, E. 2010. Biomimetic motor behavior for si-multaneous adaptation of force, impedance and trajectory ininteraction tasks. InIntl. Conf. on Robotics and Automation,2705–2711.Gomi, H., and Kawato, M. 1996. Equilibrium-point con-trol hypothesis examined by measured arm stiffness duringmultijoint movement.Science 272(5258):117–120.Gribovskaya, E.; Kheddar, A.; and Billard, A. 2011. Motionlearning and adaptive impedance for robot control duringphysical interaction with humans. InIntl. Conf. on Roboticsand Automation, 4326–4332.Groten, R.; Feth, D.; Klatzky, R.; and Peer, A. 2013. Therole of haptic feedback for the integration of intentions inshared task execution.Trans. on Haptics 6(1):94–105.Higham, N. 1988. Computing a nearest symmetric positivesemidefinite matrix. Linear Algebra and its Applications103:103–118.Hogan, N. 1985. Impedance control: An approach to ma-nipulation: Part i, ii, iii. Journal of Dynamic Systems, Mea-surement, and Control 107(1):1–24.

Kalakrishnan, M.; Righetti, L.; Pastor, P.; and Schaal, S.2011. Learning force control policies for compliant manip-ulation. In Intl. Conf. on Intelligent Robots and Systems,4639–4644.Kronander, K., and Billard, A. 2012. Online learning ofvarying stiffness through physical human-robot interaction.In Intl. Conf. on Robotics and Automation, 1842–1849.Lee, D., and Ott, C. 2011. Incremental kinesthetic teach-ing of motion primitives using the motion refinement tube.Autonomous Robots 31:115–131.Mutlu, B.; Terrell, A.; and Huang, C. 2013. Coordinationmechanisms in human-robot collaboration. InACM/IEEEIntl. Conf. on Human-Robot Interaction (HRI) - Workshopon Collaborative Manipulation, 1–6.Reed, K., and Peshkin, M. 2008. Physical collaboration ofhuman-human and human-robot teams.Trans. on Haptics1(2):108–120.Rooks, B. 2006. The harmonious robot.Industrial Robot:An International Journal 33(2):125–130.Rozo, L.; Jimenez, P.; and Torras, C. 2010. Sharpeninghaptic inputs for teaching a manipulation skill to a robot. InIntl. Conf. on Applied Bionics and Biomechanics, 370–377.Rozo, L.; Jimenez, P.; and Torras, C. 2013. A robot learningfrom demonstration framework to perform force-based ma-nipulation tasks.Intelligent Service Robotics 6(1):33–51.Schreiber, G.; Stemmer, A.; and Bischoff, R. 2010. Thefast research interface for the KUKA lightweight robot. InICRA-Workshop on Innovative Robot Control Architecturesfor Demanding (Research) Applications, 15–21.Strabala, K.; Lee, M.; Dragan, A.; Forlizzi, J.; and Srinivasa,S. 2012. Learning the communication of intent prior tophysical collaboration. InIntl. Symposium on Robot andHuman Interactive Communication, 968–973.Thobbi, A.; Gu, Y.; and Sheng, W. 2011. Using human mo-tion estimation for human-robot cooperative manipulation.In Intl. Conf. on Intelligent Robots and Systems, 2873–2878.Wilson, A., and Bobick, A. 1999. Parametric hidden Markovmodels for gesture recognition.Pattern Analysis and Ma-chine Intelligence 21(9):884–900.