new human motion control with physically plausible foot contact...

9
Vis Comput (2015) 31:883–891 DOI 10.1007/s00371-015-1097-8 ORIGINAL ARTICLE Human motion control with physically plausible foot contact models Jongmin Kim 1 · Hwangpil Park 2 · Jehee Lee 2 · Taesoo Kwon 1 Published online: 24 April 2015 © Springer-Verlag Berlin Heidelberg 2015 Abstract The foot-to-ground contact model plays an important role in the simulation of highly dynamic motions, such as turns and kicks. In this paper, we propose a method for solving dynamically cumbersome slipping contact problems, which are frequently observed in highly dynamic motions. We employ and modify a combination of two different types of cones representing the inequality constraints of a contact model: the friction cone and the velocity cone. The friction cone makes character animation physically plausible while the velocity cone allows a character to perform a sharp turn without foot-to-ground penetration. Our system effectively simulates human behavior using an inverted pendulum on a cart (IPC) model and motion capture data. In the pre- processing step, we analyze motion capture data to extract meaningful information for the IPC model. At run-time, our system produces a physically simulated character by tracking the desired motion that is predicted by the IPC model. We formulate human motion control as a quadratic programming satisfying the proposed foot-to-ground contact constraints. Our examples show that the proposed system can produce physically plausible character animation without noticeable foot-to-ground contact artifacts. Keywords Physics-based simulation · Character animation · Data-driven animation Electronic supplementary material The online version of this article (doi:10.1007/s00371-015-1097-8) contains supplementary material, which is available to authorized users. B Taesoo Kwon [email protected] 1 Division of Computer Science and Software, Hanyang University, Seoul, Republic of Korea 2 Department of Computer Science and Engineering, Seoul National University, Seoul, Republic of Korea 1 Introduction Animated characters are a crucial component of computer games, animated feature films, and virtual reality. With advances in character animation systems, their appearance has become more realistic. However, foot-to-ground con- tact issues such as foot-sticking, sliding and penetration of the feet into the ground still remain major visible arti- facts in many 3D applications. Several static and kinematic methods [12, 17, 20] have been proposed to fix such foot sliding problems. Kinematic-based approaches are efficient and reliable for resolving foot-sliding, but they cannot accu- rately model the intrinsic physical complexity of dynamic foot-to-ground contact. Recently, physical-based animation systems are becoming increasingly popular because they hold promise for emergent adaptation to changes in envi- ronmental and user inputs. Existing physics-based methods, however, usually generate an unnatural foot motion without proper rotation owing to the absence of well-defined foot-to- ground contact models for highly dynamic motions. In this paper, we focus on the dynamic foot-to-ground contact frequently observed in highly dynamic motions such as sports and dancing. For example, during a performance, a dancer may plant one foot on the ground while rotat- ing it on a vertical axis for a few seconds. To address the dynamically cumbersome foot-to-ground contact problems observed in sharp foot turns, we jointly utilize the friction cone and the velocity cone. The friction cone makes anima- tion physically plausible while the velocity cone allows the character to perform a sharp foot turn without foot-to-ground penetration. We also provide an efficient optimization-based human motion control framework while satisfying the foot- to-ground contact constraints. Our controller is built based on the controlling of an inverted pendulum on a cart (IPC). The inverted pendulum model is a simplified physical model 123

Upload: others

Post on 16-Sep-2020

0 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: New Human motion control with physically plausible foot contact …mrl.snu.ac.kr/publications/foot_2015.pdf · 2016. 9. 6. · Human motion control with physically plausible… 885

Vis Comput (2015) 31:883–891DOI 10.1007/s00371-015-1097-8

ORIGINAL ARTICLE

Human motion control with physically plausible foot contactmodels

Jongmin Kim1 · Hwangpil Park2 · Jehee Lee2 · Taesoo Kwon1

Published online: 24 April 2015© Springer-Verlag Berlin Heidelberg 2015

Abstract The foot-to-ground contact model plays animportant role in the simulation of highly dynamic motions,such as turns and kicks. In this paper,we propose amethod forsolving dynamically cumbersome slipping contact problems,which are frequently observed in highly dynamic motions.We employ and modify a combination of two different typesof cones representing the inequality constraints of a contactmodel: the friction cone and the velocity cone. The frictioncone makes character animation physically plausible whilethe velocity cone allows a character to perform a sharp turnwithout foot-to-ground penetration. Our system effectivelysimulates human behavior using an inverted pendulum ona cart (IPC) model and motion capture data. In the pre-processing step, we analyze motion capture data to extractmeaningful information for the IPC model. At run-time, oursystemproduces a physically simulated character by trackingthe desired motion that is predicted by the IPC model. Weformulate humanmotion control as a quadratic programmingsatisfying the proposed foot-to-ground contact constraints.Our examples show that the proposed system can producephysically plausible character animation without noticeablefoot-to-ground contact artifacts.

Keywords Physics-based simulation · Characteranimation · Data-driven animation

Electronic supplementary material The online version of thisarticle (doi:10.1007/s00371-015-1097-8) contains supplementarymaterial, which is available to authorized users.

B Taesoo [email protected]

1 Division of Computer Science and Software,Hanyang University, Seoul, Republic of Korea

2 Department of Computer Science and Engineering,Seoul National University, Seoul, Republic of Korea

1 Introduction

Animated characters are a crucial component of computergames, animated feature films, and virtual reality. Withadvances in character animation systems, their appearancehas become more realistic. However, foot-to-ground con-tact issues such as foot-sticking, sliding and penetrationof the feet into the ground still remain major visible arti-facts in many 3D applications. Several static and kinematicmethods [12,17,20] have been proposed to fix such footsliding problems. Kinematic-based approaches are efficientand reliable for resolving foot-sliding, but they cannot accu-rately model the intrinsic physical complexity of dynamicfoot-to-ground contact. Recently, physical-based animationsystems are becoming increasingly popular because theyhold promise for emergent adaptation to changes in envi-ronmental and user inputs. Existing physics-based methods,however, usually generate an unnatural foot motion withoutproper rotation owing to the absence of well-defined foot-to-ground contact models for highly dynamic motions.

In this paper, we focus on the dynamic foot-to-groundcontact frequently observed in highly dynamic motions suchas sports and dancing. For example, during a performance,a dancer may plant one foot on the ground while rotat-ing it on a vertical axis for a few seconds. To address thedynamically cumbersome foot-to-ground contact problemsobserved in sharp foot turns, we jointly utilize the frictioncone and the velocity cone. The friction cone makes anima-tion physically plausible while the velocity cone allows thecharacter to perform a sharp foot turn without foot-to-groundpenetration. We also provide an efficient optimization-basedhuman motion control framework while satisfying the foot-to-ground contact constraints. Our controller is built basedon the controlling of an inverted pendulum on a cart (IPC).The inverted pendulum model is a simplified physical model

123

Page 2: New Human motion control with physically plausible foot contact …mrl.snu.ac.kr/publications/foot_2015.pdf · 2016. 9. 6. · Human motion control with physically plausible… 885

884 J. Kim et al.

generated using motion capture data and closely producesthe reference human motion using a forward-dynamics sim-ulator. With this pendulum model, physically valid characteranimation is reconstructed by tracking the desired humanmotion.

A forward-dynamics simulation step is formulated as aquadratic programming (QP). The reason why we utilize theQP-based optimization framework is because of its robust-ness, effectiveness, and computational efficiency.We imposethe two types of linear constraints, which originate fromthe friction and velocity cone. We consider another set oflinear constraints derived from the Newton–Euler equation,which ensure that character animation is governed by thelaws of physics. We define the objective function as min-imizing the difference between the desired motion that ispredicted from the pendulummodel and the actual motion tobe simulated. The simulated motion is obtained by integrat-ing the acceleration obtained from a QP solver in an onlinemanner.

To generate highly believable character motion, we com-pare the results from four different foot contact models usingtwo reference motions: sharp turns and kicks. Based on theexperiments, we determine the optimal location and shape ofthe velocity cones to achieve accurate sharp foot turns alonga vertical axis while maintaining ground contact. Our resultsshow that the proposed system can produce physically plau-sible character animation while satisfying the defined footconstraints without introducing visible foot-to-ground con-tact artifacts.

2 Related work

For the past several decades, physics-based locomotioncontrol has been a challenging problem in the field of com-puter graphics and robotics. In recent years, optimization-based motion control has become an attractive strategy toimprove stability and robustness. Many optimization-basedapproaches [1,23,26,27,33] have designed to accuratelyaddress the dynamics of character, and compute joint torquesand contact forces through minimizing the defined objec-tive function while satisfying all physical constraints. Asoptimization-based approaches compute all joint torquesat once, they often achieve more robustness when com-pared with proportional derivative (PD) servo-based walkingcontrollers [10,13,18,21,31,34]. In addition, optimization-based methods can handle higher levels of dynamic con-straints to human motion to formulate physically complexproblems such as controlling cartwheels and back-flips. Inthis sense, our human motion controller is built upon anoptimization-based scheme to generate more flexible andphysically plausible human locomotion in various scenar-ios. We would like to refer readers to Geijtenbeek et al.’s

study [11] for a comprehensive overview on physics-basedlocomotion control.

Tracking the reference motion capture data are advanta-geous for reproducing believable human motion [8,14,18,21,22,27,30,31,34]. Such data-driven approaches can gen-erate high-quality and natural styles of characters becausemotion capture data contain abundant details and nuancesof original human motion. Our controller is built on a refer-ence tracking algorithm proposed by Kwon and Hogins [18],which optimized an IPC based on example data to producereference footstep trajectories. This control algorithm hasbeen employed by a few different researchers. Lee et al. [22]presented a robust, many muscle-actuated biped locomotioncontroller using online QP. Hwang et al. [14] proposed acontroller for synthesizing two animated characters whilemaintaining physical interactions. Similarly, our controlleris a combination of the IPC model for motion planning andQP for per-frame tracking optimization. The main differ-ence from previous QP-based methods is that we are ableto produce dynamic sharp turns, as we demonstrate in theexperimental results section.

Handling foot-to-ground contact is a well-known, impor-tant, and challenging problem in the fields of characteranimation, robotics, and video games. In particular, resolv-ing foot sliding is a major problem for character animationthat has been studied by several researchers [12,17,20]. Theyaddressed this problem by utilizing inverse kinematics (IK).Although these methods have shown convincing results forresolving foot-sliding, few of the suggested methods can beapplied to dynamic foot contact problems involving rapidrotational foot slipping. In terms of physics-based simula-tions, the penalty method has been widely used in many 3Dapplications to address the foot-to-ground contact problem.This method applies a normal directional force to preventpenetration of the foot into the ground. While the penaltymethod is simple and fast, occasionally feet are jittered. Thefoot-to-ground contact problem also can be formulated as alinear complementarity problem (LCP). The LCP formula-tion by Baraff et al. [2–6] is commonly composed of threetypes of constraints: a friction cone, a velocity cone and anadditional complementarity constraint. To maintain foot-to-ground contact, either the normal velocity or the contact forceshould have a zero value at the contact position. The opendynamics engine (ODE) [28] provides the LCP formulationfor handling a contact problem. While this formulation isnumerically robust, it may fail tomaintain balance in the caseof dynamic and complex foot-to-ground contact.An impulse-based simulation was proposed by Mirtich et al. [24,25]to handle contact problems in a different way. The mainidea of the impulse-based simulation is that each rigid bodyis treated independently. The contact collision is resolvedby applying impulsive contact force to the contact objects.The impulse-based methods are used in dynamics engines

123

Page 3: New Human motion control with physically plausible foot contact …mrl.snu.ac.kr/publications/foot_2015.pdf · 2016. 9. 6. · Human motion control with physically plausible… 885

Human motion control with physically plausible… 885

Bullet [7]. Although our foot-to-ground contact model isphysically compromised compared to the aforementionedLCP and impulse-based approaches, our system can generatea physically plausible sharp foot turn and robustly simulatehuman behavior in an efficient manner thanks to the QP-based per-frame optimization.

3 Overview

Our goal is to control human locomotion while satisfyingthe pre-defined foot constraints without visible artifacts in arealistic and robust manner. As shown in Fig. 1, our systemconsists of two stages: motion pre-processing and synthesis.The first stage is building an IPCmodel to follow the center ofmass (COM) of reference humanmotion. The parameters forthe IPC model are computed by an optimization process. Inthe second stage, we synthesize physically plausible humanmotion online. Our system estimates a desired IPC trajectoryin a short time period, and generates a new reference motionby modifying human motion capture data using the IK. Wethen produce a full-body character animation that maintainsbalance during dynamically cumbersome foot contact bytracking the given desired humanmotion.We design the foot-to-ground contact model to handle the sharp foot turns thatfrequently occur in highly dynamic motions. Our foot-to-ground contact model consists of friction and velocity conethat are partially related to previous successful approachesbased on the laws of Coulomb friction [1,19,22,30]. In thecontext of the geometric structure, the velocity cone is similarto the friction cone, but their roles are different. The frictioncone is used for physically plausible foot-to-ground contact.

Fig. 1 Overall workflow of our system

The velocity cone allows a character to perform a sharp footturn while preventing foot-to-ground penetration. At run-time, the defined foot-to-ground contact model is integratedinto the QP-based optimization process for motion synthesis.Our foot-to-ground contactmodel is efficient, robust, and fastenough to reproduce physically valid human motion withoutany foot-to-ground contact artifacts.

4 Motion pre-processing

In this section, we describe the character and IPC modeland a method for generating natural human motion withoutloss of balance. In the off-line stage, we analyze motion cap-ture data to extract meaningful information such as the COMand joint positions. From the obtained information, we buildthe IPC model for online human motion planning. The IPCmodel continuously predicts the trajectory of human motionin multiple future footsteps. In addition, we employ a linearquadratic regulator (LQR) to maintain balance under varioustypes of scenarios.

4.1 Character and IPC model

The character model in our system consists of 46 degrees offreedom (DOFs) including six unactuated DOFs within thepelvis. We use a 1-DOF hinge joint for each ankle and knee.The lower back and neck have 2-DOFs. We use 3-DOFs forthe remaining joints. We acquire the mass and inertia matrixfor each bone from a surfacemeshwith uniformly distributeddensity and the totalmass of the charactermodel. The surfacemesh is manually embedded into a character to fit the motioncapture marker positions as close as possible. The IPCmodelhas been widely used in dynamics and control theory [15,32]because its balancing strategy is simple and computationallyefficient. Similar to the previous method [18], we employ theIPC model and modify it to maintain the balance of the char-acter in real-time. We design the IPC as a rigid body withtwo-dimensional translation to move in the horizontal direc-tion and one rotation to swing in the opposite direction fromwhich balance is lost. The IPC model is embedded into themotion capture data such that the trajectory of the COM ofthe IPC closely approaches that of the COM of the charac-ter. While maintaining the balance of the pendulum, the IPCmodel generates a new pendulum trajectory. Our system thencontinuously converts it into full-body posture in real-time.

4.2 Balance control

We design the dynamics for the balance of the IPC model byemploying an LQR model due to its simplicity and stability.The LQR model can be represented as a time-varying linearsystem, s = As+Bu, where s is a state vector, u is a control

123

Page 4: New Human motion control with physically plausible foot contact …mrl.snu.ac.kr/publications/foot_2015.pdf · 2016. 9. 6. · Human motion control with physically plausible… 885

886 J. Kim et al.

input, andA andB are the closed-loopmatrices (see details inDorato et al.’s study [9]). In addition, we consider two LQRmodels for various directions: facing and lateral directionsof the IPC model. The LQR model maintains the balanceof the pendulum by computing the differences between thependulum’s current velocity, x, and the desired velocity, ˆx,according to the following equation:

u = −K(x − ˆx), (1)

where u denotes the control force, and K is the control gainmatrix of an LQR. In pre-processing, we obtain the desiredvelocity, ˆx, of the IPC by solving the optimization problem.We minimize the difference between the COM of the char-acter and that of the pendulum in the ground. The results ofthe optimization are a set of desired velocities for the IPCmodel, { ˆx}, for all frames of the motion capture data. Thestudy by Kwon and Hodgins [18] provides the details of theoptimization process.

5 Constraints

To produce character animation that is physically natural, weemploy a set of constraints imposed by external forces on thecharacter. One of the main external forces on the character isthe ground reaction force. Theground reaction force,which isnot pulling, but rather pushing, is produced at foot-to-groundcontact positions. We model the ground reaction force usingthe friction cone, which is based on the laws of Coulomb fric-tion. Furthermore, handling dynamic foot-to-ground contactis a crucial issue for simulation of human motion, becausesuch contact frequently occurs in turning, kicking, and othertypes of the dynamic motions. To address this issue, we pro-pose a new type of velocity cone and show that it allowsa character to perform a sharp foot turn while preservingthe foot-to-ground contact. In addition, the resulting charac-ter motion is physically accurate in that the Newton–Eulerequation is exploited.

5.1 Friction cone

We avoid foot-to-ground penetration using the ground reac-tion force that is exerted by the ground on the character’sfeet. The ground reaction forces fc can be modeled by theCoulomb friction model, which states that the ground reac-tion forces should exist inside the friction cones. We designeach friction cone as a k-sided pyramidal shape, which iscomposed of a collection of normal basis vectors, as shown inFig. 2. The ground reaction forces should be directed upwardfrom the ground to prevent foot-to-ground penetration:

fc = L(µ)λ, λ ≥ 0, (2)

Fig. 2 Illustration of our foot-to-ground contact model. The foot-to-ground contact is approximatelymodeled by two types of velocity conesand k-sided pyramidal friction cone. The velocity cones that span 180◦are aligned at the foot contact positions, while the k-sided velocity conesexist at the toe and heel position. Each k-sided pyramidal friction andvelocity cone is defined as a collection of the k normal basis vectors

where L(µ) represents a matrix that contains a set of normalbasis vectors of the friction cones, µ denotes the frictioncoefficient, and λ is a coefficient vector for the normal basesvectors of the friction cones. The torques τ c can be obtainedby multiplying the transpose of the Jacobian JTc (q) of thecharacter pose q by the ground reaction forces fc as follows:

τ c = JTc (q)fc = JTc (q)L(µ)λ, λ ≥ 0. (3)

5.2 Velocity cone

The velocity cone is a well-known kinematic foot-to-groundcontactmodel in thefield of robotics and character animation.Previous studies have designed the geometry structure of thevelocity cone to be analogous to that of the friction cone.They also aligned the locations of the velocity cones with thefriction cones. The key property of the velocity cone is thatfoot-sliding occurs when the foot velocity vector exists out-side of the velocity cone. Many previous approaches soughtto restrict the direction of the foot velocity vector to bewithinthe cone to prevent visible foot-to-ground contact artifacts.The concept of the proposed velocity cone in this paper issimilar to that of previous velocity cones. However, the loca-tion, shape, and complexity of the velocity cones vary, andourwell-designedvelocity cone allows a character to performa sharp foot turn while maintaining contact between the footand the ground. To the best of our knowledge, allowing asharp foot turn while maintaining contact with the groundvia the concept of the velocity cone has not been explored inprevious studies.

There are three reasons that we employ and modify thevelocity cone model to solve the dynamically cumbersomefoot-to-ground contact problem. First, the motion quality

123

Page 5: New Human motion control with physically plausible foot contact …mrl.snu.ac.kr/publications/foot_2015.pdf · 2016. 9. 6. · Human motion control with physically plausible… 885

Human motion control with physically plausible… 887

does not vary with different types of motions under the samevelocity cone structure. The proposed velocity cone workswell even for various types of highly dynamic motions. Sec-ond, the velocity cone can be simply formulated as a linearconstraint. We can then easily embed the linear constraintsinto the QP-based optimization which achieves robustnessand computational efficiency. Finally, we can convenientlyadjust the range of foot-sliding by manipulating the directionof the k normal basis vectors of the velocity cone. As all knormal basis vectors of the velocity cone are perpendicularwith the ground, foot-sliding occurs gradually.

In highly dynamic motions, the main foot-to-ground con-tacts at a sharp foot turn often take place close to the toe andheel position. Foot-sliding also occurs around these posi-tions. At the toe and heel position, we exploit the k-sidedpyramidal velocity cones that span less than 45◦ (see Fig. 2).The overall role of the k-sided pyramidal velocity cones isto allow foot rotation and maintain foot-to-ground contact.We also employ several velocity cones that have an openingangle of 180◦ around the toe and heel position to performfoot-sliding. We model the foot-plate as a 2D grid shape, asshown in Fig. 2, and uniformly sample the boundary pointsof the 2D grid for velocity cones that span 180◦. The locationof the k-sided pyramidal velocity cone at the toe or heel isdefined by the centroid of convex hull, which is computedfrom the boundary points of the 2D grid. We discard theinner points of convex hull except the centroid points for thek-sided pyramidal velocity cones.

The inner products between the normal basis vectors of thevelocity cones and global velocities at contact points arewrit-ten as vc = VT Jc(q)q ≥ 0, whereV is a matrix that containsthe normal basis vectors of the velocity cones and Jc(q)q is avector of the global velocities at the foot-to-ground contacts.By differentiating vc with respect to time, we can obtain theequation below:

VT Jc(q)q + VT Jc(q)q + VT Jc(q)q ≥ lc, (4)

where q is the joint acceleration vector, Jc(q) is the differen-tial of the Jacobian Jc(q), and lc is the lower boundary offsetvector. In addition, we limit joint acceleration to preventoverextension of the knee and elbow joint. This conditioncan be written as the linear inequality constraints with pre-defined upper and lower boundary. We add these constraintsto our QP-based optimization framework.

5.3 Equations of motion

The character motion generated from our system should sat-isfy the laws of physics. The Newton–Euler equations ofmotion are written as

M(q)q + C(q, q) + g(q) = τ i + τ c, (5)

where M(q) is the inertial matrix, C(q, q) represents thecentrifugal and Coriolis matrix, g(q) denotes gravitationalforces, and τ i is the internal torques applied at the joints.

6 Online motion synthesis

In this section, we describe a method for online motionsynthesis satisfying the constraints originating from thefoot-to-ground contact model. Our system enters a motionplanning step to produce the reference full-body posturesfrom multiple computed footsteps in a short time period.The resulting character animation is produced by trackingthe desired motion at every frame. We formulate the overallalgorithm as QP-based optimization.

Motion Planning At run-time, the IPC model predicts thefuture foot trajectories from estimation of the current pendu-lum state. At each foot position on the trajectory, our systemsolves the IK, as proposed in [17], for human lower bodymotion such that the reference motion consistently followsthe newly computed footprints. However, the foot trajectoriesmay be much longer or shorter than the original trajectories.To address this problem, we adjust the footprint frequency toensure that unnatural strides are prevented (see details in thestudy by Kwon and Hodgins [18]).

Optimization We employ QP-based optimization to gener-ate optimal joint accelerations, joint torques, and contactforces by tracking the desired motion at each frame. TheQP-based optimization framework consists of an objectivefunction E (q,λ, τ i ) and a set of linear constraints obtainedfrom the equations of motion, friction, and velocity cones asfollows:

minimizeq,λ,τ i

E (q,λ, τ i ) (6a)

subject to Eq. (3), (4), and (5). (6b)

Specifically, the objective function E (q,λ, τ i ) includesthree energy terms:

E (q,λ, τ i ) = wt Et + wcEc +∑

k

wke E

ke , (7)

where wt , wc, and wke are the weight values for each energy

term and k ∈ {right foot, left foot, head}. The first energyterm Et in Eq. (7) measures the errors between the currentand the desired motion:

Et = ‖q − qd‖2, (8)

where q is all of the joint accelerations and the desiredaccelerations qd is defined as follows:

qd = wp(qr − q) + wd(qr − q) + qr , (9)

123

Page 6: New Human motion control with physically plausible foot contact …mrl.snu.ac.kr/publications/foot_2015.pdf · 2016. 9. 6. · Human motion control with physically plausible… 885

888 J. Kim et al.

whereqr , qr , and qr are all of the reference joint angles, angu-lar velocities, and angular accelerations, respectively. In ourexperiments, the weight values are wp = 200 and wd = 30.Here, Ec regularizes the contact forces to prevent overshoot-ing of the forces from the ground, such that the resultingcharacter animation is able to be robustly controlled:

Ec = ‖λ‖2. (10)

To accurately follow the positions and angles of the end effec-tors of the desiredmotion,we consider an end-effector energyterm Ek

e as follows:

Eke = ‖Jk(q)q + Jk(q)q − pkd‖2, (11)

where pkd is the desired accelerations of the kth end effector,Jk(q) denotes the Jacobian of the kth end effector, and k ∈{right foot, left foot, head}.

7 Experimental results

We reproduced highly dynamic and natural motions usingour well-designed foot-to-ground contact model. Figures 4and 5 depict our simulation results of the turning and kickingmotion example. The turning motion example consists ofwalking motions that move forward, then rotation of the feetwith it planted on the ground. The kicking motion exampleincludes highly dynamicmotions such as kickingwith a sharpfoot turn motion.

We evaluated the strength and effectiveness of the pro-posed foot-to-ground contact model in various experiments.To find the most reliable foot-to-ground contact model, weset up four cases of the foot-to-ground contact models withvarying ways of employing different combinations of veloc-ity cone shapes and locations (see Fig. 3). Case 1 is a modelthat enables sharp foot turns while Case 2 is a general veloc-ity cone model, which has been widely used in previousapproaches. We also designed Case 3, which only has k-sided pyramidal velocity cones at the toe and heel position.In Case 4, several velocity cones that span 180◦ are locatedaround each heel and toe position, and the velocity cones atthe toe and heel position are absent.

In our experiments, the sharp foot turn motion reproducedin Case 1 was the most similar and accurate compared withthe reference motion among the four foot-to-ground contactmodels, as we expected (see accompanying video). In Case2, we observed visible artifacts in foot-to-ground contact,such as not allowing sharp foot turns and toeing off in theground. In Case 3, evenmore undesirable toeing off was seenduring a kicking motion. From the turning motion generatedin Case 3, we also observed artifacts such as the heels unit-ing.With the foot-to-ground contact model in Case 4, human

Fig. 3 Illustration of four different cases of the foot-to-ground contactmodels. We design four cases of velocity cones that combine k-sidedpyramidal velocity cones and velocity cones having an opening angleof 180◦

motions were reconstructed with excessive foot-sliding andunexpected rotations of the ankle joint because the veloc-ity cones that have a role in maintaining the foot-to-groundcontacts at toe and heel position were nonexistent.

We quantitatively evaluated all foot-to-ground contactmodels by measuring the difference in average error in alljoint positions and angles between the reference and simu-lated turning motion. In Fig 6, the vertical axis depicts theaverage error of the positions or angles of all joints while thehorizontal axis shows the frame number of the given motion.We also measured the total average errors and their stan-dard deviations over the test motion pertaining to all of thefoot-to-ground contact models (see Fig 6). The average errorand standard deviation of our foot-to-ground contact modelwere smaller than all other test cases. The positional errorswere measured after aligning each pose by vertical rotationusing a point cloudmatching algorithm in the study byKovaret al. [16].

8 Discussion

We have presented a new method for controlling humanmotion with handling dynamic foot-to-ground contact. Wepropose a novel foot-to-ground contact model by jointly uti-lizing the friction and velocity cone and modifying the shapeof the velocity cone to introduce a sharp foot turn. We for-mulate them as linear constraints and consider another setof linear constraints, the equation of motion for obeying the

123

Page 7: New Human motion control with physically plausible foot contact …mrl.snu.ac.kr/publications/foot_2015.pdf · 2016. 9. 6. · Human motion control with physically plausible… 885

Human motion control with physically plausible… 889

Fig. 4 The first row depicts the screenshots of the reference motion. The second row represents the turning motion generated by our method

Fig. 5 The first row represents the screenshots of the reference motion. The second row shows the kicking motion reproduced by our method

law of physics. In the pre-processing step, we design the IPCmodel for footprint generation and the LQRmodel for main-taining the balance of the character. In the motion synthesisstep, our system generates the resulting character animationby tracking the desired motion while satisfying the definedlinear constraints.

We compare our approach with previous methods per-taining to five distinct categories for human motion controlas summarized in Table 1. The criteria consist of whether

the method utilizes motion capture data, whether the IPCand LQR model are employed for human motion control,whether the method is based on the QP-based optimizationframework,whether humanmotion is simulatedwith the fric-tion and velocity cone, and whether a sharp foot turn can bereproduced. The common criterion is the use of motion cap-ture data, which enables the reproduction of plenty of detailsand nuances of original human motion. The QP-based opti-mization concept of a recent work [22] is similar to that of

123

Page 8: New Human motion control with physically plausible foot contact …mrl.snu.ac.kr/publications/foot_2015.pdf · 2016. 9. 6. · Human motion control with physically plausible… 885

890 J. Kim et al.

Fig. 6 Comparison of the accuracies of the four different cases offoot-to-ground contact models. The reconstruction errors of the jointpositions and angles between the reference and simulated turningmotion were computed using the root mean square (RMS)

Table 1 Comparison with previous methods

(a) (b) (c) Ours

Data-driven approach Y Y Y Y

IPC / LQR model Y N Y Y

QP-based optimization N N Y Y

Friction / Velocity cone N N Y Y

Sharp foot turn N N N Y

(a) Kwon et al. [18], (b) Lee et al. [21], (c) Lee et al. [22]

ours, however, the purpose is different (i.e., simulating thehuman musculoskeletal system vs. handling dynamic foot-to-ground contact). The location, shape, and complexity ofthe velocity cone are also different, such that we can gener-ate sharp foot turn motions that were not achievable usingprevious approaches [18,21,22].

Alternatively, we can consider redesigning the frictioncone instead of the velocity cone to simulate a sharp footturn. To do so, the rotational friction would be modeled byonly allowing a range proportional to the magnitude of con-tact normal force. However, this cannot be represented usinglinear constraints and some kind of decoupling is necessary,which may be undesirable for achieving a reasonable solu-tion.

Similar to the work [29], further investigation of dynamicfoot-to-ground contact could involve a system that evaluatesthe perception of a sharp foot turn. We can easily conduct

various cases of experiments in a sense that the foot turningmotion ranges from non-rotating to rotating ankle by simpleadjustment of the shape of the velocity cone. To the extent ofour knowledge, perceptual evaluation of sharp foot turns hasnot been previously explored, and we believe that this wouldbe an interesting direction for future research.

Acknowledgments We thank to the anonymous reviewers for theirhelpful comments. This research was supported by Basic ScienceResearch Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Science, ICT and Future Planning(MSIP) (NRF-2014R1A1A1038386).

References

1. Abe, Y., da Silva, M., Popovic, J.: Multiobjective controlwith frictional contacts. In: Proceedings of the 2007 ACMSIGGRAPH/Eurographics symposium on computer animation,pp. 249–258, Eurographics Association (2007)

2. Baraff, D.: Analytical methods for dynamic simulation of non-penetrating rigid bodies. ACMSIGGRAPHComput. Graph. 23(3),223–232 (1989)

3. Baraff, D.: Issues in computing contact forces for non-penetratingrigid bodies. Algorithmica 10(2–4), 292–352 (1993)

4. Baraff, D.: Fast contact force computation for nonpenetrating rigidbodies. In: Proceedings of the 21st annual conference on computergraphics and interactive techniques, pp. 23–34, ACM (1994)

5. Baraff, D.: Interactive simulation of solid rigid bodies. IEEE Com-put. Graph. Appl. 15(3), 63–75 (1995)

6. Baraff, D.: Linear-time dynamics using lagrange multipliers. In:Proceedings of the 23rd annual conference on computer graphicsand interactive techniques, pp. 137–146. ACM (1996)

7. Bullet: Bullet Physics Library (2013). http://bulletphysics.org/8. DaSilva,M.,Abe,Y., Popovic, J.: Simulationof humanmotiondata

using short-horizonmodel-predictive control. In: ComputerGraph-ics Forum, vol. 27, pp. 371–380. Wiley Online Library (2008)

9. Dorato, P., Cerone, V., Abdallah, C.: Linear-quadratic control: anintroduction. Simon & Schuster, New York (1994)

10. Faloutsos, P., Van de Panne,M., Terzopoulos, D.: Composable con-trollers for physics-based character animation. In: Proceedings ofthe 28th annual conference on computer graphics and interactivetechniques, pp. 251–260. ACM (2001)

11. Geijtenbeek, T., Pronost, N., Egges, A., Overmars, M.H.: Interac-tive character animation using simulated physics. Eurographics-state of the art reports 2 (2011)

12. Glardon, P., Boulic, R., Thalmann, D.: Robust on-line adaptivefootplant detection and enforcement for locomotion. Vis. Comput.22(3), 194–209 (2006)

13. Hodgins, J.K., Pollard, N.S.: Adapting simulated behaviors for newcharacters. In: Proceedings of the 24th annual conference on Com-puter graphics and interactive techniques, pp. 153–162.,Addison-Wesley Publishing Co, ACM Press (1997)

14. Hwang, J., Suh, I., Kwon, T.: Editing and synthesizing two-character motions using a coupled inverted pendulum model. In:Computer Graphics Forum, vol. 33, pp. 21–30. Wiley OnlineLibrary (2014)

15. Kajita, S., Nagasaki, T., Kaneko, K., Yokoi, K., Tanie, K.: A hoptowards running humanoid biped. In: Robotics and Automation,2004. Proceedings. ICRA’04. 2004 IEEE international conferenceon, vol. 1, pp. 629–635, IEEE (2004)

16. Kovar, L., Gleicher, M., Pighin, F.: Motion graphs. In: ACM trans-actions on graphics (TOG), vol. 21, pp. 473–482. ACM (2002)

123

Page 9: New Human motion control with physically plausible foot contact …mrl.snu.ac.kr/publications/foot_2015.pdf · 2016. 9. 6. · Human motion control with physically plausible… 885

Human motion control with physically plausible… 891

17. Kovar, L., Gleicher,M., Schreiner, J.: Footskate cleanup formotioncapture. In: ACM Siggraph Symposium on Computer Animation(2002)

18. Kwon, T., Hodgins, J.: Control systems for human running using aninverted pendulummodel and a referencemotion capture sequence.In: Proceedings of the 2010 ACM SIGGRAPH/EurographicsSymposium on Computer Animation, pp. 129–138. EurographicsAssociation (2010)

19. de Lasa, M., Mordatch, I., Hertzmann, A.: Feature-based locomo-tion controllers. ACM Trans. Graph. (TOG) 29(4), 131 (2010)

20. Lee, J., Shin, S.Y.: A hierarchical approach to interactive motionediting for human-like figures. In: Proceedings of the 26th annualconference on computer graphics and interactive techniques (1999)

21. Lee, Y., Kim, S., Lee, J.: Data-driven biped control. ACM Trans.Graph. (TOG) 29(4), 129 (2010)

22. Lee, Y., Park, M.S., Kwon, T., Lee, J.: Locomotion control formany-muscle humanoids. ACM Trans. Graph. (TOG) 33(6), 218(2014)

23. Macchietto, A., Zordan, V., Shelton, C.R.: Momentum control forbalance. ACM Trans. Graph. (TOG) 28, 80 (2009)

24. Mirtich, B., Canny, J.: Impulse-Based Dynamic Simulation. Com-puter Science Division (EECS), University of California, Califor-nia (1994)

25. Mirtich, B., Canny, J.: Impulse-based simulation of rigid bodies.In: Proceedings of the 1995 symposium on Interactive 3D graphics,pp. 181-ff ACM (1995)

26. Mordatch, I., De Lasa, M., Hertzmann, A.: Robust physics-basedlocomotion using low-dimensional planning. ACM Trans. Graph.(TOG) 29(4), 71 (2010)

27. Muico, U., Lee, Y., Popovic, J., Popovic, Z.: Contact-aware non-linear control of dynamic characters. ACM Trans. Graph. 28(3),81:1–81:9 (2009)

28. ODE: Open Dynamic Engine (2004). http://www.ode.org/29. Pražák, M., Hoyet, L., O’Sullivan, C.: Perceptual evaluation

of footskate cleanup. In: Proceedings of the 2011 ACM SIG-GRAPH/Eurographics Symposium on Computer Animation, pp.287–294. ACM (2011)

30. da Silva, M., Abe, Y., Popovic, J.: Interactive simulation of stylizedhuman locomotion. ACM Trans. Graph. (TOG) 27, 82 (2008)

31. Sok, K.W., Kim, M., Lee, J.: Simulating biped behaviors fromhuman motion data. ACM Trans. Graph. (TOG). 26, 107 (2007)

32. Sugihara, T.: Simulated regulator to synthesize zmp manipula-tion and foot location for autonomous control of biped robots. In:Robotics and Automation, 2008. ICRA 2008. IEEE InternationalConference on, pp. 1264–1269. IEEE (2008)

33. Wu, Jc, Popovic, Z.: Terrain-adaptive bipedal locomotion control.ACM Trans. Graph. (TOG) 29(4), 72 (2010)

34. Yin, K., Loken, K., Van de Panne, M.: Simbicon: simple bipedlocomotion control. ACM Trans. Graph. (TOG) 26, 105 (2007)

Jongmin Kim received the BSdegree in EECS from KAIST,Daejeon, Korea, in 2006, andthe PhD degree in EECS fromSeoulNationalUniversity, Seoul,Korea, in 2014. Currently, he is aresearch professor of Institute forEmbedded Software at HanyangUniversity. His research interestsinclude computer graphics, char-acter animation, machine learn-ing, and numerical optimization.

Hwangpil Park received the BSdegree in Electrical and Com-puter Engineering from SeoulNational University, Korea, in2012. He is currently work-ing toward the Ph.D. degreein the Department of ComputerScience and Engineering, SeoulNational University, Korea. Hisresearch interests include com-puter graphics and animation.

Jehee Lee received his PhDdegrees in computer sciencefrom Korean Advanced Insti-tute of Science and Technology.He is a full professor of com-puter science and engineering atSeoul National University. Hisresearch interests are in the areasof computer graphics, animation,biomechanics, and robotics. Heis interested in developing newways of understanding, repre-senting, planning, and simulatinghuman and animal movements.This involves full-body motion

analysis and synthesis, biped control and simulation, clinical gait analy-sis, motion capture, motion planning, data-driven and physically basedtechniques, interactive avatar control, crowd simulation, and controllerdesign. He cochaired ACM/EG Symposium on Computer Animationin 2012 and served on numerous program committees, including ACMSIGGRAPH, ACM SIGGRAPH Asia, ACM/EG Symposium on Com-puter Animation, Pacific Graphics, CGI, and CASA. He is leading theSNU Movement Research Laboratory.

Taesoo Kwon received the BSand MS degrees in computer sci-ence from Seoul National Uni-versity, and the PhD degree fromKAIST.After three years of post-doctral research at CarnegieMel-lon University, he is currently anassistant professor at HanyangUniversity in Korea. His recentresearch interests include com-puter graphics, data-driven andphysical-based character anima-tion, and interaction modeling.

123