a hierarchical approach to interactive motion editing for ...mrl.snu.ac.kr/papers/sig99.pdf · a...

10
A Hierarchical Approach to Interactive Motion Editing for Human-like Figures Jehee Lee Sung Yong Shin Computer Science Department Korea Advanced Institute of Science and Technology Abstract This paper presents a technique for adapting existing motion of a human-like character to have the desired features that are specified by a set of constraints. This problem can be typically formulated as a spacetime constraint problem. Our approach combines a hierar- chical curve fitting technique with a new inverse kinematics solver. Using the kinematics solver, we can adjust the configuration of an articulated figure to meet the constraints in each frame. Through the fitting technique, the motion displacement of every joint at each constrained frame is interpolated and thus smoothly propagated to frames. We are able to adaptively add motion details to satisfy the constraints within a specified tolerance by adopting a multilevel B- spline representation which also provides a speedup for the inter- polation. The performance of our system is further enhanced by the new inverse kinematics solver. We present a closed-form solu- tion to compute the joint angles of a limb linkage. This analytical method greatly reduces the burden of a numerical optimization to find the solutions for full degrees of freedom of a human-like artic- ulated figure. We demonstrate that the technique can be used for re- targetting a motion to compensate for geometric variations caused by both characters and environments. Furthermore, we can also use this technique for directly manipulating a motion clip through a graphical interface. CR Categories: I.3.7 [Computer Graphics]: Three- dimensional Graphics—Animation; G.1.2 [Numerical Analysis]: Approximation—Spline and piecewise polynomial approximation Keywords: Motion Editing, Motion Adaptation, Spacetime Con- straints, Hierarchical Techniques, Inverse Kinematics 1 Introduction Animating human-like characters is a recurring issue in computer graphics. Recently, motion capture has become one of the most promising technologies in character animation. Realistic motion data can be captured by recording the movement of a real actor with an optical or magnetic motion capture system. A motion li- brary, that is an archive of reusable motion clips, is also commer- [email protected], http://tclab.kaist.ac.kr/ jehee [email protected], http://tclab.kaist.ac.kr/ syshin KAIST, 373-1 Kusung-dong Yusung-gu, Taejon, Republic of Korea cially available. Much of the recent research in motion control has been devoted to developing various kinds of editing tools to produce a convincing motion from prerecorded motion clips. To reuse motion-captured data, animators often adapt them to a dif- ferent character, i.e., retargetting a motion from one character to another [13], or to a different environment to compensate for geo- metric variations [3, 37]. Animators also combine two motion clips in such a way that the end of one motion is seamlessly connected to the start of the other [28]. The core of the motion manipulation can be modeled as a space- time constraint problem [12, 13, 28]. Each kinematic constraint specifies the desired position or orientation of an end-effector, such as a foot and a hand, of an articulated figure at a specific time. The important features of the target motion are specified interactively as constraints, and the captured motion is deformed to satisfy those constraints. Motion data consist of a bundle of motion signals. Each signal represents a sequence of sampled values for each degree of free- dom. Those signals are sampled at a sequence of discrete time in- stances with a uniform interval to form a motion clip that consists of a sequence of frames. In each frame, the sampled values from the signals determine the configuration of an articulated figure at that frame, and thus they are related to each other by kinematic constraints. This structure yields two relationships among sampled values: inter-frame and intra-frame relationships. Through the use of an inverse kinematics solver, the intra-frame relationship, that is, the configuration of an articulated figure within each frame can be adjusted to meet the kinematic constraints. However, if each frame is considered independently, then there could be an unde- sirable jerkiness between consecutive frames. Therefore, we have to take account of the inter-frame relationship as well. For this purpose, we employ the multilevel B-spline fitting technique. We also present an efficient inverse kinematics algorithm which is used in conjunction with the fitting technique. Our approach is distin- guished from the work of Gleicher [13] who addressed the same problem. He provided a unified approach to fuse both relationships into a very large non-linear optimization problem, which is cumber- some to handle. Instead, we decouple the problem into manageable subproblems each of which can be solved very efficiently. Multilevel B-spline fitting techniques have been investigated to design smooth surfaces which interpolate scattered features within a specified tolerance [10, 20, 21, 34]. Among them, we extend the technique presented by Lee et al. [21] for adapting a motion to satisfy the constraints which are scattered over the frames. The multilevel B-splines make use of a coarse-to-fine hierarchy of knot sequences to generate a series of uniform cubic B-spline curves whose sum approaches the desired function. At each level in the hierarchy, the control points of the B-spline curve are computed locally with a least-squares method which provides an interactive performance. With this fitting technique, we cannot only manipu- late a curve adaptively to satisfy a large set of constraints within a specified error tolerance, but also edit a curve at any level of detail to allow an arbitrary portion of the motion to be affected through direct manipulation. Exploiting these favorable properties of the

Upload: others

Post on 24-May-2020

10 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: A Hierarchical Approach to Interactive Motion Editing for ...mrl.snu.ac.kr/Papers/sig99.pdf · A Hierarchical Approach to Interactive Motion Editing for Human-likeFigures Jehee Lee

A Hierarchical Approach to Interactive MotionEditing for Human-like Figures

JeheeLee�

SungYongShin�

ComputerScienceDepartmentKoreaAdvancedInstituteof ScienceandTechnology

�Abstract

This paperpresentsa techniquefor adaptingexisting motion of ahuman-likecharacterto have thedesiredfeaturesthatarespecifiedby asetof constraints.Thisproblemcanbetypically formulatedasa spacetimeconstraintproblem. Our approachcombinesa hierar-chicalcurvefitting techniquewith anew inversekinematicssolver.Usingthekinematicssolver, we canadjusttheconfigurationof anarticulatedfigure to meetthe constraintsin eachframe. Throughthefitting technique,themotiondisplacementof every joint ateachconstrainedframeis interpolatedandthussmoothlypropagatedtoframes.We areableto adaptively addmotiondetailsto satisfytheconstraintswithin aspecifiedtoleranceby adoptingamultilevel B-splinerepresentationwhich alsoprovidesa speedupfor the inter-polation. The performanceof our systemis further enhancedbythenew inversekinematicssolver. We presenta closed-formsolu-tion to computethe joint anglesof a limb linkage. This analyticalmethodgreatlyreducesthe burdenof a numericaloptimizationtofind thesolutionsfor full degreesof freedomof ahuman-likeartic-ulatedfigure.Wedemonstratethatthetechniquecanbeusedfor re-targettinga motion to compensatefor geometricvariationscausedby both charactersand environments. Furthermore,we can alsousethis techniquefor directly manipulatinga motionclip throughagraphicalinterface.

CR Categories: I.3.7 [Computer Graphics]: Three-dimensionalGraphics—Animation;G.1.2 [Numerical Analysis]:Approximation—Splineandpiecewisepolynomialapproximation

Keywords: Motion Editing,Motion Adaptation,SpacetimeCon-straints,HierarchicalTechniques,InverseKinematics

1 Introduction

Animatinghuman-like charactersis a recurringissuein computergraphics. Recently, motion capturehasbecomeone of the mostpromisingtechnologiesin characteranimation. Realisticmotiondatacan be capturedby recordingthe movementof a real actorwith an optical or magneticmotion capturesystem.A motion li-brary, that is anarchive of reusablemotionclips, is alsocommer-�

[email protected], http://tclab.kaist.ac.kr/� jehee�[email protected], http://tclab.kaist.ac.kr/� syshin�KAIST, 373-1Kusung-dongYusung-gu,Taejon,Republicof Korea

cially available. Much of the recentresearchin motion controlhasbeendevoted to developing variouskinds of editing tools toproducea convincing motion from prerecordedmotion clips. Toreusemotion-captureddata,animatorsoften adaptthemto a dif-ferent character, i.e., retargettinga motion from onecharactertoanother[13], or to a differentenvironmentto compensatefor geo-metricvariations[3, 37]. Animatorsalsocombinetwo motionclipsin suchawaythattheendof onemotionis seamlesslyconnectedtothestartof theother[28].

Thecoreof themotionmanipulationcanbemodeledasaspace-time constraintproblem[12, 13, 28]. Eachkinematicconstraintspecifiesthedesiredpositionor orientationof anend-effector, suchasa foot anda hand,of anarticulatedfigureat aspecifictime. Theimportantfeaturesof the target motion arespecifiedinteractivelyasconstraints,andthecapturedmotionis deformedto satisfythoseconstraints.

Motion dataconsistof a bundleof motionsignals.Eachsignalrepresentsa sequenceof sampledvaluesfor eachdegreeof free-dom. Thosesignalsaresampledat a sequenceof discretetime in-stanceswith a uniform interval to form a motionclip thatconsistsof a sequenceof frames. In eachframe,the sampledvaluesfromthe signalsdeterminethe configurationof an articulatedfigure atthat frame, and thus they are relatedto eachother by kinematicconstraints.Thisstructureyieldstwo relationshipsamongsampledvalues:inter-frameandintra-framerelationships. Throughtheuseof an inversekinematicssolver, the intra-framerelationship,thatis, theconfigurationof anarticulatedfigurewithin eachframecanbe adjustedto meetthe kinematicconstraints. However, if eachframe is consideredindependently, then therecould be an unde-sirablejerkinessbetweenconsecutive frames.Therefore,we haveto take accountof the inter-frame relationshipas well. For thispurpose,we employ the multilevel B-splinefitting technique.Wealsopresentanefficient inversekinematicsalgorithmwhich is usedin conjunctionwith the fitting technique.Our approachis distin-guishedfrom the work of Gleicher[13] who addressedthe sameproblem.Heprovidedaunifiedapproachto fusebothrelationshipsinto averylargenon-linearoptimizationproblem,whichis cumber-someto handle.Instead,wedecoupletheprobleminto manageablesubproblemseachof whichcanbesolvedveryefficiently.

Multilevel B-splinefitting techniqueshave beeninvestigatedtodesignsmoothsurfaceswhich interpolatescatteredfeatureswithina specifiedtolerance[10, 20, 21, 34]. Among them, we extendthe techniquepresentedby Lee et al. [21] for adaptinga motionto satisfytheconstraintswhich arescatteredover the frames.Themultilevel B-splinesmake useof a coarse-to-finehierarchy of knotsequencesto generatea seriesof uniform cubic B-spline curveswhosesumapproachesthe desiredfunction. At eachlevel in thehierarchy, the control points of the B-spline curve are computedlocally with a least-squaresmethodwhich providesan interactiveperformance.With this fitting technique,we cannotonly manipu-latea curve adaptively to satisfya largesetof constraintswithin aspecifiederrortolerance,but alsoedit a curve at any level of detailto allow an arbitraryportion of the motion to be affectedthroughdirect manipulation. Exploiting thesefavorablepropertiesof the

Page 2: A Hierarchical Approach to Interactive Motion Editing for ...mrl.snu.ac.kr/Papers/sig99.pdf · A Hierarchical Approach to Interactive Motion Editing for Human-likeFigures Jehee Lee

multilevel B-splinecurves,we convenientlyderive a hierarchy ofdisplacement� mapswhichareappliedto theoriginalmotiondatatoobtaina new, smoothlymodifiedmotion.Becauseof thisdisplace-mentmapping,thedetailcharacteristicsof theoriginal motioncanbepreserved[3, 37].

Theperformanceof ourapproachis furtherenhancedby ournewinversekinematicssolver. It is commonplaceto formulatethe in-versekinematicswith multiple targetsasa constrainednon-linearoptimizationfor whichthecomputationalcostisexpensive[28, 38].As noticedby Korein andBadler[19], we canfind a closed-formsolutionto theinversekinematicsproblemfor alimb linkagewhichconsistsof threejoints, for example,shoulder-elbow-wrist for thearm and hip-knee-anklefor the leg. We combinethis analyticalmethodwith a numericaloptimizationtechniqueto computethesolutionsfor full degreesof freedomof a human-like articulatedfigure. Our hybrid algorithmenablesusto edit themotionsof a 37DOFarticulatedfigure,interactively.

Theremainderof thepaperis organizedasfollows. After a re-view of previousworks,wegiveanintroductionto thedisplacementmappingandthemultilevel B-splinefitting techniquein Section3.In Section4, wepresentourmotioneditingtechnique.In Section5,we describetwo inversekinematicsalgorithms:Oneis designedtomanipulateageneraltree-structuredarticulatedfigureandtheotheris specializedto a human-like figure with limb linkages. In Sec-tion 6, we demonstratehow our techniquecanbe usedfor inter-active motion capture-basedanimationwhich includesadaptingamotionfrom onecharacterto another, fitting a recordedwalk ontoa roughterrainandperformingseamlesstransitionsamongmotionclips. Finally, weconcludethispaperin Section7.

2 Previous Works

2.1 Motion Editing

Therehavebeenanabundanceof researchresultsto developmotionediting tools. Bruderlin andWilliams [3] showed that techniquesfrom the signal processingdomaincan be appliedto manipulat-ing animatedmotions. They introducedthe ideaof displacementmappingto altera motionclip. Witkin andPopovic [37] presenteda motion warpingtechniquefor the samepurpose.Bruderlin andWilliams alsopresenteda multi-target interpolationwith dynamictimewarpingto blendtwo motions.Unumaetal. [33] usedFourieranalysistechniquesto interpolateandextrapolatemotiondatain thefrequency domain.Wiley andHahn[35] andGuoandRoberge[14]investigatedspatialdomaintechniquesto linearly interpolatea setof examplemotions. Roseet al. [27] adopteda multidimensionalinterpolationtechniqueto blendmultiplemotionsall together.

Witkin andKass[36] proposedaspacetimeconstrainttechniqueto producetheoptimalmotionwhichsatisfiesasetof user-specifiedconstraints.BrotmanandNetravali [2] achieveda similar resultbyemploying optimalcontrol techniques.Thespacetimeformulationleadsto a constrainednon-linearoptimizationproblem.Cohen[5]developeda spacetimecontrolsystemwhichallowsa userto inter-actively guidea numericaloptimizationprocessto find anaccept-ablesolutionin a feasibletime. Liu et al. [22] useda hierarchicalwaveletrepresentationto automaticallyaddmotiondetails.Roseetal. [28] adoptedthis approachto generatea smoothtransitionbe-tweenmotionclips. Gleicher[12] simplifiedthespacetimeproblemby removing thephysics-relatedaspectsfrom theobjectivefunctionand constraintsto achieve an interactive performancefor motionediting.Healsoappliedthis techniquefor motionretargetting[13].

2.2 Hierarchical Curve/Surface Manipulation

Thereis a vastamountof literaturedevotedto investigating hier-archicalrepresentationsof curvesandsurfaces.Schmittet al. [29]

presentedanadaptivesubdivisionmethodto produceasmoothsur-facefrom sampleddata. Forsey andBartels[9] introduceda hier-archicalB-splinerepresentationto enhancesurfacemodelingcapa-bility. This representationallows detailsto beadaptively addedtothe surfacethroughlocal refinement.They alsoemployed the hi-erarchicalrepresentationfor fitting a splinesurfaceto the regulardatasampledatgrid points[10]. WelchandWitkin [34] proposedavariationalapproachto directlymanipulatea B-splinesurfacewithscatteredfeatures,suchaspointsandcurves.Leeetal. [20, 21] sug-gestedan efficient methodfor interpolatingscattereddatapoints.They alsodemonstratedthatimagewarpingapplicationscanbecastasa surfacefitting problemby adoptingthe ideaof displacementmapping.Althoughauthorsuseddifferentterms,suchashierarchi-cal andmultilevel B-splinesurfaces,to refer to their hierarchicalstructures,their underlyingideasare the same,that is, a coarse-to-fine hierarchy of control lattices. Anotherclassof approachesis due to multiresolutionanalysisand wavelets. FinkelsteinandSalesin[8] usedB-spline waveletsfor multiresolutionediting ofcurves.Many authorshaveinvestigatedmultiresolutionanalysisformanipulatingsplinesurfacesandpolygonalmeshes[4, 6, 23,30].

2.3 Inverse Kinematics

Traditionally, inversekinematicssolverscan be divided into twocategories: analytic and numericalsolvers. Most industrial ma-nipulatorsaredesignedto have analyticsolutionsfor efficient androbustcontrol.Kahan[16] andPaden[24] independentlydiscussedmethodsto solveaninversekinematicsproblemby reducingit intoa seriesof simplersubproblemswhoseclosed-formsolutionsareknown. KoreinandBadler[19] showedthattheinversekinematicsproblemof ahumanarmandleg allowsananalyticsolution.Actualsolutionsarederivedby TolaniandBadler[32].

A numericalmethodreliesonaniterativeprocessto obtainaso-lution. GirardandMaciejewski [11] addressedthelocomotionof aleggedfigure usingJacobianmatrix andits pseudoinverse.Kogaet al. [18] madeuseof resultsfrom neurophysiologyto achieve an“experimentally”good initial guessand thenemployed a numer-ical procedurefor fine tuning. Zhao andBadler [38] formulatedtheinversekinematicsproblemof a humanfigureasa constrainednon-linearoptimizationproblem.Roseetal. [28] extendedthisfor-mulationto handlevariationalconstraintsthatholdoveranintervalof motionframes.

3 Preliminary

3.1 Displacement Mapping

The configurationof an articulatedfigure is specifiedby its jointanglesin additionto the positionandorientationof the root seg-ment. We will denotethe positionof the root by a 3-dimensionalvectorandtheothersby unit quaternions.It is well-known thatunitquaternionscanrepresent3-dimensionalorientationssmoothlyandcompactlywithout singularity [31]. This representationcan alsodescribeahumanjoint conveniently.

A motion is a time-varying function which provides the con-figurationof an articulatedfigure at a time. We denotea motionby ���� ��������� ���������� ������������ �!��� " $# , where ����� &%('*) and���+��� ,%.-!) describethe translationalandrotationalmotionof theroot segment,and ��/0��� 1%2-!) gives the rotationalmotion of the�3�456 -th joint for 798:3;8< .

A displacementmap (also called a warp function) describesthe differencebetweentwo motions[3, 37]. Gleicher [13] pro-videda goodexplanationto introducethis techniqueinto a space-time formulation. In our mathematicalsetting, the displace-ment map is definedas =���� >�?���� A@B�:CD��� , where =���� >��E C ��� �����������EF�G��� " $# and EF/���� H%>'I) for JK8.3L8.< . Thus,a new

Page 3: A Hierarchical Approach to Interactive Motion Editing for ...mrl.snu.ac.kr/Papers/sig99.pdf · A Hierarchical Approach to Interactive Motion Editing for Human-likeFigures Jehee Lee

motion canbe obtainedby applyingthe displacementmapto theoriginalM motionas ���� I�N� C ��� FO:=���� , thatis,PQQQR ����

...� �S�TTTU � PQQQR ��C���C

...���CS�TTTU O PQQQR E CEI�

...E �S�TTTU � PQQQR � C*V E C���CXW�Y[Z �EI�+

...� �C W�Y\Z �E � S�TTTUB] (1)

Here, W�Y[Z �E^ denotesa 3-dimensionalrotation about the axis_` _ ` %a'*) by angle bDE�bc%d' . We refer the readersto Kim etal. [17] for detailsof quaternionalgebraandtheexponentialmap.With thedisplacementmapping,we areableto dealwith bothpo-sitionandorientationdatain auniformway; thedisplacementmapis a homogeneousarrayof 3-dimensionalvectors,while the con-figurationof anarticulatedfigureis representedasaheterogeneousarrayof avectorandunit quaternions.

3.2 Multilevel B-spline Approximation

Leeet al. [21] proposeda multilevel B-splineapproximationtech-nique for fitting a splinesurfaceto scattereddatapoints. In thissection,we give a brief summaryto introducetheir fitting tech-nique. Sincewe needto manipulatea curve ratherthana surface,ourderivationfocusesoncurvefitting.

Let ef�ag��h%i'kj J�8N�hlN<km bea domaininterval. Considerasetof scattereddatapoints no�2gp�� / �0q / �m for � / %e . To interpo-late the datapoints,we formulatean approximationfunction r asa B-splinefunctionwhich is definedover a uniform knot sequenceoverlaid on the domain e . The function rs��� t�vu )w�x Czy w ��{4| �0}\ �~[������� wD� � can be describedin termsof its control pointsanduniformcubicB-splinebasisfunctionsy w , J18B��8f� . Here,~ / isthe 3 -th controlpoint on theknot sequencefor 4,5�8B3�8�< V 5 .With thisformulation,theproblemof deriving function r is reducedto thatof finding thecontrolpointsthatbestapproximatethedatapointsin n .

Sinceeachcontrolpoint ~�� is influencedby thedatapointsin itsneighborhood,we candefinethe proximity set n � ��gp�� / �0q / �%n�j �h4�798:� / l:� V 76m whichaffect thevalueof ~�� . Simplelinearalgebrausingpseudoinverseprovidesa least-squaressolution~��A� u�� ���$� ���0����� �G�L�/ �$� / �u � ����� ��������� � � �/ � (2)

whichminimizesalocalapproximationerror u�� � � � � � ����� � b rs�� / D4q / b � . Here, � / �t� y � � � � ��� � � �� / 4 | � / }\ comesfrom a B-splinebasisfunction,and � / � � ¡ ���+���¢H£ ¤�¥z¦^§ ¤ � ��� � �����¨��� takesaneffect to pull

thecurve towardadatapoint �� / �0q / .Thereis a trade-off betweentheshapesmoothnessandaccuracy

of theapproximationfunction. If theknot spacingof theapproxi-mationfunction is too coarse,it may leave large deviationsat thedatapoints. Conversely, if the function is definedover an exces-sively fineknotsequence,its shapewouldundulatethroughthedatapoints.Multilevel B-splineapproximationusesaseriesof B-splinefunctionswith differentknot spacingsto achieve a smoothshapewhile accuratelyapproximatinggivendatapointsin n . Thefunc-tion from thecoarsestknot sequenceprovidesa roughapproxima-tion, which is further refinedin accuracy by the functionsderivedfrom subsequentfinerknotsequences.

A multilevel B-splinefunctionis a sumof cubicB-splinefunc-tions r C ����������rp© which aredefinedover uniform knot sequencesoverlaid on domain e . The knot sequencesyield a coarse-to-finehierarchy. Without lossof generality, weassumethattheknotspac-ing of r / is coarserthanthat of r � for any 3�l�� . The multilevelB-splineapproximationbeginsby determiningthecontrolpointsof

ª �ª �ª CI« ª � « ª �ª C*« ª �ª C

Figure1: Hierarchicalcurvefitting to scattereddatathroughmulti-level B-splineapproximationrDC with Equation(2), which servesasa smoothinitial approxima-tion. r C mayleave a deviation ¬­��q / �cq / 4cr C �� / for eachpoint�� / �0q / in n . The next finer function r � is usedto approximatethedifference® � �¯g �� / ��¬­��q / �m . Then,thesum r CHV r � yieldsa smallerdeviation ¬ � q / �dq / 4fr C �� / *4fr � �� / for eachpoint�� / �0q / . At a level � of thehierarchy, function r w is derivedto ap-proximate® w ��gp�� / ��¬ w q / �m where ¬ w q / �&q / 4&u wD� �° x C r ° �� / .By repeatingthis processto thefinestlevel, we canincrementallyderive thefinal approximationfunction r��±u ©w�x C r w .4 Hierarchical Motion Editing

4.1 Basic Idea

Giventheoriginal motion � C anda set ² of constraints,our prob-lem is to derive a smoothdisplacementmap = suchthat a targetmotion �³�´� C Oi= satisfiestheconstraintsin ² . Currentmotionediting techniquesrepresentthe displacementmapasan arrayofsplinecurvesdefinedoveracommonknotsequence[13, 37]. Eachsplinecurvegivesthetime-varyingmotiondisplacementof its cor-respondingjoint. With a finer knot sequence,we canpossiblyfindasolutionthataccuratelysatisfiesall theconstraintsin ² . However,wehave to paya highercomputationalcostfor theaccuracy duetothefiner knot sequence.Ideally, we wish to determinethedensityof knotsto yield just enoughshapefreedomfor anexactsolution.However, the target motion is not known in advanceandthuswerequirethedisplacementmapwhich allows detailsto beaddedbyadaptively refiningtheknotsequence.

We adoptthehierarchicalstructure[21] reviewedin Section3.2to performthis adaptive refinement. The multilevel B-splineap-proximation techniquewas employed to derive a warp functionfor imagemorphingandgeometryreconstruction.In our context,we extend this techniqueto handlemotion data. From the dis-placementmap = , we deriveaseriesof successively finersubmaps= � ������� ��= © thatgive thecorrespondingseriesof incrementallyre-finedmotions,� � ������� ��� © .� © �±�������"��:C;O:= � �O= � �O&����� O= © ] (3)

Here, = w , 5µ8��­8�¶ , is representedby anarrayof cubicB-splinecurvesin the 3-dimensionalvectorspace.The componentcurvesof = w are definedover a commonsequence· w of knots that areuniformly spaced.The knot sequences· w , 5.8v�´8v¶ , form acoarse-to-finehierarchy. · � is placedon the coarsestlevel in thehierarchy and·[© is on thefinestlevel. Themotion � � �N� C O¸= �providesaroughapproximationto atargetmotion,which is furtherrefinedby applying = � to giveamoreaccurateapproximation� � .By applyingthesubmapsoneby one,we canincrementallyobtainthefinal motion.Beawarethatit is avery frequentmistake to have

Page 4: A Hierarchical Approach to Interactive Motion Editing for ...mrl.snu.ac.kr/Papers/sig99.pdf · A Hierarchical Approach to Interactive Motion Editing for Human-likeFigures Jehee Lee

=´� u ©w�x � = w from anerroneousderivation, that is, to substituteW�Y\Z �E � W�Y[Z �E � !����� W�Y[Z �E © for W�Y[Z �E � V E � V ����� V E © in Equa-tion (1) and(3). Thisderivationis not correct,sincethequaternionmultiplicationis notcommutative.

4.2 Constraints

To specifythedesiredfeaturesof thetargetmotion,two categoriesof constraintsareemployed:Theonesin thefirst categoryareusedto describeanarticulatedfigure itself, suchasa joint limit andananatomicalrelationshipamongjoints. Thosein theothercategoryarefor placingend-effectorsof thefigureatparticularpositionsandorientationswhich are interactively specifiedby the useror auto-matically derived from the interactionbetweenthe figure and itsenvironment. For example,we first specify the contactpoint be-tweenthefoot andthegroundthroughagraphicalinterfaceandau-tomaticallymodify thepoint laterin accordancewith thegeometricvariationof theground.We assumethata constraintin eithercate-gory is definedat a particularinstanceof time. A variationalcon-straintthatholdsover aninterval of motionframescanberealizedby a sequenceof constraintsfor thetime interval. An orderedpair����D��¹I�� specifiestheset ¹I� of constraintsat a frame��� .4.3 Motion Fitting

In order to computea displacementmap, it is necessaryto esti-matethe motion displacementof eachjoint at every constrainedframe. Thedisplacementof the joint at a particularframeis inter-polatedby thecorrespondingcomponentcurveof thedisplacementmapandthussmoothlypropagatedto theneighboringframes.Sup-posethat we arenow at the � -th level for 5:8º�&8º¶ . At eachconstrainedframe � � , our inversekinematicssolver givesthecon-figuration� �»�

of thecharacter, thatmeetsagivensetof constraints����D��¹I�� . Sincetheremay exist many possibleconfigurationsthatsatisfyall constraintsin ¹I� , we consistentlychoosetheonethat isminimally deviated from the motion � wD� � at the previous level.Thatis, weminimizebD= �»� b �¼ �fb�� �½� @:� w�� � ����D �b �¼ (4)� �¾ / x C!¿ / bDE / b � �where= �»� �±�E C ����������EF�G . Wecancontroltherigidity of anindi-vidual joint by adjustingits weightvalue.

Combiningthe inversekinematicssolver with the hierarchicalstructuregivenin Equation(3),wegivethefollowingmotionfittingalgorithm:

Algorithm 1 HierarchicalmotionfittingINPUT : theoriginalmotion � C , theset² of constraintsOUTPUT: anew motion � ©

1: for �µÀ��±5 to ¶ do2: ®ÁÀ¨�Â3: for each ����D��¹I�� *%�² do4: � � � À�� IK solver�¹I����� wD� � ����� " 5: = �»� À¨�N� �½� @i� w�� � ����� 6: ®ÁÀ¨�®�Ãt����D��= �½� 7: end for8: Compute= w by curvefitting to ®9: � w À¨�N� wD� � O= w

10: end for

This algorithmevaluates= w , 5K8o�Ä8Ŷ , in the coarse-to-fineorder. At eachlevel � in the hierarchy, we computethe motion

Figure2: A live-capturedwalking motion wasinteractively mod-ified at themiddle framesuchthat thecharacterbentforwardandloweredthepelvis.Thecharacteris depictedat themodifiedframe.Therangeof deformationis determinedby thedensityof theknotsequences.The knots in · � arespacedevery (top) 4, (middle) 6,and(bottom)12 frames.

displacement= �½� �¯� �½� @c� wD� � ����� for all ����D��¹I�� in ² usingthe inversekinematicssolver (Seelines 2–7). Here, � w�� � is ei-ther the original motion ( ��Æ5 ) or hasbeenalreadyobtainedinthepreviouslevel ( �1Ç�5 ). Thenewly computeddisplacementsin®È�³gp����D��= �½� �m areusedas the keyframesfor deriving the dis-placementmap = w (Seeline 8). Several techniquesare availablefor computing= w thatinterpolatesthedisplacements.Onepossiblesolutionis to employ an iterative numericalmethodthatmaygivean optimal solution. However, we choosea curve fitting methodgiven in Equation(2) to achieve an interactive performance.Ourmethodis extremelyfast,sincethesolutioncanbeobtainedanalyt-ically unlike thenumericalmethodthatnormally requiresa heavycomputationalcostfor iterations.In general,localcurvefitting withB-splinesmayhaveseveraldrawbacks.Theresultingcurvemaybelessaccurateandcouldhaveundulationsbecauseof thelack of theglobalpropagationof a displacement.Fortunately, our hierarchicalstructurecan compensatefor suchdrawbacksby globally propa-gating the displacementat a coarselevel andperformingthe latertuningatfine levels.

4.4 Knot Spacing

For simplicity, our implementationdoublesthe densityof a knotsequencefrom onelevel to thenext. Therefore,if · w has �< V �  control points on it, then the next finer knot sequence· w � � willhave "7+< V �  controlpoints. Thedensityof a knot sequence· w ,5Ä8É�:8(¶ , determinesthe rangeof influenceof a constraintonthe displacementmap at a level � . This is of great importancefor direct manipulation. For example,considerthe situationthatwe interactively adjusttheconfigurationof anarticulatedfigurebydraggingoneof its segmentsat a certainframethrougha graphi-cal interface.Theuserinput is interpretedasconstraintswhich areimmediatelyaddedto the setof prescribedconstraints.Then,oursystemsmoothlydeformsa portionof themotionclip aroundthismodifiedframe. Here,therangeof influenceon themotionclip ismainlydependentonthespacingof · w . Thelargerspacingbetween

Page 5: A Hierarchical Approach to Interactive Motion Editing for ...mrl.snu.ac.kr/Papers/sig99.pdf · A Hierarchical Approach to Interactive Motion Editing for Human-likeFigures Jehee Lee

3 DOF

3 DOF1 DOF

3 DOF

3 DOF

1 DOF

Figure3: A human-like figure thathasexplicit redundanciesat itslimb linkages

knotsyieldsthewider rangeof deformation(SeeFigure2). There-fore, the displacementmap = � , that is derived from the coarsestsequence· � , hasnon-zerovaluesoverthewidestrangeto smoothlypropagatethechangeof themotion.Thesubsequentfinerdisplace-mentmaps= w , 718Á�¸8Á¶ , performsuccessive tuningsto satisfytheconstraints.

Thedensityof thefinestknotsequence· © controlstheprecisionof the final motion � © . If · © is sufficiently fine to accommodatethedistribution of constraintsin the time domain, ��© canexactlysatisfyall constraints.However, our algorithmmay leave a smalldeviation for eachconstraintin ² even with several levels in thehierarchy. In our experiments,we needjust four or five levels forvisuallypleasingresults,thatcanbefurtherenhancedto achieveanexactinterpolationby enforcingeachconstraintindependentlywiththeinversekinematicssolver.

4.5 Initial Guesses

For a spacetimeproblem,a goodinitial guesson thedesiredsolu-tion is veryimportantto improveboththeconvergenceof numericaloptimizationandthequality of theresult[13]. We obtainaninitialguessfor motionfitting by shifting thepositionof therootsegmentin theoriginal motion. To motivatethis, considerthewalking mo-tion that is adaptedto the roughterrainasshown in Figures5 (a)and(b). Thepositionof astancefoot, thattouchesthesurfaceof theterrain,is pulledupwardat a smallhill on theterrain,andthusthecharacteris unwantedlyforcedto squat. Even thoughthe inversekinematicssolver triesto minimizethedeviation of joint angles,itcannotpreventthekneebendingcompletely. To reducethisartifact,wechangethepositionof therootsegmentdueto thechangeof ge-ometry. Specifically, we displacethe root segmentby theaverageshift of the contactpositionsat eachframe. The shift of the rootsegmentpositionata framecanalsobesmoothlypropagatedto theneighboringframesusingthemultilevel B-splinefitting method.

5 Inverse Kinematics

The most time consumingcomponentin the motion fitting algo-rithm is the inversekinematicssolver which is invoked very fre-quentlyat eachlevel of thefitting hierarchy. Therefore,theoverallperformanceof a hierarchicalfitting critically dependson theper-formanceof theinversekinematicssolver. Wedescribe,in thissec-tion, two inversekinematicsalgorithms. In Section5.1, we intro-ducean inversekinematicsalgorithmfor a generaltree-structuredfigurewith sphericaljoints basedon numericaloptimizationtech-niques.In Section5.2,wepresentafasterspecializedalgorithmfora human-like figurewith limb linkages.Thelatteralgorithmcom-binesthenumericaltechniqueswith ananalyticalmethodillustratedin Section5.3.

5.1 A Numerical Approach

Our inversekinematicssolver is basedon a constrainednon-linearoptimizationtechniquethat minimizesthe objective function sub-ject to a set ¹ of constraints.We have anadditionalburdento en-forcetheunitarinessof eachquaternionparameter�K�±�ʵ�0q���Ë �0Ì! .Onepossiblesolutionwould be to augmentthe set ¹ with a newconstraint Ê � V q � V Ë � V Ì � 4Á5o�ÈJ for the unit quater-nion parameter. We circumvent this extra constraintfrom the ob-servation that every orientation � can be expressedas a rotationW�Y[Z �E^ Í%N-!) from a fixed referenceorientation� C %´-!) , that is,���� C W�Y\Z �E^ . Therefore,we canparameterize����+���+������� �����G by a simplevector Î.�d�qFCD������� �0q )$� � � {%'*)$� � ) usingthedis-placement=Ä�±�E C ����������EF�! from agivenreferenceconfiguration�� C ��� �C ����������� �C asfollows:�>�N� C*V E C � and� / �N� /CXW�Y[Z �E / �� 598:3;8<*� (5)

where E / �É�q )"/ ��q )$/ � � �0q )$/ � � for J&8�3Ï8�< . Letting the ref-erenceconfigurationbe � wD� � ����� in Equation(4), we reducetheobjective functionto a quadraticform of Î . Accordingly, our con-strainedoptimizationproblemis formulatedasfollows:

minimize rs�Î� I� 57 Î #FÐ Î^�subjectto Ñ / �ÎF I�:JÒ�Ó3;%�Ô1Õ��Ñ / �ÎF *ÇiJÒ�Ó3;%�Ô / �

where Ð is a diagonalmatrix thatdeterminestherigidity of indi-vidualparametersin Î .

A typical approachto the constrainedoptimizationis to trans-form the constrainedprobleminto an unconstrainedversionwithextra parameters(the Lagrangemultipliers) or extra energy terms(penaltyfunctions).We avoid illegal configurationsby employingthe penaltymethodthat allows us to handlethe equality and in-equalityconstraintsin a uniform way [25]. Theobjective functionfor theunconstrainedversionisÖ �ÎF I�frs�ÎF V ¾/ �\×�Ø � / Ñ / �ÎF � V ¾/ �\×�� � / ½ÙÏÚ¨Û^�Ñ / �ÎF ���Js " � � (6)

where� / weightseachindividual constraint.We adopttheconju-gategradientmethodto minimizethis function[26].

5.2 A Hybrid Approach

Themajordifficulty of solvinganinversekinematicsproblemstemsfrom theexcessiveDOFsof anarticulatedfigure.A reasonablehu-manmodelmayhaveabout40DOFsfor computeranimation,whilewespecifymuchfewerconstraintsfor manipulatingthefigure.Fora figureof < DOFs,we canremove Ñ of thoseDOFswith a setofÑ independentconstraintsimposedon it. The remaining �<4�Ñ6 DOFsspanthesolutionspaceof theproblem.

A reduced-coordinateformulationparameterizesthe redundantDOFswith a reducedsetof �<>4¸Ñ6 variables.Oneexplicit redun-dancy in thehumanbody is the “elbow circle” thatwasfirst men-tioned in Korein andBadler [19]. Even thoughthe shoulderandthewrist arefirmly planted,we canstill afford to move theelbowalonga circle with its axisthroughtheshoulderandthewrist (SeeFigure3). The humanfigure hasfour limbs, two from armsandtwo from legs.TheredundantDOFfor the 3 -th limb linkagecanbeparameterizedwith a rotationangleÜ / , 598:3;8:Ý , abouttheaxis.

Without loss of generality, we assumethat the positionsandorientationsof handsand feet are fixed by constraints. If thereis a free hand or foot, the DOFs in the correspondinglimb

Page 6: A Hierarchical Approach to Interactive Motion Editing for ...mrl.snu.ac.kr/Papers/sig99.pdf · A Hierarchical Approach to Interactive Motion Editing for Human-likeFigures Jehee Lee

are left unchanged. Let � �Þ��I������������� ����ß�����ß � ���������Ò�+���G be theà configuration of a human-like figure. Its rear part���ß � ��������� �����G denotestheDOFsfor the limbs andthe fore part��I��� � ������� �+� ß doesthe remainingDOFs. Sincethe constraintsrestraintheDOFsin thelimb linkages,thereducedsetof parame-ters ��I�����+������������ß��0Ü � ������� �0Ü[á� spanall possibleconfigurationsofthefigureundertheconstraints.

Incorporatingthe idea of reduced-coordinateformulation intothe numericaloptimization framework, we can solve an inversekinematicsproblem using a fewer number of optimization pa-rameters âÎã�ä�q C ������� ��q )$ß � � �0Ü � ������� �0Ü[á� Å%å'*)$ß � æ . Notethat we have replacedthe rear part of Î with the elbow circleparametersÜ � ������� �0Ü á for limb linkages. Whenever we evalu-ate the objective function with new parametersâÎ , the parameters��I��� � ������� �+� ß arecomputedfirst by Equation(5), andthentheothersfor ���ß � ���������������G areuniquelydeterminedby ananalyti-calsolverwhichtakes ��I��� � ������� ��� ß andÜ / , 598:3;8iÝ , asinput.Then,we extracttheunknown part �q )$ß � ) ������� �0q )"� � � of Î from���ß � ��������� �����G to evaluatetheobjective functionin Equation(6).Thereduced-coordinateformulationusesafewernumberof param-etersto yield fasterconvergenceandfewer iterationsto enhancetheoverallperformance.

5.3 Arm and Leg Postures

Consideralimb linkage,for example,anarmlinkage.Startingfroman initial configuration,we sequentiallyadjustthe joint anglesforthe elbow, the shoulderandthe wrist of the arm linkageto placethehandat thedesiredpositionandorientation.Weassumethatthetorsoandtheshoulderpositionsaregiven. Let ç � , ç � , è � , è � and ébedefinedasfollows(SeeFigure4(a)):ç � � thelengthof theupperarm�ç � � thelengthof theforearm�è � � thedistancefrom theelbow rotationaxisto theshoulder�è � � thedistancefrom theelbow rotationaxisto thewrist, andé´� thedistancebetweentheshoulderandthewrist]To placethewrist at apositiondistantfrom theshoulderby é (SeeFigure4(b)), theangle ê betweenupperandlowerarmsis givenbyê��:ë�ì í � �hî ç � � V ç �� V 7Gï ç � � 4¸è �� ï ç �� 4�è �� 4Ké �7+è � è � ð � (7)

asillustratedin theappendix.Then,we bring thewrist to thegoalpositionby adjustingtheshoulderangles(SeeFigure4(c)). In thesubsequentstep,werotatethewrist anglesto coincidewith thegoalorientation.Onceonefeasiblesolutionis given,theothersolutionscan be obtainedby rotating the elbow aboutthe axis that passesthroughtheshoulderandthewrist positions.Given Ü / , we cande-terminethearmpostureuniquely(SeeFigure4(d)). Similarly, wecandeterminea leg posture.

If é is longerthanthearmlength,ç � V ç � , theelbow stretchesasfaraspossible.On theotherhand,if é is toosmall,thentheelbowanglecouldviolate its lower limit andthusis pulledbackinto theallowable range. In both cases,we cannotplacethe wrist at theexactpositionandthusthecorrespondingpenaltyfunctionyieldsapositivevaluefor thegiventorsoconfiguration.

6 Experimental Results

Our humanmodelhas6 DOFsfor thepelvispositionandorienta-tion, 3 DOFsfor the(eitherrigid or flexible) spine,and7 DOFsforeachlimb to yield the total of 37 parametersfor the inversekine-maticsproblem. Otherparametersfor thehead,the neck,andthe

ñ �goalposition& orientation

ò �ó ò �ñ �

ô /(d) Redundancy

(a) Initial configuration

(c) Shoulder& Wrist rotation

(b) Elbow rotation

õ

Figure4: Theprocessfor adjustingthearmposture

fingersarenot usedfor resolvingkinematicconstraints.Throughthe reduced-coordinateformulation, we can remove 6 DOFs foreachlimb andthuswe have at most13 DOFsto be computedbya numericaloptimizationmethod.Themotionclips for our experi-mentshavebeensampledat therateof 30 framespersecond.

Thewalking motionof Figure5(a)is producedby performingasequenceof transitionsamongasetof motion-capturedclips,whichinclude“walk straight”,“turn left”, “turn right”, “start”, and“stop”.We interactively specify the momentsof heel-strikesandtoe-offsfor themotionclips. This informationis usedfor establishingthekinematicconstraintsthat enforcethe foot contactsfor the entiremotion. Theterrainof Figure5(b) is representedasa NURBSsur-faceof whichcontrolpointsareplacedonaregulargridwith aspac-ing of 80 % of theheightof thecharacter, andtheir y-coordinates(heights)arerandomlyperturbedwithin 120 % of the height. Toadaptthemotionontotheroughterrainwith doorways,wefirst ad-just theconstraintssuchthatthecontactpositionsareshiftedalongthe y-axis to be placedon the terrain,andaddnew constraintstobendthecharacterunderthedoorways. Then,we useour motionfitting algorithmto warpthemotionto satisfytheconstraints.Theoriginal andthe adaptedmotionsaredepictedin Figures5(a) and5(b), respectively.

The“climbing a rope” examplein Figure5(c) givesconstraintson bothhandsandfeet. A physically simulatedropeis usedto ex-plicitly illustratethemomentsof graspingandreleasingtheropebya hand,which correspondto theinitiation andtermination,respec-tively, of a variationalconstraintfor thathand. We adaptthis mo-tion to adifferentcharacterwith longerlegsandashorterbodyandarms. For the charactermorphingexampleshown in Figure5(d),the size of a charactersmoothlychangesto have extremely longlegsanda shortbody, andthento have extremelyshortlegsandalong body. The original walking motion is warpedto preserve itsuniformstrideagainstthechangeof charactersize.

Ourmotionfitting methodis alsousefulfor generatingasmoothtransitionbetweenmotionclips. Figure5(e)shows the transitionsfrom walkingto sneakingandfrom sneakingto walking. Thebasicapproachis very similar to the onepresentedby Roseet al. [27]We seamlesslyconnectthe motion databy fading one out whilefadingthe other in. Over the fadingduration,Hermite interpola-tion andtime warpingtechniquesareusedto smoothlyblendthejoint parametersof themotiondata.Sincejoint parameterblendingmaycausefoot sliding,weenforcefoot contactconstraintswith themotionfitting method.

Page 7: A Hierarchical Approach to Interactive Motion Editing for ...mrl.snu.ac.kr/Papers/sig99.pdf · A Hierarchical Approach to Interactive Motion Editing for Human-likeFigures Jehee Lee

(e) Transitions between walking and sneaking

(d) Character morphing(c) Climbing a rope for different characters

(b) The adapted motion for the rough terrain(a) The original walking motion on the flat ground

Figure5: Examples

Page 8: A Hierarchical Approach to Interactive Motion Editing for ...mrl.snu.ac.kr/Papers/sig99.pdf · A Hierarchical Approach to Interactive Motion Editing for Human-likeFigures Jehee Lee

Figure6: Graphicaluserinterface

Table 1 givesa performancesummaryof the examples. Tim-ing informationis obtainedon a SGI Indigo2workstationwith anR10000195MHz processor. Theexecutiontime for eachexampleis not only influencedby quantitative factorssuchas the numberof frames,constraintsandparameters,but alsoby qualitative fac-torssuchasthedifficulty of achieving desiredfeaturesspecifiedbyconstraintsandthequality of initial estimates.In particular, well-choseninitial estimatesprovide greatspeedupsfor mostof exam-ples. Onepromisingobservation is that both executiontimesandmaximumerrorsrapidly decreaselevel by level. This implies thattheperformanceof our algorithmis not critically dependenton theerrortolerance.In all examples,every constraintis satisfiedwithinor slightlyover1 % of theheightof thecharacterby thehierarchicalfitting of four levels.A few morelevelsmayresultin a moreaccu-ratesolution.As shown in experimentaldata,wecananticipatethatthe computationcostfor an additionallevel is muchcheaperthanthecostfor theprior level.

Our prototypesystemis implementedin C++ on top of X-windows/MOTIF ö�÷ andOpenInventorö�÷ thatfacilitatesdevelop-ing interactive 3D userinterfaces.In particular, OpenInventorö�÷providesconvenienttoolkits to supportdirect manipulationin the3D space.With this toolkits, ananimatorcaninteractively modifytheposeof a characterby draggingoneof its segments.Our userinterfaceallows theuserto edit a desiredportionof themotionbyadjustingthespacingof knotsover which thedisplacementmapisdefined.SinceweusecubicB-splinesto representthedisplacementmap,thechangeof theposeata framecouldaffect theposesat theneighboringframescoveredby seven knotsat the currentlevel inthehierarchy [1, 7].

7 Discussion

In thissection,wecompareourmotionfitting algorithmto thepre-viousapproachesatseveralviewpoints.

Interpolating splines vs. Multilevel B-splines: An inter-polatingspline is a possiblechoiceto representthe displacementmap [3, 37]. However, the interpolatingspline could undulaterapidly for a densedistribution of constraintsso that it often failsto preservethedetailcharacteristicsof theoriginalmotion.Instead,we usea seriesof uniform B-splinesthat form a hierarchy of mo-tion displacements.SinceuniformB-splinesapproximatedifferent

detailsof displacementsaccordingto knot spacings,we areabletoedit the motion at any level of detail; fitting at the coarsestlevelmakesa grossdeformationandthenfine detailsareincrementallyaddedat thefiner levels.

Global vs. Local least-squares: Fitting a B-spline curveto scattereddatapoints can be formulatedas a least-squaresap-proximation problem for solving an over-determinedor under-determinedsystemof linearequations[26]. To obtaintheoptimalsolutionin theleast-squaressense,wecanusethepseudoinverseofa largematrix thataccommodatesthedatapointsall together[15].However, this globalmethodoftensuffers from over-shootingthatmaygive anundesirablecurve shape.Ironically, the lessaccuratelocal methodin Equation(2) is preferredin a hierarchicalframe-work. Sinceapproximationerrorsat one level are incrementallycanceledout in the later levels, the accuracy at eachlevel is notcritical. This localmethodis muchmoreefficientandlesspronetoover-shootingthantheglobalmethod.

Single large optimization vs. Many small optimizations:Gleicher[13] castmotionretargettingasa largeoptimizationprob-lem. Basedon the“divide-and-conquer”strategy, however, wepar-tition hisoptimizationprobleminto many smallerinversekinemat-ics optimizationsand thensolve eachof themvery efficiently byadoptingspecializedtechniquessuchas the hybrid inversekine-maticssolver. Finally, we combinetheposesat constrainedframesemploying thehierarchicalcurvefitting technique.Thisdivide-and-conquerapproachprovidesa noticeablespeedupto satisfytheper-formancerequirementof our interactivemotioneditingsystem.

Limitation of our approach: Ourmotionfitting algorithmre-quiresadditionalwork to handlean inter-frameconstraintthaten-forcesthe relationshipamongparametersscatteredover multipleframes.This kind of constraintsareusedfor avoiding foot slidingbetweentheheel-strike of a foot andits toe-off while allowing theabsolutecoordinatesof thefoot to bealtered.Weaddressthisprob-lemby incorpolatingtheintra-frameinversekinematicsconstraintsat thoseco-relatedframesinto a larger optimizationproblemthatincludesinter-frameconstraintsamongthoseframesaswell. In theextremecasewhereall framesare relatedto eachother, this ap-proachis reducedto Gleicher’s so thatwe have to solve onelargeoptimizationproblemto derive thedisplacementmapat eachlevelin thehierarchy.

8 Conclusion

Wehavepresentedanew approachto adaptingexistingmotionof ahuman-like characterto have desiredfeaturesthatarespecifiedbya setof constraints.Thekey ideaof our approachis to introduceahierarchicalmotionrepresentationby whichwecannotonly manip-ulatea motionadaptively to satisfya largesetof constraintswithina specifiederror tolerance,but alsoedit anarbitraryportionof themotionthroughdirectmanipulation.

The performanceof our algorithmis greatly improved by em-ploying acurvefitting techniquethatminimizesa localapproxima-tion error. Thehierarchicalstructurecompensatesfor thepossibledrawbacksof thelocalapproximationmethodbygloballypropagat-ing thedisplacementatacoarselevel andlatertuningatfine levels.Furtherperformancegain is achievedby thenew inversekinemat-ics solver. Our hybrid algorithmperformsmuchfasterthana purenumericalalgorithm. We alsopresenta simpleyet effective for-mulationto optimizeorientationparameterswithout yielding extraconstraints.

Page 9: A Hierarchical Approach to Interactive Motion Editing for ...mrl.snu.ac.kr/Papers/sig99.pdf · A Hierarchical Approach to Interactive Motion Editing for Human-likeFigures Jehee Lee

Table1: Performancedata.# of parameterscountstheDOFsof acharacterusedfor resolvingkinematicconstraints.A numberin ( ) indicatesthatof reducedparameters.Themaximumerroris measuredin percentagesof theheightof thecharacter.

walkingon roughterrain climbinga rope# of parameters 20 (8) 37 (13)# of frames 464 275# of constraints(exceptjoint limits) 5568 5898preprocessingtime(CPUsec.) 0.17 0.21level 1st 2nd 3rd 4th 1st 2nd 3rd 4th# of frames/knot 16 8 4 2 16 8 4 2executiontime(CPUsec.) 7 ]ùø J 5 ]�ú û 5 ] � 5 5 ] 5�J ú6]�ø J � ]�ø 5 5 ] �67 J ]�û Jmaximumerror(%) 5�Ý ] �65 ü ] 7�Ý � ] Ýsý 5 ] J ú 5 5 ] Jsü û ]�ø 5 � ]ùø6û 5 ] 5 5

charactermorphing transition# of parameters 20 (8) 20 (8)# of frames 62 119# of constraints(exceptjoint limits) 420 834preprocessingtime(CPUsec.) 0.02 �level 1st 2nd 3rd 4th 1st 2nd 3rd 4th# of frames/knot 16 8 4 2 16 8 4 2executiontime(CPUsec.) J ] 56ý J ] 5 � J ] Jsü J ] J ú J ] 5 ý J ] 5 ø J ] 5 ø J ] 5�Ýmaximumerror(%) ú\] ü6ý ø6] � ø � ] Jsý 5 ] 5�J û6] 5 û Ý ]�û 5 7 ] 5+Ý J ]�û Ý

Acknowledgments

We would like to thankMichaelCohenat Microsoft Researchforhis carefulreadingandconstructive criticism. We arealsogratefulto Hyun JoonShinandMin Gyu Choi who helpededit theaccom-panying videotape.We thankKoreanBroadcastingSystem(KBS)for supplyingthemodelin Figures5(a)and5(b). Thisresearchwassupportedin partby theMinistry of InformationandCommunica-tion (MIC) andtheElectronicsandTelecommunicationsResearchInstitute(ETRI).

References

[1] R. H. Bartels,J.C. Beatty, andB. Barsky. An IntroductiontoSplinesfor Usein ComputerGraphicsandGeometricModel-ing. MorganKaufmann,1979.

[2] L. S. BrotmanandA. N. Netravali. Motion interpolationbyoptimal control. ComputerGraphics(Proceedingsof SIG-GRAPH88), 22(4):309–315,August1988.

[3] A. Bruderlin and L. Williams. Motion signal processing.ComputerGraphics(Proceedingsof SIGGRAPH95), 29:97–104,August1995.

[4] A. Certain,J. Popovic, T. DeRose,T. Duchamp,D. Salesin,andW. Stuetzle.Interactive multiresolutionsurfaceviewing.ComputerGraphics(Proceedingsof SIGGRAPH96), 30:97–115,August1996.

[5] M. F. Cohen. Interactive spacetimecontrol for anima-tion. ComputerGraphics(Proceedingsof SIGGRAPH92),26(2):293–302,July1992.

[6] M. Eck, T. DeRose,T. Duchamp,H. Hoppe, M. Louns-bery, andW. Stuetzle. Multiresolutionanalysisof arbitrarymeshes. ComputerGraphics (Proceedingsof SIGGRAPH95), 29:173–182,August1995.

[7] G.Farin.CurvesandSurfacesfor ComputerAidedGeometricDesign– A Practical Guide. AcademicPress,4th edition,1997.

[8] A. Finkelstein and D. H. Salesin. Multiresolutioncurves.ComputerGraphics(Proceedingsof SIGGRAPH94),28:261–268,July1994.

[9] D. R. Forsey andR. H. Bartels.HierarchicalB-splinerefine-ment. ComputerGraphics(Proceedingsof SIGGRAPH88),22(4):205–212,August1988.

[10] D. R. Forsey andR. H. Bartels.Surfacefitting with hierarchi-cal splines. ACM Transactionsof Graphics, 14(2):134–161,April 1995.

[11] M. Girard and A. A. Maciejewski. Computationalmodel-ing for thecomputeranimationof leggedfigures. ComputerGraphics(Proceedingsof SIGGRAPH85), pages263–270,July1985.

[12] M. Gleicher. Motion editing with spacetimeconstraints.InProceedingsof SymposiumonInteractive3D Graphics, pages139–148,1997.

[13] M. Gleicher. Retargettingmotion to new characters.Com-puter Graphics(Proceedingsof SIGGRAPH98), 32:33–42,July1998.

[14] S. GuoandJ. Roberge. A high-level controlmechanismforhumanlocomotionbasedon parametricframespaceinterpo-lation. In Proceedingsof ComputerAnimationand Simula-tion ’96, EurographicsAnimationWorkshop, pages95–107.Springer-Verlag,1996.

[15] W. M. Hsu, J. F. Hughes,andH. Kaufman. Direct manipu-lation of free-formdeformations.ComputerGraphics(Pro-ceedingsof SIGGRAPH92), 26:177–184,July1992.

[16] W. Kahan. Lectureson computationalaspectsof geometry.Unpublishedmanuscripts, 1983.

[17] M. J. Kim, M. S. Kim, andS. Y. Shin. A generalconstruc-tion schemefor unit quaternioncurveswith simplehighorderderivatives.ComputerGraphics(Proceedingsof SIGGRAPH95), 29:369–376,August1995.

Page 10: A Hierarchical Approach to Interactive Motion Editing for ...mrl.snu.ac.kr/Papers/sig99.pdf · A Hierarchical Approach to Interactive Motion Editing for Human-likeFigures Jehee Lee

[18] Y. Koga,K. Kondo,J.Kuffer, andJ.Latombe.Planningmo-tions with intentions. ComputerGraphics(ProceedingsofSIGGRAPH94), 28:395–408,July1994.

[19] J. U. KoreinandN. I. Badler. Techniquesfor generatingthegoal-directedmotionof articulatedstructures.IEEE CG&A,pages71–81,Nov. 1982.

[20] S.Lee,K.-Y. Chwa,S.Y. Shin,andG. Wolberg. Imagemeta-morphosisusingsnakesand free-formdeformations. Com-puterGraphics(Proceedingsof SIGGRAPH95), 29:439–448,August1995.

[21] S.Lee,G. Wolberg, andS.Y. Shin. Scattereddatainterpola-tion with multilevel b-splines.IEEE Transactionson Visual-izationandComputerGraphics, 3(3):228–244,1997.

[22] Z. Liu, S.G.Gortler, andM. F. Cohen.Hierarchicalspacetimecontrol.ComputerGraphics(Proceedingsof SIGGRAPH94),pages35–42,July1994.

[23] M. Lounsbery, T. D. DeRose,and J. Warren. Multiresolu-tion analysisfor surfacesof arbitrarytopologicaltype. ACMTransactionsonGraphics, 16(1):34–73,1997.

[24] B. Paden.KinematicsandControl RobotManipulators. PhDthesis,Universityof California,Berkeley, 1986.

[25] J. C. Platt andA. H. Barr. Constraintmethodsfor flexiblemodels.ComputerGraphics(Proceedingsof SIGGRAPH88),22(4):279–288,August1988.

[26] W. H. Press,SaulA. Teukolsky, W. T. Vetterling,andB. P.Flannery. NumericalRecipesin C: TheArt of ScientificCom-puting. CambridgeUniversityPress,secondedition,1992.

[27] C. Rose,M. F. Cohen,andB. Bodenheimer. VerbsandAd-verbs:Multidimensionalmotioninterpolation.IEEE CG&A,18(5):32–40,October1998.

[28] C. Rose,B. Guenter, B. Bodenheimer, andM. F. Cohen.Ef-ficient generationof motion transitionsusingspacetimecon-straints.ComputerGraphics(Proceedingsof SIGGRAPH96),30:147–154,August1996.

[29] F. J.M. Schmitt,B. A. Barsky, andW. Du. An adaptive sub-division methodfor surface-fittingfrom sampleddata. Com-puterGraphics(Proceedingsof SIGGRAPH86), 20(4):179–188,August1986.

[30] P. SchroderandW. Sweldens.Sphericalwavelets:Efficientlyrepresentingfunctionson the sphere. ComputerGraphics(Proceedingsof SIGGRAPH95), 29:161–172,August1995.

[31] K. Shoemake. Animating rotation with quaternioncurves.ComputerGraphics(Proceedingsof SIGGRAPH85),19:245–254,1985.

[32] D. Tolani andN. I. Badler. Real-timeinversekinematicsofthehumanarm. Presence, 5(4):393–401,1996.

[33] M. Unuma,K. Anjyo, andR. Takeuchi.Fourierprinciplesforemotion-basedhumanfigureanimation.ComputerGraphics(Proceedingsof SIGGRAPH95), 29:91–96,August1995.

[34] W. WelchandA. Witkin. Variationalsurfacemodeling.Com-puterGraphics(Proceedingsof SIGGRAPH92), 26(2):157–166,July1992.

[35] D. J. Wiley andJ.K. Hahn. Interpolationsynthesisfor artic-ulatedfiguremotion. In Proceedingsof IEEE Virtual RealityAnnual InternationalSymposium’97, pages157–160.IEEEComputerSocietyPress,1997.

[36] A. Witkin andM. Kass. Spacetimeconstraints. ComputerGraphics (Proceedingsof SIGGRAPH88), 22(4):159–168,August1988.

[37] A. Witkin and Z. Popovic. Motion warping. ComputerGraphics(Proceedingsof SIGGRAPH95), 29:105–108,Au-gust1995.

[38] J.ZhaoandN. I. Badler. Inversekinematicspositioningusingnonlinearprogrammingfor highly articulatedfigures. ACMTransactionsonGraphics, 13(4):313–336,1994.

Appendix: Derivation for Equation (7)ò �ñ � þ � þ �ò � óõ ññ �To identify theanglebetweentheupperandfore arms,we projectthejoint positionsontoa planeperpendicularto theelbow rotationaxis.Then,theprojectionsfor theshoulderandthewrist areplacedon two concentriccircleswhosecentercoincideswith the projec-tion for theelbow. Thedistanceè betweentheprojectionsis givenin termsof è � , è � andtheangle ê betweenthem.è � ��è �� V è �� 4�7+è � è � ë�ì6í�ê ]Letting ÿ � � ï ç �� 4Kè �� and ÿ � � ï ç �� 4¸è �� , respectively, thedis-tanceé betweentheshoulderandthewrist positionsisé � �±�ÿ � V ÿ � � V è ��ç �� V ç �� V 7�� ç �� 4Kè �� � ç �� 4Kè �� 4Kè �� 4¸è �� V è ��ç �� V ç �� V 7�� ç �� 4Kè �� � ç �� 4Kè �� 4�7+è � è � ë�ì í����è � è � ]Therefore,ë�ì í�ê­� ° � � � ° �� � ��� ° � � � ß � � � ° �� � ß �� ��� �� ß � ß � ]