Alternative Representation of Dynamics During Motor Learning
17
The Journal of Neuroscience, May 1994, 74(5) : 32083224 Adaptive Representation of Dynamics during Learning of a Motor Task Reza Shadmehr and Ferdinand0 A. Mussa-lvaldi Department of Brain and Cogni tive Sciences , Mas sachusetts Institute of Technology, Cambridge, Massachuse tts 02139 We investigated ho w the CNS learns to control movements in different dynami cal conditions, and how this learned be- havior is represented. In particular, we considered the task of making reaching movements in the presen ce of externa lly imposed for ces from a mechanical environment. This envi- ronmen t wa s a force field produced by a robot manip uland- urn, and the subje cts made reaching movements whil e hold- ing the end-effe ctor of this mani pula ndum . Since the force field significantly changed the dynamics of the task, sub- jects’ init ial movemen ts in the force field wer e grossly dis- torted compared to their move ment s in free space. However, with practice, hand trajectories in the force field converged to a path very similar to that observed in free space. This indicated that for reaching movements, there wa s a kine- matic plan independent of dynamical condition s. The recovery of performance within the changed me- chanical environment is motor adaptation. In order to inves - tigate the mechanism underlying this adaptation, we con- sidered the response to the sudden removal of the field afte r a training phase. Th e resulting trajectories, name d afte fef- fects, were approxi mately mirror ima ges of those that wer e observed when the subjects wer e initi ally exposed to the field. This suggested that the motor controller wa s gradually composing a model of the for ce fiel d, a model that the ner- imposed by the environment. In order to explore the struct ure of the model, we investigat ed whethe r adaptation to a force field, a s presented in a sma ll region, led to aftere ffects in other regions of the workspace . We found that indeed there were aftereffects in workspace regions where no exposure to the field had t aken place; that is, there wa s transfe r be- yond the boundary of the training data. This observation rules out the hypothesis that the subject’s mod el of the force field wa s constructed as a narro w association between vis- ited states and experienced forces ; that is, adaptation was not via composition of a look-up table. In contra st, subjec ts modeled the for ce field b y a combination of computational elements whos e output wa s broadly tuned across the motor Received July 23, 1993; revised Oct. 1, 1993; accepted Nov . 1, 1993. This work has been greatly enriched because ofour interactions with Drs . Em ilio Bizzi, Tomaso Poggio, Simon Giszt er, Richard Held, Neville Hogan, Mike Jordan, and Eric Loeb. We are oarticularlv arateful for the time and attention given to this project by Prof. Bi&. Finan cial support wa s provided in part by grants from the NIH (NS09343 an d AR26710) and the ONR (N0001 4/90/5/194 6). R.S. was supported by the McDonnell-Pew Center for Cognit ive Neurosci ences and the Center for I%ological and Computational Learning at MIT. Send correspondence to Dr. Reza Shadmehr, Room E25-201, Department of Brain and Cogn itive Sciences, Massa chuse tts Institute of Techn ology, Cambridge , MA 02139. Copyright 0 1994 Society for Neuroscience 0270.6474/94/143208-17$05.00/O state space. T hese elements formed a model that extra p- olated to outside the training region in a coordinate system similar to that of the joints and muscles rathe r than end-point forces. This geometric property suggests that the eleme nts of the adaptive process represent dynamics of a motor task in terms of the intrinsic coordinate system of the sensors and actuators. [Key wo rds : motor learning, reaching movements, internal model s, force fields, virtual environm ents, generaliz ation, motor control] Childr en start to reach f or objects that interest them at about the age of 3 months. These goal-directed movements often ac- company a “flail ing” action of the arm. From a system s point of view, flailing can be seen as an attempt to excite the dynamics of the arm: to make a reaching movement successfu lly, the motor controller needs to find the appropriate force so that the skeletal system makes the desired motion. Effectivel y, this op- eration correspo nds to inverting a dynamical transformation that relates an input force to an output motion. A controller may impleme nt this “inverse transformation” via a combina- tion of feedback and feedfor ward mech anism s: usua lly, the feed- forward component provides some estima te ofthe inverse trans- formation-called the “inverse model” or simp ly the “internal model”-while the feedback component compe nsates for the errors of this estimation and stabilizes the system about the desired behavior (cf. Slotine, 1985). Therefore, the internal model refers to an approximation ofthe inverse dynamics ofthe system being con trolled . In the case of the infant, the action of flailing may be an attempt to explore this dynam ics and build an internal model. During development, bones grow and muscl e mass increases, changing the dynamics of the arm significantly. In addition to such gradual variations, the arm dynamics change in a shorter time scale when we grasp objects and per for m manipulation. The changing dynamics of the a rm make it so that the same muscle forces produce a variety of motor behaviors. It follows that to maintain a desired performan ce, the controller needs to be “robust” to changes in the dyna mics of the arm. This ro- bustn ess may be achieved through an updating , or adaptatio n, of the internal model. Indeed, humans excel in t he ability to adapt ra pidly to the variable dyna mics of their arm as the hand inter acts with the environment. Therefore, a task where the hand inter acts with a novel mechanica l environment might be a good candidate for studying how the CNS updates its internal model and learns dynamics. The particular tas k that we hav e co nsidered is one where a subject makes a reaching moveme nt while the hand interacts
