embodied language - a review of the role of the motor

8/10/2019 Embodied Language - A Review of the Role of the Motor

1/27

PLEASE SCROLL DOWN FOR ARTICLE

This article was downloaded by: [Illinois State University] On: 13 January 2011Access details: Access Details: [subscription number 788830241] Publisher Psychology Press Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

The Quarterly Journal of Experimental PsychologyPublication details, including instructions for authors and subscription information:http://www.informaworld.com/smpp/title~content=t716100704

Embodied language A review of the role of the motor system in languagecomprehensionMartin H. Fischer a; Rolf A. Zwaan ba University of Dundee, Dundee, UK b Erasmus University, Rotterdam, The Netherlands

First published on: 30 January 2008

To cite this Article Fischer, Martin H. and Zwaan, Rolf A.(2008) 'Embodied language: A review of the role of the motorsystem in language comprehension', The Quarterly Journal of Experimental Psychology, 61: 6, 825 850, First publishedon: 30 January 2008 (iFirst)To link to this Article DOI 10.1080/17470210701623605URL http://dx.doi.org/10.1080/17470210701623605

Full terms and conditions of use: http://www.informaworld.com/terms-and-conditions-of-access.pdf

This article may be used for research, teaching and private study purposes. Any substantial orsystematic reproduction, re-distribution, re-selling, loan or sub-licensing, systematic supply ordistribution in any form to anyone is expressly forbidden.

The publisher does not give any warranty express or implied or make any representation that the contentswill be complete or accurate or up to date. The accuracy of any instructions, formulae and drug dosesshould be independently verified with primary sources. The publisher shall not be liable for any loss,actions, claims, proceedings, demand or costs or damages whatsoever or howsoever caused arising directlyor indirectly in connection with or arising out of the use of this material.
http://www.informaworld.com/smpp/title~content=t716100704http://dx.doi.org/10.1080/17470210701623605http://www.informaworld.com/terms-and-conditions-of-access.pdfhttp://www.informaworld.com/terms-and-conditions-of-access.pdfhttp://dx.doi.org/10.1080/17470210701623605http://www.informaworld.com/smpp/title~content=t716100704


2/27

Embodied language: A review of the role of the motor system in language comprehension

Martin H. Fischer University of Dundee, Dundee, UK

Rolf A. Zwaan Erasmus University, Rotterdam, The Netherlands

A growing body of research suggests that comprehending verbal descriptions of actions relies on aninternal simulation of the described action. To assess this motor resonance account of language com-prehension, we rst review recent developments in the literature on perception and action, with a view towards language processing. We then examine studies of language processing from an action simu-lation perspective. We conclude by discussing several criteria that might be helpful with regard toassessing the role of motor resonance during language comprehension.

Keywords: Embodied cognition; Language processing; Mirror neurons; Motor resonance; Perceptionand action.

Taking the right actions is the result of efcientcognition. Yet, for much of the time spent study-ing human cognition, actions have been con-sidered as trivial appendages to the seemingly

more sophisticated mental operations subservinghigher level cognition, such as object identi-cation, language comprehension, or decisionmaking. Traditionally, movement-related pro-cesses have been reduced to simple buttonpresses in order to isolate, as much as possible,the central cognitive processes that were of interest(see Abrams & Balota, 1991, for an early critique

of this approach). As a result of this neglect of motor control in the science of mental life andbehavior (Rosenbaum, 2005), several recentreports of effects of movement-related processes

on higher level cognition have had the advantageof surprise and have gained much attention in acommunity interested in a more general under-standing of human cognition. Examples of suchndings include action effect blindness (Musseler& Hommel, 1997), where the preparation of alateralizedresponse temporarilyhinders perceptionof stimulus attributes with the same lateralized

Correspondence should be addressed to Martin Fischer, School of Psychology, University of Dundee, Dundee DD1 4HN, UK,or to Rolf A. Zwaan, Department of Psychology, Erasmus University, 3000 DR Rotterdam, The Netherlands. E-mail:[email protected] or [email protected]

MHF is a member of the Marie Curie Research and Training Network: Language and Brain (RTN: LAB), which is funded by the European Commission (MRTN-CT-2004512141) as part of its Sixth Framework Program (for details see http:// www.hull.ac.uk / RTN-LAB / ). RAZ was supported by Grants MH-63972 from the National Institutes of Health and BCS-0446637 from theNational Science Foundation. The authors thank Art Glenberg, Marc Jeannerod, and two anonymous reviewers for their helpfulcomments.

# 2008 The Experimental Psychology Society 825http://www.psypress.com/qjep DOI:10.1080/17470210701623605

THE QUARTERLY JOURNAL OF EXPERIMENTAL PSYCHOLOGY 2008, 61 (6), 825850


3/27

features, or the action sentence compatibility effect(ACE; Glenberg & Kaschak, 2002), where the verbal description of spatially directional actionsfacilitates movements in the same direction. Thelatter nding, and others to be reviewed below,illustrates the intriguing possibility that languagecomprehension may incorporate, and possibly even require as an essential component, someactivity of the motor system that could be charac-terized as motor resonance (Zwaan & Taylor,2006). Interactions between language processingand motor processing, and the idea of motor res-onance in particular, are the focus of the presentreview.

Inspired by ndings of unexpected facilitationand inhibition, and motivated by a need to incor-porate such effects into a valid understanding of

human cognition, the increasingly prevalent view today is that perceptual and action-related pro-cesses are tightly linked to each other, as well asto more abstract cognition (e.g., Barsalou, 1999,2007). In turn, a proper account of cognition ispredicated on an understanding of its links withperception and action. This notion, that cognitionis grounded in perception and action, is encapsu-lated in the term embodiment: A principledunderstanding of cognition ought to relate cogni-tion to the mechanisms that govern the perceptual

processes feeding into cognition, as well as theactions selected by and guided through cognition. The physical instantiation of the cognitive appar-atus as a brain inside a body provides such prin-ciples, be it physical laws or biomechanicalconstraints (e.g., Shepard, 1984, 1994).

The purpose of this article is to provide a sys-tematic overview of how motor processes, and inparticular the metaphorical motor resonance,may subserve visual cognition, action understand-ing, and especially language comprehension. TheSection 1 of this article describes some current views of the relationship between perception andaction, including the two-visual-pathways theory,the theory of event coding, the mirror systemhypothesis, and the recently proposed view of motor cognition. We then discuss in Section 2possible mechanisms for motor resonance in twodomains of visual cognitionnamely, affordance

computation during object recognition andmotor resonance during action observation, witha view towards the suitability of the resonancemetaphor for language processing. In Section 3 we review the current evidence of motor resonancein language processing, organized in increasinglevels of language complexity. The articleconcludes with an outlook on future research.

1. Current views of the relationshipbetween perception and actionIn order to better understand the current debateabout possible links between action-related andlanguage-related processes, a brief review of recent theoretical developments is in order. This will also help us to introduce some of the key

facts that constrain this discussion, some of which will be revisited towards the end of thisreview.

1.1. The two-visual-pathways theoryVisual information processing has traditionally been understood as the input stage to higherlevel cognition, with action being the mere reec-tion and implementation of the outcomes of thiscentral process. However, in recent years thisnotion of a single and serial information proces-

sing chain has been replaced by a more differen-tiated conceptualization in which the same visualinformation is processed differently for perceptualand action-related tasks. Based on pioneeringanimal studies by Schneider (1969) and by Ungerleider and Mishkin (1982), and furtherinspired by performance dissociations in a patient with visual agnosia (Goodale & Milner, 2004;Milner & Goodale, 1995/ 2006), this view dis-tinguishes two visual pathways in the brain, onefor conscious perception of objects and the othersubserving the visual control of ongoing actions.In this model, vision for action relies on egocentri-cally coded information about the location andother spatial attributes of targets, whereas objectidentication uses allocentrically coded or scene-based information. These two processes are con-sidered to be largely segregated into distinctparts of each hemisphere in the primate brain,

826 THE QUARTERLY JOURNAL OF EXPERIMENTAL PSYCHOLOGY, 2008, 61 (6)

FISCHER AND ZWAAN


4/27

with object identication in the temporal lobes or ventral stream and action control in the parietallobes or dorsal stream.

Research with normal participants has providedsome surprising support for the two-visual-pathways hypothesis. For example, visual contextcan induce distorted perceptual impressions aboutthe size of objects while at the same time leavingaction control largely unaffected (for recent review and discussion, see Glover, 2004). As documentedin Bridgemans short review (Bridgeman & Hoover,2008 this issue), such dissociations between percep-tual reports and visually guided motor action can befound in normal observers, both in the laboratory and in the real world. We discuss some examples of this work below (see Section 3.2). For now, we notethat this view of the relationship between perception,

cognition, and action as originating from a commoninput stage but subsequently being anatomically andfunctionally segregated seems to further isolatecognition from perception and action by takinghigher level planning processes out of the loop,rather than embedding cognition and providing itsexperiential grounding. Nevertheless, the two- visual-pathways view places important constraintson the idea that we simulate internally the actionsdescribed by language. For example, it assumes thataction control relies on rapid visuo-motor updating

that creates short-lived but veridical representationsof the environment to ensure successful motor per-formance. The exact time course of this updatingmechanism is a matter of contention, but it seemsclear that spatial visual changes can modify ongoingactions within 150 ms and that such rapid modi-cations can occur without conscious control (cf.Desmurget, Pelisson, Rossetti, & Prablanc, 1998;Pisella et al., 2000). As we see later, this estimate isin harmony with ndings from brain imagingstudies on the time course of motor activation from verbal processing (see Section 2.3). What is at issueis whether these results can be taken as convergingevidence for the notion that action simulation ispart of language comprehension.

1.2. The theory of event coding A competing view of the relationship betweenperception and action has recently been offered

by the theory of event coding (TEC; e.g.,Hommel, 2004; Hommel, Musseler,Aschersleben, & Prinz, 2001; Knoblich & Prinz,2005). This theoretical development is based onthe notion that actions are cognitively representedin terms of their perceived effects. To overcomethe conceptual schism between separate represen-tational domains for perception and action, TECproposes a common representational medium forperceptual and action features (a commoncode). To illustrate the idea of common coding,consider action effect blindness, where the prep-aration of a spatially selective action (e.g., a left key press) temporarily reduces perceptual sensitivity selectively for visual features with the samespatial attribute (e.g., a left pointing arrow;Musseler & Hommel, 1997). To understand

such surprising effects of action planning on per-ception, the proponents of TEC abandon strictly serial models of sequentially ordered and discreteinformation processing stages. Instead, they assume that motor planning operates on feature-based representations in the same way as percep-tion and also requires some binding mechanismto generate goal-directed actions. When featuressuch as the left code of a response becomebound into an action plan, they are temporarily unavailable to code corresponding visual infor-

mation about the location or direction of a stimu-lus, thus leading to the inability to properly represent and perceive particular visual stimuli.

TEC is inspired by the idea that actions are theresult of the anticipation of sensory consequences,a proposal that originates with ideo-motor the-ories of cognition (for a short historical review,see Stock & Stock, 2004). According to this view, an agent already knows what to expectfrom a forthcoming action, and this anticipatedgoal state helps to select and guide the action.Such predictive knowledge might well be theresult of an internal action simulation process(Wolpert, Doya, & Kawato, 2003). One importantadvantage of using the ideo-motor principle toconceptualize action planning and action simu-lation is the resulting emphasis on learning anddevelopment. Clearly, any knowledge of sensory consequences to be expected from ones own

THE QUARTERLY JOURNAL OF EXPERIMENTAL PSYCHOLOGY, 2008, 61 (6) 827

MOTOR SYSTEM IN LANGUAGE


5/27

action must be due to having previously experi-enced the same contingencies between ones ownactions and their consequences. The agents learn-ing history can explain the bidirectionality of theassociations between sensory and motor represen-tations, which is necessary for the concept of action simulation to work. Moreover, this associ-ation learning principle can be extended tolanguage-based action simulations as well (seeGlenberg et al., 2008 this issue) and can thusaccount for intriguing associations that haverecently been documented between lexical andmotor processing (see Section 3.2).

However, in a recent elaboration on their orig-inal nding of action effect blindness, Hommeland Musseler (2006) showed that preparing a verbal utterance such as left or right had a det-

rimental effect only on the visual perception of words but not on the visual perception of arrows. Thus, a domain-specic overlap between stimulusfeatures and response features is necessary for thisblinding from binding to occur, whereas themore abstract semantic overlap of features (which would be a characteristic of language processing)appears to be ineffective.

Depending on ones interpretation, the commoncoding view, as represented in TEC, either offers aneven more reductive role for cognition (because all

goal-directed planning eventually dissolves into thecommon feature representations for perception andaction), or else successfully abandons the conun-drumof howtraditionally separatedrepresentationaldomains can communicate (because it abolishes thedistinction between perception, cognition, andaction). In any event, the use of TEC for a betterunderstanding of language-induced motor reson-ance seems limited in the light of its rejection of semantic mediation of feature overlap (Hommel & Musseler, 2006, p. 520). However, Kaschak andBorregine (2008 this issue) provide an example of how it may still be possible to account for the timecourse of motor resonance fromlanguage processingby using this framework.

1.3. Mirror neurons The idea of a common code for perception andaction as proposed by TEC has received strong

support from an important neurophysiological dis-covery. Using single-cell recordings, Rizzolatti andhis colleagues discovered so-called mirrorneurons in the ventral premotor area F5 of themacaque monkey (e.g., Di Pellegrino, Fadiga,Fogassi, Gallese, & Rizzolatti, 1992; Gallese,Fadiga, Fogassi, & Rizzolatti, 1996; Rizzolatti,Fogassi, & Gallese, 2001). In addition, area PF /PFG in the inferior parietal cortex has also beenidentied as containing such neurons. Theseneurons are sensitive to movement as well as to vision. Specically, they increase their ring rates when the monkey performs object-directedactions, as well as when it observes an exper-imenter or conspecic performing a similaraction. In this sense, the mirror neurons are per-spective independent. The degree to which the

observed and executed actions must be similar toeach other to be coded by a single mirror neuronis a matter of debate. This debate has led tofurther subdivisions of mirror neurons intobroadly congruent and strictly congruenttypes, depending on whether only the overallgoal (e.g., picking up an object) or also themeans of achieving it (e.g., with a power grip ora precision grip) is coded by the neuron (forreview, see Rizzolatti & Craighero, 2004).Mirror neurons do not respond to the

observation of an object alone (neurons withthis type of selectivity are called canonicalneurons), nor to the mere sight of a handmimicking an action in the absence of anobject. Thus, all three components of a transitiveaction (agent, patient, and action) are necessary to activate a mirror neuron in the monkeysbrain. This feature of mirror neurons may be rel-evant in the light of the effectiveness of differentsyntactic constructions in inducing motor reson-ance (see Section 3.3). However, Ferrari,Gallese, Rizzolatti, and Fogassi (2003) showedthat some mirror neurons discharge in responseto intransitive actions such as lip smacking ortongue protrusion, actions that are not objectdirected (although one possibility is that,during the observation of an intransitive action,the monkey reconstructs internally a goal-directed action).


FISCHER AND ZWAAN


6/27

Interestingly, some degree of abstraction fromthe visually given information can be tolerated by mirror neurons; they are activated by the sight of actions towards hidden objects as long as themonkey knows they are present (Umilta` et al.,2001). Other such abstraction properties of mirror neurons include the fact that actionsounds, such as the sound of tearing paper orcrushing peanuts, are sufcient to drive mirrorneuron activation (Keysers et al., 2003; Kohleret al., 2002) and their ability to discriminate themeaning of a given action on the basis of thecontext within which it is performed (Fogassiet al., 2005). Today it is widely believed thatseveral areas in the monkey brain encompass amirror system for action observation and under-standing. This system encompasses parietal and

frontal areas of the brain, in particular area F5(the premotor cortex) and also relies on additionalactivation from the superior temporal sulcus(which contains neurons tuned to body parts butnot movement).

Recent PET (positron emission tomography)and fMRI (functional magnetic resonanceimaging) studies suggest that humans alsopossess a mirror system (e.g., Lamm, Fischer, & Decety, 2007; Koski et al., 2002) and that thissystem might be somatotopically organized (e.g.,

Aziz-Zadeh, Wilson, Rizzolatti, & Iacoboni,2006; Buccino et al., 2001; Gazzola, Aziz-Zadeh, & Keysers, 2006). These studies show that mere observation of actions activates theBroca area, an inferior frontal brain area con-sidered to be homologous to area F5 in themacaque monkey. In contrast to monkey neurons, human mirror neurons seem to be morebroadly tuned, and this ability to abstract fromspecic, visually available situations is of key importance for the notion of motor resonance tolanguage input. One main observation implyinga broad and abstract tuning of the human mirrorsystem is that it codes intransitive (gestural) as well as transitive (object-directed) actions, asdocumented in an inuential TMS (transcranialmagnetic stimulation) study by Fadiga, Fogassi,Pavesi, and Rizzolatti (1995). The authors rstdetermined the minimal amount of left motor

cortical stimulation required to detect its presencein muscle evoked potentials (MEPs) over the rightarm. Then they investigated whether passiveobservation of various video sequences (of objects, arm actions, object-directed actions, andluminance changes) would selectively modulatethe transduction characteristics of the nervoussystem. It was found that observing both transitiveand intransitive arm actions increased the excit-ability of the observers motor cortex and thusimproved the gating of cortical stimulation intothe periphery (see Fadiga, Craighero, & Olivier,2005, for recent review).

Another important difference between thehuman and monkey mirror system appears to bethat static images that only imply an action seemto be sufcient to trigger the human mirror

system. This was rst illustrated by Nishitaniand Hari (2002) whose participants watchedphotographs of a person adopting various lip pos-tures while their brain activity was measured withhigh temporal and spatial resolution throughMEG (magneto-encephalography). Both speech-related and nonverbal lip postures triggered aseries of activations in successive processingstages of the brains of the viewers, and both theiramount and their timing were very similar underpassive viewing and active imitation conditions.

This result indicates that considerable motor res-onance resulted from viewing static facial posturesof others (see also Urgesi, Moro, Candidi, & Aglioti, 2006, described in Section 2.3).

Overall, these results obtained in humans indi-cate that their mirror neurons are even less depen-dent on specic, visually accessible actioninformation than are the monkeys mirrorneurons. Their ability to abstract from the visually given makes mirror neurons interesting to thosetrying to understand language-induced action sim-ulations. But there may be a cost associated withthe abstractness of language-based compared to vision-based input to the mirror system. Buccinoet al. (2005) made this point when comparingthe processing of sentences with the processingof pictures by means of brain activation in themirror system. They suggested that, in the lattercase, there is less ambiguity about details of the




7/27

action to be imagined, so that there can be facili-tation from vision. In the case of verbal descrip-tions, however, there is always a comparative lack of precision, thus either activating too many com-peting actions, or not activating any action simu-lation strongly enough. We describe Buccinoet al.s (2005) study in more detail below (seeSection 3.3).

Illustrating yet another possible differencebetween human and animal mirror systems,Buccino et al. (2005) suggested that motor reson-ance for oral communication in humans may belimited to conspecics. In this brain imagingstudy, they showed their participants mouth-related action videos of other humans (lip-reading), of monkeys (lip smacking), and of dogs(barking). There was a systematic reduction of

brain activity in the frontal operculum, withmost activation for observing the actions of con-specics, less activation for observing monkeys,and no activation for dogs. The fact that barkingdid not induce any frontal lobe activation overand above baseline suggests that motor resonancein the human mirror system could be the resultof matching the observed actions against theobservers action repertoire, with the degree of match reected in the corresponding increase of mirror system activity. However, Buccino et al.

(2005) used static images as a baseline and sub-tracted their associated activation from activationin the experimental conditions. In the light of ndings by Nishitani and Hari (2002, see alsoFischer, Prinz, & Lotz, 2008 this issue; Urgesiet al., 2006), this leaves open a possibility thateven actions outside ones action repertoire caninduce motor resonance.

A more compelling examination of therelationship between motor repertoire and motorresonance was recently reported by Calvo-Merino, Glaser, Grezes, Passingham, andHaggard (2005) who compared the brain activity of dancers who had expertise in either capoeira(a martial arts dance style) or classical ballet. Allparticipants watched dancers performing move-ments that either were or were not part of theobservers specialized dance repertoire. Theauthors found more activation in motor areas of

the brain, including parietal and prefrontal areasof the mirror system, when the visual input andmotor expertise overlapped. A follow-up study ruled out the possibility that this resonance effectmerely reected visual familiarity and not truly motor familiarity. These two are usually con-founded, either because one dances in front of amirror or because ones fellow dancers are practis-ing the same style. Calvo-Merino, Gre`zes, Glaser,Passingham, and Haggard (2006) utilized the factthat there are gender-specic movements that areperformed by either male or female balletdancers. This implies that male dancers whomostly perform with female dancers have amotor repertoire that is distinct from their visualrepertoire for ballet movements. When suchdance movements were presented to participants

in a brain scanner, stronger premotor, parietal,and also cerebellar activity was again found whenthe visual stimuli matched the observers actionrepertoires. Most recently, Cross, Hamilton, andGrafton (2007) trained their participants toperform novel dance moves and subsequently found a positive correlation between their per-ceived motor expertise and activation in brainareas typically involved in both action observationand action simulation.

The existence of mirror neurons provides

direct evidence for common coding at the neuro-physiological level. The characteristics of thehuman mirror system include perspective inde-pendence as well as sensitivity to impliedmotion and to communicative gestures regardlessof whether they are transitive or not. Importantly,the human mirror system also includes Brocasarea, the left-lateralized centre for speech pro-duction (Gallese & Goldman, 1998). Thesefeatures make it a plausible candidate forlanguage-based action simulation and have ledto new predictions for behavioural testing of theembodiment of cognition (e.g., Gentilucci & Dalla Volta, 2008 this issue; Glenberg et al.,2008 this issue; Masson, Bub, & Newton- Taylor, 2008 this issue). Given that languageacquisition precedes reading acquisition, theneural implementation of part of the humanmirror system in a speech area might also


FISCHER AND ZWAAN


8/27

provide a mechanism mediating motor resonancein reading (see Kaschak & Borreggine, 2008 thisissue; Nazir et al., 2008 this issue; Taylor & Zwaan, 2008 this issue).

1.4. Motor cognitionA nal and very recent theoretical development with implications for our understanding of thelink between language processing and motorcontrol is the emerging eld of motor cognition(Haggard, 2005; Jeannerod, 2001, 2006;Sommerville & Decety, 2006). Dened by Jeannerod (2006, p. v), motor cognition studiesthe way actions are thought, planned, intended,organized, perceived, understood, learned, imi-tated, attributed, or in a word, the way they arerepresented. This new perspective on human cog-

nition from a motor perspective has historical rootsin pragmatist psychology and the philosophy of language, as well as in cybernetics. It also ties inclosely with mental chronometry and with recentdevelopments in cognitive neuroscience andneural modelling.

At the heart of motor cognition is the idea that we simulate our own as well as other peoples beha- viour as part of understanding it. The same evidencethat we have reviewed so far is taken by proponentsof motor cognition to underline the experiential

grounding of all representations, regardless of whether they remain unconscious and subserveefcient action control, or whether they becomeconscious and inuence the attribution of agency.Motor cognition differs from the two-visual-pathways theory in that it distinguishes semantic frompragmatic representations within both perceptionand action, instead of strictly separating perceptionfrom action. From this perspective all languagebehaviour has both semantic and pragmaticaspects, and while speech and also sign languageare overt motor behaviours, inner speech andthought are examples of covert movements oraction simulations. A key role in language proces-sing is assigned to Brocas area as a productioncomprehension interface, with a ventral-to-dorsalgradient of semantic, syntactic, and phonologicalprocessing, respectively ( Jeannerod, 2006, p. 156).Clearly, the motor cognition perspective attributes

the various motor resonance phenomena discussedabove to action simulation as a necessary com-ponent in language comprehension.

2. Mechanisms of action simulationIn this section of our review we briey describetwo candidate mechanisms involved in actionsimulationnamely, the computation of affor-dances during object recognition and motorresonance during action observation. Both of thesemechanisms may be involved in language compre-hension, so we highlight evidence of interactionsbetween these mechanisms andlanguage processing. We also look at evidence regarding the time courseof such simulations, in order to provide further

constraints for the hypothesized link betweenaction simulation and language comprehension.

2.1. Affordance computation during object recognitionEffects of covert action simulation are abundant inthe literature on object perception, where thespeed and accuracy of lateralized responses areinuenced by the distance between an objectstypical contact point (e.g., a handle) and the observers hands. For example, when pressing buttons to

classify pictures of objects as right side up or upsidedown, the responses of the hands are facilitated when they are placed next to the protrudinghandles (e.g., Phillips & Ward, 2002). Thisrobust nding suggests that the motor systemspontaneously uses object information tocompute possible actions in the light of onescurrent posture and to select favourable responses. Whether these so-called affordances are computeddoes not depend on the object being response rel-evant. For example, Fischer and Dahl (2007)recently obtained powerful affordance effects when participants pressed left or right buttons inresponse to colour changes of a xation point while a cup with a protruding handle continuously rotated in the background. Reaction times weregradually modulated by the irrelevant handleposition, leading to two phase-shifted sine-wave-shaped reaction time functions, one for the left




9/27

and the other for the right hand. An importantand novel control condition, with object trans-lation equivalent to the size and opposite to theside of the handle, ruled out the possibility thatthis response bias merely reected the visualasymmetry of the object.

In addition to lateralized (left vs. right side)response biases, affordance effects can also inu-ence different responses within one hand. Tuckerand Ellis (2001) recorded the speed of precisionand power grasp responses of participants whoclassied objects as natural or man-made. Thetwo responses were made either by pressing aswitch between thumb and index nger, or by squeezing a stick with the remaining ngers of the right hand. There was a congruency effectbecause response selection was biased by the size

of the object. Electrophysiological and brainimaging work shows that also attentional processesare biased toward objects that can be manipulated(Grezes, Armony, Rowe, & Passingham, 2003;Grezes & Decety, 2002; Handy et al., 2005;Handy, Grafton, Shroff, Ketay, & Gazzaniga,2003).

How are visual affordances related to languageprocessing? Interestingly, affordance effects alsoemerge when participants are only briey exposed to a perceptually degraded object or

when they only see (Tucker & Ellis, 2004) orhear (Richardson, Spivey, & Cheung, 2001) a word denoting the object. Thus, continuous visual availability is not necessary to create a biasin the motor system, and concept activationseems to be sufcient to do so. These recent nd-ings require a revision of the original idea thataffordance computations bypass higher cognitivelevels of representation. We provide a more in-depth discussion of this topic in Section 3.2.

A study by Creem and Proftt (2001) providesfurther information about the possible interactionsbetween affordance computation and languageprocessing. The authors imposed either a semanticor a spatial secondary task on their normal partici-pants as they picked up household tools that wererandomly placed in front of them. When they performed a spatial imagery task they spon-taneously picked up all tools by their handles and

thus exhibited an affordance effect. In contrast, when they produced semantic associations theaffordance effect was clearly reduced. Both second-ary tasks were comparable in difculty and verbaloutput modality, so the authors concluded thatthere is an obligatory connection between seman-tic processing and action control.

These ndings of concept-modulated actioneffects challenge the two-visual-pathways ideathat vision is isolated from cognition and rapidly updates the representation of our environment toprovide veridical information for on-line actioncontrol. A possible solution to this challengemight be to distinguish between an early planningcomponent and a late control component of goal-directed motor responses. Biases in response selec-tion (such as the affordance effects) would be

attributed to early motor planning while the sub-sequent control phase would reect veridical andunbiased vision (Glover, 2004). Again, this ideais elaborated in Section 3.2.

The affordance studies mentioned so fardescribe systematic inuences of object perceptionon motor planning. Consider now the oppositerelationship of motor activity biasing object per-ception. This is an important issue, given theidea that action simulation might rely on theanticipation of outcomes and thus on learned,

bidirectional links between actions and goals (seeSection 1.2). A number of studies show that theintention to make goal-directed movements leadsto systematic biases in visual perception. Inaddition to the already-mentioned action effectblindness (see Section 1.2), there is also evidencefor facilitation. For example, planning to make agoal-directed eye or hand movement leads toimproved perception at the target location of themovement (e.g., Deubel, Schneider, & Paprotta,1998; Fischer, 1997, 1999). This is also true forsequences of eye movements (Gersch, Kowler, & Dosher, 2004; Gersch, Schnitzer, Sanghvi,Dosher, & Kowler, 2006) and sequences of handmovements (Baldauf, Wolf, & Deubel, 2006).Recent evidence for a dependence of suchmotor-visual priming on the motor repertoire of the observers comes from work by Casile andGiese (2006). In their study, the visual recognition


FISCHER AND ZWAAN


10/27

of gait patterns from point-light stimuli wasassessed before and after their participants hadlearned a novel movement that matched one of the visual test patterns. Despite the absence of vision during training, visual recognition of thelearned movement was selectively improved aftertraining and correlated with the acquired motorskill level. These results show that motor learningmodulates visual recognition of the learned move-ment and further suggests that the mirror system with its resonance mechanism might be involvedin the effect. We list this particular study herefor its methodological demonstration of an effectof action repertoire on action perception. Theidea of experience-dependent action processing(what I can do determines what I can under-stand) places important constraints on motor res-

onance from language processing. For example, itmight predict that the processing of negative andcounterfactual statements should not lead tomotor resonance. In potential conict with thisprediction, however, Kaup and Zwaan (2003; seealso Kaup, Yaxley, Madden, Zwaan, & Ludtke,2007) recently demonstrated that negative state-ments seem to induce a simulation of the positive version of their contents, which may then supportmotor resonance phenomena.

2.2. Motor resonance during action observationGiven the hypothesis of a mirror system in humansthat matches observed actions against ones ownaction repertoire to discover goals or intentions,it appears that the domain of signed languagecommunication and gesturing would be an idealeld to study motor resonance in the context of language processing. Unlike speech perception orreading, sign language processing requires thedirect mapping of body movements onto abstractmeaning representations. However, a recentreview of the relationship between sign languageand the human mirror system (Corina & Knapp,2006) concludes that this link is not as tight asexpected. For example, after damage to Brocasarea patients can exhibit a dissociation of impairedsign production with intact sign language compre-hension, while the opposite dissociation can follow damage to left parietal areas extending into

supramarginal gyrus (e.g., Poizner, Klima, & Bellugi, 1987). While these lesion sites are encom-passed by the human mirror system, the doubledissociation pattern argues against the notion of a mirror matching mechanism for language com-prehension. Furthermore, sign language pro-duction seems to rely predominantly on leftfrontal brain areas while sign language compre-hension also involves right frontal areas (Corina& Knapp, 2006, p. 534). This bilateral patternprobably reects the life-long associationbetween language concepts and space for nativesigners (Emmorey et al., 2005), but there is alsoa pervasive association of language productionand manual gesturing in all language users, even when listeners cannot observe the gestures, as inphone conversations or conversations between

blind speakers (Iverson & Goldin-Meadow,1998). Gentilucci and Dalla Volta (2008 thisissue) discuss further interactions between gestur-ing and language production in the context of themirror neuron theory. We now take a broader look at evidence for biases in the motor system as aresult of action observation.

Clear evidence for motor resonance duringaction observations comes from the work of Flanagan and Johansson (2003). These authorsset out to test the direct matching hypothesis of

action understanding, according to which weunderstand another persons intentions on thebasis of the similarity of their actions with ourown repertoire of previously performed actions.Specically, they focused on the tight link thatexists between our hand movements and the eyemovements that we make to guide our hands. Their participants either actively stacked blocks with their right hand to replicate a model, orpassively observed an actor completing the sametask while sitting across the table from them.Hand and eye movements were recorded in bothtasks. The key nding was that, in the passivecondition, the observers eyes looked ahead of the actors hand to the goal of each reaching move-ment, much as they did in the active condition as aresult of normal eyehand coordination. Flanaganand Johansson (2003) inferred that the actionrepertoire of the observer was engaged to predict




11/27

what the actor intended to do, rather than tomerely passively observe (see Rotman, Troje, Johansson, & Flanagan, 2006, for a replication with uncertainty about forthcoming actions).Interestingly, such motor resonance effects canalready be observed in the eye movement patternsof 12-month-old children (but not in 6-month-olds; Falck-Ytter, Gredeback, & von Hofsten,2006).

Eye movements in response to language proces-sing have been thoroughly studied in the visual world paradigm, where participants view arraysof objects while listening to auditory statementsabout the depicted objects. A consistent ndingis that they spontaneously xate on the objectthat is named next, as soon as the auditory evi-dence becomes unambiguous (for a recent review,

see Altmann & Kamide, 2004). This importantobservation has led to a urry of eye movementstudies in attempts to understand the nature of language processing and suggests that the simu-lation mechanism driving the eyes might be very fast. It is also memory driven in the sense thateye xations are made to blank locations on thescreen where the mentioned object had beendepicted earlier (Altmann, 2004; see alsoKennedy, 1983). A similar nding was reportedby Spivey and Geng (2001) who monitored eye

movements as participants looked at a blank walland listened to verbal reports of objects andevents. Interestingly, when they heard a descrip-tion of a tall building, their eye movements werepredominantly vertical. These observations are inharmony with the earlier nding that mirrorneurons can be driven by the memory of anobject (Umilta et al., 2001). As expected on thebasis of the evidence for abstraction in the mirrorsystem (see Section 1.3), the eye movementsystem can engage in action simulation even inthe absence of visible actionsfor example, when listening to descriptions.

2.3. The time course of motor simulationAs described above (see Section 1.3), Fadiga andcolleagues (2005; Fadiga et al., 1995) had shownthat a TMS pulse would pass through the nervoussystems of their observers much better when they

watched grasping actions instead of other percep-tual events at the time of stimulation. Building onthis method, Gangitano, Mottaghy, and Pascual-Leone (2004) titrated the time course of motor res-onance through mirror neurons. They asked how much of the TMS probe would be gated to theirobservers arms at various times during actionobservation, or when the observed action wouldbe perturbed to various degrees and at differenttimes during the movement. Delaying the ngeraperture in the action video by only 600 ms abol-ished motor resonance from the earliest time of probing (1,200 ms) and throughout the observationperiod. This result indicates that observers weresensitive to the absence of small aperture changesbetween index nger and thumb even at movementonset. It also suggests that there is an all-or-none

criterion for activating motor resonance.Further expanding on this simulationstimulation approach to embodied cognition,Urgesi et al. (2006) recently presented to theirparticipants static pictures of a resting hand, agrasping hand in (implied) motion, or a handadopting a nal precision grip posture (butalways without objects). The authors found TMS gating effects exclusively for the impliedmotion picture, not for the start or end postures. They conclude that pictures of a hand in action

conveyed dynamic information about forwardand backward action paths, whereas the nalposture hand provided information only aboutbackward action paths. This would suggest thatthe motor system was maximally activated by the extrapolation of the future trajectory of body actions (p. 7948). We note that, in contrastto Gangitano et al.s participants, who decidedduring each movie whether a tone occurred while the moving hand was to the left or rightof their xation cross, participants in Urgesiet al.s study had no motor task and hence wereunlikely to be engaged in any kind of responsepreparation. Also, the fact that their resultsobtained without objects being present in any of the hand pictures underlines the fact that actiongoals can be supplied through top-down cognitiveinferences (i.e., action simulations), just as they were in the Umilta et al.s (2001) study of


FISCHER AND ZWAAN


12/27

single-cell activity with actions towards hiddenobjects and in Kohler et al.s (2002; see alsoKeysers et al., 2003) single-cell study withaction sounds.

The conclusions drawn by Urgesi et al. (2006)nicely converge with ndings from a recent study of the time course of joint attention fromhand postures (Fischer & Szymkowiak, 2004).Observers saw pictures of a static hand, eitherpointing at or grasping one of three horizontally arranged tangerines. After the hand had returnedto a neutral resting position, and following arandom delay of 300 700 ms, a visual probeappeared unpredictably over the left or right tan-gerine, thus creating three trial categories: validtrials, where the probe appeared over the manipu-lated tangerine, neutral trials, where the hand had

previously interacted with the central tangerine,and invalid trials, where the probe appeared onthe tangerine opposite to the primed tangerine. The observers simply pressed a button to indicatethey had detected the probes. Importantly, in thepointing condition they were faster when theprobe appeared in the valid than in the neutraland invalid trials, whereas in the grasping con-dition, they detected the probes faster in theinvalid than in neutral and valid trials. Theauthors suggested that these differences between

observing a pointing and a grasping action weredue to rapid anticipation (or simulation) of likely future states. Specically, a pointing hand promptsthe observers attention because it indicates thelikely target of a future action, whereas a graspinghand cannot prompt the observers attentionbecause it depicts an already completed action(but see Fischer et al., 2008 this issue).

In a follow-up study Nuku et al. (2007) showedthat low-level perceptual factors contributed to theobserved inhibitory pattern for grasping in Fischerand Szymkowiaks (2004) study: Spatial separ-ations between hand postures and objects weresmaller in the grasping than in the pointing con-ditions, thus leading to inhibition of return of attention (e.g., Pratt, Kingstone, & Khoe, 1997). When distances were equally short for pointingand grasping, inhibition in valid trials waspresent for both pointing and grasping. Thus,

visually similar conditions are important toensure that any differences can safely be attributedto an internal motor simulation. Interestingly,however, when Nuku et al. (2007) changed theto-be-detected event from an arbitrary onset to adisplacement of one of the objects, higher levelaction simulation in valid trials overruled thelow-level inhibition. Thus, it appears that actionsimulation benets when observers can establisha causal relation between an observed action andan observed effect.

These ndings support the idea of a rapid, obli-gatory and visually driven action simulation fromobserving others grasping postures. As a resultof this action simulation, observers allocate theirattention to the most plausible target object of the forthcoming action, just as they do when

they are about to perform the same action them-selves (Baldauf et al., 2006; Deubel et al., 1998). The discrepancy between the positive ndings with static end postures in Fischer and Nukusstudies and the negative ndings for similar endpostures in Urgesi et al.s (2006) recent TMSstudy suggests that the visual presence of objectsmay be advantageous for action simulation afterall. Whether such a need to explicitly mentionthe objects of actions also exists for language-induced motor simulations remains to be investi-

gated. Together, these behavioural results reveala complex pattern of activation and inhibitionduring action simulation from implied actions,making it hard to derive clear prediction forlanguage-induced action simulation.

The use of static pictures instead of dynamicsequences in Nishitani and Haris (2002) study,as well as their use of MEG as a brain imagingtechnique with high spatial and temporal resol-ution, allowed these authors to pinpoint the timecourse and neural implementation of motor reson-ance in great detail. They found rapid cortical res-onance to static lip postures, regardless of theobservers intention to observe or imitate, startingat occipital visual areas (around 110 ms afterstimulus onset) and progressing successively through superior temporal sulcus (180 ms),inferior parietal areas (200 ms), and inferiorfrontal areas (Brocas area, 250 ms) into primary




13/27

motor cortex (by 320 ms; see also Nishitani & Hari, 2002). These estimates underline the rapid-ity of motor resonance phenomena and are in line with recent evidence (also obtained with MEG)on language-induced motor resonance that isreported in Section 3.2.

Further converging evidence that motor reson-ance also occurs when the result of an action isbeing viewed comes from the nding that hand- written letters, as compared to printed letters,activate the hand area of the motor cortex(Longcamp, Tanskanen, & Hari, 2006). In other words, viewing the result of an action activatesthe processes that would bring about that result. This interpretation is consistent with TEC (seeSection 1.2), according to which bidirectionallinks between actions and their anticipated effects

must be activated as part of action planning.

3. Motor processes in languagecomprehensionAlthough it is tempting to view the comprehen-sion of action sentences as a kind of action observation, it is important not to overlook thedifferences between the direct observation of anaction and the comprehension of a description of that action. Directly observing an action provides

analogue online temporal information about theactions unfolding. This is not necessarily truefor language comprehension. For example, inhearing a sentence such as He turned the page ,the speed with which the sentence is processed ispredominantly determined by the speakers rateof speaking, rather than by how quickly theaction is performed. However, a recent study shows that speech rate is correlated with thespeed of the described event (Shintel, Nusbaum,& Okrent, 2006). Similarly, in reading, thespeed with which a described event is processedis codetermined by a number of factors unrelatedto the manner of the action (lexical access, syntac-tic processes, and so on). But again, an authorsspeech rate inuences the reading speed for theirtext (Kosslyn & Matt, 1977).

Secondly, even if there were a close correspon-dence between information acquisition via direct

observation and processing times for a correspond-ing sentence, this would be irrelevant. The actiondoes not unfold in the comprehenders mind aseach word is being processed. For example, wedont know what the protagonist is turning until we encounter the noun page . For all we know, heor she could be turning a corner, turning his lifearound, or turning green with envy, among otherpossibilities. This lack of determinacy means thatmotor resonance should arise at specic points inthe sentence, rather than throughout. It is there-fore important to examine how the modulationof motor resonance is modulated by incoming lin-guistic input. A few recent studies have taken thisstep (Glenberg et al., 2008 this issue; Taylor & Zwaan, 2008 this issue; Zwaan & Taylor, 2006).

Another difference between action observation

and language comprehension is that a great dealof information can be omitted from the latterthat is explicit in the former. A sentence doesnot have to include a description of the actor orthe patient, nor of the manner in which theaction is being performed. For instance, theexample sentence He turned the page does notspecify who is turning the page other than that itis a male individual. Also, the sentence does notspecify of what larger entity the page is a partabook, magazine, newspaper, calendar? Finally,

the sentence does not specify how the action isperformed (carefully, quickly, nonchalantly?). This presumably means that language-bornmotor resonance is more diffuse than motor reson-ance evoked during action observation or imitation(see also Buccino et al., 2005). More evidenceregarding this issue is necessary, however, beforermer conclusions can be reached.

From these preliminary reections it is already clear that language processing places different con-straints on motor resonance than does actionobservation. With this in mind, it is useful toreview the extant ndings. The review is organizedin terms of level of linguistic units, moving fromsyllables to discourse.

Before reviewing the literature, it is importantto distinguish two types of motor resonance thatmight occur during language comprehension.Communicative motor resonance occurs when the


FISCHER AND ZWAAN


14/27

motor system responds to the communicative actitself, whereas referential motor resonance occurs when the motor system responds to the contentof the communication. An example may clarify this distinction. If a listeners speech motorsystem responds to hearing the word kick, thenthis would be an example of communicativemotor resonance; the motor system is simulatingthe production of the utterance. However, if theleg area of the premotor cortex responds, this would indicate referential motor resonance; themotor system is simulating the action that isbeing described by the utterance rather than theproduction of the utterance itself. The two typesof motor resonance presumably occur simul-taneously during skilled language comprehension. We speculate that communicative resonance helps

the comprehender anticipate what the speaker isgoing to say next, whereas referential motor reson-ance will tell the comprehender what is going tohappen next in the situation that is beingdescribed. The two types of simulation are pre-sumably mutually constraining (see Pickering & Garrod, 2007, and Zwaan & Kaschak, in press,for further discussion).

3.1. Phonological processing The role of the motor system in speech perception

has long been recognized, particularly in the form of the motor theory of speech perception (Liberman,Cooper, Shankweiler, & Studdert-Kennedy,1967). According to this theory, the objects of speech perception are the intended phonetic ges-tures of the speaker, represented in the brain asinvariant motor commands (Liberman & Mattingly, 1985, p. 2; see Galantucci, Fowler, & Turvey, 2006, for a recent review). A great deal of recent evidence supports the notion of motor involvement in speech perception. There is evidence thatmotor resonance occurs at the level of speech pro-duction. For example, Fadiga, Craighero,Buccino, and Rizzolatti (2002) applied a TMSpulse over the motor area controlling tonguemuscles while participants listened to Italian words and pseudowords containing a tongue-trilled, double-r sound or a nontrilled, double-f sound. The authors found that MEPs recorded

from tongue muscles were signicantly larger when listening to double-r stimuli than double-f stimuli. In other words, listening to linguisticstimuli produced the phoneme-specic activationof speech motor centres (see also Gentilucci & Dalla Volta, 2008 this issue).

These ndings suggest that online observationof an action is not necessary for motor resonance,a conclusion supported by nding auditory activation of the mirror system (Keysers et al., 2003;Kohler et al., 2002; Lewis, Phinney, Brefczynski-Lewis, & DeYoe, 2006). Instead, images of implied actions seem to sufce (see also Lammet al., 2007; Fischer, 2005). This conclusion is inaccordance with the earlier observation that affor-dance effects can be induced by perceptually degraded object pictures or object words.

Apparently, the activation of an action concept issufcient for motor resonance, as long as theinitial input is visual and hence concrete, asopposed to linguistic/ abstract.

However, it is surprising to nd that a goaldepiction leads to the activation of a particularset of muscles, when other muscle activations canlead to the same goal. The link between motoractivity and action outcomes is never uniquely specied: Many different actions can lead to thesame goal (this is known as equinality or motor

equivalence). Thus, additional assumptions wouldhave to be introduced, such as selection criteriathat reduce the degrees of freedom problem by evaluating efciency criteria (e.g., required effort,end state comfort, or time to reach the goal) orcontextual constraints. The contribution of suchconsiderations in motor cognition has been docu-mented by Jeannerod (2006).

In summary, there is strong evidence formotor resonance during speech processing. Inaddition, there is evidence that the results of linguistic action may produce motor resonance.Importantly,bothof thesecases areexamplesof com-municative motor resonance . The rest of our review ismostly concerned with referential motor resonance.

3.2. Lexical accessSeveral behavioural studies have examined whether motor representations are evoked by




15/27

single words. Semantic effects have been found ongrasp aperture. For example, a word printed on anobject of xed size can affect the movementdirected towards that object. Thus, for Italian subjects, the word GRANDE (large) evokes alarger maximum grip aperture than the wordPICCOLO (small; Gentilucci, Benuzzi,Bertolani, Daprati, & Gangitano, 2000, Exp. 4). This nding suggests that concurrent semanticinformation interferes with action performance,but given that the eventual grasp aperture wasaccurate, the interference was compensated forduring the movement. Glover and colleaguesexplain this pattern of effects by assuming thatsemantic activation interfered with the planningof the action rather than with its online control(Glover, Rosenbaum, Graham, & Dixon, 2004).

Glover et al. hypothesized that planning andmotor control are differentially affected by the words. Specically, the planning system relies ona visual representation, which is susceptible tointerference from cognitive and perceptual vari-ables, leading to large and systematic errors inaction planning. On the other hand, the controlsystem relies on the visuospatial properties of thetarget itself, uncontaminated by other cognitiveand contextual variables; this allows it to correctfor the inuences of these variables in ight.

Glover and colleagues (2004) tested thishypothesis by using nouns denoting concreteobjects, associated with a specic hand aperture. This contrasts with Gentilucci and colleagues(2000), who used adjectives that explicitly men-tioned size. Glover et al.s subjects reached forblocks of different sizes after having silently reada word from a computer screen one secondearlier. The words referred to either small orlarge objects (e.g., GRAPE or APPLE).Subjects were told to pay attention to the words,because they would receive a memory test later. This test was never administered, but subjects were asked instead if they noticed anythingspecial about the experiment. Only one subjectnoticed the relevance of the size of the objectsdenoted by the words, but this subjects data fell within one standard deviation of the other subjects data and were therefore not omitted from

the analyses. As such, it appears that reading anobject name interferes with the planning of agrasping movement. Moreover, it appears thatthis occurs outside the subjects conscious aware-ness. These data do not show, however, thatmotor resonance occurs in the absence of amotor planning task.

Other studies have examined whether qualitat-ively different types of hand postures (rather thanmerely hand aperture) afforded by objects are activated by exposure to the associated words. Forexample, subjects who judged whether objectsshown in pictures were natural or man-made by manipulating an input device that required eithera power grip or a precision grip exhibited aresponse compatibility effect (Tucker & Ellis,2004). Moreover, power grip responses were

faster to pictures and words denoting objects thatrequire a power grip than to pictures and wordsdenoting objects requiring a precision grip, whereas the reverse was true for precision gripresponses (Tucker & Ellis, 2004, Exp. 3; see alsoSection 2.1).

Other experiments have shown primingbetween words denoting similar motor actions(Myung, Blumstein, & Sedivy, 2006; Exp. 1).For example, playing the piano and using a type- writer involve similar manual actions. Accordingly,

the corresponding words were found to primeone another in a lexical-decision experiment.Objects affording similar actions tend to besimilar in shape, such that priming might be dueto visual similarity. To address this problem,Myung et al. had selected words denoting objectsthat were rated as having high similarity in termsof manipulation, but low visual similarity. Thus,they explained the priming effect by assumingthat the words automatically evoked action rep-resentations, which mediated the priming fromthe prime to the target.

This ensemble of behavioural ndings sup-ports the conclusion that motor resonanceoccurs automatically during exposure to action-related words (nouns, verbs, adjectives).Evidence from neuroimaging studies and studiesusing TMS supports these conclusions. Severalof these studies have shown that the naming of


FISCHER AND ZWAAN


16/27

tools, as opposed to the naming of animals, dif-ferentially activates the left middle temporalgyrus, which is also activated by action generationtasks, and the left premotor cortex, which is gen-erally activated when participants imagine them-selves grasping objects with their dominanthand (e.g., Martin, Wiggs, Ungerleider, & Haxby, 1996). However, stronger evidence forthe role of motor resonance in language compre-hension would show that exposure to action andtool words evokes motor resonance, whichshould manifest as activation in motor areas of the brain. Several studies have indeed producedsuch evidence. Exposure to action verbs andtool words semantically related to actions elicitsstronger fronto-central cortical activation thandoes exposure to object words (Martin et al.,

1996; Preissl, Pulvermuller, Lutzenberger, & Birbaumer, 1995; Pulvermuller, Lutzenberger, & Preissl, 1999). More specically, the action words related to movements of the face, arm, orleg activated fronto-central cortex in a somatoto-pic fashion (Hauk, Johnsrude, & Pulvermuller,2004; Shtyrov, Hauk, & Pulvermuller, 2004)consistent with the claim that sensorimotorcortex processes action-related aspects of wordmeaning (Pulvermuller, 2005).

Further evidence for the automatic activation

of motor representations upon exposure toaction verbs comes from a study using high-density MEG (Pulvermuller, Shtyrov, & Ilmoniemi, 2005b). Subjects were engaged in adistractor task while listening to words denotingactions involving the leg or face. Different pat-terns of cortical activation were found for legand face words in premotor areas. Face-wordstimuli activated inferior frontocentral areasmore strongly than leg words, whereas thereverse was found at superior central sites.Importantly, these activations occurred within170 ms after onset of the words, thus making itunlikely that strategic factors contributed to theresult. Pulvermuller and colleagues interpret thisto reect the cortical somatotopy of motoractions signied by the words. The resultsfrom this study show that meaning access inaction word recognition is an early automatic

process reected by spatiotemporal signatures of word-evoked activity.

If motor resonance is automatically evoked by words, then it might be hypothesized that theprocess can run in reverse as well. That is, recog-nition of these words should be facilitated by stimulation of the relevant motor and premotorareas. A recent study provides support for thishypothesis (Pulvermuller, Hauk, Nikulin, & Ilmoniemi, 2005a). Hand and leg areas in theleft hemisphere were stimulated 150 ms after word onset using magnetic pulses below motorthreshold while healthy right-handed nativespeakers of English read common arm- and leg-related words intermixed with meaninglesspseudowords briey ashed on a computerscreen. Subjects had to respond by a brisk lip

movement only when recognizing a meaningful word. Lip movements were chosen for lexicaldecision responses to minimize interferencebetween semantic and motor preparation pro-cesses. In addition to observing somatotopically mapped facilitation, the authors also reportedthat this effect only emerged after left hemisphericstimulation, in agreement with the typicallanguage dominance of this hemisphere.

These data provide preliminary evidence that wordrecognition is facilitated by referent-congruent

motor activation. However, it should be empha-sized that motor activation occurred while the word was already being processed for 150 ms. Itis important for future research to examine moreclosely the temporal relation between the facili-tation of lexical access and motor resonance (cf.Nazir et al., 2008 this issue). One caveat aboutthese ndings is that they dont necessarily impli-cate the effector per se. They could implicate thetypical goals that one tries to accomplish with aparticular effector. For example, we typically actupon objects with our arms and hands (e.g., grasp , fold , bend ), whereas we use our legs for loco-motion (e.g., walk, trot , run). Nevertheless, thestudy by Pulvermuller et al. (2005a) is important,because it demonstrates that accessing action-related words not only evokes motor resonance,but that the process is bidirectional: Lexicalaccess of action words is facilitated by motor




17/27

resonance. Such bidirectionality is a key assump-tion made by TEC (see Section 1.2).

A similar conclusion can be reached based on arecent behavioural study (Lindemann, Stenneken, van Schie, & Bekkering, 2006). In these exper-iments, participants made lexical decisions to words in a go/ no-go task paradigm, after havingprepared for a specic action. They had to pick up either a magnifying glass and bring it to theeye or a cup and bring it to the mouth. A letterstring was presented after action onset. Thetarget words were eye and mouth. Preparing for acongruent action (e.g., picking up the magnifyingglass for eye ) facilitated lexical decisions on theassociated word. A subsequent experiment ruledout visual associations as a factor. These ndingssuggest that preparing for an action activates

semantic information associated with the goal of the action and thereby primes access to a lexicalrepresentation associated with that goal.However, these ndings should be interpreted cau-tiously given that there were just two target words, which were repeated over and over and could thushave given rise to expectancy effects. The resultsalso seem to be in conict with the previously dis-cussed action effect blindness (see Section 1.2)and suggest that the outcome of the interplay between language comprehension and action simu-

lation crucially depends on their relative timing.A recent study found further evidence that thetiming between the planning of a movement andlexical processing is important for the directionof the motor resonance effect. Boulenger et al.(2006) carried out two experiments in which par-ticipants had to perform a reaching movement.In the rst experiment a letter string appearedright after the onset of their movement. In thesecond experiment the letter string was presentedas the go signal. Participants were instructed toreturn their hand to the starting position if thestring was not a word. The results showed thataction verbs affected overt motor performancesignicantly compared to nouns: Within 200 msafter movement onset, processing action verbsinterfered with a concurrent reaching movement. This was evident in a slightly reduced peak wristacceleration but not in other kinematic

parameters, such as movement time. In thesecond experiment, the same words assisted thereaching task when processed before movementonset. This priming became evident 550580 msafter word onset, again only in wrist peak accelera-tion (see Nazir et al., 2008 this issue, for furtherevidence).

3.3. Sentence comprehensionSeveral studies have examined motor resonanceevoked during sensibility judgements of sentences. The key manipulation is that the physical responsemade by the subjects shares some characteristics with the action performed by the subject. In anearly study on motor involvement in sentence (oractually, word-pair) processing, subjects judgedthe sensibility of verbnoun pairs (e.g., squeeze

tomato) after having been primed with a handshape (Klatzky, Pellegrino, McCloskey, & Doherty, 1989). An action-appropriate handshape was found to prime the comprehensionof word pairs describing the manipulation of objects. For example, the sensibility of Throwing a dart was judged more quickly when subjects had their hands in the appropriateshape for throwing darts than when not.Crucially, lexical associations were ruled out as acause of this effect.

A more recent study of embodied sentence com-prehension was already mentioned above. In judging the sensibility of sentences such as Close the drawer , subjects moved their right hand fromone button to another one that was either closerto or further away from their body. In this case,the action performed by the subject would be com-patible with the sentence if the hand moved away from the bodybecause our hand usually movesaway from our body when we close a drawer whereas responses toward the subjects body would be incompatible with the described action. The nding was that action-compatible responses were faster than action-incompatible responseshence the actionsentence compatibility effect, orACE (Glenberg & Kaschak, 2002). The ACEhas been found not only for imperatives, but alsofor descriptive sentences of two types: doubleobject (e.g., Mike handed you the pizza) and dative


FISCHER AND ZWAAN


18/27

constructions ( Mike handed the pizza to you ).Moreover, the effect was also found for abstracttransfer sentences, such as Liz told you the story.At rst sight, this provides evidence of the general-ity of motor resonance. On further reection,however, it poses some problems for the interpret-ation of the effect as one of mental simulation. Whereas moving the hand away from the body can be plausibly said to be an integral part of closing a drawer, it is not an integral part of telling someone a story. This raises two relatedquestions. First, how did the ACE come about inthe latter type of sentence if not as a result of mental simulation, and second, what does itimply about mental simulation?

A potential answer to the rst question can befound by distinguishing between lexical items

and grammatical constructions. For example,some of the sentences used by Glenberg andKaschak (2002) had a double-object construction(e.g., Mike handed you the pizza). Such structuresmay have become associated with movementtoward or away from the body during early language acquisition, because these structures areassociated with verbs of manual transfer, such as give or bring (Tomasello, 2003). As such, it ispossible that the ACE was evoked as a result of the syntactic structure. In part, this is Glenberg

and Kaschaks point, but it is important to notethat this does then not implicate mental simu-lation as an explanation of the ACE in theseexperiments. Rather, the syntactic construction isassociated with a certain type of motor resonanceindependent of the linguistic content. Evidencefor a neural dissociation between lexical andconstructional representations is emerging(Kemmerer, 2006). If this reasoning is correct,then the motor resonance that is evoked by sen-tences may be a complex phenomenon. First,there is the action-specic motor resonanceinvoked by individual words or word combinations(see next paragraph); and second, there is moregeneral motor resonance evoked by the linguisticconstruction. Presumably, the ACE reported inGlenberg and Kaschak reects a combination of both of these aspects in the concrete sentencesand only the latter aspect in the more abstract

sentences (see also Glenberg et al., 2008 thisissue).

In addition to the syntactic issue with theGlenberg and Kaschak (2002) study mentionedearlier, there is another potential limitation tointerpreting the ACE as evidence for mental simu-lation during language comprehension. It cannotbe ruled out that motor resonance is a functionof making a sensibility judgement rather thancomprehending the sentence. To address thisissue, recent studies have tried to unpack theACE by examining the modulation of motor res-onance during sentence comprehension. Forexample, Zwaan and Taylor (2006, Exp. 4) hadsubjects read sentences like Because / the music /was too loud,/ he / turned down/ the / volume . Thesentences were presented in segments, here indi-

cated by slashes. The subjects read the sentencesby rotating a knob, with each 5 degrees of rotationresulting in the presentation of the next segment. The reading-rotation direction either matched ormismatched the rotation direction implied by thesentence. Results indicated that motor resonanceoccurred on the verb (e.g., turned down), but haddissipated on the next words. Zwaan and Taylorhypothesized that motor resonance might besubject to linguistic focus, rather than beinglimited to the verb. They observed that in their

experimental sentences, the focus shifted fromthe action to the patient or the result of theaction. They reasoned that maintaining focus onthe action should result in continued motor reson-ance. In order to test this prediction, in a morerecent study (Taylor & Zwaan, 2008 this issue),they created sentences such as The runner / was very / thirsty./ A fan/ handed him/ a bottle / of cold / water / which he / opened / quickly . Here, thefocus on the action is maintained by the adverb, which modies the action. In accordance withthe linguistic-focus prediction, the match advan-tage on the verb was now extended to the sub-sequent word, the adverb.

There is another aspect to these results that is worth noting: They show that motor resonancearises during comprehension, rather than merely during lexical access. For example, by itself a word such as open does not necessarily imply




19/27

(counterclockwise) wrist rotation. One can openones eyes, open a sunroof, open a bank account,or open a lead in a race, none of which necessarily involves wrist rotation. Moreover, adverbs such asquickly and gradually do not at all imply wristrotation without further context. In other words,the ndings of Zwaan and Taylor (2006) show that motor resonance occurs when information isintegrated. A limitation of these studies is thatby themselves they do not implicate motor reson-ance in comprehension, given the special nature of the reading task, which by necessity involvesmanual rotation. However, they do show thatmotor resonance is associated with linguisticcontent in a very precise manner.

Glenberg et al. (2008 this issue) also examinedmotor resonance during online sentence compre-

hension, using TMS. Their results are consistent with those of Zwaan and Taylor (2006) in thatthey found motor resonance on the verb. Inother words, motor resonance occurred duringcomprehension, rather than as the result of anextraneous task such as making sensibility judge-ments. Moreover, like the study of Zwaan and Taylor, this study also shows that motor resonanceoccurs as the result of integration, rather thanmerely activation.

Kaschak and Borregine (2008 this issue) pro-

vided further evidence that timing is importantin motor resonance studies. They found a facilita-tory effect on a secondary motor task early on, which reversed into a mismatch advantage atlater testing intervals (cf. also the results reportedby Boulenger et al., 2006). They explain theirresults by referring to the theory of event coding, TEC (Hommel et al., 2001). According to thisexplanation, comprehending the sentence grabsthe feature coding for towards or away, whichnow cannot be used for planning the motorresponse, thus resulting in an interference effect.

Evidence that affordances have an immediateinuence on sentence comprehension comesfrom a study employing the visual-world paradigm(Chambers, Tanenhaus, & Magnusson, 2004; seealso Section 2.2 above). Subjects listened to sen-tences describing simple spatial scenes while viewing a display composed of elements relevant

to the sentence and holding or not holding atool. Chambers and colleagues monitored how the subjects xations on the scene were modulatedby the linguistic input as a function of holding ornot holding the tool. Holding the tool affectedsyntactic parsing. Specically, it changed theamount of time spent looking at a (possible) goallocation that is likely under one parse or unlikely under the alternative parse.

Of course, this research does not show directly that motor resonance is evoked during normallanguage comprehension. However, it does show that motor resonance occurs very rapidly duringcomprehension, even before the associated linguis-tic constituent has been fully processed.

Buccino et al. (2005) used TMS and a behavioural paradigm to assess whether listening to

action-related sentences modulates the activity of the motor system. By means of single-pulse TMS, either the hand or the foot/ leg motor areain the left hemisphere was stimulated in distinctexperimental sessions, while participants were lis-tening to sentences expressing hand and footactions. Listening to abstract content sentencesserved as a control. MEPs were recorded fromhand and foot muscles. Results showed thatMEPs recorded from hand muscles were speci-cally modulated by listening to hand-action-

related sentences, whereas MEPs recorded fromfoot muscles were modulated by listening tofoot-action-related sentences. This modulationconsisted of an amplitude decrease of the recordedMEPs. In the behavioural task, participantsresponded with the hand or the foot while listen-ing to actions expressing hand and foot actions, ascompared to abstract sentences. In accordance with the TMS results, hand responses wereslower during hand-action-related sentences thanduring abstract sentences, and foot responses were slower during foot-action-related sentences.

A recent neuroimaging study found thatregions that were active during action observation were also active during sentence comprehension,in an effector-dependent manner (Aziz-Zadehet al., 2006). Thus areas in the premotor areathat were active during the observation of handactions were also active during the comprehension


FISCHER AND ZWAAN


20/27


21/27

that the following questions be considered infuture work.

4.1. The association questionAre motor representations associated with the per-formance of a cognitive task? That is, does theactivation of the motor system co-occur systemati-cally with the performance of a cognitive task? Tofurther specify this question, it could be asked whether the cognitive task performance precedesthe onset of motor activation, or whether theonset of motor activation precedes cognitive task performance. Answers to this question can beobtained in behavioural experiments, as well as inneuroimaging studies, for example by showingthat motor representations are activated duringpassive viewing. An important step in this direc-

tion has been taken by Pulvermuller et al.(2005a). If further systematic evidence can beobtained that activation of the motor system pre-cedes the activation of higher level cognitive pro-cesses, then this provides a step towardaddressing the next question, which is the neces-sity question.

4.2. The necessity questionIs the activation of motor representations necessary for language comprehension? Pertinent evidence

to answer this question can be obtained in lesionstudies with neuropsychological patients, or inarticial lesion studies using repetitive TMSstimulation to temporarily deactivate selectedbrain areas. If a brain area known to subserve par-ticular actions is damaged or incapacitated by a(temporary) lesion, and this lesion eliminates orat least signicantly impairs the performance of these actions, then it should also signicantly impair the comprehension of words, sentences,and discourse describing that action.

4.3. The sufciency questionIs the activation of motor representations sufcientfor the comprehension of action words, sentences,discourse? To answer this question, it would berequired to obtain an exhaustive listing of allother representations and neural substrates thatcould be involved in the task and then to assess

comprehension by selectively incapacitating thesemechanisms (e.g., by articial lesions or in dual-task paradigms). An afrmative answer to the suf-ciency questions would be in order if it could bedemonstrated that comprehension still occursdespite the fact that all other potentially usefulsources of information except the motor systemhave been successfully disabled. It might beargued that such demonstrations are a logicalimpossibility, given that it may be impossible toknow beforehand all of the sources of informationthat may potentially be useful to comprehend aparticular segment of linguistic input. On theother hand, it is not implausible to anticipatethat future theories will at some point convergeon a catalogue of information sources that may contribute to an understanding of linguistic seg-

ments. At such a juncture in the literature, itmay be possible to address the sufciency question.In more general terms, this goal has already beenaccomplished by the theory of motor cognition(Jeannerod, 2006; Sommerville & Decety, 2006).From the viewpoint of motor cognition languageis abstract goal-directed action, and all languagecomprehension is through covert reenacting.

By systematically addressing the association,necessity, and sufciency questions, it should bepossible to build a theory of the role of motor pro-

cesses in cognitive performance generally andlanguage comprehension in particular. There areseveral issues related to language processing for which competing theories would make distinctpredictions. Among them are the narrative per-spective (rst vs. third person), the mappingbetween the comprehenders action repertoireand the content of the discourse, the level of abstractness of the narrative, and the presence of specic object descriptions, or its degree of special-ization. With respect to the relationship betweeneye movements and language processing, readingresearch has made substantial progress (forreview, see Rayner, 1998). In reading, there isgood evidence that readers internally simulatethe events that are described in a text (Barsalou,1999; Bower & Morrow, 1990; Zwaan, 2004;Zwaan & Radvansky, 1998). However, we arenot aware of attempts to use the spatial


FISCHER AND ZWAAN


22/27

characteristics of eye movements to assess spatialaspects of internal action simulations duringreading. The different theoretical views on therelationship between perception and action seemto be differentially suited for this future researchon embodied language processing. Although thetwo-visual-pathways theory seems to impose toostrict a separation between perception and cogni-tion, it might be taken to make some predictionsfor action simulation during language processing.A corollary of the theory is the sharp distinctionbetween egocentric and allocentric coding foraction and perception, respectively, a claim thatis currently being reassessed (Schenk, 2006).Nonetheless, the theory should predict an advan-tage for simulating rst-person over third-personnarratives. To our knowledge this prediction has

not yet been tested. The theory of event coding has been identied asan alternative framework but recent new evidencemay lead to a revision of the extent to whichlanguage-based cognition can be incorporated.Clearly, the mirror neuron theory has already inspired substantial work on language-basedmotor processes, and the results, although oftenmore conicting than converging, provide startingpoints to constrain language-based action simu-lation. Finally, the theory of motor cognition

appears to be the most promising theoretical framework within which to study the link between move-ment-related processes and higher level cognition.

In the light of the studies reviewed above, it isfair to say that the bulk of current evidence forthe role of motor processes can be taken toprovide an afrmative answer to the associationquestion. At the same time, the studies comparingexperts and nonexperts or conspecics and non-conspecics discussed earlier have demonstratedthat motor resonance occurs when the observedactions are part of the observers own motor reper-toire. But does this mean that actions are incom-prehensible to us when we do not have them inour own repertoire? This seems implausible. Atsome level, we do understand the action, presum-ably relying on perceptual processes only, orperhaps also by in addition relying on some very general motor processes that may not be strong

enough to be detected by current neuroimagingand chronometric methods. For example, eventhough we may not show motor resonance when we see or hear a dog barking, we still understand what the dog is doing. Likewise, although ourmotor system may not be active when we observea ballerina perform a jump, we certainly under-stand the action as being part of a performancethat we have chosen to watch, and our motorsystem may resonate at some level to the generalaction o

embodied language - a review of the role of the motor

Documents