grasping ideas with the motor system: semantic somatotopy in idiom comprehension

20
Grasping ideas with the motor system: Semantic somatotopy in idiom comprehension Véronique Boulenger MRC Cognitive and Brain Sciences Unit, Cambridge (UK) L2C2 Institut des Sciences Cognitives CNRS, Lyon (France)

Upload: nickan

Post on 20-Mar-2016

59 views

Category:

Documents


0 download

DESCRIPTION

Grasping ideas with the motor system: Semantic somatotopy in idiom comprehension. Véronique Boulenger. MRC Cognitive and Brain Sciences Unit, Cambridge (UK) L2C2 Institut des Sciences Cognitives CNRS, Lyon (France). Embodied Cognition. - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Grasping ideas with the motor system: Semantic somatotopy in idiom comprehension

Grasping ideas with the motor system: Semantic

somatotopy in idiom comprehension

Véronique Boulenger

MRC Cognitive and Brain Sciences Unit, Cambridge (UK)

L2C2 Institut des Sciences Cognitives CNRS, Lyon (France)

Page 2: Grasping ideas with the motor system: Semantic somatotopy in idiom comprehension

Semantic mechanisms draw upon and are grounded in action-perception systems of the brain (Barsalou, 2007; Glenberg, 2007; Martin, 2007; Pulvermüller, 2005)

Embodied Cognition

Page 3: Grasping ideas with the motor system: Semantic somatotopy in idiom comprehension

Tettamanti et al. (2005)

Somatotopic activation during processing of action-related words

and sentences

Leg

Arm

Mouth

Hauk et al. (2004)

Page 4: Grasping ideas with the motor system: Semantic somatotopy in idiom comprehension

But…

Only simple referential meaning of words and sentences has been examined.

Does semantic somatotopy persist during processing of abstract/idiomatic sentence meaning?

Page 5: Grasping ideas with the motor system: Semantic somatotopy in idiom comprehension

Predictions

If the grounding of semantics in the sensory-motor system is a universal feature of the human cognitive system:action-perception information should play a role in determining the landscape of semantic brain activation to sentences, even if their meaning is highly abstract.

Page 6: Grasping ideas with the motor system: Semantic somatotopy in idiom comprehension

Our study

Pattern of fMRI activation during reading of idioms that include arm- and leg-related action verbs « He grasped the idea » « He kicked the habit »

Page 7: Grasping ideas with the motor system: Semantic somatotopy in idiom comprehension

MethodsParticipants18 healthy right-handed English native speakers

Stimuli76 English idiomatic expressions76 literal sentences including arm- or leg-related action verbs

« John grasped the IDEA » vs. « John grasped the OBJECT ».« Mary kicked the HABIT » vs. « Mary kicked the BALL».“Critical word”Action word

Page 8: Grasping ideas with the motor system: Semantic somatotopy in idiom comprehension

Stimuli

Critical words were matched for relevant psycholinguistic variables.

Arm- and leg-related action words were matched for the same variables.

76 baseline stimuli (## #### ##)

6 probe sentences (e.g. « He opened the door »)

Methods

Page 9: Grasping ideas with the motor system: Semantic somatotopy in idiom comprehension

Experimental procedure

Silent reading task + Answer to simple yes/no questions about probe sentences (e.g. « Did he open the door? ») with left fingers - measure of error rates

Sentences were presented word by word (500 ms)

Motor Localizer task perform small movements of left and right index fingers and feet (4 times for 20 s each time)

Methods

Page 10: Grasping ideas with the motor system: Semantic somatotopy in idiom comprehension

fMRI data analysis

Two time-windows for analysis: Early: HRF response time-locked to the onset of critical words Late: HRF response time-locked to a point delayed by 3s after critical words

Metabolic activity related to semantic integration at the sentence level was found to be maximal at about 6-8 s after sentence completion (Humphries et al., 2007) or even later (Simmons et al., in press).

Methods

Page 11: Grasping ideas with the motor system: Semantic somatotopy in idiom comprehension

Four-way ANOVA : Time-Window (early vs. late) x Idiomaticity (idiomatic vs. literal) x ROI (hand area vs. foot area) x Semantic Reference of verbs (arm vs. leg)

2 left-lateralized ROIs from the « motor localizer task » « hand motor area » « foot motor area »

MethodsfMRI data analysis (1)

Page 12: Grasping ideas with the motor system: Semantic somatotopy in idiom comprehension

2x9 ROIs along the motorstrip (central sulcus and precentral gyrus; 25 ≤ z ≤ 76 mm)

Four-way ANOVA: Time-Window (early vs. late) x Dorsality (dorsal vs. lateral regions) x Idiomaticity (idiomatic vs. literal) x Semantic Reference of verbs (arm vs. leg)

MethodsfMRI data analysis (2)

Page 13: Grasping ideas with the motor system: Semantic somatotopy in idiom comprehension

Results

Behavioral Results

Error rates for probe sentences = 8.3 % (SE 2.45)

Participants paid attention to the task

Page 14: Grasping ideas with the motor system: Semantic somatotopy in idiom comprehension

fMRI Results: « motor localizer »

EARLY ANALYSIS WINDOW

Arm Idiomatic Leg Idiomatic Arm Idiomatic Leg Idiomatic

LATE ANALYSIS WINDOWEARLY ANALYSIS WINDOW

Arm Idiomatic Leg Idiomatic Arm Idiomatic Leg Idiomatic

LATE ANALYSIS WINDOW

Time-Window x ROI x Reference interaction (p = .013)ROI x Reference interaction (p = .014) in the late analysis window

Results

Page 15: Grasping ideas with the motor system: Semantic somatotopy in idiom comprehension

fMRI Results: « motor localizer »Idiomatic sentences including leg-related action words activated more strongly the left foot motor area vs.Idioms including arm-related action words activated more strongly the left hand motor area

Results

Page 16: Grasping ideas with the motor system: Semantic somatotopy in idiom comprehension

fMRI Results: Motorstrip

Time-Window x Dorsality x Reference interaction (p < .001)

Cortical activity along the motorstrip was modulated by semantic reference of the verbs differently for the 2 time-windows

Dorsality x Reference interaction (p <.001) in the late analysis window

Results

Page 17: Grasping ideas with the motor system: Semantic somatotopy in idiom comprehension

Dorsal regions ( z ~ 75 mm): cortical activity was stonger for sentences including leg-related action verbs vs.Lateral regions (25 ≤ z ≤ 50 mm): stronger activty was observed for arm-related sentences

Arm-relatedLeg-related

-5, 75-15, 76-25, 73-33, 68

-42, 60

-51, 52

-54, 44

0

0.2

0.4

0.6

0.8

1 -58, 35

-59, 25

Arm-relatedLeg-relatedArm-relatedLeg-related

-5, 75-15, 76-25, 73-33, 68

-42, 60

-51, 52

-54, 44

0

0.2

0.4

0.6

0.8

1 -58, 35

-59, 25

Results

fMRI Results: Motorstrip

Page 18: Grasping ideas with the motor system: Semantic somatotopy in idiom comprehension

Discussion

Silent reading of idioms that include action words recruits the motor cortex somatotopically Semantic somatotopy was evident in the late analysis window, where sentence level meaning is computed Our results support theories that view abstract semantics as grounded in action-perception systems (Barsalou, 2007; Pulvermüller, 2005) Time-course of motor activity during idiom processing using MEG

Page 19: Grasping ideas with the motor system: Semantic somatotopy in idiom comprehension

The orchestration of abstract meaning is not solely explained by the activation of unspecific semantic centers in fronto-temporal cortex, but it involves late complementary activations in the motor system. These referentially grounded activations may play a specific role in the composition of sentence meaning.

Conclusion

Page 20: Grasping ideas with the motor system: Semantic somatotopy in idiom comprehension

Thank you