l bildbenennung l wortgenerierung(z.b. nennen sie möglichst viele tiere!) l wortlesen (hund) l...

Bildbenennung

Wortgenerierung (z.B. Nennen Sie möglichst viele Tiere!)

Wortlesen (HUND)

Pseudowortlesen (HUNG)

Analyse von 82 Hirnaktivierungsxperimenten mit vier verschiedenenWortproduktionsaufgaben:

Talairach & Tournoux (1988) Lateral and medial view of reference brain

Reported at least onceReported at least once

Estimate of probability of overlap under the assumption of a random distribution of activated regions

number of regions: 110

mean number of activated regions: r

chance probability for a region to be reportedas activated in a single experiment (p1): r/110

chance probability for a region to be reported as activated in n1 out of n experiments:

21 n1

n1

21)p(1p!n!n

n!p (with n1 + n2 = n)

Reliability criterion: p < 0.1 cut-off point in binomial distribution

Example region 1

Number of experiments: 82

Mean number of reported regions: 12.4

Reliably activated: 12 or more experiments

Reliably not activated: 4 or less experiments

Example region 2

Number of experiments: 23

Mean number of reported regions: 10.4

Reliably activated: 5 or more experiments

Reliably not activated: -

Zuverlässig aktivierte (rot) und nicht aktivierte (blau) Hirngebiete (basierend auf allen 82 Studien)

TASK ANALYSIS

Many tasks were not just word production tasks; they involved other operations as well.

For instance, when you name the picture of a horse, you not only produce the word 'horse', but you also look at the picture and recognize it. Such additional 'lead-in' operations involve the activation of additional brain regions. These should be filtered out.

That requires a systematic task analysis, a distinction between 'lead-in' and 'core' operations of word production.

Responses during Verb Generation Task

BANANATROUSERSCHAIRGLASSESTRUMPETPENCILBUTTONBIRDEARDOOR

peel, slip on, eat up, plantput on, wash, mend, buy, warmsit, build, nail, sell, work, learnclean, put on, step on, buy, seeblow, make music, put away, hear, playsharpen, break, put away, drawtear off, close, openfly, eat up, singhear, pinchopen, close, kick against

Konzeptuelle Vorbereitung

lexikalische Selektion

lexikalisches Konzept

Lemma

Wortformzugriff

Wortform

Syllabifizierung

phonologisches Wort

phonetische Enkodierung

abstraktes Motorprogramm

Artikulation

gesprochenes Wort

visuelle Objekt-erkennung

Einleitungsprozesse KernprozesseAufgabe

Worterkennung Objektvorstellung Gedächtnis etc.

visuelle Worterkennung

Graphem/PhonemKonversion

Bildbenennung

Wortgenerierung

Wortlesen

Pseudowortlesen

Selb

stmo

nito

ring

aussprechen vs. Wort “denken”

Bildbenennung

Wortgenerierung

Bildbenennung (grün), Wortgenerierung (blau), gemeinsame Gebiete (rot)

Gemeinsame Aktivierungsgebiete von Bildbenennung und Wortgenerierung




Lemma

Wortformzugriff

Wortform

Syllabifizierung

phonologisches Wort



Artikulation

gesprochenes Wort






Bildbenennung

Wortgenerierung

Wortlesen

Pseudowortlesen

Selb

stmo

nito

ring


Gemeinsame Aktivierungsgebiete von Bildbenennung, Wortgenerierung und Wortlesen




Lemma

Wortformzugriff

Wortform

Syllabifizierung

phonologisches Wort



Artikulation

gesprochenes Wort






Bildbenennung

Wortgenerierung

Wortlesen

Pseudowortlesen

Selb

stmo

nito

ring


Gemeinsame Aktivierungsgebiete aller Aufgaben

Aussprechen im Vergleich zu Wort “denken”

Schematische Darstellung des Ergebnisses der Meta-Analyse von 82 Hirnaktivierungsstudien

Indefrey, P. and Levelt, W.J.M. (2004) Cognition

Picture naming

Study Hernandez Hernandez De Bleser Vingerhoets Rodriguez-FornellsYear 2000 2001 2003 (non-cognates) 2003 2005Journal BrainLang Neuroimage Neuroimage Neuroimage JOCNNo. Subj. 6 6 11 12 11Method fMRI fMRI PET fMRI fMRIactive condition cov picture naming cov picture naming cov picture naming cov picture naming onsetdec on picturescontrol condition rest rest fixation scrambled pictures fixationadditional tasksL1 Spa Spa Fle Dut SpaL2 Eng Eng Fre Fre/Eng GerL2 onset <5 <5 10 10.3/13.5 3L2 duration 18 >16 8-11 17/14 >20L2 proficiency high high high good balancedL2 use L2 dominant L2 dominant n.g low/high L2 dominant

found in L2 onset

L2 proficiency L2 exposure

De Bleser 2003 10 good – very good ?

Vingerhoets 2003 10-14 mixed low/high

not found in

Rodrigues- Fornells 2005

3 balanced dominant

Hernandez 2001 <5 high dominant

Hernandez 2000 <5 high dominant

mean no. areas: 1.4

Picture naming

Stronger activation in L2 as compared to L1

Lucas II et al., J Neurosurgery 2004, Fig. 4

L2

L1

shared

Conclusions

L1 and L2 word production recruit the same set of areas (but only at the group level).

No areas are more strongly recruited in L1.

The left posterior inferior frontal gyrus may be recruited more strongly in L2 speakers with late L2 onset and/or low proficiency.

This region contains L1 specific but no L2 specific sites that are necessary for word production.

This region is the most likely candidate for post-lexical phonological encoding (syllabification) in L1 word production.

Left posterior IFG optimized for L1 phonotactic constraints and less efficient for early L2?

The cognitive architecture of listening to language

speech signal

interpretation

decoding

segmenting

speech code

phonemes, syllables

phonological processing

word recognition

syntactic analysis

thematic analysis

integration with other knowledge sources

Reversed speech versus silence

Word lists versus silence

Study Stimulus #

Belin 1998 200ms frequency transition, 60/min 1

Belin 1998 40ms frequency transition, 60/min 2

Belin 1999 synthetic diphthong, 6/min 3

Binder 2000 tones, different frequencies, 90/min 4

Bookheimer 1998 pseudowords, 9/min 5

Celsis 1999 syllables, 180/min 6

Celsis 1999 tones, 500 + 700Hz, 180/min 7

di Salle 2001 tones, 1000Hz, 6/min 8

Engelien 1995 environmental sounds, 10/min 9

Fiez 1996 pseudowords, 60/min 10

Fiez 1996 words, 60/min 11

Giraud 2000 vowels vs. expecting vowels, 120/min 12

Holcomb 1998 tones, 1500Hz + lower tones, 30/min 13

Jäncke 1999 tones, 1000Hz, 60/min 14

Lockwood 1999 tones, 500 + 4000Hz, 60/min 15

Mellet 1996 words, 30/min 16

Mirz 1999 music 17

Mirz 1999 sentences 18

Study Stimulus #

Mirz 1999 tones, 1000Hz 19

Mirz 1999 tones, 1000 + 4000Hz 20

Mirz 1999 words 21

Müller 1997 sentences, 12/min 22

Petersen 1988 words, 60/min 23

Price 1996 words, 40/min 24

Price 1996 words, different rates 25

Suzuki 2002a words, 60/min 26

Suzuki 2002b tones, 1000Hz, 60/min 27

Thivard 2000tones with spectral maxima, 60/min

28

Warburton 1996 words, 4/min 29

Wise 1991 pseudowords, 40 or 60/min 30

Wong 1999 reversed sentences, 30/min 31

Wong 1999 sentences, 30/min 32

Wong 1999 words, 30/min 33

Wong 2002 reversed words, 15/min 34

Wong 2002 sentences, 12/min 35

Wong 2002 words, 15/min 36

Indefrey & Cutler, 2004

Studies comparing auditory stimuli to silent baseline conditions

Study Stimulus vs. control stimulus #

Benson 2001 CVC > CV > V 1

Binder 1996 words vs. tones 2

Binder 2000 pseudo vs. tones 3

Binder 2000 reversed words vs. tones 4

Binder 2000 words vs. tones 5

Giraud 2000 amplitude modulated noise vs. noise 6

Giraud 2000 sentences vs. vowels 7

Giraud 2000 words vs. vowels 8

Hall 2002 frequency modulated vs. static tone 9

Hall 2002 harmonic vs. single tone 10

Jäncke 2002 syllables vs. 350 ms white noise bursts 11

Jäncke 2002 syllables vs. steady state portion of vowel 12

Jäncke 2002 syllables vs. tones 13

Müller 2002 90% 1000Hz + 10% 500Hz vs. 1000Hz 14

Mummery 1999 words vs. signal correlated noise 15

Price 1996 words vs. reversed words 16

Schlosser 1998 sentences vs. unknown language 17

Scott 2000 sentences vs. rotated sentences 18

Thivard 2000 frequency transition vs. stationary tone 19

Indefrey & Cutler, 2004

Studies comparing auditory stimuli to simpler auditory stimuli

Silent control

Listening to speech without an additional task induces extensive bilateral temporal activation but no reliable activation of Broca’s area.

Summary

With increasing linguistic complexity of stimuli, the distance of activation maxima from the primary auditory cortex increases; particularly in the left hemisphere.

It seems to be the highest linguistic processing level that leads to the most significant activation difference compared to a silent control.

Summary

The left hemisphere shows a clearer stimulus-specific differentiation of activation maxima.

Areas that seem to be especially related to (post-) lexical and sentence level processing can be identified.

Summary

bilateral posterior STG: phonology

left posterior STS: lexical phonology

left anterior STS: possibly lexical and sentential prosody, possibly lexical and sentential meaning

Summary

Neuroimaging studies on syntactic processing

Approach A: Syntax versus no syntax

The cat is chasing the mouse. (Rest)The cat is chasing the mouse. &%$#@ &%#$@ %2#The cat is chasing the mouse. dgjrt hgjtrdf frt fpg hgrfbdwThe cat is chasing the mouse. mouse the chasing cat is the

ACTIVATION CONTROL

Advantage: Syntactic parsing not subtracted out

Disadvantage: Activated areas may be related to

nonsyntactic processing components


Approach B: More syntax versus less syntax

The cat are chasing the mouse.

The cat is chasing the mouse.


The mouse is chased by the cat.


The cat is chasing the rat.

The mouse that the cat chased stole the cheese.

The cat chased the mouse that stole the cheese.

ACTIVATION CONTROL

Advantage: Nonsyntactic processing components well controlled

Disadvantage: Syntactic parsing (in part) subtracted out


Indefrey (2004)

Talairach & Tournoux (1988) Lateral and medial view of reference brain

How many studies must agreeon a certain area?

number of regions = 110

mean number of activated regions per experiment = 5.1

chance probability for a region to be reported as activated in a single experiment: p1 = 5.1 / 110 = 0.046

chance probability for a region to be reported as activated in n1 out of n experiments:21 n

1n

121

)p(1p!n!n

n!p (with n1 + n2 = n)

Sentence processing studies:Summary

Indefrey (2004)

Sentences vs. control below sentence level

Syntactically more vs. less demanding sentences

Semantically more vs. less demanding sentences

53 57 51Reading Listening Reading Listening Reading Listening

23 33 42 18 37 15Note: some studies reported sentence reading and listening data

Hagoort & Indefrey (2014)

Interim summary

Compared to low-level control conditions, sentence processing activates left posterior inferior frontal (BA 44, 45, 47) and left temporal cortex

Sentence listening activates bilateral temporal cortices

For passive sentence listening or word-level control conditions BA 44 (pars opercularis) is no longer reliably found

> understanding simple sentences may not involve (detectable) syntactic processing

wegstossen-Animation(1)

wegstossen-Animation(2)

Condition1: Sentences

Der rote Kreis stößt die grüne Ellipse weg.

(The red circle pushes the green ellipse away.)

Condition 2: Noun phrases

roter Kreis, grüne Ellipse, wegstoßen

(red circle, green ellipse, push away)

Condition 3: Single words

Kreis, rot, Ellipse, grün, wegstoßen

(circle, red, ellipse, green, push away)

All conditions at slow (6/min) and fast (8/min) rate.

Sentences vs. Single Words

Activation maximum at -60,14,12

Indefrey et al. (2004) Brain & Language

Activation maximum at -54,6,10

Indefrey et al. (2001) PNAS

S and NP production vs. control (W)

Indefrey, Hellwig, Herzog, Seitz & Hagoort (2004) Brain & Language

Sentences vs. control below sentence level

Syntactically more vs. less demanding sentences

Semantically more vs. less demanding sentences

53 57 51Reading Listening Reading Listening Reading Listening

23 33 42 18 37 15Note: some studies reported sentence reading and listening data

Hagoort & Indefrey (2014)

Syntactically demanding Violation Inflection

The test is being explain/explained* Grammatical category The dance is being not too seriously rehearsal/rehearsed* (Cooke et al., 2006)

Ambiguity He noticed that landing planes frightens some new pilots. (Rodd et al., 2010)

Complexity Relative clauses: The reporter who the senator attacked admitted the error. The reporter who attacked the senator admitted the error. (Just et al. , 1996) Non-canonicity: The red book John gave to the professor from Oxford. John gave the red book to the professor from Oxford. (Ben-Shachar et al., 2004)

Semantically demanding Violation Selection restrictions:

Dutch trains are sour. World knowledge: Dutch trains are white. (Hagoort et al. 2004)

Ambiguity The reporter commented that modern compounds react unpredictably. (Rodd et al., 2010)

Complexity Metaphor: A sailboat is a floating leaf. (Diaz & Hogstrom, 2011) Metonymy, coercion, causal relationships Africa is hungry/arid. (Rapp et al., 2011) The novelist began/wrote the book. (Husband et al., 2011) The boys were having an argument. They became more and more angry./They began hitting each other. The next day they had bruises. (Kuperberg et al., 2006) Indirect question/reply, Irony: Did you like my presentation?/ How hard is it to give a good presentation? It is hard to give a good presentation. (Bašnáková et al., 2013) Ann promised to keep her party dress clean. She came home covered in mud. Her mom said:”Thanks for staying so clean.” (Eviatar & Just, 2006)

Resting state connectivity patterns reported by Xiang et al. (2010, Cerebral Cortex)

Summary

Both syntactic and semantic compositional processing recruit frontal and temporal regions

In both frontal and temporal cortex there is a gradient with syntactic processes activating more dorsal regions and semantic processes activating more ventral regions.

this pattern speaks against reducing syntactic processing to some aspect of semantic processing

Frontal and temporal cortex activations dissociate for violations this finding supports a division of labor with frontal cortex subserving

compositional processing as such and temporal cortex having a role in the retrieval of lexically stored semantic and syntactic information

Understanding non-literal meaning, in particular ‚speaker meaning‘ requires the recruitment of additional regions, such as the medial prefrontal cortex, supporting non-linguistic Theory-of-Mind processing.

Sentence comprehension

Study Chee Hasegawa Luke Frenck-Mestre RueschemeyerYear 1999 (exp1) 2002 2002 2005 2005 (exp2, corr sent)Journal Neuron Neuroimage HBM Neuroreport HBMNo. Subj. 15 10 7 6 18/14Method fMRI fMRI fMRI fMRI fMRIactive condition sentence reading sentence listening VP reading overt sentence readingsentence listeningcontrol condition fixation fixation fontsizedec consonant strings restadditional tasks comprehension probe verification synt or sem dec judgmentL1 Chi/Eng Jap Chi Eng Ger/RusL2 Chi/Eng Eng Eng Fre GerL2 onset <6 12 >10 >12 n.g.L2 duration 13-20 14 10-21 >15 5L2 proficiency high high high(selfrating) high n.g.L2 use daily high(in L2 env.) moderate? high (in L2 env.) high (in L2 env.)

Narrative comprehension

Study Perani Perani Perani Nakai Nakada VingerhoetsYear 1996 1998 1998 1999 2001 2003Journal Neuroreport Brain Brain Neurosc letters Neurosc Research NeuroimageNo. Subj. 9 9 12 4 10 12Method PET PET PET fMRI fMRI fMRIactive condition story listening story listening story listening story listening paragraph reading story readingcontrol condition reversed speech reversed speech reversed speech rest false fonts pseudoword listsadditional tasksL1 Ita Ita Spa/Cat Jap Jap/Eng DutL2 Eng Eng Spa/Cat Eng Eng/Jap Fre/EngL2 onset 7 10 2 n.g. 10 10.3/13.5L2 duration 14-25 9-40 17-25 n.g. >10 17/14L2 proficiency moderate high high n.g. high goodL2 use low daily daily n.g. n.g. low/high

found in L2 onset

L2 proficiency

L2 exposure

Hasegawa 2002 12 high high

Luke 2002 >10 high ?

not found in

Nakai 1999 ? ? ?

Perani 1998 10 high high


Perani 1996 7 moderate low


Nakada 2001 >10 high ?

Rüschemeyer 2005 ? ? high

Chee 1999 <6 high high

Frenck-Mestre 2005 >12 high highmean no. areas: 2.4

Sentence listening / reading


found in L2 onset

L2 proficiency

L2 exposure

Nakai 1999 ? ? ?



not found in











found in L2 onset

L2 proficiency

L2 exposure

Nakai 1999 ? ? ?


not found in











found in L2 onset

L2 proficiency

L2 exposure



not found in

Nakai 1999 ? ? ?










found in L2 onset

L2 proficiency

L2 exposure

Nakai 1999 ? ? ?


not found in








Frenck-Mestre 2005 >12 high high

mean no. areas: 2.4



found in L2 onset

L2 proficiency

L2 exposure



not found in

Nakai 1999 ? ? ?











Conclusions

L1 and L2 sentence level comprehension recruit the same set of areas.

Within this set of areas, there are to date no reliable activation level differences between L1 and L2 narrative comprehension.

Stronger L2 activation of the left posterior IFG may be found for speakers with late L2 onset when additional judgment tasks are used.

The left posterior IFG is the most likely candidate area for syntactic processing.

Left posterior IFG optimized for L1 syntax and less efficient for L2?

(taking into account word production data) Left posterior IFG optimized for L1 compositional processes and less efficient for L2?

L2 proficiency

minutes weeks years

Perani et al. 1996, 1998

Wartenburger et al. 2003

L2 proficiency

minutes weeks years

Longitudinal study

in NL Lessons started to learn NL

TS April 03 school (6 hours/week) Febr 04

CX Jan 04 school (6 hours/day) Jan 04

ZY Jan 04 school (6 hours/day) Jan 04

HQ Jan 03 self study Jan 04

CJ April 02 school (6 hours/week) Jan 04

JX March 03 with a colleague (3 hours/week) Febr 04

Longitudinal study: Participants

Longitudinal Study: Methods

Test battery at 3, 6, 9, 15, 18, and 24 months

Behavioural testing ‘nonverbal’ intelligence test (Raven Progressive Matrices) handedness test (only at 0 months) standard Dutch proficiency test language questionnaire syntactic judgment test

fMRI experiment on syntactic processing

ERP experiment on semantic and syntactic violations

S: Het blauwe vierkant wordt door de gele cirkel weggestoten. (correct)Lan2 fang1kuai4 bei4 huang2 yuan2quan1 tui1zou3.The blue square is pushed away by the yellow circle.

De gele cirkel wordt door het blauwe vierkant weggestoten. (incorrect)Huang2 yuan2quan1 bei4 lan2 fang1kuai4 tui1zou3 The yellow circle is pushed away by the blue square away

W: (correct)

(incorrect)

vierkantfang1kuai4square

cirkelyuan2quan1circle

blauwlan2se4blue

geelhuang2se4yellow



geelhuang2se4yellow

blauwlan2se4blue

wegstotentui1zou3push away

wegstotentui1zou3push away

Indefrey, Hellwig, Davidson, & Gullberg 2005

Dutch sentences versus words (Dutch listeners)

Chinese sentences versus words (Chinese listeners)

Dutch sentences versus words (Chinese listeners, 3 months)

Ducth sentences versus words (Chinese listeners, 9 months)

Conclusions

In L2-comprehension, both frontal and temporal areas show syntactic-processing related activation similar to L1 already after a few months of exposure.

The emergence of this activation precedes the ability to detect number or gender violations, but follows the ability to detect at least some types of word order violations.

So far, there does not seem to be a correlation between behavioral data and the enhanced hemodynamic response to sentences compared to word lists.

l bildbenennung l wortgenerierung(z.b. nennen sie möglichst viele tiere!) l wortlesen (hund) l...

Documents

bildbenennung slide

studien slide

wort denken slide

number of experiments

n experiments

gemeinsame gebiete rot

experiments example

wortgenerierung blau