ReadingReading in the brain
NeuronalNeuronal mechanisms of a lt l i ticultural invention
Stanislas DehaeneCollège de France
andand INSERM‐CEA
Cognitive Neuroimaging UnitNeuroSpin Center, Saclay, France
www.unicog.org Jean‐Honoré Fragonard, La liseuse (1772)National Gallery of Art, Washington
What is reading?“A conversation with the deceased
(…) [a way to] listen to the dead with my eyes”with my eyes
Francisco de Quevedo
“The art of communicating thoughts to the mind, through the eye ‐‐the great invention of the world ”the great invention of the world.
Abraham Lincoln
“We are absurdly accustomed to the miracle of a few written signs being able to contain immortalbeing able to contain immortal imagery, involutions of thought, new worlds with live people, speaking weeping laughing ”speaking, weeping, laughing.
Vladimir Nabokov, Pale Fire
Writing systems: cultural diversity and biological unityAt first sight, writing systems seemto vary enormously in their forms.
Yet in this talk I will defend the viewthat
- they are all fundamentallyrelated
th ll ll th b i- they all call upon the same braincircuits
this cross cultural universality- this cross-cultural universalitycan be explained:
During reading acquisition we learnDuring reading acquisition, we learnto recycle evolutionarily oldercircuits for visual and gesturerecognition (Neuronal recyclingrecognition (Neuronal recyclingmodel)
The time course of reading
K. Marinkovic, A. Dale, E. Halgren
The time course of reading
K. Marinkovic, A. Dale, E. Halgren
i d i i
The brain architecture for readingLearning to read consists in:
‐ creating an invariant visual representation of written words‐ connecting it to brain areas coding for speech sounds and meaning
Access to pronunciationand articulation
Visual inputs
Visual word form areaAccess to meaning
Perisylvian language areas are alreadyPerisylvian language areas are already activated by spoken language in two‐
month‐old babiesmonth old babies
L1
Adult (for reference)
•Left perisylvian temporal and inferior frontal regions are activated by speech
Two‐month old infant
•Left perisylvian temporal and inferior frontal regions are activated by speech
•Activation unfolds according to a temporal hierarchy.
•A left‐hemispheric asymmetry is present in Planum Temporale for speech but not for music
Dehaene‐Lambertz et al, Science, 2002; PNAS, 2006; Brain & Language, 2009
A left hemispheric asymmetry is present in Planum Temporale, for speech but not for music.
The brain architecture for readingBefore the child learns to read, the major systems for vision and
speech recognition are already in place.An interface must be created between vision and language
Access to pronunciationand articulation
Visual inputs
Visual word form areaAccess to meaning
The visual word form area (VWFA)(VWFA)
• A reproducible site of activation duringreading in all cultures (e.g. Bolger, Perfetti & Schneider, g2005)
• Always located at the same coordinatesin the left lateral occipito-temporal sulcus
• Whose lesion can cause pure alexia, an acquired selective disability in reading (e.g. Déjerine, 1892; Gaillard et al., 2006)
• Which activates to known scripts more than to other categories of visual stimuli (e.g. Baker et al., 2007)
• A high-level visual area, invariant for location and case in word identification (e.g. Dehaene et al., 2001; Cohen et al., 2002)
• An automated system, capable of activating even to subliminal stimuli (e.g. Dehaene et al., 2001, 2004)
The reading paradox:Why do we seem to have a brainWhy do we seem to have a brain
“organ” for reading, given that they was no pressure for it to evolve? p
The visual wordThe visual wordform area(VWFA)
WORDSWORDS
The neuronal recycling hypothesis
The architecture of the human brain is tightly constrained by its past evolution.constrained by its past evolution.
New cultural acquisitions are only possible if they fit within this pre‐existing architecture.
Each cultural objectmust find its neuronal niche ‐‐ a set of circuits that are sufficiently close to the required function and sufficiently q yplastic to be partially “recycled”.
We inherit specialized circuits for vision, auditory processing and spoken languageprocessing, and spoken language.
The invention of writing consisted in recycling these regions so that they could process written signs.
The brain never evolved “for reading”, but writing systems evolved “for the brain”.writing systems evolved for the brain .
Recycling the brain for readingThe ventral visual pathway of all primates contains
evolved circuitry for invariant visual recognition.
Human brain Macaque monkey
After size normalization
Visual recognition Visual recognitionof objects, faces;and written words
of objects and faces
Neurons in this region respond to an alphabet of shape g p p pdescriptors.
We adopted many of these shapes as the foundation of our writing systems.
Jean‐Baptiste Siméon Chardin, Le singe antiquaire
our writing systems.
The properties of this area can explain puzzling features of reading acquisition
Organization of the primate inferotemporal visual pathwayinferotemporal visual pathway
Gross Tanaka Logothetis Poggio Perrett Orban Rolls etcGross, Tanaka, Logothetis, Poggio, Perrett, Orban, Rolls, etc.
From Tamura H & Tanaka K (2001) Cerebral CortexFrom Rolls, Neuron 2000 From Tamura, H., & Tanaka, K. (2001). Cerebral Cortex.,
A putative precursor of the visual word form:The « alphabet » of object features in macaqueThe « alphabet » of object features in macaque
infero‐temporal cortex
A putative precursor of the visual word form:The « alphabet » of object features in macaqueThe « alphabet » of object features in macaque
infero‐temporal cortex
K. Tanaka, 1996
Brincat & Connor, 2004
Origins of the universal « alphabet » of shapes :The importance of non‐accidental properties in visionp p p
Different types of line junctions (T, L, Y…) provide essential information for visual shaperecognition.g
TT
YE
Biederman, Psychological Review, 1987
The topology of strokes in written symbolsobeys a universal statistical distributionobeys a universal statistical distribution
Changizi’s universal distribution
Symboles
Changizi’s 9 most frequent configurations
Changizi & Shimojo (2005)Changizi et al (2006)
A hypothesis for the reproducible localization of the VWFASzwed, Dehaene, et al., Neuroimage 2011
• Fusiform, but not lateral occipital object areas are sensitive to the presence of line junctions rather than line segments.
• This factor together with other factors (excentricity connections to
junctions > segmentsin objects
• This factor, together with other factors (excentricity, connections to language cortex) may explain the reproducible location of the VFWA.
object areas
reading areasz=-16
Lateral Occipital (LO)object areas >
F if
vertex>midsegment in objects
Fusiformobject areas* >
peak of line junctions > segments in words*
Another source of constraints:The left‐hemispheric lateralization of spoken languageThe left hemispheric lateralization of spoken language
- The « letter box » visual word form area varies in its lateralization across individuals. In individuals with unusual- Spoken language lateralization predicts itslateralization.
In individuals with unusualright-hemisphere language, the VWFA is also right-lateralized
Pinel & Dehaene, J. Cognitive Neuroscience (2009)
lateralization map for sentence reading
Colateralization
Cai et al., Cerebral Cortex (2009):
Colateralization--- within reading--- within speech listening
f li t i t di--- from listening to readingtypical
lateralizationatypical
lateralization
Involvement of the VWFA and early visual cortices in readingSzwed, Dehaene, et al., Neuroimage 2011
words - controls
- Once controlling for low-level visual features, early visual areas are more active for words than pictures
Interaction(words – controls)
> VP/V4
2 2
Left hemisphere Right hemisphere
z=-16
(pictures – controls)
0
1
2
0
1
2
pictures - controlsV1/V2
2
-1
2
-1
-
-1
0
1
2
-1
0
1
2
-2 -2
pictures – scrambled pictures
words – scrambled wordsExpertise and retinotopic
t l l i i l
z=-16
pictures scrambled pictures
word gestalts – scrambled words
perceptual learning in early visual cortex?
Directly visualizing the brain changes due to literacyDehaene Pegado Braga Ventura Nunes Filho Jobert Dehaene Lambertz Kolinsky Morais and CohenDehaene, Pegado, Braga, Ventura, Nunes Filho, Jobert, Dehaene‐Lambertz, Kolinsky, Morais and Cohen
How learning to read changes the cortical networks for vision and language. Science, 2010
What is changed in the visual system whenwe learn to read?
• In the VWFA?
• At a lower visual level?• At a lower visual level?
• Elsewhere?
What stimuli activate the VWFA before welearn to read?
Do we lose or do we gain when learning to read? Is there cortical competition?
Juan de Roelas, Saint Ann teaching Virgin Mary to read (1615, Sevilla)
read? Is there cortical competition?
Can the same changes be induced by learning to read in adulthood?
fMRI and high‐density ERP recordings in six groups of subjects
– Illiterates (Brasil only, n=10)
– Ex‐illiterates (Brasil, n=10, Portugal, n=11)Ex illiterates (Brasil, n 10, Portugal, n 11)
– Literates (Brasil: high SES, n=10; low SES, n=10; Portugal, n=12)
Six populations of subjects and their performances
Reading speed is highly variable.Given enough time, all groups (except the illiterates) can read.
The illiterates scanning protocol• Three fMRI experiments:
– Localizer with checkers, spoken and written sentences (Pinel et al. 2007)
10 short sentences 10 subtractions 10 motor instructions 20 Flashing
the river
across
fifteen
from
button
left
Visual Stimuli
10 short sentences 10 subtractions 10 motor instructions 20 Flashingcheckerboards
many bridges
There are
seven
Subtract
on the
Press
time
10 short sentences 10 subtractions 10 motor instructions
– Visual categories (with a minimal task of detecting occasional stars)
Audio. Stimuli “we easily find ataxi in Paris”
“press theleft button”
“subtract nineto eleven”
10 short sentences 10 subtractions 10 motor instructions
– Visual categories (with a minimal task of detecting occasional stars)Faces Houses Tools Strings False fonts Checkers
– Spoken words and pseudowords during lexical decisionp p g• with consistent or inconsistent spelling, testing for the orthographic consistency effect
(Ziegler and Ferrand, 1998)
Increases in activation to written sentences withliteracy
LB1LP
4
Response at [-44, -50, -14]
VWFA
6Response at [-46, -2, 52]
Left frontalz =‐14 The left ventral
visual pathway i itLP
LB2EXB
ILBEXP
1
2
3
0
2
4increases its responses to written words
Checkers Spoken Written Checkers Spoken Written
x =‐48
0
Response at [-54, 26, -6]Response at [24, -84, -10]
Occipital
L R
In language areas,
0
1
2
4
In language areas, written wordsattain the samelevel of activation
Checkers Spoken Written
Response at [-50, 12, -24]
-1
Response at [-50, -44, 6] Response at [-54, -12, -12]Left superior temporal sulcus
0Checkers Spoken Written
level of activation as spoken words.
1
2
4
6
1
2Literates (LB1)Literates (LP)
Ex‐illiterates (EXB)Literates (LB2)
Checkers Spoken Written Checkers Spoken WrittenCheckers Spoken Written
00
2
C S
0Ex‐illiterates (EXP)Ex illiterates (EXB)
Illiterates (ILB)
Directly visualizing the brain changes due to literacy
Literacy enhances the brain activation to written sentences.
Brainactivation
2
3
4
Ex‐illiterates
Literates(schooled in childhood)
0
1
2 (learned to read in adulthood)
Illiterates
0 50 1000
Words read per minute
A massive enhancement in the visual word form area in response to letter strings.
This effect exists even in ex‐illiterates who learned to read in adulthoodThis effect exists even in ex‐illiterates who learned to read in adulthood.No critical period.
What does the visual word form area do prior to reading?Literates (LB1)
li i f h li ff i h l i It responds mostly to faces
(and somewhat less to objects and checkers)2
Literates (LP)
Ex‐illiterates (EXP)Ex‐illiterates (EXB)
Illiterates (ILB)
Literates (LB2)
Replication of the literacy effect with letter strings
z =‐14
objects and checkers)
Responses to faces 1
1.5
2 Illiterates (ILB)
pand checkersdecrease with
literacy.F H T S FF Ck
0
0.5
L R F H T S FF Ck
VWFA activation to :
L R
2 2 2 2 2 2
Faces Houses Tools Letter strings False fonts CheckersVWFA activation to :
0.5
1
1.5
0.5
1
1.5
0.5
1
1.5
0.5
1
1.5
0.5
1
1.5
0.5
1
1.5
0 50 100 1500
0 50 100 1500
0 50 100 1500
0 50 100 1500
0 50 100 1500
0 50 100 1500
Words read per minute
What does the visual word form area do prior to reading?Literates (LB1)
li i f h li ff i h l i It responds mostly to faces
(and somewhat less to objects and checkers)2
Literates (LP)
Ex‐illiterates (EXP)Ex‐illiterates (EXB)
Illiterates (ILB)
Literates (LB2)
Replication of the literacy effect with letter strings
z =‐14
objects and checkers)
Responses to faces 1
1.5
2 Illiterates (ILB)
pand checkersdecrease with
literacy.F H T S FF Ck
0
0.5
L R
In the right anteriorfusiform gyrus, 4
F H T S FF CkL R
responses to faces increasewith literacy.
faces are 2
3
« pushed » towardsthe right hemisphere1
2
F H T S FF Ck
A competition between written words and faces
The peak response to faces is not changed by literacy.
The competition between faces and words occurs at the boundary: as greater space is gained by words, less space is occupied by faces.
Shells centered on peak response to faces – houses
Subject specific peakf f h faces houses
p=0.015
Mary Cassatt, Young girl reading (Smithsonian)
e.g. for faces ‐ house
p=0.037
nsnsns
C t i h ll
shell radius in voxels
peak 2 4 6 8Concentric « shells » around the peak
Literacy mostly has a positive impact on the visual system 10 Response at [24, -84, -10]z =‐10
6
Response at [24, 84, 10]Literates (LB1)Literates (LP)
Ex‐illiterates (EXB)Literates (LB2)
2
4
Responses to
Ex‐illiterates (EXP)Ex illiterates (EXB)
Illiterates (ILB)
L R
pall categories of
stimuli increase in the
Response at [16, -88, 2]Response at [-12, -88, 2] z = 2
occipital cortex.
4
64Even the primaryvisual cortex selectivelyi i
0
2
4
0
2
L R
increases itsresponse to
horizontal shapes( h k )
Horiz Verti0
Horiz Verti(checkers)
Literacy also changes spoken language processing
Activation of Planum Temporale to spoken language is doubled in literates
x =‐38
6Response at [-38, -28, 18]
4
6 LB1LP
2
4z =18 LB2
EXB
0
2
ILBEXP
0Checkers Spoken Written
L R
The main brain changes induced by reading
Regionsinvolved in spokenl
Phonemicrepresentation
Planum temporalelanguage Planum temporale
Visual cortex
l fVisual word form area:The brain’s « letterbox »
How do single neurons tune to reading?
Model of object or face recognitionSize and structureof receptive fields Sample preferred stimuli
Model of word recognition
E
vraimentCOMMENT
En
EE
+- -
Shimon Ullman
Testing the predicted hierarchical organization of the visual word form areathe visual word form area
False fonts Infrequent letters Frequent letters Bigrams Quadrigrams Words
100%
0%
Percent activation relative to words in the occipitotemporal cortexAverage of non-word stimuliVinckier, Cohen , Dehaene et al, Neuron 2007
Bigram coding can explain some puzzles of reading(Grainger & Whitney, 2004)
Aoccdrnig to rseaecrh at Cmabrigde Uinervtisy, it deosn't mttaer in waht oredr the ltteers in a wrod are the olny iprmoatnt tihng iswaht oredr the ltteers in a wrod are, the olny iprmoatnt tihng istaht the frist and lsat ltteer be at the rghit pclae. The rset can be a toatl mses and you can sitll raed it wouthit porbelm. Tihs isy pbcuseae the huamn mnid deos not raed ervey lteter by istlef, but the wrod as a wlohe.
Matt Davis, http://www.mrc‐cbu.cam.ac.uk/~mattd/Cmabrigde/, p // / / g /
• If words are coded by bigrams (ordered pairs of letters)
BADGE =BADGE =
BA + BD + BG + BE + AD + AG + AE + DG + DE + GE
then « BAGDE » and « BADGE » have very similar codes(Perea & Lupker, 2003; Schoonbaert & Grainger, 2004)
• The proportion of shared bigrams predicts similarity (Grainger et al, 2006)
There is no such thing as « whole‐word reading »
A classical debate in education: « whole‐words » or « phonics » ?
In adults, reading is so fast that it seems instantaneous.The time to read a word is independent of the number of letters.These findings give an illusion of whole‐word reading.
ds)
millisecon
d
In reality:The adult brain relies on letters
ng time (m ‐ The adult brain relies on letters,
but they are processed in parallel‐ In young children, reading relies on
i l l tt b l tt d di
Read
in serial, letter‐by‐letter decoding‐ Learning with the whole‐wordmethod ismuch slower and trains
Number of letters in the wordthe wrong brain area in the right hemisphere.
Explaining another puzzle of reading acquisition:mirror reading and writing
Mirror writing in children The reinvention of « boustrophedon »
g g
All children show a « mirror stage » in learning to read
Mirror reading and writing are neverexplicitly taught.
% hild
How can youngchildren be more competent than
100
normalmirror
% children able to write their name
competent thanadults?
50
mirror
0<5 y 5-6 y 6-7 y 7-8 y > 8 y
(Data from Cornell, 1985)
Children’s age
Mirror generalization : the Panda’s thumb of reading?
• We have evolved a symmetrymechanism that helps to recognizefaces and objects regardless of theirfaces and objects regardless of theirorientation
•This « symmetry generalization » isun‐learned when we learn to read.
Infero‐temporal neurons generalize across mirror images
Infero‐temporal neurons of macaque monkeys respond more similarly tomonkeys respond more similarly to horizontal mirror‐image pairs than to vertical mirror‐image pairs (Rollenhagen & Olson, Science, 2000).(Rollenhagen & Olson, Science, 2000).
Following training with a specific random wireframe shape infero temporal neurons in
Preferred view Mirror generalization
‐72°‐108°‐144° ‐36° 0° +36° +72° +108° +144° +180°
Following training with a specific, random wireframe shape, infero‐temporal neurons in the macaque spontaneously generalize to its mirror image (Logothetis, 1995).
Mirror invariance in a face patch in macaquesFreiwald & Tsao Science 2010Freiwald & Tsao, Science 2010
View‐point selectiveselective
Mirror invariance
View‐invariant responseto specific individuals
Mirror invariance ispresent in the majority of neuronsin patch AL.
Mirror invariance in the human brainDehaene, Nakamura et al., NeuroImage, 2009
30
Normal pairs
Mirror
Same
Different
SameX
Priming for words Priming for pictures25
30 Mirror pairs
Same
Different
BUT NOT their mirror images AND their mirror images
15
20
5
10
0
Words Pictures
Replication with simpler images, matched to lettersPegado, Nakamura, Cohen and Dehaene, NeuroImage, 2011
Surprisingly, we learn to readprecisely with the brain area thatbest recognizes mirror pictures.g No wonder that we have trouble learning the letters p, q, b or d
No relation to dyslexia
g
30
Normal pairs
Mirror
Same
Different
SameX
No relation to dyslexia
Priming for words Priming for pictures25
30 Mirror pairs
Same
Different
BUT NOT their mirror images AND their mirror images
15
20
5
10
0
Words Pictures
illiterates perform relatively better in mirror‐image judgements(Pegado et al., in revision)
Letter stringsPictures False fontsAre these images « the same »?
pictures false fonts stringspictures false‐fonts strings
RT Mirror Cost Index
The cost for decidingthat two mirrorimages are the sameincreases with literacy
r2=7.7%;p<0.05
r2=18.1%;p<0.001
r2=23.6%;p<0.0001
Reading score (words and pseudowords read per minute)
The flip side: illiterates have problems in making left‐right distinctionsilliterates have problems in making left right distinctions
Kolinsky et al., Journal of Experimental Psychology: General, 2011
Conclusion: Understanding the « miracle of reading »We can acquire literacy because:We can acquire literacy because:
‐We inherit a structured and efficient visual recognition system, attuned to line junctions and other invariants
‐ This system is sufficient plastic to bepartially reoriented towards thepartially reoriented towards the particular shapes of written words
‐ Generations of scribes selected writtenh h fi b i hishapes that fit our brain architecture.
We now understand that the changes induced by literacy go much beyond the y y g yvisual word form area.
A neural understanding of this system canexplain many of the quirks of this
Pierre‐Auguste Renoir, Portrait de Claude Monet
explain many of the quirks of thiscultural invention (e.g. mirror reading).
It also clarifies how reading should betaught (letters and graphemes, not whole words).
A final word on culture and brain
Can the neuronal recycling model be extended to other domains of human culture? Mathematics, tool use, art, religion, language…, , , g , g g
Brain research leads to new educational toolsBrem et al., PNAS 2010
Lyytinen and colleagues have developed the graphogame, free software that teaches the knowledge of letter-sound correspondencesg p
Kindergarten pre-reader children were trained for less than four hours (spread over 8 weeks) with the graphograme or with a number control gamegraphograme or with a number control game.
fMRI and ERPs revealed the emergence of the visual word form and other brain responses to printp p(words versus false-fonts)