language development and the brain: a phonological perspective usha goswami centre for neuroscience...

Language Development and the Brain:A Phonological Perspective

Usha GoswamiCentre for Neuroscience in EducationUniversity of Cambridge

How does the Brain encode Speech?

-Auditory signal

-Visual dynamics

-Encoding via oscillations?

The Brain and the Speech Signal

Phonetics – ba / da

Syllable structureRhythmProsody

NINE SEVEN TWO THREE TWO

Behavioural Science

Babies use speech rhythm as an early segmentation cue

Can also distinguish e.g. ba / pa, syllable stress

What is happening in the Brain?Envelope and Fine Structure

Smith, Oxenham & Delgutte, 2002: Auditory Chimera

Method for contrasting envelope (slowly varying) and fine structure (rapidly varying) componentsof speech (or other sounds) experimentally

CHIMERA = Envelope cues from one sentenceFine structure cues from another sentence

Which sentence will listeners hear?

Auditory Chimera Sentences

Perception mainly from fine structureNot much from envelope

Perception mainly from envelopeNot much from fine structure

Envelope information mostimportant for speech intelligibility

Early language acquisition:critical role for envelope?

Envelope information mostimportant for speech intelligibility

Emphasises importance of speechrhythm and syllable structure

Envelope: Importance of Rise Times

Spectro-temporal profile (STeP)

Spectrogram + Waveform

Spectro-Temporal Profile (STeP) Spectrogram+Waveform

Greenberg et al. (2003)

Combination of Cues Important“SEVEN”

Amplitude Envelope and Stressed Syllables

[s]

[eh]

[vx]

[en]

juncture accented syllable

unaccented syllable

“Seven”

mean duration

Full-spectrumperspective

Greenberg 2002

[s] [eh] [vx] [en]

Rise times

How Does the Brain Encode Modulation?

Language Representations

Neurons

Networks

Neuroscience

Simple neural codingmechanisms, likeoscillation at different rhythmic rates, appearto be critical.

Delta: 0.5 – 4 HzTheta: 4 – 8 HzGamma: 20 – 50 Hz

Speech signal

Rapid modulationsGamma networks

20 – 50 Hz

Slow modulationsTheta networks

4 – 8 Hz

Binding forspeech perception

The Brain Samples Information in DifferentFrequency Bands (Theta, Gamma ..)

“syllables”“phonemes”

Speech signal

Rapid modulationsGamma networks

20 – 50 Hz

Slow modulationsTheta networks

4 – 8 Hz

Speech percept

Phonetics:ba - pa

RhythmSyllablesMeter

Rise Times Enable Phase Locking?

Hickok, Poeppel

Alignment with“edges” in signal

2 Hz

10 Hz

Hamalainen, Rupp et al. – Phase Locking

Speech signal

Rapid modulationsGamma networks

20 – 50 Hz

Slow modulationsTheta networks

4 – 8 Hz

Speech percept

Phoneticsba - pa

RhythmSyllablesMeter

Early language experiences entrain the oscillators?

Infant-directed speech

Perceptual magnets

Implications for Language Development Prior to Schooling

Entraining the oscillators – activities that emphasise rhythms and metrical structure of speech

- nursery rhymes- poetry- music and singing- other rhythmic experience – dancing

marchingplaying instruments

Link To Phonology:Word Representations in the Brain

semantic representation

phonologicalrepresentation

motorprogramme

Contribution of Brain Imaging

Brain(mental lexicon)

Phonology

Acousticsignal

How does the phonological system develop?

Need to understand how basic auditory processingand learning to talk contribute to the development ofwell-specified phonological representations

Phonological Representation

What are the Brain Representations of Language like Prior to Reading?

semantic representation

phonologicalrepresentation

motorprogramme

The Mental Lexicon

Age 1 year produce 50 – 100 words

Age 6 years produce 6000 wordscomprehend 14,000 words

Developmentally, requires fine-grained representations of similarities and differences in sound

Spoken Language Processing

Lexicon

Acoustics

PhoneticsPhonology

Spoken Language Processing

Lexicon/Reading

Acoustics

PhoneticsPhonology

Bottom-Up

Spoken Language Processing

Lexicon

Acoustics

PhoneticsPhonology

Top-down

The Development of Phonological Awareness

Children gradually become “aware” of soundunits within words as they develop theirlanguage skills

A universal developmental sequence is found acrosslanguages

Levels of Phonological Awareness

Syllable (butterfly, wigwam, soap)

Rhyme (onset-rime: str - eet, fl - eet)

Phoneme (smallest units of sound that change meaning: cat-pat, cat-cot, cat-cap

but pin … spoon)

Basic Unit of Speech Processing – The Syllable

g r

gr

a s p

asp

graspsyllable

onset rime

phonemes

(develop largely via reading)

The Development of Phonological Awareness

Oddity Task: Bradley & Bryant, 1983

Alliteration: hill, pig, pin

Rhyme 1: cot, hat, pot

Rhyme 2: sit, pin, win

Oddity Task: English, German, Chinese

0

20

40

60

80

English German Chinese

% c

orr

ec

t

onset rime

Oddity Task: Dyslexic Children

(Bradley & Bryant, 1978)

0

20

40

60

80

100

Allit Rhyme1 Rhyme2

%c

orr

ec

t

Dys RL

Gains in Reading and Spelling after 2 yrs,Adjusting for Age and I.Q.

60

80

100

Read Acc ReadComp

SpellAd

just

ed s

tan

d s

core

in m

on

ths Train Ctrl1 Ctrl2

Predicting Reading Acquisition

BrainLanguage

Reading

Phonology

Language playNursery rhymes

“AuditoryOrganization”

Phonological& orthographicconnections

Measuring Syllable and Phoneme Awareness

e.g., tapping out sounds using a drumstickcounting out sounds using plastic tokens

Syllable level

popsicle 3 tapsbutter 2 tapssoap 1 tap

Phoneme level

book 3 tapsup 2 tapsI 1 tap

Counting Tasks: Syllable vs Phoneme

0

20

40

60

80

100

Nor Ger Fre Tur Gr

% c

orr

ec

t

phoneme syllable

Rate of Phonemic Development

Languages vary in the degree to which letters have a 1:1mapping to sound

GreekFinnishGermanItalianSpanish

SwedishDutchIcelandicNorwegian

FrenchPortugueseDanish

English

Phoneme counting at end of Grade 1

% correct

Greek (Porpodas) 100Italian (Cossu et al.) 97Turkish (Durgunoglu & Oney) 94German (Wimmer et al.) 92Norwegian (Hoien et al.) 83French (Demont & Gombert) 61English (Perfetti et al.: Grade 2) 65

Predicting Reading AcquisitionAcross Languages

BrainLanguage

Reading

Phonology SyllablesRhymesPhonemes

The Development of Phonological Representations

Infancy

- physical changes in signal where languages placephonetic boundaries

- statistical learning of prototypes and phonotactics

- use of prosodic cues to segment syllables and words

Within first year have proto-lexical representationsthat encode stress and segmental information

The Development of Phonological Representations

Early Childhood

- enormous increase in vocabulary

- pressure for segmental specificity

- syllable structure varies across languages

- phonological “neighbourhood density” variesacross languages

PHONOLOGICALREPRESENTATIONS

OF WORDS

Vocabulary size andrate of expansion

Speech processingskills (input and

output)Rhythmic cues Phonological

neighbourhooddensity

(language specific)

Linguistic factorseg, sonority profile(language specific)

Word frequency/familiarity/

ageof acquisition

Speech-readingeg, lip shape

Syllable Structure(languagespecific)

How Reading Changes the Brain

PHONOLOGICALREPRESENTATIONS

OF WORDSchair tray tip

Vocabulary size andrate of expansion

Speech processingskills (input and

output)Rhythmic cues? Phonological

neighbourhooddensity

(language specific)

Linguistic factorseg, sonority profile(language specific)

Word frequency/familiarity/

ageof acquisition

Speech-readingeg, lip shape

Reading and spelling acquisition

(phoneme level)

Syllablestructure(language Specific)

Development of Phonological Awareness: Language Universal?

Preschool: large units

syllablesrhymes

With schooling: small units

phonemes

English Monosyllables: Approx. 4000

CV sea 5%

CVC cat 43%

CVCC desk 21%

CCVC trip 15%

CCVCC grasp 6%

German: 1400 French: 2500

Representing Phonemes Varies Across Languages

GreekFinnishGermanItalianSpanish

SwedishDutchIcelandicNorwegian

FrenchPortugueseDanish

English

Languages vary in the degree to which letters have a 1:1 mapping to sound, and this affects how rapidly phonology gets “re-mapped” in the brain

EU Study: Word and Nonword Reading (cvc)

0

20

40

60

80

100

Gk Fin Ger It Sp Sw Du Ic N Fr Po Da Eng

% c

orr

ect

NW "vuf" Word "cup"

Rates of development largely explained by spelling transparency across languages

E.g., ‘a’

German English

Hand hand

Ball ball

Garten garden

Rates of Development across Languages

- development of phonemic representation slower in less consistent orthographies

- development of grapheme-phoneme recoding skills(“sine qua non” of reading acquisition) alsoslower in less consistent orthographies

Educational Neuroscience

Importance of understanding how the brain encodes information

Importance of understanding how environment shapes the basic encoding

Focus on causal mechanisms of development

Deeper understanding of development

Implications for Language Development Prior to Schooling

Need to hear as much language as possible

Importance of “rich” language

Rich language enhanced in story reading interactions

Entraining the oscillators – activities that emphasise rhythms and metrical structure of speech

- nursery rhymes- poetry- music and singing