An optical imaging study on language recognition in the first year of life

Susan Hespos

Northwestern University

Developmental Cognitive Neuroscience

• Many neuroimaging methods can be applied to the developing human brain

• How does this method contribute to knowledge of language acquisition?

• Where and when particular patterns of neural activity occur

Why do imaging on infants?

• We can look at continuity and change over time

• Is it the same behavior outcome and different underlying mechanisms?

• Are there different behavior outcomes and the same underlying mechanism?

• Rich data, low task demands, holding the task constant across ages

Infants

Adults

BehavioralResearch

ImagingResearch

Phonetic contrastsStatistical learningLanguage-specific perception & production

Bilingual activation Phonetic contrastsSentence comprehension

Phonetic contrastsStatistical learningLanguage-specific perception & production

From Kuhl 2004 Nature Reviews Neuroscience

Near Infrared Spectroscopy (NIRS)

• Based on pulse oximetry

• Measurement of temporal changes in both oxyhemoglobin and deoxyhemoglobin

Principles of NIRS

About NIRS

• Pros– Pulse ox technology

is used widely– No injections– Silent – Minimal restraint – Records oxy and

deoxy– Portable

• Cons– Measures surface

cortical only– Not many users yet– Analyses techniques

vary

Key research secrets: chin straps, bubbles, ace bandage

Previous research using NIRS on infants

• Baird et al. (2002)– Longitudinal 5 to 12 month olds– Piagetian search tasks– Significantly more frontal activity after success

• Taga et al. (2003)– 2 to 4 month olds– Occipital areas show increase to flickering checker

• Peña et al. (2003)– Neonates sleeping– LH superiority to speech, but not backward speech

or silence

Our Questions

• Is there LH superiority to language stimuli over the course of the first year?

• Are there non-language stimuli that show LH superiority?

• Are the responses similar across development (e.g., young vs. old infants compared to adults)?

Experiment

• Participants– Infants n = 80

• 40 – ‘young’ (3 to 7.5 months)• 40 – ‘old’ (7.5 to 10.5 months)

– 16 adults

• 5 possible conditions• English, Scrambled English • Korean, Scrambled Korean • Tone (continuous sine wave)

Scrambled Conditions

• Very speech like– Preserved segmental consonants and

vowels

• Not like speech at all– Violates continuity and prosody

Comparison to Peña et al.

• State

• Age

• DV

• Path length

• Language

• Stimuli features

ENG ENGENG16 sec 24 sec 32 sec

TON TONTON16 sec 24 sec 32 sec

SCRKOR

SCRKOR

SCRKOR16 sec 24 sec 32 sec

SCRENG

SCRENG

SCRENG16 sec 24 sec 32 sec

KOR KORKOR16 sec 24 sec 32 sec

Infant: continuous

Male infant

Oxy

Total

Female infant

Deoxy

Infants hearing English

0

0.5

1

1.5

2

2.5

3

3.5

Left Right

Hemisphere

Number of Voxels w/Sig

Activation

upper

lower

Infants hearing English

0

0.5

1

1.5

2

2.5

3

3.5

Left Right

Hemisphere

Number of Voxels w/Sig

Activation

upperlower

Infants hearing Scrambled English

0

0.5

1

1.5

2

2.5

3

3.5

Left Right

Hemisphere

Number of voxels w/Sig

Activation

upper

lower

Infants hearing English

0

0.5

1

1.5

2

2.5

3

3.5

Left Right

Hemisphere

Number of Voxels w/Sig

Activation

upperlower

Infants hearing Scrambled English

0

0.5

1

1.5

2

2.5

3

3.5

Left Right

Hemisphere

Number of voxels w/Sig

Activation

upperlower

Infants hearing Korean

0

0.5

1

1.5

2

2.5

3

3.5

Left Right

Hemisphere

Number of Voxels w/ Sig

Activation

upper

lower

Infants hearing Korean

0

0.5

1

1.5

2

2.5

3

3.5

Left Right

Hemisphere

Number of Voxels w/ Sig

Activation

upperlower

Infants hearing English

0

0.5

1

1.5

2

2.5

3

3.5

Left Right

Hemisphere

Number of Voxels w/Sig

Activation

upperlower

Infants hearing Scrambled English

0

0.5

1

1.5

2

2.5

3

3.5

Left Right

Hemisphere

Number of voxels w/Sig

Activation

upperlower

Infants hearing Scrambled Korean

0

0.5

1

1.5

2

2.5

3

3.5

Left Right

Hemisphere

Number of voxels w/Sig

Activation

upper

lower

Infants hearing Korean

0

0.5

1

1.5

2

2.5

3

3.5

Left Right

Hemisphere

Number of Voxels w/ Sig

Activation

upperlower

Infants hearing Scrambled Korean

0

0.5

1

1.5

2

2.5

3

3.5

Left Right

Hemisphere

Number of voxels w/Sig

Activation

upperlower

Infants hearing English

0

0.5

1

1.5

2

2.5

3

3.5

Left Right

Hemisphere

Number of Voxels w/Sig

Activation

upperlower

Infants hearing Scrambled English

0

0.5

1

1.5

2

2.5

3

3.5

Left Right

Hemisphere

Number of voxels w/Sig

Activation

upperlower

Infants hearing Tone

0

0.5

1

1.5

2

2.5

3

3.5

Left Right

Hemisphere

Number of Voxels w/ Sign

Activation

upper

lower

Infants hearing Tone

0

0.5

1

1.5

2

2.5

3

3.5

Left Right

Hemisphere

Number of Voxels w/ Sig

Activation

upperlower

Infants hearing Korean

0

0.5

1

1.5

2

2.5

3

3.5

Left Right

Hemisphere

Number of Voxels w/ Sig

Activation

upperlower

Infants hearing Scrambled Korean

0

0.5

1

1.5

2

2.5

3

3.5

Left Right

Hemisphere

Number of voxels w/Sig

Activation

upperlower

Infants hearing English

0

0.5

1

1.5

2

2.5

3

3.5

Left Right

Hemisphere

Number of Voxels w/Sig

Activation

upperlower

Infants hearing Scrambled English

0

0.5

1

1.5

2

2.5

3

3.5

Left Right

Hemisphere

Number of voxels w/Sig

Activation

upperlower

Infant Results

• LH superiority across all language conditions

• Optical imaging can detect differences in auditory cortex– Across conditions– Between hemispheres– Between groups of channels

Age difference in infants• Young infants: most activation to

English+Scrambled Eng compared to other conditions

• Older infants: most activation to straight compared to scrambled and tone conditions

0

0.5

1

1.5

2

2.5

3

3.5

Eng Kor Ton Sc Eng

Sc Kor

Eng Kor Ton Sc Eng

Sc Kor

Ave

rag

e #

of

Vo

xels

Sh

ow

ing

sig

act

iva

tion

Young Infants Old Infants

0

0.5

1

1.5

2

2.5

3

3.5

Left Right

0

0.5

1

1.5

2

2.5

3

3.5

Left Right

Adults hearing Korean

Hemisphere

Number of Voxels w/ Sig

Activation

Adults hearing Tone

Hemisphere

Number of Voxels w/ Sig

Activation

0

0.5

1

1.5

2

2.5

3

3.5

Left Right

Adults hearing English

Hemisphere

Number of Voxels w/ Sig

Activation

upperlower

0

0.5

1

1.5

2

2.5

3

3.5

Left Right

Adults hearing Scrambled English

Hemisphere

Number of Voxels w/ Sig

Activation

upperlower

0

0.5

1

1.5

2

2.5

3

3.5

Left Right

Adults hearing Scrambled Korean

Hemisphere

Number of Voxels w/ Sig

Activation

upperlower

upperlower

upperlower

Adult Results

• LH superiority to English and Korean

• RH superiority to Scrambled conditions

• Bilateral and low activation to Tone

Comparisons between infants and adults

• Language conditions only– Young infants are significantly different from adults– Old infants are not significantly different from

adultsYoung Adults

0

0.5

1

1.5

2

2.5

3

3.5

4

Left Hemisphere Right Hemisphere

English

Korean

0

0.5

1

1.5

2

2.5

3

3.5

4

English

Korean

Left Hemisphere Right Hemisphere

This comparison collapses across straight/scrambled factor

Individual differences

• English (n = 62 infants)– LH Superiority: 71%– RH Superiority: 11%– Equal activation: 2%– No activation: 16%

Discussion

• There is LH superiority to language over the course of the first year

• Young infants show LH superiority to our scrambled stimuli

• Developmental differences are measurable across infants and adults

Speculations• Prosodic sensitivity is not in place by 6 months

(Jusczyk et al. 1993; 1994)– Perhaps that is related to the young infants LH superiority

across all language conditions

• Prosodic sensitivity is in place by 7.5 months (Jusczyk et al. 1999; Newsome & Jusczyk, 1995)– Perhaps older infants and adults are sensitive to violations of

the spectral quality and prosody and responded differently to the straight versus scrambled speech.

• Our findings are consistent with Native Language Neural Commitment (Kuhl, 2004)

Thanks!

• John Gore, Chris Cannestraci, and Sohee Park at Vanderbilt University

• Anna Lane for heroic efforts in data analyses!

• McDonnell Foundation and Discovery grants

Asleep Awake

English

Korean

7, 23 old females

KOREAN

Time

-5 0 5 10 15 20 25

% signal change

-4

-2

0

2

4

6

8

AwakeAsleep

ENGLISH

Time

-5 0 5 10 15 20 25

% signal change

-10

-5

0

5

10

15

20

25

30

AwakeAsleep

Sleeping vs Waking Hemodynamic Lines

Asleep Awake

Korean

ScrKorean

Same male, same visit

Korean

Time

-5 0 5 10 15 20 25

% signal change

-10

-8

-6

-4

-2

0

2

4

6

8

AwakeAsleep

Sleeping vs Waking Hemodynamic Lines(same voxel)

SCRAMBLED KOREAN

Time

-5 0 5 10 15 20 25

% signal change

-50

-40

-30

-20

-10

0

10

20

AwakeAsleep

Adult Average

Time

-5 0 5 10 15 20 25-6

-4

-2

0

2

4

6

8

10

12TonalEnglishKorean

Young English and Korean

Time-50510152025% signal change-4-202468101214 EnglishKorean

Average Old English and Korean

Time-50510152025% signal change-6-4-20246EnglishKorean

HemodynamicCurves

Mohinish’s Question

Time

-5 0 5 10 15 20 25

-8

-6

-4

-2

0

2

4

6

8

10

12

Adult probesInfant probes

Principles of NIRS

What does the data look like?

• 4 parts of the signal– Heart rate– Respiration – Mayer wave– Functional change

• Analysis– Modified Beer Lambert Law

• Known distance light traveled through

• Same absorbency assumed

Experiment 1

• Participants– Cross sectional 39

• 11 – 4 to 6 months (M = 5 months)• 14 – 7 to 9 months (M = 8 months)• 14 adults (M = 23 years)

– Longitudinal• 2 infants 8 visits between 1 and 3 months

– Additional• 18 did one condition but not both• 3 fussed out

Experiment 1

• Apparatus– Hitachi ETG 100, 780 and 830 nm– 24 source/detector pairs– Path length for adults 3 cm baby 2 cm

• Data Analysis– Filtering done in Matlab, down sampling, applied

modified Beer-Lamberts– Brain Voyager QX used for linear drift correction

and statistical analysis

Vibrating Toy Activity

Visual Flicker

Visual CortexMotor Cortex

No Activity

ActivityNo Activity

Stimuli and Design

Individual Results

4-6 mos female 4-6 mos male

7-9 mos female 7-9 mos male

Average Voxels Active

4 to 6 mos 7 to 9 mos adults

Motor Visual Motor Visual Motor Visual

Toy 5 1 5 0 12 5

Video 2 10 0 12 1 12

Characteristics of the BOLD response for different cortical areas

Motor Cortex for Young Group

Time (sec)

-5 0 5 10 15

% signal change

-3

-2

-1

0

1

2

3

4

Motor TaskVisual Task

Motor Cortex for Older Group

Time (sec)

-5 0 5 10 15

% signal change

-4

-3

-2

-1

0

1

2

3

4

Motor TaskVisual Task

Motor Cortex for Adult Group

Time (sec)

-5 0 5 10 15

% signal change

-10

-5

0

5

10

15

20

25

Motor TaskVisual Task

Visual Cortex for Young Group

Time (sec)

-5 0 5 10 15

% signal change

-6

-4

-2

0

2

4

6

8

Motor TaskVisual Task

Visual Cortex for Older Group

Time (sec)

-5 0 5 10 15

% signal change

-6

-4

-2

0

2

4

6

8

Motor TaskVisual Task

Visual Cortex for Adult Group

Time (sec)

-5 0 5 10 15

% signal change

-20

-10

0

10

20

30

40

Motor TaskVisual Task

Mot

orV

isua

l

4-6 mos 7-9 mos Adult

Mean time to peak 13s after stimulus onset

Mean time to peak 6s after stimulus onset

Individual differences

Motor Cortex

Younger Group

% signal change

-10

-5

0

5

10

15

Motor TaskVisual Task

Motor Cortex

Older Group

% signal change

-10

-8

-6

-4

-2

0

2

4

6

8

10

12

Motor TaskVisual Task

Motor Cortex

Adult Group

% signal change

-20

-10

0

10

20

30

40

Motor TaskVisual Task

Visual Cortex

Younger Group

% signal change

-20

-10

0

10

20

30

Motor TaskVisual Task

Visual Cortex

Older Group

% signal change

-15

-10

-5

0

5

10

15

20

Motor TaskVisual Task

Visual Cortex

Adult Group

% signal change

-40

-20

0

20

40

60

80

Motor TaskVisual Task

Mot

orV

isua

l

4-6 mos 7-9 mos Adult

Discussion

• Global similarities across ages in motor and visual stimulation

• All participants showed increase blood flow due to stimulation

• Nuanced differences across ages

• Individual data suggest quantity of variance

Cheers

• We tested 4x as many subjects

• Infant friendly design - low drop out rate and better quality data

• Design that doesn’t require ‘rest’

• Double dissociation in the design