The Development and Neural Bases of Face Recognition

Charles A. Nelson, Ph.D.1,2,3

University of Minnesota

1 Institute of Child Development2 Department of Pediatrics3 Center for Neurobehavioral Development

Adult Models of Face Processing

• Kanwisher– Suggests that the ventral temporal cortex

contains a limited number of separable areas that are specialized for representing specific categories of stimuli (faces or objects)

• Haxby– Neural representations of faces and different

categories of objects are widely distributed and overlapping (not one area devoted to processing faces and objects)

• Gauthier– Activation of the fusiform area is not specific to faces

but to any category of stimulus in which expertise has been obtained.

• Can studying development help us resolve this debate?

Identity

Changeableaspects

Why is the study of the development of face processing interesting?

• Prior to the onset of language (0-2 years), most communication is non-verbal; thus accurately decoding facial signals is important– e.g., facial identity (e.g., who is this?); facial expression (e.g., what

is this person feeling?)

How does this develop?• Even with language, attending to the face occurs naturally

(thought experiment: carry on a conversation with someone without looking at the face). Why?

• Speculation that it may be adaptive (from reproductive fitness perspective) for young infant to recognize caregiver

Why is the study of the development of face processing interesting? (Con’t)

• Processing some faces (e.g., familiar ones) can be selectively knocked out or perturbed in various disorders (e.g., prosopagnosia, autism), suggesting to some face recognition is “hard wired.” Only studies of development can address this hypothesis. Similarly…

• May provide a model system for teasing apart experiential from nonexperiential effects. – For example… Debate continues over whether face recognition

represents • a domain specific “module” that is innate and does not develop or • an experience-expectant and activity-dependent ability that does develop

– Former would argue that neural architecture for face recognition would be present very early in life (which I will demonstrate is true);

– Later would suggest remodeling of this architecture over months and years (which I will also demonstrate is true).

My charge• Review the available evidence, both behavioral

and neural, which describes the development of face processing.– Will start with critique of newborn work, then move to

behavioral work with older infants and children – Then turn to developmental cognitive neuroscience

work

• Will then provide examples of three prominent models of development of face processing– Johnson– deSchonen– Nelson

Behavioral Studies - Newborns• History: Bowlby (e.g., 1969) argued that would be

adaptive (from reproductive fitness perspective) for newborns to recognize caregiver; thus, newborns should show visual preference for face vs. non-face (mother in particular)

• Archival Data: Some studies (e.g., Goren, Sarty, & Wu, 1975) supported Bowlby’s prediction - newborns would look longer at moving schematic face vs. moving schematic non-face. Others did not (e.g., Hershenson, 1965; Thomas, 1965). All were plagued by methodological problems.

• Modern Data: Johnson, Slater, Simion, etc. have demonstrated that newborns show visual preference for face-like stimuli vs. non faces.

Newborn Studies, Con’t• BUT, what are infants really

preferring (i.e., is it “facedness” per se or something else)?

– Simion, Macchi-Cassia and colleagues (2002, 2003): a stimulus with an oval perimeter and internal features situated in the upper half of the oval are preferred over the same oval with features situated in the lower half; moreover, these features do not need to resemble faces, they can also simply be dot patterns.

>

-Johnson & Morton, 1991-Valenxa, Simion, Macchi Cassia, & Umilta, 1996

Simion, Valenza, Macchi Cassia, Turati,&

Umilta, in press

• Newborns prefer geometrical, non-facelike stimuli with more elements in the upper part over stimuli in which more elements are in the lower part.

>

>

>

…and of course, these effects also obtain with “real” faces...

How Robust is this effect?• Caldara et al. (2003) presented these same stimuli

to adults; reported that the head-shaped patterns with a greater number of elements in the upper visual field showed greater fusiform activation than other stimuli

• Conclusion: newborn preference for face-like stimuli probably not driven by “facedness” per se, as much as arrangement of perceptual elements, which in turn influenced by tuning properties and limitations of newborn visual system.

Behavioral Studies – Older Infants

• Archival Data: Fagan (1972) demonstrated that around 4 months, infants begin to show facial inversion effect, pointing to development of face “schema.”

• Modern Data: is rapid improvement in face processing skills >4-6 months. Examples: Infants <1 year can categorize facial gender, some facial expressions. However, no evidence of categorizing negative facial expressions till 1-2 years; furthermore, recognition of negative facial expressions still lags behind adult levels through (at least) middle childhood.

Neural Bases of Face Recognition

• Relatively new field of inquiry. Why?– Most developmental psychologists less interested in the

neurobiological bases of behavioral development than in describing changes in behavior across time.

– Tools to study development of neural bases more limited than what is available to study adults

• Can’t use PET (see French [Mazoyer] exception)• Can’t easily use fMRI <5 (or so) years (see French [Dehaene]

exception)

– Infants and young children typically do not suffer from the kinds of infarcts/injuries that afflict adults and that selectively knock out face circuits; thus, use of classic “lesion” approach limited.

Neuropsychological Studies

de Schonen and her colleagues have demonstrated that 4- to 9-month-old infants show a right hemisphere (left visual field; LVF) bias towards processing faces, similar to what is observed in the adult (i.e., is long history of research using divided field presentations, and more recent fMRI studies, point to right>left hemisphere specialization)

Neuroimaging Studies: PET

• Neurologically compromised 2-month-old infants were studied using PET*– Infants were presented with faces or flashing red and

green diodes. – A broad array of areas were activated to faces that

seemingly had little to do with face processing per se (e.g., left superior temporal and inferior frontal gyri).

– However, activity was also observed in the fusiform gyrus, similar to what has been observed in adults (see next slide)

* Mazoyer, N., de Schonen, S., Quinton, O., Crivello, F., Reutter, B., & Mazoyer, B. (1999). NeuroImage, 9, S346.

Neuroimaging Studies: Event-Related Potentials

• de Haan & Nelson* used event-related potentials (ERPs) to examine discrimination of familiar and novel faces and objects in 6-month-old infants: – Clear ERP differences when mother contrasted to stranger, specifically

• NC (see next slide) larger to mother vs. stranger• Greater ERP activity was observed at the right vs. left temporal scalp,

consistent with de Schonen observation (and adult data).– Observed a component over occipital scalp that had a shorter latency to

faces than to objects, leading to speculation that this component (P400) may be a) a precursor to the adult N170 (see next slide) and b) specific to faces

* de Haan, M., & Nelson, C.A. (1997). Child Development, 68, 187-210. de Haan, M. & Nelson, C.A. (1999). Developmental Psychology, 35, 1113-1121

N170 and P400 “Face” Components

6-MONTH-OLD

-10-6

-22

610

141822

26

-100 0 100 200 300 400

ms

µv

N290

P400

ADULT

-8

-6

-4

-2

0

2

4

6

8

-100 0 100 200 300 400

ms

µv

N170

What about differences in face vs. object processing?

• Difficult studies to do with infants, as all the relevant control conditions are hard to implement (see Gauthier & Nelson, 2001* for discussion).

• Caveat notwithstanding….

* Gauthier, I., & Nelson, C.A. (2001). The development of face expertise. Current Opinions in Neurobiology, 11, 219-224.

Neural correlates of preferential-looking in 6-month olds

(Kelly Snyder, in preparation)

Experimental Design

•Encoding Phase (ERPs)Test Phase

(Preferential-looking)

Familiar

Novel

Familiar

Novel

Snyder (in preparation).

250 500 750 1000 1250 1500

5

0

-5

-10

10

Memory Effects

Snyder (in preparation).

NP FP

NPFP

Effect: Novelty preferences associated with a more negative Nc (but only for face stimuli), possibly reflecting greater attention to the stimulus during encoding.

* NP < FP

NPFP

Memory Effects

-10

250 500 750 1000 1250 1500

5

0

-5

10

Snyder (in preparation).

Effect: Novelty preferences associated with a reduction in the amplitude of the slow wave (but only for face stimuli), possibly reflecting a more fully encoded stimulus (one that requires less updating)

FP

NP

* NP < FP

NPFP

Memory Effects

250 500 750 1000 1250

5

0

-5

-10

10

250 500 750 1000 1250 1500

5

0

-5

-10

10

FPNP

Effect: Novelty Preferences associated with a reduction in amplitude of several components over occipital scalp regions (but only for object stimuli), possibly reflecting the habituation of visual perceptual responses.

FPNP

* NP < FP

* NP < FPNPFP

Summary & Conclusions

• Is clear association between stimulus encoding (as inferred by ERPs) and recognition memory (as inferred from looking time)

• Are clear differences in face vs. object processing by 6 months, both behaviorally (e.g., infants look longer at faces) and electrophysiologically.

• Points to early specialization of this ability• …but how developed is this function and

how specific is it?

The Other-Species Effect

• Pascalis & Bachevalier (1998) report that both non-human primates and humans are better at recognizing faces of their own species. – Monkeys look longer at novel monkey

faces but not human faces and humans look longer at novel human faces but not monkey faces.

The Other-Species Effect

• Pascalis, de Haan & Nelson (2002)* directly tested theory of perceptual narrowing** using the Visual Paired Comparison procedure in 6 and 9 month old infants and adults.

• Results indicate that adults and 9 month olds looked longer at novel human faces, but looked equally long at monkey novel and familiar faces.

• In contrast, 6-month-olds showed novelty preferences in both the human and the monkey conditions.

• Thus, perceptual window through which faces are viewed initially tuned broadly, then narrows with experience (accounting for why 6 month olds are “better” at discriminating monkey faces than are 9 month olds and adults)

*Pascalis, O., de Haan, M., & Nelson, C.A. (2002). Science, 296, 1321-1323. ** Nelson, C.A. (2001). Infant and Child Development, 10, 3-18

The Other-Species Effect

• deHaan and Johnson* studied adults and 6-month-olds in a human and monkey face recognition task while they recorded ERPs.

• Would adults and infants show the same cortical specificity during face processing?

• Adults and infants were shown both monkey and human faces in upright and inverted orientations.

* de Haan, M., Pascalis, O., & Johnson, M.J. (2002). Journal of Cognitive Neuroscience, 14, 199-209.

Other-Species Effect

• In adults, all stimuli evoked an N170 over temporal and occipital leads.

• This N170 was larger in amplitude and longer in latency for upright monkey compared to upright human faces.

• Inversion effects were apparent in the human but not the monkey conditions (increased amplitude and latency to inverted faces).

The Other-Species Effect

• No component of the infant ERP showed the same specificity as the adult N170.

• 6-month-olds did show sensitivity to both inversion and species but it was distributed across two components.

• Nc (species), P400 (orientation)

• 3-month-olds did not show the same specificity for human faces that 12 month olds and adults exhibit. – 12 month olds exhibited a negative

component (N290) that was larger in amplitude and longer in latency for human than monkey faces

– This same component showed an inversion effect (larger amplitude to inverted human faces only).

– The P400 showed longer latency to human and inverted faces.

– 3-month olds did not show the same specificity for human faces.

Halit et al., in press

Summary

• Both the N290 and the P400 seem like reasonable candidates for the adult face-specific N170….although much more work remains to be done.

Scott, Shannon & Nelson, in preparation

Infant ERP Task

Adult ERP and Behavioral Study

Scott, Shannon, & Nelson, in preparation

• Adult Study: – Adults performed significantly better on human

compared to monkey task. – P2 amplitude differentiated species; P2 latency

differentiated orientation; N300/400 differentiated familiarity.

• Infant Study (9-month-olds):

O2 O1

HumanMonkey

T6 T5

O2

FrontalProfile

O1T6 T5

O2 O1T5

Familiar Unfamiliar

T6

LEFT HEMISPHERE RIGHT HEMISPHEREP400

N290

Summary

• Infants’ ERPs reflect processing differences along all three dimensions – species, orientation, and familiarity.

• Infants’ ERPs are more defined in the right hemisphere compared to the left hemisphere, indicating cortical specialization by 9 months.

Summary of Other Species Effect

• Advantage in processing faces from same species develops over the first year of life

• Typical development characterized by the loss of ability

• Presumably experience drives perceptual expertise that is acquired towards processing faces from same species.

• In theory, exposure to faces from other species would keep perceptual window open (work ongoing by Pascalis/Scott, Shannon & Nelson)

Summary

• The neural substrate for face recognition is clearly not fully developed within the first year of life.

• It appears that this development is activity-dependent and experience-driven.

What Develops Beyond Infancy?

• Carver et al. (2003)*:

– Presented familiar (mother) and novel faces and familiar and novel objects to 18-54 month old typically developing children

• 18-24 months• 24-45 months• 45-54 months

– Recorded ERPs from 64 channels

* Carver, L.J., Dawson, G., Panagiotides, H., Meltzoff, A.N., McPartland, J., Gray, J., & Munson, J. (2002). Developmental Psychobiology, 42, 148-159.

Face Processing

Object Processing

Carver et al.• Conclusions:

– Consistent with deHaan/Nelson/Johnson, is segregation of face vs. object processing

• Responses to familiar face but NOT object varied as a function of age. • More specifically, all age groups showed a larger response for the Nc

and P400 to unfamiliar versus familiar objects, but only children between 18-24 months of age showed greater ERP response to the mother’s face compared to a strangers face.

– Ability to recognize faces continues to be refined through 5 years

– Since in previous work demonstrated that children with autism fail to show ERP differentiation to mother/stranger face (but do show typical profile to familiar/novel objects), suggests that face recognition “system” can be corrupted by something other than lack of experience.

What Develops Beyond the Preschool Years?

• Passarotti et al. (2003)*. – Presented 10-12 year old children and adults with face matching

vs. location matching task– fMRI performed at 1.5Tesla– RESULTS:

Behavioral Data: accuracy and RT improve with agefMRI Data: Relative to adults, children showed more distributed pattern of activation in both right and left temporal regions (i.e., increased activation in areas lateral to fusiform gyrus and anterior to fusiform in middle temporal gyrus) (see next slide)

* Passarotti, A.M., Brianna, M.P., Bussiere, J.R., Buxton, R.B., Wong, E.C., & Stiles, J. (2003). Developmental Science, 6, 100-117.

Me1

3

0

4

2

5

A. Medio-lateral fusiform regions

Adults N=16

Z=-17

Children N=12 children N=12

B. Middle Temporal region

Adults N=16

Z=-9

RH LH RH LH RH LH RH LH

Children N=12

1

3

0

4

2

5

Figure 2

Face-matching Task

Passarotti et al.

• Conclusions: – Children have a more diffuse and distributed

area of activation in both face and location processing compared to adults.

– Areas involved in face processing are still undergoing refinement as “late” as 10-12 years.

General Conclusions About Face Recognition

• The bulk of the evidence supports a rapid improvement in face recognition abilities through infancy, with more subtle changes occurring well beyond infancy.

• Neurophysiological (ERP) and metabolic (fMRI) data, not surprisingly, also show changes in neural substrate across first decade of life.

• Presumably these changes are experience-dependent, and do not reflect any sort of “innate” module.

What about Disorders of Face Recognition?

• The study of typical development can often be informed by the study of children who deviate from a normal trajectory. – Prosopagnosia– Autism

Prosopagnosia

• Special nature of face processing originates from reports of patients who are reportedly unable to recognize familiar faces while maintaining the ability to recognize objects. – This impairment is often accompanied by focal

damage to the ventral occipitotemporal and temporal cortices.

– Reported to process faces similar to objects (sometimes absence of inversion effects).

Prosopagnosia

• Marotta, Genovese, & Behrmann (2001)– fMRI of prosopagnosic patient did not show

normal activation of fusiform. – Did show left hemisphere posterior fusiform

activation, suggesting faces are being processed featurally.

Developmental Prosopagnosia

• There are a small number of developmental prosopagnosic cases. – Most have more general visual processing deficits

– Farah et al (2000) report a case study of a boy who sustained brain damage at 1 day of age. They report that this boy (now late teens) has damage to occipitotemporal pathway and is severely impaired in face compared to object recognition.

• Evidence that face processing specified in genome? No experience necessary?

Summary of Developmental Prosopagnosia Literature

• When review all available studies, appears that children can develop a facial agnosia, but may be less specific than occurs in adults

• Surprisingly, seems to be little plasticity in this system; that is, once agnosia develops is little recovery of function over time. This, in turn, points to a sensitive period for developing facial expertise

Summary Developmental Prosopagnosia Literature (Con’t)

• Recent work by Mondloch/Maurer on children with congenital catarcts provides partial support for this hypothesis; specifically, restoration of vision following cataract surgery leads to dramatic improvement in vision generally, but persistent deficits in face processing.

Autism

• Children with autism are often reported as having deficits in face processing abilities.

• Schultz et al (2000) found decreased activation in the fusiform region and increased activation in the inferior temporal gyrus in autistic/aspergers syndrome patients compared to normal controls (similar to object processing)

Autism

• Osterling and Dawson (1994)– Home videos of 1 year old birthdays indicate

that autistic children fail to attend to other people’s faces compared to controls

• Dawson et al (2002) recorded ERPs in 3 and 4 year olds with autism. – Children with autism did not differentiate

familiar and unfamiliar faces (but did differentiate objects)

Dawson et al (2002)

ControlAutism

Faces

Objects

McPartland, Dawson et al (2001a;b)

• Recorded ERPs from high functioning autistic adolescents– Latency to peak response of the N170

component was longer than controls and did not show the typical differences between upright and inverted faces.

– Not right lateralized.

Why do individuals with autism have face processing deficits?

• Elgar and Campbell (2001)– Aberrant affective/social “drivers”

• mediated by the ventromedial parts of the inferior frontal cortex, which mediates activity flowing from the cortex, amygdala and thalamus.

• Marcus and Nelson (2001) possibilities:– Lack of “expected” visual stimuli (faces)– Faces may be present but not getting in, due to other

deficits (attending to social stimuli; reduced social interest)

– Faces treated as objects, thus only one system develops

Models of the Neural Basis of the Development of Face Processing

Model #1 (Johnson & Morton, 1991; Morton & Johnson, 1991; Johnson, 1997)

• Infants less than 2 months old will track face-like objects because of a subcortical visuomotor mechanism (involving the superior colliculus) referred to as CONSPEC

• CONSPEC is essentially an innate structure that facilities attention to faces (or face-like objects), possibly because this subcortical region responds preferentially to movement and to objects in the periphery

Johnson (Con’t)

• After about 2 months, influence of CONSPEC begins to wane

• At this time a cortical mechanism called CONLERN comes into play.

• CONLERN benefits from experience and thus, further development in face recognition reflects an experience-dependent process

Model #2 (de Schonen & Mathivet, 1989)

• Face recognition generally is an experience-dependent process

• Differences in the type of visual information processed by the left vs. right hemispheres account for right hemisphere bias towards face recognition.

• Specifically…

de Schonen Model (Con’t)

• The right hemisphere is better suited than the left in processing configural information because such information consists predominantly of low spatial frequencies.

• The processing of low spatial frequencies is optimal for the young infant, given limitations of their visual systems (i.e., relatively poor contrast sensitivity function early in life)

• As infants’ contrast sensitivity function improves, experience with faces further drives the development of the right hemisphere, leading to increased neural specialization.

de Schonen (Con’t)

• With time the left hemisphere (which can also process faces, albeit less well) also benefits from this experience, and in so doing it becomes possible to account for

• the LVF/right hemisphere bias for processing faces • recent neuroimaging studies reporting bilateral

activation of regions like the fusiform gyrus.

Model #3 (Nelson, 1993, 2001)•Face perception represents an experience-expectant process that develops through activity-dependent mechanisms.•Experience with faces drives the development of the cortical circuitry putatively involved in face recognition (e.g., fusiform gyrus). •Initially, this circuitry is broadly tuned, but with experience, there is a narrowing of the perceptual window through which faces are observed, and cortical specialization occurs.

Model #3 (Con’t)• Evidence supporting perceptual narrowing:

• Adults have difficulty discriminating inverted faces, a phenomenon that appears to emerge by about 4 months of age.

• The “other race” effect, in which adults, more so than children, find it harder to recognize faces from races other than their own.

• Maltreated children respond differentially to certain facial expressions (e.g., anger) relative to nonmaltreated children. Similarly, children of depressed mothers differ from children of non-depressed mothers in recognizing certain facial expressions.

• Both monkeys and human adults are better at recognizing faces from their own species, although early in development infants are as good at discriminating monkey faces as human faces (see next slide).

Visual Discrimination of Human & Monkey Faces

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

6-month 9-month Adult

Pro

po

rtio

n o

f L

oo

kin

g

Familiar

Novel

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

6-month 9-month Adult

Pro

po

rtio

n o

f L

oo

kin

g

Familiar

Novel

From: Pascalis, de Haan & Nelson, Science, 2002, 296, 1321-1323.

45

Problems common to all models

• Failure to specify the specific experiences that are required to drive development

• The timing of experience is underspecified

• Failure to account for why the regions of the brain that ultimately become specialized for face recognition are targeted vs. some other area (i.e., why the fusiform gyrus vs. ???)

Conclusions

• Regardless of which model proves to provide the best fit to the data, the bulk of the evidence favors the perspective that face-selective neural circuits become specialized via experience with faces.

• Unclear if there is a sensitive period for this to occur. Work on congenital cataracts, and on developmental prospopagnosia suggests there might be.

• Challenge: what precisely is the nature of the critical experiences, and what is the timing whereby these experiences must occur? And, what about plasticity???

THE END