Attention and emotion: Attention and emotion: From data to conceptual issuesFrom data to conceptual issues

Luiz PessoaDepartment of Psychology

University of Maryland, College Park

Background

• 1990s: work showing limitations of visual processing and the need for attention Change blindness Attentional blink

Background

• Processing of emotion-laden information is prioritized Independent of awareness

RL

Morris et al. (1998)Whalen et al. (1998)

LeDoux

“Automatic” Processing

Amygdala

Research goal

• Understand the role of attention and awareness during the processing of emotional visual items

Employ strong attentional manipulations

Evaluate awareness with Signal Detection Theory

Role of spatial attention

• Is activity evoked by emotional faces automatic?

OR

• Does activity evoked by emotional faces require attention?

200 ms

Same/differentDifficult: 64% correct

Not drawn to scale

200 ms

Attended Faces

Unattended Faces

Spatial attention

Male/femaleEasy: 91% correct

Attention is required for the expression of valence (N = 21)

• Strong valence X attention interaction: Effect of valence depends on attention

Fear UNATT

Happy UNATT

Neutral UNATT

Happy ATTNeutral ATTFear ATT

-0.10 2 4 6 8 10 12

-0.05

0

0.05

0.1

0.15

0.2

0.25

Right Amygdala

Re

spo

nse

Am

plit

ude

Seconds

Pessoa et al. (2002): PNAS

X

L R

Emotional perception requires attention

• Attention parametrically manipulated within the same task

• Affective significance enhanced via conditioning

Lim et al. (2008): NeuropsychologiaTask: find X

Easy Hard

Pessoa et al. (2005): NeuroimageHsu and Pessoa (2007): Neuropsychologia

Role of visual awareness

Pessoa et al. (2005): Emotion

Target

Mask

Yes/No

1-3 scaleConfidence?

67 ms

33 ms

Visual awareness: Signal Detection

67 ms

Y = -4

AWARE

L R

33 ms

Y = -4

UNAWARE

L R

Amygdala responses

VS.

Pessoa et al. (2006): Cerebral Cortex

• Many participants can detect fearful faces even at 17 ms

Behavioral results: Individual differences

Szczepanowski and Pessoa et al. (2007): Journal of Vision

17 ms

Fear stimulus > Neutral stimulusAmygdala

Pessoa et al. (2006): Cerebral Cortex

67 ms

“Normals”

N = 19Y = -4

AWARE

L R

“Detecters”

N = 8Y = -6

AWARE

L R

33 ms

Y = -4

UNAWARE

L R

Y = -6

AWARE

L R

x

.

Role of temporal attention/awareness

. . .

. . .

100 ms

100 ms

Lim, Padmala, and Pessoa (2009): PNAS

2 s

T2T1 CS+ vs. CS–

Attentional blink: Behavior (N = 30)

• Enhanced perception of CS+: Reduced blink

T1T2

50

55

60

65

70

75

80

CS+ CS-scene category

T2 a

ccur

ay (%

)CS+

CS–Building or House?

Role of attention/awareness

. . .

. . .

Parahippocampal gyrus

T2

Miss trials

. . .

. . .

T2

Parahippocampal gyrus

Role of attention/awareness

• Miss trials: no differences observed between CS+ and CS- trials

Visual ctx

Amygdala

Time

CS+CS-

Conceptual issues

Impasse

• While a great deal has been learned about the extent and limits of affective visual processing, two camps have opposing and entrenched views

Impasse

• While a great deal has been learned about the extent and limits of affective visual processing, two camps have opposing and entrenched views

Capacity-limited Capacity-Unlimited

Impasse

• While a great deal has been learned about the extent and limits of affective visual processing, two camps have opposing and entrenched views

Capacity-limited Capacity-Unlimited

Not too surprising…

• Emotional stimuli are sufficiently potent that they exhibit a host of properties that do not appear to occur with neutral items

They are processed when unattended

• Affective processing is subject to capacity limitations, as revealed by several experimental manipulations

Attentional blink

Impasse: will it go away?

• Advocates of limited processing can claim that processing resources have not been consumed

“If the manipulation were stronger, the impact of affective items would go away…”

Impasse

• Showing that the emotional effect has disappeared is always subject to the “null problem”

Arguing for the absence of an effect

Power vs. strength of manipulation

Fear unatt

Happy unatt

Neutral unatt

Happy att

Neutral att

Fear att

0 2 4 6 8 10 12

Right Amygdala

Seconds

Res

pons

e A

mpl

itude

Left Amygdala

0 2 4 6 8 10 12-0.1

-0.05

0

0.05

0.1

0.15

0.2

0.25

Seconds

Pessoa et al. (2002): PNAS

X

processing resources

Processing resources

• Given the limited capacity of mental processes, performance is impaired if demands are greater than available resources

Easy/efficient

Hard/inefficient

Norman and Bobrow (1975)

Processing resources

processing resources

processing resources

“automatic”

Nakayama and Joseph (1998)

Processing resources

processing resources

processing resources

Capacity limitation Dual-tasks

Nakayama and Joseph (1998)

Processing resources

• Moors and De Houwer (2006): Every process is uncontrolled, efficient, unconcious, and fast

Processing resources

• Moors and De Houwer (2006): Every process is uncontrolled, efficient, unconcious, and fast, to some degree…

• Relative to what?

• Affective processing: relative to neutral not enough Fine comparisons needed (e.g., abrupt onsets,

search, etc.) Broad set of comparison tasks

Two camps

Capacity-limited Capacity-Unlimited

Preattentive-attentive model

t1t2

Stage 1:Preattentive

Stage 2:Attentive

boundary

Preattentive-attentive model

• Some features are processed pre-attentively in virtue of the fact that they are optimally matched to properties of the early visual system (e.g., orientation)

• Affective processing: Sub-cortical pathway Superior colliculus pulvinar amygdala

Dynamic model

Multiple interactive “stages”

Dynamic model

• Processing is not pre-attentive or attentive, but a gradient of processing efficiency is hypothesized to exist

• Gradient based on the properties of early visual areas

• But critically, gradient is dynamically configured based on task demands

• Configuring is suggested to depend on parietal and frontal cortex

Dynamic model

• Multiple “gates”

• Variable permeability

Less susceptible to capacity limitations

More susceptible to capacity limitations

“bottlenecks”

Dynamic model

• Hierarchical and “short-cut” connections

Multiple waves

• Initial processing of visual information proceeds simultaneously along parallel channels

• “Multiple waves” of activation across visual cortex and beyond

• The multiple waves are engaged dynamically based on task requirements

Subcortical processing

Subcortical processing

Pessoa and Adolphs, Nat. Rev. Neurosci (2010)

“passive”

“integrative”

Subcortical processing

Pessoa and Adolphs, Nat. Rev. Neurosci (2010)

Processing architecture and attention

Task 1

Task 2

Collaborators

Ralph Adolphs Jan Engelmann Shruti Japee Shen-Mou Hsu Seung-Lark Lim Srikanth Padmala Remik Szczepanowski Leslie Ungerleider

National Institute of Mental Health

emotioncognition.org