neural correlates of object indeterminacy in art...
TRANSCRIPT
Neural Correlates of Object Indeterminacy in Art Compositions
Alumit Ishai
University of Zurich, Switzerland
The Key, Jackson Pollock, 1946
Detail from the Sistine Chapel by Michelangelo
Succulus, Robert Pepperell, 2005
Turner, Vision of Medea Pittoni, The Annunciation
Pepperell, Exhaust Pepperell, Felt
Behavioral Study
Object Recognition Task
% Yes Responses
0
20
40
60
80
100
Color Gray
Representational (VA) Indeterminate (RP)
Response Latencies (msec)
0
500
1000
1500
2000
2500
Color Gray Color Gray
VA RP
Yes, VA No, RPYes, RP
* *
It took subjects significantly longer to decide whether or not indeterminate paintings contain any familiar objects
Ishai et al., 2007
Judgment of Aesthetic Affect Task“how strongly did this painting affect you?”
Ratings (%)
Color Gray
Not Little Fair Very
VA RP VA RP0
20
40
60
Response Latencies (msec)
0
500
1000
1500
2000
2500
VA RP VA RP
Color Gray
* *
Aesthetic rating of ALL painting was virtually identical, but it took subjects longer to rate the indeterminate paintings
Conclusions
• Perception and subsequent memory of art compositions depend onsemantic aspects (content), whereas aesthetic affect depends on visualfeatures (form / color).
• Indeterminate art compositions comprise a special class of stimuli,with which visual perception and memory can be further investigated.
fMRI StudyRepresentational Indeterminate Abstract Scrambled
Task: Do you recognize any familiar objects?
Hypotheses: i) Abstract paintings would activate more posterior regions in the ventral stream than indeterminate or representational compositions.
ii) Subjects would use mental imagery to resolve object indeterminacy.
Behavioral Data (N=12)
% Yes Responses
0
20
40
60
80
100
VA RP AB SC
Reaction Time (msec)
YES NO
0
400
800
1200
1600
2000
VA RP AB RP AB SC
* * **
Fairhall & Ishai, 2008
Representational “VA” Indeterminate “RP” Abstract “AB” Scrambled “SC”
Fairhall & Ishai, 2008
Main Effect: paintings vs. scrambled
144 t value
fMRI Data
L R
Representational “VA” Indeterminate “RP” Abstract “AB” Scrambled “SC”
144 t value
Representational Indeterminate Abstract
0
10
20
30
40
IOG FGDOC IPS IFG Hipp
PE
*
*
Region-of-Interest Analysis
Fairhall & Ishai, 2008
X=49
The Effect of Meaningful Content
3.6 6
high-order associations in the temporo-parietal junction, which mediates the binding of visual features and spatial locations.
>
Representational Indeterminate
Fairhall & Ishai, 2008
3.6 5
The Indeterminacy Effect
activation in the precuneus and IFG, which mediate imagery and memory retrieval and likely reflects the subject’s strategy to resolve the object indeterminacy.
>
Scrambled Indeterminate
X=0
1.7 3
>
Abstract Indeterminate
X=-3
Fairhall & Ishai, 2008
Picasso, The Dresser, 1910
mirror
glass withtoothbrush
locketkey in drawer
key and lock
Braque, House in La Roche-Guyon, 1909
Roof of house
Wall of house
Tree trunk
Branches of tree
Gris, Still life with bottles and knife, 1911-1912
Bottle
Knife
Bottle
1. To what extent titles influence the understanding and appreciation of artworks?
The Formalists: titles are “identification tags” and should not function as sources of meaning for the viewer (Bell, Fry).
The Anti-Formalists: Titles are important contextual factors that provide explicit directives for interpretation and engage the attention of the viewer (Gombrich, Fisher).
The debate in scientific terms: do titles provide essential ‘top-down’ cues?
2. Can naive viewers learn to recognize familiar content in abstract compositions?
Cubist PaintingsBraque GrisPicasso Scrambled
“Vase of Flowers”
1.5 sec
8 sec
3.5 sec3 sec
How many objects did you see?0123+
Did you recognize any familiar objects?
“Untitled”How many objects did you see?0123+
Experimental Design
Two groups of subjects: trained / untrained
TrainedControl
Titled Untitled
*
TrainedControl0
20
40
60
80
100
Did you recognize any familiar objects?
% of Yes Responses
Trained subjects recognized objects in more titled than untitled paintings
Behavioral Data (N=24)
Wiesmann & Ishai, 2010
Trained TrainedControl Control
Titled Untitled
Yes No Yes No Yes No Yes No0
0.5
1.0
1.5
2.0
2.5 ** ***
The Effect of TitleDid you recognize any familiar objects?
Reaction Time (sec)
It took trained subjects significantly longer to decide that both titled and untitled paintings do not contain any objects
“How many familiar objects did you recognize?”
Trained subjects recognized more objects than control subjects as reflected by proportion of Less ”0” and more “2” responses, especially for titled paintings
Reaction Time (msec)
control trained
0 1 2 30 1 2 30
500
1000
1500
0
10
20
30
40
50
control trained
0 1 2 3 0 1 2 3
% Responses
*
*
main effect: paintings vs. scrambled
L-8.00
-4.35
t(11)p < 0.001
8.00
4.35
y=-73 y=-53 y=14
L
R
R
control subjects (N=12)
trained subjects (N=12)
fMRI Data
Wiesmann & Ishai, 2010
Activation in the Parahippocampal Cortex
L R
t(11)p < 0.001
8.00
4.35y=-40 y=-40
Control subjectsN=12
trained subjectsN=12
time (sec)
% s
igna
l cha
nge
-0.2
0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
1.6
2 4 86 1210 1814 160 2 4 86 1210 1814 160
0 1 2 3
number of recognized objects:
Correlations between Reaction Time & Brain Activation
-8.00
-2.67
t(11)p < 0.01
8.00
2.67
X=37
trained subjects (N=12)
The longer response latencies associated with “No, I did not recognize any familiar objects” were correlated with activation in a network of brain regions: MTG, IPS, MFG, IFG, and the insula.
Wiesmann & Ishai, 2010
Summary
Short training on object recognition in cubist paintings resulted in significant behavioral and neural changes:
• Trained subjects recognized more familiar objects in more paintings.
• Trained subjects were slower to report not recognizing any objects.
• Meaningful titles facilitated object recognition in cubist paintings, but only when some prior knowledge/experience existed.
• Trained subjects showed enhanced and differential activation in the PHC.
• In trained subjects response latencies were correlated with activation in the fronto-parietal network for spatial attention.
Conclusions• Art compositions elicit activation in a distributed cortical network.
• Observers likely resolve visual indeterminacy by using contextual associations, mental imagery and memory retrieval.
• Our studies provide empirical evidence for:
i) The proactive brain framework, according to which the human brain uses associations to generate predictions.
Gombrich: when we view works of art we use schemas, namely stored structures of knowledge, in order to form expectations.
ii) Bayesian inference in the human brain: trained subjects are more likely than control subjects to suppress errors and establish a match between the indeterminate bottom-up input and their top-down hypotheses.
Viewing art is not a passive process, but rather a dynamic cognitive function that engages distributed cortical networks activated during the allocation of attention, contextual associations, mental imagery, and retrieval from memory.
Ishai, 2010
Coda
Acknowledgment
Scott Fairhall
Robert Pepperell
Martin Wiesmann