mnemonic influences on perception as revealed by visual...
TRANSCRIPT
Mnemonic Influences on Perception as Revealed by Visual Aftereffects
Jae-Jin Ryu
Department of Psychology
McGill University
Montreal, Quebec, Canada
April 2009
A thesis submitted to the Faculty of Graduate Studies and Research in
partial fulfillment of the requirements of the degree of Doctor of Philosophy
Jae-Jin Ryu, 2009
Table of Contents
Abstract ............................................................................................................. 2
Résumé.............................................................................................................. 5
Acknowledgments ............................................................................................ 6
ORIGINAL CONTRIBUTIONS TO KNOWLEDGE............................................. 8
CONTRIBUTION OF AUTHORS...................................................................... 10
Chapter 1 General Introduction ..................................................................... 11
Chapter 2 Literature review: Top-down influences on vision ..................... 14 Visual Search........................................................................................................... 15 Top-down influence: perception of ambiguous images ...................................... 16 Top-down influence on perception of simple stimulus features and perceptual learning ................................................................................................. 18
Chapter 3 Representations of familiar and unfamiliar faces as revealed by viewpoint-aftereffects ................................................................ 22
Abstract .................................................................................................................... 22 Introduction.............................................................................................................. 23 Methods.................................................................................................................... 27
Participants............................................................................................................ 27 Apparatus and stimuli ............................................................................................ 27 Procedure .............................................................................................................. 28
Results...................................................................................................................... 30 Experiment 1 ......................................................................................................... 30 Experiment 2 ......................................................................................................... 32
Discussion ............................................................................................................... 33 Conclusions ............................................................................................................. 36 Acknowledgments................................................................................................... 38 Figure Legends........................................................................................................ 38
Chapter 4 Imagine Jane and Identify John: Face Identity Aftereffects Induced by Imagined Faces ...................................................... 43
Abstract .................................................................................................................... 43 Introduction.............................................................................................................. 45 Materials and Methods............................................................................................ 48
Participants............................................................................................................ 48 Apparatus and stimuli ............................................................................................ 48 Procedures ............................................................................................................ 49
Results...................................................................................................................... 52 Aftereffect tasks..................................................................................................... 52 Discrimination Task ............................................................................................... 54
Discussion ............................................................................................................... 54 Conclusions ............................................................................................................. 59 Figure Legends........................................................................................................ 61
Chapter 5 Dynamic motion aftereffects induced by static images previously associated with unidirectional motion ....................................... 68
2
3
Abstract .................................................................................................................... 68 Introduction.............................................................................................................. 69 Methods.................................................................................................................... 72
Participants............................................................................................................ 72 Apparatus and stimuli ............................................................................................ 73 Procedures ............................................................................................................ 74
Results...................................................................................................................... 77 Experiment 1 ......................................................................................................... 77 Experiment 2 ......................................................................................................... 78 Experiment 3 ......................................................................................................... 79
Discussion ............................................................................................................... 80 Figure Legends........................................................................................................ 84
Chapter 6 Concluding Remarks..................................................................... 91 Summary .................................................................................................................. 91 Limitations ............................................................................................................... 93 Possible neural mechanisms mediating mnemonic influence on perception of complex images............................................................................... 95 Mnenmonic influence and perception of simple stimulus features ................... 96 Perception as results of Interactions amongst different visual areas ............... 98
References..................................................................................................... 100
Abstract
Perceiving a visual object often leads to the formation of its
representation in memory. In this case, the role of visual perception in
memory is emphasized, but it is also conceivable that memory plays a
role in the processing of a visual stimulus in a top-down manner. One
way to study whether memory does influence visual perception is to
make use of the selective adaptation method designed to produce
aftereffects. In a typical selective adaptation experiment, a stimulus is
presented for an extended period of time (adapting stimulus) and this
results in a temporary distortion in the perception of subsequent stimuli
(aftereffects). The selective adaptation method has mainly been used to
behaviorally elucidate neural mechanisms involved in the processing of
the adapting stimulus. However, it also is a useful tool to study possible
influences of memory processes on visual perception, because it leads
to the hypothesis that adapting stimuli with different and similar
mnemonic contents should produce different and similar visual
aftereffects, respectively. Results described in the current thesis show
that visual processing of motion and faces, both believed to recruit
specialized areas in the visual cortex are subject to mnemonic
influences.
4
Résumé
La perception visuelle d’un objet mène souvent à la formation de sa
représentation dans la mémoire. Dans ce cas, le rôle de la perception
visuelle est importante, mais il demeure possible que la mémoire est
impliquée dans le traitement d’un stimulus visuel d’une manière
descendante. Une façon d’étudier l’influence de la mémoire sur la
perception visuelle est d’utiliser la méthode d’adaptation sélective
conçue pour produire des effets consécutifs. Lors d’une expérience
d’adaptation sélective typique, un stimulus est présenté pour une
période prolongée (stimulus d’adaptation) et ceci mène à une distorsion
temporaire de la perception des stimuli subséquents (effets consécutifs).
La méthode d’adaptation sélective est surtout utilisée pour élucider au
niveau du comportement les mécanismes neuronnes impliqués dans le
traitement du stimulus d’adaptation. Cependant, c’est aussi un outil utile
pour étudier les influences possibles du procesus de la mémoire sur la
perception visuelle. Ceci mène à l’hypothèse que des stimuli
d’adaptation avec des effets mnémoniques différents et similaires
devraient produire des effets consécutifs différents et similaires,
respectivement. Les résultats de cette thèse montrent que le traitement
visuel de la motion et des visages, qui implique des régions spécialisées
du cortex visuel, est soumis à l’influence mnémonique.
5
Acknowledgments
I am very fortunate to have the following people in my life, who
have helped, inspired, and challenged me throughout the whole
process.
I would first like to thank my supervisor, Prof. Avi Chaudhuri, who
allowed me to pursue my own questions and curiosity in his lab. I am
grateful for the support and encouragements that he has given me,
whether he was in Montreal, or halfway across the globe.
I am thoroughly indebted to my colleagues at CVL who have now
become my friends. They not only helped me to mature as a scientist,
but also made my time at the lab immensely enjoyable. I would
especially like to thank Carmelo Milo for being the best lab manager in
the world, and Reza Farivar for being my scientific inspiration. I would
also like to thank Karen Borrmann, Pascal Lachance, Caitlin Mouri and
Dana Hayward for their invaluable friendship. I will miss you guys very
much.
I am eternally grateful to Jung-Kyong Kim for understanding
everything and making a lizard out of a snake, Lucia Yoon for being so
proud of me, Clara Yoo for just being her, and Sophia Koukoui for
making my life in Montreal twinkle with glamour.
6
I would also like to thank my parents for believing in me, and my
sister and brother for being there. Words cannot describe how much
they mean to me. My newest family member is my husband, Jae-Hun
Kim, who basically made everything possible. I love you very much.
When I was five, my maternal grandparents always talked about
how much they wished me to become a “bak-sa”, which is Korean for
someone with a Ph. D. degree. Therefore, it was probably not a
coincidence that I decided to pursue graduate studies. For that, I thank
them.
7
Original Contribution to Knowledge
This doctoral thesis presents a number of original contributions
about how different mnemonic processes influence visual perception.
Chapter 1 presents the rationale for the present investigations
and explains why the classical adaptation approach is a useful tool for
the purpose of the current thesis.
Chapter 2 briefly reviews previous studies that examined top-
down influences on visual processing.
Chapter 3 presents the results from the investigation that
examined the role of familiarity on the perception of viewpoints of faces.
The psychophysical data suggest that neurons that process viewpoint
information are also involved in the representations of familiar faces,
which are traditionally thought to be view-invariant. This chapter is
based on the following published manuscript.
Ryu, J. J., & Chaudhuri, A. (2006). Representations of familiar
and unfamiliar faces as revealed by viewpoint-aftereffects. Vision
Research, 46(23), 4059- 4063.
8
Chapter 4 describes the study that investigated whether similar
neural networks are involved in the perception and imagery of familiar
face identities. This chapter is based on the following published
manuscript.
Ryu, J.J., Borrmann, K., & Chaudhuri, A. (2008). Imagine Jane
and identify John: face identity aftereffects induced by imagined
faces. PLoS ONE, 3(5), e2195
Finally, Chapter 5 reports that mnemonic processes can influence
the perception of motion, as demonstrated by the dynamic motion
aftereffect induced by static images previously associated with motion.
This chapter is based on the following manuscript.
Ryu, J.J., & Chaudhuri, A. (2009). Top-down influence on motion
perception: Dynamic motion aftereffects induced by static images
previously associated with unidirectional motion.
9
10
C ontribution of Authors
The contribution of the authors to the manuscripts on which this
thesis is based is as follows. Jae-Jin Ryu conceived the research
questions, designed and conducted the experiments, analyzed the data
and wrote the majority of the manuscripts. Karen Borrmann analyzed
the data described in Chapter 4 and wrote parts of the corresponding
manuscript. The supervisor, Dr. Avi Chaudhuri provided guidance
throughout the research process.
Chapter 1 General Introduction
Chapter 1
General Introduction
According to the bottom-up view of mnemonic processes,
perception of a visual input is often the first step in the construction of its
representations in memory (Magnussen, Greenlee, Asplund, & Dyrnes,
1991; Schacter, Norman, & Koutstaal, 1998). In this view, the effect of
visual perception on memory formation is emphasized. However, it is
also possible that once the representation of a visual object is formed in
memory, this representation could, in turn, influence the perception of
subsequent objects in a top-down manner. Compared to the body of
research investigating the bottom-up processes relating perception and
memory (Bentin, Moscovitch, & Nirhod, 1998; Busey & Loftus, 1998;
Craik, 2002; Magnussen et al., 1991; Medendorp, Tweed, & Crawford,
2003; Slotnick & Schacter, 2004; Sperling et al., 2001; Suzuki, Zola-
Morgan, Squire, & Amaral, 1993; Wagner, Koutstaal, & Schacter, 1999),
few have directly examined possible mnemonic influences on the
processing of incoming visual information. Therefore, the present thesis
sought to study ways in which different mnemonic processes affect
visual perception.
11
Chapter 1 General Introduction
Processing of visual information is believed to occur in a
hierarchical manner in multiple brain structures (Felleman & Van Essen,
1991; Konen & Kastner, 2008; Nassi & Callaway, 2006). If mnemonic
processes do indeed influence visual processing, one could ask further
questions regarding locations at which this interaction may occur. For
example, do mnemonic processes exclusively influence a selected
visual area or is this effect exerted across multiple regions? In order to
answer these questions, it is necessary to employ a method that is
applicable to various stages of visual processing in a consistent manner.
If mnemonic processes do influence perception, one can expect
that this influence would result in changes in visual perception. One
simple, yet powerful way to reveal transient changes in visual
processing is through the use of classical adaptation method. In a
typical adaptation experiment, an “adapting” stimulus is presented for an
extended period of time, producing a temporary distortion in the
perception of subsequent stimuli. This perceptual distortion, or
aftereffect, is attributed to the overall shift in response profile of neural
networks involved in the processing of the adapting stimulus.
Aftereffects have been found with a wide range of visual stimuli, from
simple lines (Gibson & Radner, 1937), to complex patterns such as
faces (Leopold, O'Toole, Vetter, & Blanz, 2001; Webster, Kaping,
Mizokami, & Duhamel, 2004). Indeed, the method of selective
12
Chapter 1 General Introduction
13
adaptation is often referred to as the psychologist’s microelectrode
because it allows researchers to make inferences about the activity of a
specific neural ensemble engaged in the processing of the adapting
stimulus (Frisby, 1980).
The adaptation method is a suitable tool for the purposes of the
current topic for two reasons. First, it can be used with various types of
visual stimuli that recruit different processing areas in the visual stream.
Second, it leads to the hypothesis that adapting stimuli with different and
similar mnemonic contents should produce different and similar visual
aftereffects, respectively. Despite these advantages, this method has
mainly been used to elucidate neural mechanisms mediating visual
perception and rarely has it been applied to investigate possible top-
down influences on visual perception.
In Chapter 2, I will review previous studies that examined other
top-down influences on visual processing. In Chapters 3, 4 and 5, I will
present how different manifestations of mnemonic processes, such as
familiarity, imagery and associative learning can influence perception of
various types of visual stimuli. The present thesis focuses on memory
processes that are mediated by the medial temporal lobe, which result in
conscious, explicit mnemonic representations (Tulving, 1987).
Chapter 2 Top-down Influences on Vision
Chapter 2
Literature review: Top‐down influences on vision
According to the bottom-up processing theory of sensory
information, it is the incoming perceptual input that gives rise to higher
cognitive processes. For example, conscious perception of a complex
image activates a number of brain regions involved in visual processing,
and this often results in the formation of its representation in memory.
However, it is also possible that these cognitive influences affect the
processing of sensory information in a top-down manner. In the current
chapter, different ways in which these top-down processes can have an
effect on visual perception are discussed.
Top-down processes refer to the general mechanisms that modify
or constrain processing of incoming perceptual information (Puce,
Allison, & McCarthy, 1999). In this context, the phrase “top-down
processes” encompasses a wide range of neural and behavioral
phenomena, ranging from mnemonic influences stemming from prior
knowledge and experiences, to introspective, subjective factors
including expectancies and motivation. Existing investigations show that
multiple aspects of visual perception are subject to these top-down
influences.
14
Chapter 2 Top-down Influences on Vision
Visual Search
One experimental manipulation that clearly reveals top-down
influences on perception is to limit the amount of time during which
visual stimuli are presented. This is often the case in a visual-search
study, in which participants are required to locate a target among
multiple distractor-items. The presentation duration of these stimuli is
often in the range of 100-200 ms, prompting participants to make a
quick response. The efficiency of visual search is reflected in decreased
response time.
The effect of top-down signals on visual search can be either
short-term or long-term. Response time for a search that was resumed
after a brief interruption is shorter than that of a newly initiated search
(Lleras, Rensink, & Enns, 2005). This response time benefit is attributed
to the retrieval of perceptual information stored in memory during the
initial search. This case of top-down influences on perception can be
considered short term because the top-down representations are
developed in a relatively short period of time (less than a few seconds).
Top-down influences on perception are also observed after long-
term learning, during which the nature of visual search changes from
15
Chapter 2 Top-down Influences on Vision
“serial” to “parallel” (Treisman & Gelade, 1980). When visual search is
serial, the reaction time is positively correlated with the number of
distractor-items in the visual display. However, when visual search
becomes parallel after extensive training, the detection of a target
occurs almost instantly, as if the target perceptually “pops-out” amongst
multiple distractor-items. Furthermore, the number of distractor-items
exerts relatively little impact on the reaction time. The representations of
the target item, developed after long-term, extensive prior learning,
enable almost instantaneous detection of the target in visual search
(Wang, Cavanagh, & Green, 1994 1994). The acquired representations
of target items that influence visual search are often perceptual in
nature, which are distinct from those acquired through explicit
associative learning (Korner & Gilchrist, 2008).
Top‐down influence: perception of ambiguous images
The influence of top-down signals also becomes conspicuous
when perception is ambiguous. The effect of prior knowledge and
experience on conscious perception is demonstrated in studies
examining visual identification of degraded or incomprehensible images
(Ramachandran, Ruskin, Cobb, Rogers-Ramachandran, & Tyler, 1994;
Snodgrass & Feenan, 1990; Snodgrass & Hirshman, 1994). These
studies commonly show that the identification of degraded images of an
16
Chapter 2 Top-down Influences on Vision
object is facilitated if an intact image of the same object is previously
presented.
Additional evidence showing the effect of top-down influences on
perception comes from studies reporting multi-stable phenomena.
These phenomena occur when multiple percepts are produced from a
single visual stimulus. A well-known visual stimulus producing multi-
stable percept is Boring’s my wife and my mother-in-law figure (Boring,
1930). The percepts produced by these images are highly subjective,
relying heavily on an individual’s expectations and prior knowledge
(Leopold & Logothetis, 1999).
Although both the perception of degraded images and multi-
stable phenomena are subject to top-down influences, the extents to
which the primary visual cortex mediates these processes appear to
differ significantly. The identification of objects in degraded images is
mainly modulated by the activity of high-level visual processing areas.
Neuroimaging investigations have revealed that the activity in medial
parietal cortex and the fusiform gyrus was increased, but there was no
change in the level of activity of the primary visual cortex when
degraded images of objects were presented before and after the
presentation of intact images (Dolan et al., 1997; Eger, Schweinberger,
Dolan, & Henson, 2005). Furthermore, the activity of these high-level
visual processing areas is specific to the type of stimuli shown. For
17
Chapter 2 Top-down Influences on Vision
example, when participants were asked to detect faces in progressively
degrading pure-noise images, face-specific fusiform face area (FFA)
showed a significant increase in activation. The activity of FFA was not
significantly different from baseline when non-face stimuli were detected
(Zhang et al., 2008). Top-down influences thus enhance the perception
of degraded images by modulating the activity of high-level visual
processing areas in a stimulus-specific manner.
In contrast to the patterns of activity produced by the perception
of degraded images, the primary visual cortex is actively engaged during
the perception of multi-stable images. Various studies report that the
activities of early visual areas are closely correlated with the switching of
percepts produced by these images (Parkkonen, Andersson,
Hamalainen, & Hari, 2008; Shulman et al., 1997). Similar patterns of
results have also been shown in the cases of binocular rivalry, in which
alternating percepts are produced due to incongruent inputs to the two
eyes (Leopold & Logothetis, 1999; Tong, Meng, & Blake, 2006).
Top‐down influence on perception of simple stimulus features and perceptual learning
Processing of simple stimulus features, such as depth, motion
and orientation is traditionally believed to be stimulus-driven and largely
immune to top-down influences (Pylyshyn, 1999). However, more recent
investigations show that perception of simple stimulus features can be
18
Chapter 2 Top-down Influences on Vision
influenced by previous experience. Evidence supporting top-down
influence on perception of simple stimulus features mainly comes from
studies examining perceptual learning. Perceptual learning refers to
improvement in perception of a sensory attribute through repeated
discriminatory trainings. The neural substrates of this type of implicit
learning are found outside of the medial temporal cortex (Gilbert &
Sigman, 2007). Consequently, perceptual learning is distinct from
explicit, conscious learning which requires the involvement of structures
in the medial temporal lobe.
Improved ability to discriminate a visual attribute due to extensive
training has been shown in the perception of orientation (Schoups,
Vogels, Qian, & Orban, 2001), direction of motion (Zohary, Celebrini,
Britten, & Newsome, 1994), and depth (Ramachandran & Braddick,
1973; Westheimer & Truong, 1988). In many cases, perceptual
sensitivity was accompanied by corresponding improvement in neuronal
sensitivity. For example, Zohary and colleagues measured behavioral
and neural responses to dynamic random dot stimuli with varying
coherence levels (Zohary et al., 1994). They report that perceptual
sensitivity to motion improved as a result of extensive training and that
this increase in motion sensitivity was accompanied by increased
sensitivity of motion-selective MT neurons. The increased perceptual
sensitivity accompanied with increased neuronal sensitivity has also
19
Chapter 2 Top-down Influences on Vision
been found with the perception of orientation and orientation-selective
V1 neurons (Schoups et al., 2001). These findings show that top-down
influence on visual processing occurs at both behavioral and
neurophysiological levels.
The mechanisms underlying the modified V1 activity due to top-
down influence appear to operate at two different levels. First, top-down
influences can produce intrinsic changes in response properties of
individual neurons. For example, in the case of orientation-selective
neurons, perceptual learning can cause sharpening or narrowing of their
tuning curves (Schoups et al., 2001). In addition, top-down signals can
also alter contextual tuning of V1 neurons, by changing the nature of the
lateral interactions in response to stimuli placed outside of the receptive
field (Crist, Li, & Gilbert, 2001; Li, Piech, & Gilbert, 2004). These
changes result in context-dependent response to the identical visual
stimuli. In V1, top-down signals mediate the intrinsic response properties
of visual neurons, as well as modulating networks that act on individual
neurons.
Results from previous studies show that visual processing is
affected by top-down influences. Out of many cognitive aspects that
comprise top-down influences, the current investigations focus on the
influence of mnemonic processes on perception. Visual perception is
20
Chapter 2 Top-down Influences on Vision
21
closely linked to the formation of memory (Baddeley, 1992; Magnussen
et al., 1991; Schacter et al., 1998). Using a selective adaptation method
to measure transient changes in perception, the influence of long-term,
declarative mnemonic representations and related processes on visual
processing is examined.
Chapter 3 Familiarity and Viewpoint Perception
Chapter 3
Representations of familiar and unfamiliar faces as revealed by viewpoint‐aftereffects
A bstract
A viewpoint-dependent aftereffect occurs after prolonged viewing of a
stimulus of a particular orientation, with the result that the test image is
perceived to be facing away from the adapting orientation. Prior
psychophysical work has led to the suggestion that the visual brain
encodes a limited range of viewpoint information with regard to complex
images. In this study, we investigated whether familiar faces were
susceptible to a viewpoint aftereffect. Familiar faces are believed to be
represented in a view-invariant manner, whereas unfamiliar faces are
represented in a viewpoint-dependent manner. Adaptation to both
familiar and unfamiliar faces influenced the perception of viewpoint of
subsequent face images. However, category-specific transfer of a
repulsive viewpoint-dependent aftereffect was observed with unfamiliar
faces. Our results suggest that neural networks that mediate viewpoint
information are also involved in view-invariant representation of familiar
faces.
22
Chapter 3 Familiarity and Viewpoint Perception
I ntroduction
Multiple encounters with faces rarely occur from identical vantage
points in real-life situations. However, humans are often able to
recognize the face of a familiar person despite significant changes in
viewpoint. The ability to recognize faces from different viewpoints is
limited when the observer is not familiar with the face (Burton, 1999;
Hancock, Bruce, & Burton, 2000; O'Toole, Deffenbacher, Valentin, &
Abdi, 1994). This differential ability to recognize familiar and unfamiliar
faces across different viewpoints has led to the suggestion that they are
represented in qualitatively different ways in the brain. Familiar faces are
believed to be represented in a view-invariant or abstract manner
whereas unfamiliar faces are represented in a viewpoint-dependent
manner (Bruce & Young, 1986; Burton, 1999; Eger et al., 2005; Hill,
Schyns, & Akamatsu, 1997).
It has been postulated that facial familiarity is acquired largely
through two processes – multiple exposures to a face and acquisition of
semantic information about the face (Bruce and Young, 1986; Burton et
al., 1999; Pourtois, Schwartz, Seghier, Lazeyras, & Vuilleumier, 2005).
In experimental settings, familiar faces are often equated with famous
faces whose semantic information can be easily retrieved (e.g., the face
of an actor or a well-known politician). In this context, the representation
23
Chapter 3 Familiarity and Viewpoint Perception
of a familiar face is believed to be linked to semantic information about
the identity of that face. Therefore, the abstract nature of
representations of familiar faces may be partly due to a strong cognitive
link to semantic information that is separate from visually driven
perceptual information. Representations of unfamiliar faces, on the other
hand, are more dependent on viewpoint because they are reliant upon
images obtained from prior encounters. The viewpoint from which these
encounters occurred may then determine how perceptual
representations of unfamiliar faces are formed.
The abstract nature of familiar face representations is
emphasized in several cognitive models of face processing (Bruce &
Young, 1986; Ellis, 1992; Valentine & Bruce, 1986). One influential
model has been proposed by Bruce and colleagues (Bruce & Young,
1986; Burton, 1999), in which representations of familiar faces are
composed of different units or nodes, with each node being responsible
for processing different types of information, including visual structure of
the face and its identity. Among the nodes is a pool of cognitive units
that is responsible for familiar-face recognition, known as Face
Recognition Units (FRUs). A notable feature of FRU is that they are
view-independent.
Recent findings from neuroimaging studies report distinct
patterns of activation in response to familiar and unfamiliar faces.
24
Chapter 3 Familiarity and Viewpoint Perception
Familiar faces produced a greater response in several brain areas,
including the left anterior middle temporal gyrus (Gorno-Tempini & Price,
2001), as well as other areas in the left hemisphere (Leube, Erb, Grodd,
Bartels, & Kircher, 2003; Paller et al., 2003). Interestingly, it appears that
areas sensitive to changes in viewpoints are different for familiar and
unfamiliar faces, possibly reflecting the different weights associated with
viewpoint-relevant information in facial representations. Pourtois and
colleagues (2005) conducted a study in which different images of
familiar and unfamiliar faces were shown. They reported that repeated
presentations of unfamiliar faces with varying viewpoints produced
selective repetition decreases in a medial portion of the right fusiform
gyrus, whereas repeated presentations of familiar faces from different
viewpoints produced a similar pattern of responses in the left middle
temporal and interior frontal cortex. These results reinforce behavioral
data as well as current models that suggest distinct encoding of
viewpoint information of familiar and unfamiliar faces.
One way to explore the behavioral relevance of viewpoint-
dependent versus viewpoint-independent representation is through a
classical adaptation approach. Recently, Fang and He (2005) showed
that adaptation to complex images of a particular orientation produced
an aftereffect that altered the perception of viewpoint. Their viewpoint
aftereffects were obtained with objects within the same categories and
25
Chapter 3 Familiarity and Viewpoint Perception
were greater when the adapting and test images were of the same
object or identity. What is particularly noteworthy is that they obtained
similar results with unfamiliar faces, suggesting that neural assemblies
that encode this information are susceptible to viewpoint-dependent
stimulus adaptation. The question then remains as to whether a similar
phenomenon arises with familiar faces, which has not been previously
examined.
Based on currently accepted theories of abstract representation
of familiar faces, we hypothesized that familiar faces are not susceptible
to a similar viewpoint-dependent aftereffect as was shown to be the
case for unfamiliar faces. If so, then the question arises as to the nature
of the aftereffect with familiar faces and whether it applies across
alternate exemplars within the same category. We show here that use of
a selective adaptation procedure produces view-dependent aftereffects
with familiar faces that are distinctly different than those with unfamiliar
faces, suggesting that neural assemblies that process viewpoint
information are recruited in the representation of familiar faces.
26
Chapter 3 Familiarity and Viewpoint Perception
Methods
P articipants
Six undergraduate students from McGill University participated in
each experiment (2 males, mean age of 21 for experiment 1; 1 male,
mean age of 21 for experiment 2). Participants were naïve to the
purpose of the experiment. All had normal or corrected-to-normal vision.
The study was reviewed and approved by an institutional ethics board
for human psychophysical studies. Written consent was acquired from
each participant prior to the experimental session.
A pparatus and stimuli
All stimuli were presented on an LG flat-screen monitor with 1024
x 768 resolution and 85 HZ refresh rate. The stimuli were presented on
a uniform grey background of 18.6 cd/m2. The presentation sequence
was programmed in MATLAB software using the Psychophysics
Toolbox extensions (Brainard, 1997). A chinrest was used to stabilize
the head position at a distance of 57 cm from the monitor surface.
Face images were acquired from the Max-Plank face database
(http://faces.kyb.tuebinggen.mpg.de). Adapting and test face stimuli
were created by projecting the 3-D images onto a two-dimensional plane
27
Chapter 3 Familiarity and Viewpoint Perception
with different in-depth orientation angles. Adapting stimuli were face
images oriented 300 to the left or right. The degree of orientation of the
adapting stimuli was chosen based on a previous study on objects and
unfamiliar faces (Fang & He, 2005), which reported the maximum
viewpoint-dependent aftereffect to occur at this orientation value. The
test stimuli included images in frontal view as well as off-frontal
orientation at 30 and 60 to the left and right. The size of all stimuli was 70
x 8.50.
P rocedure
Each participant completed three sessions – familiar, unfamiliar
and baseline. The familiar session began with a learning phase during
which four faces were repeatedly presented along with their fictional
names and occupations. Nine different views of each face were created
(frontal and 300, 450, 600, and 900 rotated to the left or right) and
presented in a sequential manner, twice clockwise and twice counter-
clockwise (Fig.1). Each image was presented for 1s. At the end of the
learning phase, a recognition test was conducted to verify the
participant’s familiarity with the faces. All participants were able to
achieve 100% person recognition before proceeding to the aftereffect
task.
28
Chapter 3 Familiarity and Viewpoint Perception
The aftereffect task consisted of the following regime.
Participants were first exposed for 5s to an adapting face image chosen
randomly from the four previously learned faces. A central fixation point
appeared at the end of this exposure for 2 s. Test stimuli were randomly
chosen from the five orientations (frontal; 30 and 60 to the left or right)
and presented at one of the four corners (upper left, upper right, lower
left and lower right) of the monitor for 400 ms in order to avoid possible
low-level, location-specific aftereffects. The center of the test image was
located at approximately 10.50 away from the central fixation point, at
one of four following angles = 450, 1350, 2250, or 3150. Participants were
allowed to alter their fixation to the test stimulus and report whether they
perceived it to be oriented to the left or right by way of a key press. An
inter-trial interval of 5s was used.
The adapting and test stimuli during the familiar session
consisted only of the four previously learned faces. In the unfamiliar
session, a battery of 16 novel faces was used. In Experiment 1, the
adapting and test stimuli for both familiar and unfamiliar sessions within
each trial were of the same face identity (e.g., the face of Joe, shown
from different viewpoints). In Experiment 2, the test faces in both
sessions were different from the adapting faces.
In the first experiment, each session consisted of 320 trials as
follows – 80 presentations of each familiar face during the familiar, and
29
Chapter 3 Familiarity and Viewpoint Perception
20 presentations of each unfamiliar face during the unfamiliar session,
both divided equally between right and left adapting orientations. In the
second experiment, the familiar session consisted of 360 trials – 90
presentations of each familiar face, divided equally between right and
left adapting orientations. As with the first experiment, each unfamiliar
face in the unfamiliar session was presented 20 times for a total of 320
trials. The order of familiar and unfamiliar sessions was counterbalanced
across participants.
The faces used in both sessions were presented without
adaptation during the baseline session. Participants were asked to
decide which direction the test stimuli were facing (left or right). The
baseline session was only administered after the two sessions were
completed.
Results
E xperiment 1
The proportion of trials in which the test stimuli were perceived to
be facing the opposite direction relative to the adapting stimuli is plotted
against orientation angles of test stimuli and shown in Fig. 2. The
logistic function, 1/(1+exp( - *)) was fitted to the data. and are
free parameters that determined the midpoint and the slope of the
30
Chapter 3 Familiarity and Viewpoint Perception
psychometric function. The orientation angles of the test stimuli were
labeled with respect to those of the adapting stimuli such that they were
the same or opposite to the direction of adapting stimuli. The points of
subject equality were extrapolated at the threshold of 0.5 from the
psychometric function for each experimental session and indicated as U
for the unfamiliar and F for the familiar session.
Baseline scores were calculated based on orientation-
discrimination accuracy. Paired T-tests between accuracy scores
obtained for test stimuli oriented 30 to the left and right, and 60 to the left
and right revealed no significant differences in the baseline perception of
these stimuli.
The bias in perception produced by a viewpoint-dependent
aftereffect has been found to be in the opposite direction to the adapted
viewpoint (Fang & He, 2005). Therefore, the repulsive bias in perception
was more likely to be observed with test stimuli that are in frontal view or
those oriented in the same direction as adapting stimuli. Indeed, with
these test stimuli, a consistent leftward shift from baseline scores was
observed in both experimental sessions.
In order to examine possible differences in viewpoint-dependent
aftereffects observed in familiar and unfamiliar sessions, the respective
differences from baseline at the selected test stimuli (Same 6, Same 3,
0) were submitted to a two-way ANOVA (session x test stimuli). Main
31
Chapter 3 Familiarity and Viewpoint Perception
effects of session and test stimuli were both significant (F(1,5) = 114.86,
p < 0.001 for session; F(2, 10) = 5.31, p < 0.05 for test stimuli;
Greenhouse-Geisser correction). Interaction between the two factors
was not significant.
Test stimuli oriented in the same direction as the adapting stimuli
were more likely to be perceived to be facing away from the adapting
stimuli. Adapting to both familiar and unfamiliar faces produced
repulsive viewpoint-dependent aftereffects. However, this shift in
perception of viewpoints was shown to be greater following adaptation
to a familiar face. These viewpoint-dependent aftereffects were obtained
when the adapting and test stimuli were of the same face identities.
A notable feature of the viewpoint-dependent aftereffect is that it
transfers across different exemplars within the same category (Fang &
He, 2005). A second experiment was therefore conducted to investigate
whether the viewpoint-dependent aftereffect induced by a familiar face
influences the subsequent perception of a different familiar face.
E xperiment 2
In this experiment, the adapting and test images were of different
identities in both familiar and unfamiliar sessions. The familiar session
consisted of 360 trials, and the unfamiliar, 320 trials. All other aspects
and parameters of the experiment were identical to Experiment 1.
32
Chapter 3 Familiarity and Viewpoint Perception
The proportion of trials in which test stimuli were perceived to be
oriented in the opposite direction to the adapting stimuli is shown in Fig.
3. The psychometric function for the performance in the unfamiliar
session showed a consistent leftward shift from the baseline
performance across all test stimuli.
Familiar adapting faces, on the other hand, appeared to have
induced a significant disruption in the subsequent perception of
viewpoint, as the performance was near the chance level across all test
stimuli. Indeed, a logistic function was not able to fit the data due to the
relatively constant level of performance across the test stimuli. A
repeated ANOVA on the performance from the familiar session revealed
a non-significant main effect of the test stimuli.
D iscussion
We investigated the effect of familiarity on the viewpoint
aftereffect phenomena by using a selective adaptation approach with
both familiar and unfamiliar faces. When adapting and test stimuli were
of the same identity, adaptation to familiar and unfamiliar faces viewed
from a particular angle produced similar shifts in the subsequent
perception of viewpoint. However, when different faces were used for
adapting and test stimuli, familiar adapting images produced a
33
Chapter 3 Familiarity and Viewpoint Perception
viewpoint-dependent aftereffect that was qualitatively distinct from that
produced by unfamiliar adapting images, Category-specific transfer of a
systematic viewpoint-dependent aftereffects was observed with
unfamiliar faces. Together, our results suggest that repulsive viewpoint-
dependent aftereffects produced by familiar faces are identity-specific.
Our failure to obtain a systematic within-category transfer of the
viewpoint-dependent aftereffect with familiar faces may be attributed to
the additional processing of the changed, familiar identities. The
presentation of a different, yet familiar, test face after prolonged
exposure to a familiar adapting face may cause activation of semantic
information associated with the newly presented face. This new
activation of information may have interfered with the processing of the
viewpoint information, thus producing the near-chance performance
when the identities of the adapting and test faces were different.
Familiar faces are believed to be represented in a view-invariant
manner (Bruce & Young, 1986). Given the discovery of viewpoint-
dependent aftereffects with complex images (Fang & He, 2005), we
asked whether a similar effect persists with familiar faces and if so,
could the nature of the phenomenon provide further insight into the
neural mechanisms that mediate the abstract nature of familiar faces.
Fang and He (2005) suggested on the basis of their results that neurons
34
Chapter 3 Familiarity and Viewpoint Perception
mediating the perception of viewpoint are organized in a manner similar
to orientation-selective neurons in earlier cortical areas.
We sought to examine whether the existence of view-invariant
representations of familiar faces are susceptible to viewpoint-dependent
aftereffects. Our finding that selective adaptation of familiar faces
influenced the subsequent perception of viewpoint suggests that the
neural assemblies mediating familiar face perception are functionally
linked to biological processing of viewpoint information. However, the
manner in which these neurons are activated in response to familiar and
unfamiliar faces appears to differ, as suggested by the distinct nature of
the viewpoint-dependent aftereffect induced by these separate images.
A systematic transfer of the aftereffect to other faces was observed with
unfamiliar faces, thus replicating findings from a previous study (Fang &
He, 2005), which suggested category-specificity of a viewpoint-
aftereffect. However, a similar within-category transfer of the aftereffect
was not observed with familiar faces.
The bias in perception induced by adaptation has long been
attributed to decreased sensitivity of neurons selectively recruited during
adaptation (McCollough, 1965; Wenderoth & Johnstone, 1987; Yoshida,
1978). The brief impairment in perception of orientation following
selective adaptation to familiar face images suggests that neurons
processing information about viewpoints were involved in the processing
35
Chapter 3 Familiarity and Viewpoint Perception
of familiar adaptor images. Significantly reduced orientation-judgment
performance following adaptation to different but familiar face images
provides support for this argument.
Facial familiarity is achieved through the acquisition of semantic
information and multiple exposures to the images under different
viewing conditions (Bruce & Young, 1986). The accumulation of different
images of a familiar face is likely to be crucial in the formation of an
abstract representation of the face. Once the abstract, view-invariant
representation has formed, the overall activation of viewpoint-selective
neurons may provide easier access to semantic information, enabling
identification of the face despite alterations in viewpoint. An important
and unanswered question in face perception research concerns the
transitional nature of the neural representation as unfamiliar faces
become familiar and the corresponding conversion from a view-
dependent to a view-invariant representation.
C onclusions
When adapting and test stimuli were of the same identity,
adaptation to familiar and unfamiliar faces produced similar viewpoint-
dependent aftereffects, suggesting the involvement of viewpoint-
selective neurons in the processing of both types of face stimuli.
36
Chapter 3 Familiarity and Viewpoint Perception
However, category-specific transfer of a systematic viewpoint aftereffect
was observed only with unfamiliar faces. This may be due to the shift in
attention to the changed, familiar identity, subsequently disrupting the
perception of viewpoint. The present results provide behavioral support
for the notion of differential weights attached to viewpoint information
contained in representations of familiar and unfamiliar faces (Pourtois et
al., 2005).
When an unfamiliar identity becomes familiar, semantic
information concerning that identity (names, occupation) is often
associated with the corresponding visual image. Accordingly, multiple
encounters with the familiar identity eventually lead to the generation of
visual imagery of the face when relevant semantic information about the
identity is presented. Considerable evidence from neurophysiological
investigations shows that biological basis for imagery of familiar faces is
similar to those mediating visual perception (Ishai & Sagi, 1995;
Kosslyn, Thompson, & Alpert, 1997; Kreiman, Koch, & Fried, 2000;
O'Craven & Kanwisher, 2000). The purpose of the following study is to
further probe neural networks underlying perception and imagery of
familiar face identities using the selective adaptation method.
37
Chapter 3 Familiarity and Viewpoint Perception
A cknowledgments
We wish to thank Dr. Alain Mignault for help in creating the face
stimuli, and Karen Borrmann for helpful discussions. We thank two
anonymous reviewers for their extremely helpful comments on an earlier
version of this paper. This study was funded by operating grants from
the Canadian Institutes of Health Research (CIHR) to A.C.
Figure Legends
Figure 1. Different images of the same face presented during the
learning phase (Familiar session).
Figure 2. The mean psychometric functions for viewpoint judgments
under each viewing condition. Proportion of trials in which test images
were perceived to be facing opposite to the direction of the adapting
stimuli was plotted against different test stimuli. The solid horizontal line
indicates threshold for the point of subjective equality (.5). The point of
subjective equality was extrapolated for each experimental session (U
for unfamiliar, F for familiar). Bars indicate standard errors.
38
Chapter 3 Familiarity and Viewpoint Perception
Figure 3. Results from Experiment 2 in which adaptor and test images
were of different faces. The mean psychometric functions for viewpoint
judgments for the baseline and unfamiliar conditions. A logistic function
was not able to fit the data from the familiar condition. The solid
horizontal line indicates the threshold for the point of subjective equality
(.5). The point of subjective equality was extrapolated for the unfamiliar
condition only. Bars indicate standard errors.
39
Chapter 3 Familiarity and Viewpoint Perception
Figure 1
40
Chapter 3 Familiarity and Viewpoint Perception
Figure 2
41
Chapter 3 Familiarity and Viewpoint Perception
42
Figure 3
Chapter 4 Face Identity Aftereffects and Imagery
Chapter 4
Imagine Jane and Identify John: Face Identity Aftereffects Induced by Imagined Faces
A bstract
It is not known whether prolonged exposure to perceived and imagined
complex visual images produces similar shifts in subsequent perception
through selective adaptation. This question is important because a
positive finding would suggest that perception and imagery of visual
stimuli are mediated by shared neural networks. In this study, we used a
selective adaptation procedure designed to induce high-level face-
identity aftereffects – a phenomenon in which extended exposure to a
particular face facilitates recognition of subsequent faces with opposite
features while impairing recognition of all other faces. We report here
that adaptation to either real or imagined faces produces a similar shift
in perception and that identity boundaries represented in real and
imagined faces are equivalent. Together, our results show that identity
information contained in imagined and real faces produce similar
behavioral outcomes. Our findings of high-level visual aftereffects
induced by imagined stimuli can be taken as evidence for the
43
Chapter 4 Face Identity Aftereffects and Imagery
44
involvement of shared neural networks that mediate perception and
imagery of complex visual stimuli.
Chapter 4 Face Identity Aftereffects and Imagery
I ntroduction
An encounter with a familiar person’s name often generates the image of
that person in our mind. The process by which an image is created without
actual retinal input is referred to as visual imagery. Although there are reports of
patients with deficits of perception but intact imagery (Behrmann, Moscovitch, &
Winocur, 1994; Michelon & Biederman, 2003), multiple lines of evidence from
behavioral (Farah, 1985; Ishai & Sagi, 1995), and neuroimaging (Ishai,
Ungerleider, & Haxby, 2000; Kosslyn, Thompson, & Alpert, 1997; O'Craven &
Kanwisher, 2000) studies suggest that the properties and neural substrates of
imagery are similar to those of perception. The similarities in neural structures
underling imagery and perception are further corroborated by an
electrophysiological study showing that single neurons in the human medial
temporal lobe respond to both imagery and perception (Kreiman, Koch, & Fried,
2000).
It is possible also to use psychophysical methods to study the neural
mechanisms underlying perception and imagery. One approach is to use
selective adaptation to directly probe the biological basis of cognitive function. In
a selective adaptation experiment, an adapting stimulus is presented for an
extended period of time, resulting in a temporary perceptual distortion, or
aftereffect (Koehler & Wallach, 1944). Aftereffects have been found with a wide
45
Chapter 4 Face Identity Aftereffects and Imagery
range of visual stimuli, from simple lines (Gibson, 1933; Gibson & Radner,
1937), to complex patterns such as faces (Leopold, O'Toole, Vetter, & Blanz,
2001; Webster, Kaping, Mizokami, & Duhamel, 2004). It is believed that
sustained activity of neurons during adaptation causes a shift in their
subsequent response level, leading to a perceptual bias towards opposite (or
complementary) stimulus attributes (Frisby, 1980). The method of selective
adaptation is often referred to as the psychologist’s microelectrode because it
allows researchers to make inferences on the activity of neurons engaged in the
processing of adapting stimuli (Frisby, 1980).
The extant physiological evidence that similar neural structures are
involved in perception and imagery leads to the hypothesis that perceived and
imagined stimuli should produce similar behavioral results in a selective
adaptation study. Despite this expectation, the results from several prior
experiments have been inconsistent, reflecting the difficulty associated with
using imagined visual stimuli in experimental settings (Broerse & Crassini, 1980;
Finke & Schmidt, 1977, 1978; Moradi, Koch, & Shimojo, 2005; Over & Broerse,
1972; Singer & Sheehan, 1965). Some of these studies only examined
aftereffects induced by imagined stimuli with simple visual attributes, such as
color and orientation, which are believed to activate early areas in the visual
processing stream (Hubel & Wiesel, 1968). It is experimentally challenging to
control various visual attributes such as precise hue, orientation and size of
imagined stimuli (Finke & Schmidt, 1978; Singer & Sheehan, 1965).
46
Chapter 4 Face Identity Aftereffects and Imagery
Furthermore, low-level visual aftereffects are sensitive to a variety of
manipulations including changes in the size and position of adapting stimuli
(Koehler & Wallach, 1944). While problems that are inherent in the use of simple
stimuli may partly explain some of the inconsistencies found in the extant
literature, a previous study which did not find significant visual aftereffects
induced by imagined complex stimuli such as faces (Moradi et al., 2005)
warrants further considerations of other factors such as participants’ familiarity of
the experimental tasks and stimuli.
In this study, we sought to directly probe the neural networks that underlie
visual imagery and perception by inducing high-level, face-identity aftereffects
(FIA) through selective adaptation. FIA occurs when adaptation to a particular
face facilitates identification of subsequent faces with opposite features (anti-
faces) while impairing identification of unrelated faces (Leopold et al., 2001).
Unlike aftereffects induced by low-level stimuli, the biological mechanisms
mediating FIA are invariant to changes in stimulus size, position, and orientation
(Leopold et al., 2001; Watson & Clifford, 2003).
For proper comparison of high-level aftereffects induced by imagined and
perceived stimuli, the task requirements in both conditions should be identical. In
order to ensure that the difficulty of imagining complex stimuli would not interfere
with task performance, we used a fixed-order, within-subject design in which the
aftereffect task with perceived stimuli preceded that with imagined stimuli. In
addition, we added a discrimination task to ensure the identity information
47
Chapter 4 Face Identity Aftereffects and Imagery
contained in perceived and imagined faces was equivalent. We hypothesized
that if perception and imagery were indeed mediated by shared neural networks
then adaptation to real and imagined faces would produce a similar bias in
perception of subsequent faces.
Materials and Methods
P articipants
Ten undergraduate students from McGill University participated in the
study (2 males, mean age of 19.1 years). Participants were naïve to the purpose
of the experiment. All had normal or corrected-to-normal vision. The study was
reviewed and approved by an institutional ethics board for human
psychophysical studies. Written consent was acquired from each participant
prior to the experimental session. Data from two participants whose baseline
identification accuracy of test faces with 45% identity strength did not exceed
75% during the aftereffect tasks were removed from analysis.
A pparatus and stimuli
All stimuli were presented on an LG flat-screen monitor with 1024 x 768
resolution and 85 Hz refresh rate. The stimuli were centrally presented on a
uniform black background. The presentation sequence was programmed in
MATLAB software using the Psychophysics Toolbox extensions (Brainard,
48
Chapter 4 Face Identity Aftereffects and Imagery
1997). A chinrest was used to stabilize head position at a distance of 57 cm from
the monitor surface. All experiments were carried out in a dark testing room. The
size of stimuli did not exceed 7 x 10.5 degrees.
Three face/anti-face pairs were taken from those used in the previous
study reporting FIA (Leopold et al., 2001). According to the face space model,
the anti-face of a face is located on the same identity axis but on the other side
of the mean on the computationally derived, multi-dimensional face space
(Leopold et al., 2001; T. Valentine, 1991). Therefore, the facial features of the
face/anti-face pair are completely opposite to each other. The identity strength of
each face was manipulated by adjusting its distance from the average face on
the face space. Identity strength of the face stimuli ranged from 0 (the average
face) to 45%. Due to the computational processes through which anti-faces are
generated, the maximal identity strength of an anti-face was 45%.
P rocedures
The experiment was divided into real- and imagined- stimulus conditions.
Both conditions began with a training session in which participants were
familiarized with the face stimuli. The identity strength of the faces was 45%.
Each face appeared for five seconds on the monitor along with its fictional name
(6 faces in total). The serial presentation of the face was repeated seven times
and the order of presentation was randomized. The participants’ familiarity with
49
Chapter 4 Face Identity Aftereffects and Imagery
the faces was probed with a verbal identification task in which each face was
presented without its name. The presentation of the faces and identification task
were repeated until 100% accuracy on the identification task was achieved.
In order to maximize the participants’ familiarity with the anti-faces during
the imagined-stimulus condition, all participants completed the real-stimulus
condition before proceeding to the imagined-stimulus condition. In the imagined-
stimulus condition, a discrimination task (described below) was added and was
administered before the aftereffect task. In both real- and imagined-stimulus
conditions, the baseline task, in which participants were required to identify test
faces without adaptation, was completed after the aftereffect task.
Aftereffect tasks
In each trial of the aftereffect task, a five-second presentation of the
adapting face was followed by a brief presentation (400 ms) of a morphed test
face (Figure 1). There was no ISI between the adapting and test images. The
adapting face was one of the three anti-faces. The identity strength of a test face
used in the aftereffect task ranged from 0 to 45%. The identity strength of anti-
faces was 45%. In the real-stimulus condition, the adaptor was a true face
image, whereas in the imagined-stimulus condition a name served as a cue
prompting participants to vividly visualize the corresponding face with their eyes
open. After adaptation, observers were asked to identify the test face among
three shown face names and indicate their answers by pressing the appropriate
50
Chapter 4 Face Identity Aftereffects and Imagery
button on the keyboard. Each adapting anti-face was shown 100 times during
the aftereffect task, for the total of 300 trials in each condition.
The aftereffect tasks were composed of “matching” trials, in which the
features of adapting and test stimuli were opposite to each other (face/anti-face
pair) and “non-matching” trials, in which the two face stimuli were not
perceptually related. An equal number (150) of matching and non-matching trials
was randomly interleaved. To reduce a possible learning effect, we administered
a baseline task after the respective aftereffect tasks. The baseline task required
participants to identify test faces without adaptation. The test faces, as well as
the number of trials presented during the baseline task, were identical to those
shown during the aftereffect task.
Discrimination Task
To assess possible differences in the degree of face identity contained in
real and imagined stimuli, we sought to measure the difference threshold for
identities in real and imaged faces in a discrimination task (Figure 2). In this
task, participants were instructed to compare either the real or the mental
images of the anti-faces (Face 1) with subsequent real face (Face 2).
Participants indicated whether the two face stimuli (Faces 1 & 2) belong to the
same person with a key press. Face 2 matched the identity of Face 1 (i.e., both
faces can be found on the same identity trajectory on the face-space), but the
identity strengths of Face 2 stimuli were less than those of the Face 1 stimuli:
The identity strengths of Face 1 stimuli were 45% whereas the identity strengths
51
Chapter 4 Face Identity Aftereffects and Imagery
of Face 2 stimuli varied from 5 to 40%. Each Face 2 stimulus appeared 5 times
with each of the real and imagined Face 1 stimuli. Participants completed 180
trials in total.
R esults
Data were averaged over 8 participants whose baseline identification
accuracy of test faces with 45% identity strength exceeded 75%. A logistic
function 0.333+0.667*1/(1+exp(-(x-c)/a)) was fitted to the data, in which ‘a’ and
‘c’ are free parameters that determine the slope and midpoint of the
psychometric function.
A ftereffect tasks
At the end of the imagined-stimulus condition, all participants reported to
be able to imagine a corresponding face during the adapting period. The
imagined-stimulus condition was preceded by the real-stimulus condition for all
participants. Therefore, their familiarity with the face stimuli was expected to be
greater in the imagined-stimulus condition. The baseline tasks in both conditions
involved identification of the morphed test faces without adaptation. The
increased familiarity with the stimuli in the imagined-stimulus condition was
reflected by the significantly increased performance on the baseline task
compared to that of the real-stimulus condition (Greenhouse-Giesser correction
for sphericity, F (1,7) = 8.313, p < 0.05). Due to the difference in the baseline
52
Chapter 4 Face Identity Aftereffects and Imagery
performance between the real- and imagined- stimulus conditions, the
differential effects of adaptation to matching (adapting face is the anti-face of the
test face) and non-matching (adapting and test faces do not have opposite
features) faces were analyzed for each stimulus condition.
In the real-stimulus condition, there was a significant increase in
identification performance after adaptation to “matching” faces compared with
the baseline condition. In contrast, identification performance after adaptation to
“non-matching” faces was diminished compared to baseline (Figure 3). The
fractions of trials in which participants correctly identified the test face in
matching and non-matching trials were compared to the baseline performance.
The respective differences in identification performance between the trial types
(matching and non-matching) and baseline were subjected to a repeated-
measure two-way analysis of variance (ANOVA) with trial type and identity
strength as factors. The main effect of trial type (F (1,7) = 61.536, p < 0.001),
and interaction effect (F (2,832, 19.824) = 11.364, p < 0.001) were significant.
The main effect of identity strength was marginally significant (F (4.467, 31,266)
= 3.651, p< 0.05).
The imagined-stimulus condition produced a similar pattern of results to
the real-stimulus condition (Figure 4). An ANOVA revealed a significant main
effect of trial type (F(1,7) = 28.613, p < 0.01) and interaction effect (F (3.981,
27.864) = 4.051, p < 0.001). The main effect of identity strength was not
significant (F(3.607, 25.251) = 1.397, p > 0.05).
53
Chapter 4 Face Identity Aftereffects and Imagery
D iscrimination Task
The proportion of trials in which participants perceived the second face
(Face 2) differently from the anti-face (Face 1) on the discrimination task was
plotted against the difference in the identity strengths of the two faces (Figure 5).
The anti-face was either perceived or imagined. The data was fitted to a logistic
function in order to extrapolate difference thresholds (75% different responses)
for identifying visual and imagined faces. The difference thresholds were
identical at 13%.
D iscussion
We have used selective adaptation methods designed to induce face-
identity aftereffects (FIA) to test psychophysically whether face perception and
imagery are processed by shared neuronal ensembles. We found that
adaptation to physically presented matching anti-face images enhanced the
recognition of test faces, whereas adaptation to anti-face images that were not
matched to the test face resulted in reduced recognition performance compared
to baseline. These effects can be seen from the corresponding shifts of the data
points in Figure 3. Adaptation to imagined matching and non-matching anti-
faces produced similar results – i.e., a significant increase in identification of test
54
Chapter 4 Face Identity Aftereffects and Imagery
faces after adaptation to imagined ‘matching’ anti-faces and reduced
identification performance relative to baseline after adaptation to non-matching
imagined anti- faces (Figure 4).
Compared to the condition in which adapting stimuli were physically
presented, the magnitude of shift from the corresponding baseline was much
smaller when participants were asked to imagine the adapting stimuli (Figures
3a & 4a). The apparent reduction of the aftereffect following adaptation to
imagined stimuli could be explained by the difficulty in visualizing a complex
image in a sustained, coherent manner. This may have led to diminished
activation of neuronal ensembles that otherwise show greater response to visual
stimuli. The decreased activation of these neurons may have produced a
smaller net adaptation leading to a smaller aftereffect.
We also measured the respective difference thresholds for identity
strength in a discrimination task to investigate this possibility and examine
potential differences between the properties of real and mental images of
learned faces. If imagined faces indeed had wider identity boundaries than real
images of those same faces, this should be reflected in larger difference
thresholds for imagined faces. However, we found that difference thresholds for
real and imagined faces were the same, thus showing that the identity
represented in the imagined-stimulus condition was similar to that contained in
the real-stimulus condition.
55
Chapter 4 Face Identity Aftereffects and Imagery
We used a fixed-order, within-subject design to achieve participants’
maximum familiarity with the experimental task and stimuli during adaptation to
imagined stimuli. It is possible that the order in which the stimulus conditions
were presented could have influenced participants’ performance. Indeed, the
increased familiarity with the face stimuli during the imagined-stimulus condition
could have contributed to the reduced magnitude of aftereffect, as the baseline
performance during the imagined-stimulus condition was significantly increased.
However, despite the increased familiarity with the face stimuli, adaptation to
non-matching imagined faces still reduced recognition performance, suggesting
that the adaptation effect of non-matching imagined faces was comparable to
that of real faces.
It is interesting to note that a similar study conducted by Moradi et al.,
(2005) failed to report a significant high-level aftereffect induced by imagined
faces. Due to the top-down nature of imagery, it is absolutely necessary to
minimize possible cognitive and perceptual interferences during adaptation to
imagined stimuli. In our task, the presentation of the name of an adapting face
was brief, merely serving as cue for participants to imagine the corresponding
face. Consequently, no visual stimulus was presented during adaptation. On the
other hand, the name of an adapting face continued to be shown during
adaptation in the study by Moradi et al. It is possible that continuous visual input
during adaptation could have interfered with imagery, resulting in non-significant
aftereffects.
56
Chapter 4 Face Identity Aftereffects and Imagery
This notion that neuronal responses are diminished during adaptation to
an imagined stimulus is reinforced by a neuroimaging study that compared
fusiform face area (FFA) and parahippocampal place area (PPA) activation
during viewing or imagery of faces and places (O'Craven & Kanwisher, 2000).
This study showed that both perception and imagery of faces selectively
activated portions of FFA, whereas viewing and imagery of places produced
greater activation in PPA. Interestingly, within a region responding more strongly
to a given stimulus category, O’Craven and Kanwisher (2000) also reported
stronger levels of activation for real compared to imagined stimuli of that
category. This finding is consistent with our finding of a larger magnitude of FIA
for real as compared to imagined faces.
We have shown that adaptation to a visually presented anti-face and to
an imagined anti-face produces similar perceptual aftereffects. The occurrence
of such a high-level visual aftereffect from a purely mental image reveals that a
close neural interaction exists between visual perception and imagery. The
processing of complex visual stimuli such as faces is believed to be specialized
in the later stages of the occipito-temporal visual processing stream (Ishai,
Ungerleider, Martin, Schouten, & Haxby, 1999; Kanwisher, McDermott, & Chun,
1997). Accordingly, a recent neuroimaging study investigating the neural activity
underlying FIA reported that areas in the anterior temporal lobe are involved in
the mediation of the effect (Furl, van Rijsbergen, Treves, & Dolan, 2007). Given
the similarity in the shift of perception following adaptation to imagined and real
57
Chapter 4 Face Identity Aftereffects and Imagery
faces, these areas are likely to be involved in producing FIA induced by
imagined faces.
Representations of a familiar face not only contain information about its
visual attributes, but also semantic information regarding its identity. It appears
that the two types of information are closely linked together. Support for this idea
comes from a single-neuron study that reported neurons in the human medial
temporal cortex that responded to both the presentation of a familiar face as well
as the proper name associated with the face (Quiroga, Reddy, Kreiman, Koch, &
Fried, 2005). Since our aftereffect tasks involved perception and identification of
familiar faces, some aspects of memory processes may have contributed to our
results. It is likely that both identity and visual information are activated during
adaptation to perceived and imagined familiar faces.
Our results are consistent with neuroimaging studies that have shown
selective activation of stimulus-specific brain regions in extrastriate cortex
following exposure to both real and imagined stimuli. Similar activity following
presentations of real and imagined stimuli have been found in face- (Ishai et al.,
2000; O'Craven & Kanwisher, 2000), object- (Kosslyn et al., 1997), and place-
selective (O'Craven & Kanwisher, 2000) regions. However, our evidence
supporting the commonality between neural structures underlying perception
and imagery appears to be in conflict with previous reports of patients with
dissociable deficits (Bartolomeo et al., 1998; Behrmann et al., 1994; Michelon &
Biederman, 2003). These patients showed severely impaired perception but
58
Chapter 4 Face Identity Aftereffects and Imagery
more intact imagery of complex objects. It is possible, however, that intact
imagery of these patients was mainly due to the retrieval of representations of
objects acquired before the lesion. In that case, imagery tasks are likely to
measure the patient’s ability to remember, rather than to imagine what was just
perceived. Although these studies provide interesting insight into the overall
brain networks supporting perception and imagery, the dissociable deficits found
in patients do not necessarily provide support for separate neural structures
mediating these two experiences.
Our finding of equivalent identity boundaries for visual and mental images
suggests that face imagery activates robust and accurate face representations
that are similar to those produced by visual stimulation. We propose that the
similarity in identity contained in imagined and real faces is produced by
activation of shared neural networks that code for these representations.
C onclusions
Evidence from the present study suggests that overlapping neural
networks mediate perception and imagery of familiar face identities. As
mentioned before, familiarity with a facial identity develops as a result of formed
association between a face image and corresponding semantic information
about the identity (i.e., the name of the face). Therefore, visual imagery of the
familiar face upon the presentation semantic information becomes possible.
59
Chapter 4 Face Identity Aftereffects and Imagery
Associative learning can be considered an essential process responsible
for building mnemonic representations. The current study showed that
association formed between semantic and visual information about face
identities ultimately influences perception of face images. If semantic information
and a visual image can be associated in memory, it may also be possible that
similar associations are formed between two different visual images. Indeed,
various behavioral and neurophysiological studies have provided evidence for
this type of association (Messinger, Squire, Zola, & Albright, 2001; Miyashita,
1988; Miyashita, Kameyama, Hasegawa, & Fukushima, 1998; Schlack &
Albright, 2007). However, what remains unanswered is whether association
between different visual images also exerts similar influence on perception in a
selective adaptation task. The following chapter describes a study that
attempted to provide answers for this question.
60
Chapter 4 Face Identity Aftereffects and Imagery
Figure Legends
Figure 1. Trial sequence in the aftereffect tasks during the real- and imagined-
stimulus conditions. A five-second presentation of the adapting stimulus was
followed by a brief presentation of a test face. After adaptation, observers were
asked to identify the test face among three shown faces.
Figure 2. A trial in the discrimination tasks on real and imagined stimuli.
Participants were asked to judge whether either the real or imagined anti-face
and test face belonged to the same person. The identity strengths of the test
faces were varied to be less than those of the anti-faces.
Figure 3. Sensitivity to face identity in real-stimulus conditions.
The logistic function 0.333+0.667*1/(/(1+exp(-(x-c)/a)) was fitted to the data, in
which ‘a’ and ‘c’ are free parameters that determine the midpoint and the slope
of the psychometric function. The fraction of trials in which participants correctly
identified the test face is plotted in relation to the identity percentage contained
in test faces. Data from baseline with no adaptation (squares), adaptation to
matching anti-face (triangles), and adaptation to non-matching anti-face (circles)
are shown. The recognition threshold for each condition was taken to be the
61
Chapter 4 Face Identity Aftereffects and Imagery
inflection point of the corresponding cure and shown accordingly. (a) Average of
all participants. Standard errors are shown. (b and c) Individual data from two
participants.
Figure 4. Results from imagined-stimulus conditions.
(a) average of all participants, with corresponding threshold for each condition.
Standard error of mean (SEM) are shown. (b & c) Data from two individual
participants. (b & c) Data from two individual participants.
Figure 5. Difference thresholds for real (triangles) and imagined (squares) faces.
The fraction of trials in which participants perceived the test face to be different
from the previously learned face is plotted in relation to identity difference
between the anti- and test faces. Difference thresholds (75% different
responses) for the two psychometric functions for visual and imagined faces are
identical.
62
Chapter 4 Face Identity Aftereffects and Imagery
Figure 1
63
Chapter 4 Face Identity Aftereffects and Imagery
Figure 2
64
Chapter 4 Face Identity Aftereffects and Imagery
Figure 3
65
Chapter 4 Face Identity Aftereffects and Imagery
Figure 4
66
Chapter 4 Face Identity Aftereffects and Imagery
67
Figure 5
Chapter 5 Motion Aftereffects and Stable Images
Chapter 5
Dynamic motion aftereffects induced by static images previously associated with unidirectional motion
A bstract
Current neurophysiological evidence suggests that it is possible to alter stimulus
selectivity of neurons of MT through associative learning, such that they show
increased firing with stable images that were previously paired with motion
(Schlack & Albright, 2007). This finding leads to the question of whether the
neurological association between static and dynamic stimuli has a
corresponding impact on perceived motion. We measured changes in perceived
motion after adaptation to static shapes that were previously associated with
unidirectional motion. We report that a dynamic motion aftereffect was evident
after adaptation to the static images. A delay of 3.5 seconds following adaptation
to the static images or moving dots significantly decreased the magnitude of the
effect. A dynamic MAE was also produced after adaptation to static images of
arrows without any explicit associative learning. Our results show that
associative influence can alter the perception of motion in a top-down manner.
68
Chapter 5 Motion Aftereffects and Stable Images
I ntroduction
It is well known that the visual area MT is crucial for the processing of
dynamic stimuli. Various neurophysiological studies have shown that this area is
active during the perception of motion (Albright, 1984; Maunsell & Van Essen,
1983; Tootell et al., 1995), and that damages to MT produce a significant
impairment in the perception of dynamic stimuli (Newsome & Pare, 1988;
Newsome, Wurtz, Dursteler, & Mikami, 1985). Area MT neurons display strong
directional selectivity, with stable images normally eliciting a poor response
(Albright, Desimone, & Gross, 1984; Born & Bradley, 2005).
However, recent studies have shown that static stimuli that imply motion
are also capable of activating similar motion-processing mechanisms in humans
(Jellema & Perrett, 2003; Kourtzi & Kanwisher, 2000; Lorteije et al., 2006;
Winawer, Huk, & Boroditsky, 2008). These stimuli are often static images of
natural objects motion (e.g., flying bird, running dog, etc.). Unlike dynamic visual
stimuli that automatically produce perceptual sensation of motion, processing of
the static images of objects in motion may engage additional high-level cognitive
processes that link these stimuli to motion. For example, the activity of the
motion-selective mechanisms in response to a picture of a running athlete may
be modulated by the top-down signals resulting from the activation of the
69
Chapter 5 Motion Aftereffects and Stable Images
dynamic context (i.e., position of the athlete in moments leading up to, and
following the time point at which the picture of taken) associated with the image.
Interestingly, it appears that the dynamic information associated with a
static image can be acquired through forced associative learning. For example,
a static, abstract image that does not represent motion in natural settings can
come to elicit responses in directionally selective neurons in MT. In a study by
Schlack and Albright (2007), the activity of neurons in visual area MT were
recorded in awake, behaving animals prior to and after a training period during
which they learned to associate moving stimuli with static images of arrows. The
authors showed that after the associative learning trials, area MT neurons
responded to the stable images in a manner that was consistent with the nature
of the learned associations.
This intriguing finding shows that the range of static stimuli that can
potentially activate motion-processing mechanisms is not limited to images of
natural objects in motion. It led us to question whether static abstract images
that are previously associated with motion are capable of influencing perception
of real motion. In the current experiment, we employed the selective adaptation
method designed to induce dynamic motion aftereffects to capture transient
changes in perception of motion. During a selective adaptation experiment, a
stimulus is presented for an extended period of time, resulting in temporary
perceptual distortions (or aftereffects). Dynamic MAE makes use of dynamic
random dot stimuli (Hiris & Blake, 1992; Newsome & Pare, 1988), which are
70
Chapter 5 Motion Aftereffects and Stable Images
groups of dots moving in different directions over time. The directional
coherence of these stimuli is determined by the percentage of dots moving in
the same direction, which can be varied from 0 (no global direction) to 100%
(total correlated motion). The dynamic MAE is produced after extended
exposure to dots moving in one direction, resulting in heightened perceptual bias
to movement in the opposite direction. It is believed that MT is a key visual area
mediating aftereffects related to motion (Antal et al., 2004; Rees, 2001;
Tikhonov, Handel, Haarmeier, Lutzenberger, & Thier, 2007).
If an abstract, static stimulus previously associated with a particular
direction of motion can activate motion-processing mechanisms, then prolonged
exposure to that stable stimulus alone should be sufficient to influence the
perception of subsequent motion. We show here that adaptation to a static
image associated with unidirectional movement subsequently increased
sensitivity to movement in the direction opposite to the associated direction in
our first two experiments. In our last experiment, we also report that a dynamic
MAE was produced after adaptation to stable arrow images without any recent
associative learning in experimental settings. It is presumed that the meaning of
abstract shapes such as arrows is explicitly achieved through prior associative
experience in humans. In this experiment, there was no learning phase during
which the images of arrows became explicitly associated with motion and
instead relied on an intrinsic or preexisting semantic representation of arrow
shape and movement direction. Together, our results show that adaptation to
71
Chapter 5 Motion Aftereffects and Stable Images
both static stimuli associated with motion and dynamic stimuli can influence
subsequent perception of motion in similar manners.
Methods
P articipants
A total of ten observers participated in Experiment 1(mean age = 24.6; five
males). Following the completion of Experiment 1, the same group of observers
participated in Experiment 2. Five new subjects participated in Experiment 3
(mean age = 25 yrs; one male). All participants had previously participated in
psychophysical experiments but were naïve to the purpose of the present
experiment. All had normal or corrected-to-normal vision. The study was
reviewed and approved by the McGill institutional ethics board for human
psychophysical studies. Written consent was acquired from each participant
prior to the experimental sessions. The data from two participants who
participated in both Experiments 1 and 2 were excluded from analysis because
their motion discrimination of moving dots at the 30% coherence level during the
control condition did not exceed chance (50% accuracy).
72
Chapter 5 Motion Aftereffects and Stable Images
A pparatus and stimuli
All stimuli were presented on an LG flat-screen, CRT monitor with 1024 x 768
resolution and 85 Hz refresh rate. The stimuli were centrally presented on a
uniform black background. The presentation sequence was programmed in
MATLAB software using the Psychophysics Toolbox extension (Brainard, 1997).
A chinrest was used to maintain head position at a distance of 57 cm from the
monitor surface. Gaze direction was monitored by an eye-movement tracking
camera (Arrington Research, AZ) positioned under the right eye to ensure
proper fixation. All experiments were carried out in a dark testing room.
The stable images in Experiment 1 and 2 were made up of four filled
geometric shapes (circle, square, diamond and hexagon) whereas those for
Experiment 3 were arrows pointing in one of four directions (up, down, left and
right). The size of the stable images did not exceed 7 x 7 degrees. The moving
dots appeared within a circular window of 15 degrees diameter. The dot density
was 16.7 dots/deg2 and the size of each dot was approximately 3 x 3 pixels. The
center of the window was marked with a cross. A subset of dots, depending on
the coherence level, was repositioned from the original location in one of the
four directions (left, right, up and down), with the remaining dots being randomly
repositioned in incoherent directions. The speed of dots was 6.0 deg/s. During
73
Chapter 5 Motion Aftereffects and Stable Images
associative learning and adaptation, dot coherence was 80% and the direction
of the dots was either left or right. The coherence of test dots varied from 0 to
30%.
P rocedures
Experiment 1
Participants completed an associative learning phase before proceeding to the
motion aftereffect tasks. During the associative learning phase, two of the four
geometric shapes (circle, square, diamond or hexagon) were randomly selected
for each subject. Subjects then learned that each shape was paired with one
direction of motion (left or right). These stimuli are illustrated in Figure 1. The
pairing of stable images and directions of motion was randomized across
participants. The presentations of the stimuli within a pair occurred in a
sequential manner. The presentation of the abstract shape always preceded that
of the moving dots. Each stimulus in the pair was presented for 5 seconds. The
two pairs were presented a minimum of 10 times in randomized order
throughout the associative learning phase. Participants were asked to maintain
fixation on the central cross during the presentation of the paired stimuli. At the
end of the presentation period, participants were asked to verbally describe the
direction of motion associated with each shape. The presentation of the pairs
was repeated until 100% accuracy in verbal description was achieved.
74
Chapter 5 Motion Aftereffects and Stable Images
The motion aftereffect tasks were presented immediately after the
associative learning phase. As shown in Figure 2, there were three adaptation
conditions depending on the type of the adapting stimuli: one of the two abstract
shapes presented during the associative learning phase (stable-image
condition), a group of dots moving in one of the two directions (moving-dots
condition), or one of the two new abstract shapes with no associated direction of
motion (control condition). Each adaptation condition was tested by way of two
blocks of 140 trials and each adapting stimulus appeared for one block of trials.
The adapting stimulus would remain constant within each block (e.g., one
abstract shape or one movement direction per block). The duration of the
adapting stimulus was 10 seconds for the first trial of each block, and 5 seconds
for all subsequent trials. For the stable-image condition, participants were asked
to simultaneously imagine the direction of motion associated with the image. A
total of six blocks, two from each condition, were presented in an intermixed,
pseudo-randomized order.
The adapting stimulus was followed by brief presentation of a central
fixation cross (500 ms) and a 1 second presentation of the test stimulus (Fig. 2).
Test stimuli were a group of moving dots that could have one of seven
coherence levels (0 – 30%) in two different directions (same or opposite to the
direction represented by the adapting stimulus). Participants were made aware
of these two directional possibilities. Their task was to identify the direction of
global motion of the test stimulus with a key press (left or right). Each stimulus
75
Chapter 5 Motion Aftereffects and Stable Images
condition (control, stable-image and moving –dots) was composed of two
blocks, with each block containing 140 trials. A total of ten presentations were
made at each of the seven coherence levels and two directions (total = 140
trials) within each block.
Participants were asked to fixate on the central cross throughout the task.
For all trials, the participants’ eye movements were monitored to ensure
constant fixation.
Experiment 2
Upon the completion of Experiment 1, the same participants also
participated in Experiment 2. Each participant completed the same associative
learning task described in Experiment 1, as a reminder, before proceeding to the
MAE tasks. There were two adaptation conditions in Experiment 2: stable-image
and moving-dots. The MAE tasks were identical to those in Experiment 1,
except that the inter-stimulus interval (ISI) was increased to 3.5 seconds. A
central fixation cross was presented for the duration of ISI.
Experiment 3
In this experiment, the participants completed the MAE tasks without any
associative learning. As with Experiment 1, there were three adaptation
conditions (stable-image, moving-dots and control). However, the adapting
76
Chapter 5 Motion Aftereffects and Stable Images
stimuli in the stable-image condition images were composed of arrows pointing
in one of four directions (up, down, left, and right). The adapting stimuli in the
control condition were composed of the four abstract shapes described in
Experiment 1. All participants completed the three conditions in the following
order – stable-image, moving-dots and control. All other parameters were
identical to those of Experiment 1.
Results
E xperiment 1
All participants whose data were included for analysis achieved 100%
accuracy in describing the relationship between stable images and directions of
motion after the first session of the associative learning task (10 presentations of
each pair). Data from eight participants were used for analysis. The performance
data were fitted to a logistic function.
Fig. 3a shows averaged proportions of trials in which participants
perceived the test stimulus to be moving in a direction opposite to that
represented by the adapting stimulus, plotted against the coherence level of the
test stimulus in two directions (same or opposite to adapting). A logistic function
was fitted to the average of the raw data. Compared to the control condition
during which adapting stimuli were stable images with no associated motion,
adaptation to either moving dots or stable images that were previously
77
Chapter 5 Motion Aftereffects and Stable Images
associated with unidirectional motion caused a leftward shift, suggesting a
perceptual bias towards motion in the opposite direction. We observed a greater
leftward shift with the moving-dots condition compared to the stable-image
condition. The data from two representative individuals in this experiment are
separately presented in Figures. 3b and 3c.
To assess the magnitude of the aftereffect produced in the different
adaptation conditions, the coherence of test dots that produced no perceived
global motion (nulling percentage) was calculated, and compared to that of the
control condition. A paired t-test revealed a significant shift in the nulling
percentage from the control condition in both adaptation conditions (moving-dots
condition [t(7) = 10.859, p = .000, prep =.99 one- tailed], mean shift in nulling
percentage = 18.130, SEM= 1.670; stable-image condition [t(7) = 4.919, p
=0.001, prep =.99 one- tailed], mean shift in nulling percentage = 8.070, SEM =
1.640).
E xperiment 2
In Experiment 2, we increased the inter-stimulus interval between the
adapting and test stimuli to 3.5 seconds in both moving-dots and stable-images
conditions. Data from the same 8 participants of Experiment 1 were included for
analysis. Group averages in moving-dots and stable-images conditions were
compared to those in the respective conditions from Experiment 1. Data from
the control condition from Experiment 1 are shown to provide reference points.
78
Chapter 5 Motion Aftereffects and Stable Images
Figure 4a shows the response profiles in directional judgments following
adaptation to moving-dots obtained from Experiments 1 and 2. Compared to
those from Experiment 1 (ISI of 0.5 s), the ISI of 3.5s in Experiment 2 caused a
decrease in the magnitude of the aftereffect, as indicated by the decreased
shifts in the nulling percentage (mean shift of 10.159, SEM = 1.935, compared
to the mean shift of 18.129 in Experiment 1).
Similar patterns were observed in performances following adaptation to
stable images previously associated with motion. As shown in Figure 4b, the
increased ISI produced a decrease in the magnitude of the aftereffect (mean
shift of 2.518, SEM = 1.096, compared to the mean shift of 8.070 in Experiment
1).
E xperiment 3
In this experiment, the adapting stimulus in the stable-image condition
consisted of a set of arrows randomly presented in one of the four cardinal
directions. There was no associative learning involved. The moving-dots
condition was the same as in Experiment 1. Fig. 5a shows that the pattern of
results based on the performance of five participants was similar to that found in
Experiment 1. Adaptation to arrow images and unidirectional moving dots
produced a leftward shift from the control condition. The shift in nulling
79
Chapter 5 Motion Aftereffects and Stable Images
percentages caused by adaptation to the arrow images was statistically
significant [t(4) = 4.102, p = 0.003, prep =.97, one- tailed].
D iscussion
We have used dynamic random dot displays with varying coherence
levels to measure transient changes in perceived motion after exposure to static
adapting stimuli composed of various geometric shapes. Subjects first
completed a learning phase in which they associated a given shape with a
particular direction of motion. We discovered that subsequent adaptation to a
given static image produced a bias in perceived motion opposite in direction to
that associated with the object. This motion aftereffect, which was generated by
a static object, was significantly distinguishable from control (no associative
learning) but not as strong as the MAE produced by actual moving dots. In a
subsequent experiment, we discovered that the magnitude of the effect is
significantly decreased after a delay of 3.5 seconds, suggesting that a common
mechanism mediates the MAE following adaptation to unidirectional motion and
stable images previously associated with motion.
Previous studies have shown that static images of objects in motion (e.g.,
flying bird, running dog, etc.) activate motion-processing mechanisms (Kourtzi &
Kanwisher, 2000; Lorteije et al., 2006; Winawer, Huk, & Boroditsky, 2008). In
our third experiment, we further explored whether abstract shapes implying a
80
Chapter 5 Motion Aftereffects and Stable Images
particular motion direction can also generate an MAE in the absence of recent
associative learning. Unlike images of natural objects in motion, the meaning of
abstract shapes such as arrows is achieved through prior associative
experience. In this experiment, there was no learning phase during which the
images of arrows became explicitly associated with motion and instead relied on
an intrinsic or preexisting semantic representation of arrow shape and
movement direction. Adaptation to stable images of arrows produced an MAE,
suggesting that the phenomenon of implied motion can be extended to include
abstract images indicating directions of motion.
The results of our study extend the findings of Schlack and Albright
(2007) into the human behavioral domain. We have shown that adaptation to
static images previously associated with movement direction produces a
perceptual bias that is similar in nature to that observed at the biological level.
Compared to adaptation to moving dots, the magnitude of the perceptual MAE
as measured by shifts in nulling percentages was smaller when the adapting
stimulus was a stable image. This result is also consistent with previous reports
of reduced activity of motion-sensitive neurons when stimulated by images
associated with motion (Schlack & Albright, 2008). It appears that the response
selectivity of a neuron can be altered through associative learning, but the level
of response may not be equivalent to that elicited by its inherently preferred
stimulus.
81
Chapter 5 Motion Aftereffects and Stable Images
One interesting finding in our study concerns the MAEs induced by arrow
images, which occurred in the absence of any recent associative learning. It
appears that the presentation of an arrow produced automatic activation of its
semantic representation that in turn may have primed motion-selective neurons
during adaptation. In contrast, Schlack and Albright (2007) found that area MT
neurons do not show a response bias to arrow presentation prior to associative
learning. This is to be expected given that monkeys are not likely to have pre-
existing knowledge of arrow connotations whereas prior human association with
such stimuli likely accounts for the effects we observed.
The magnitude of the aftereffect following adaptation to moving dots and
stable images with associated motion was significantly decreased when the ISI
between adapting and test stimuli was increased. As pointed out by Winawer
and colleagues (2008) whose study reported motion aftereffects from
photographs depicting motion, a short delay between the presentations of
adapting and test stimuli would not influence the size of the aftereffect, if the
effect were based purely on cognitive processes. The observed decrease in the
aftereffect magnitude caused by the delay suggests that the bias following
adaptation to stable images previously associated with motion has a perceptual
basis.
While viewing stable images previously associated with motion during
adaptation, participants were asked to mentally recall the direction of associated
motion. Therefore, it is possible that the imagery of moving dots may have
82
Chapter 5 Motion Aftereffects and Stable Images
contributed to the effect. Indeed, imagery of motion has been shown to activate
brain areas involved in processing of real motion (Grossman & Blake, 2001).
Nonetheless, continuous, strong visual inputs have been suggested to interrupt
imagery (Pearson, Clifford, & Tong, 2008; Ryu, Borrmann, & Chaudhuri, 2008).
Since stable images remained present during the adaptation period, imagery
alone cannot account for the subsequent perceptual bias reported here.
Our results lead to the question of what brain structures may be
responsible for the neural basis of object–motion associations. Motion
aftereffects induced by stable images suggest the involvement of neural
mechanisms that specialize in the processing of both motion and simple shapes.
One possible region, at least in humans, where such integration may occur can
be found in the posterior middle temporal gyrus. Neuroimaging studies have
revealed increased activity in this area in response to stable images implying
motion (Kourtzi & Kanwisher, 2000; Krekelberg, Vatakis, & Kourtzi, 2005),
mental imagery of moving objects (Grossman & Blake, 2001), and words that
represent motion (Martin, Haxby, Lalonde, Wiggs, & Ungerleider, 1995;
(Wallentin, Lund, Ostergaard, Ostergaard, & Roepstorff, 2005). Located anterior
to the human MT area, this area may also be involved in integrating static cues
and motion, and possibly modulates the activity of area MT neurons in a top-
down manner.
83
Chapter 5 Motion Aftereffects and Stable Images
Figure Legends
Figure 1. a. Basic geometric shapes used in stable-images and control
conditions. Two geometric shapes were randomly chosen from the four basic
shapes for each participant. Participants then underwent the learning phase
prior to the motion aftereffect task, during which they were asked to associate
each geometric shape with a direction of motion (shown in b).
Figure 2. Trial sequence during the motion aftereffect task in Experiment 1. The
adapting stimulus was a geometric shape previously associated with
unidirectional motion in the stable-images condition, a group of dots moving in
one direction (left or right) in the moving-dots condition, or a geometric shape
with no associated motion in the control condition. The adapting stimulus was
followed by brief presentation of a central fixation cross (500 ms) and a 1
second presentation of the test stimulus. Test stimuli were a group of moving
dots that could have one of seven coherence levels (0 – 30%) in two different
directions (same or opposite to the direction represented by the adapting
stimulus). Participants were asked to indicate the overall direction of the test
stimulus.
Figure 3. Performance on the motion aftereffect tasks in Experiment 1.
Proportions of trials in which participants perceived the test stimulus to be
84
Chapter 5 Motion Aftereffects and Stable Images
moving in a direction opposite to that represented by the adapting stimulus
plotted against coherence level of the test stimulus in two directions (same or
opposite to adapting). A logistic function [1/(1+exp(-(x-c)/a))] was fitted to the
data, in which ‘a’ and ‘c’ are free parameters that determine the midpoint and
slope of the psychometric function. Data from control (no associative learning),
stable-image and moving-dots conditions are shown. a. Averaged data from
eight participants. Standard errors of mean are indicated. b & c. Data from two
individuals.
Figure 4. Performance on the motion aftereffect tasks in Experiment 2 in which
the ISI between adapting and test stimuli was increased to 3.5 seconds (Delay).
Participants from Experiment 1 completed the tasks in Experiment 2. Data from
the control and corresponding conditions from Experiment 1 are shown to
provide reference points. Standard errors of the mean are shown. a. Group data
from the moving-dots conditions from Experiments 1 and 2. b. Group data from
the stable-images conditions from Experiments 1 and 2.
Figure 5. Performance on the motion aftereffect tasks in Experiment 3. During
the stable-images condition, static images of arrows were presented as adapting
stimuli. a. Averaged data from five participants. b & c. Data from two individuals.
85
Chapter 5 Motion Aftereffects and Stable Images
Figure 1
86
Chapter 5 Motion Aftereffects and Stable Images
Figure 2
87
Chapter 5 Motion Aftereffects and Stable Images
Figure 3
88
Chapter 5 Motion Aftereffects and Stable Images
Figure 4
89
Chapter 5 Motion Aftereffects and Stable Images
90
Figure 5
Chapter 6 Concluding Remarks
Chapter 6
Concluding Remarks
Summary
The present thesis describes experiments that attempted to examine how
mnemonic processes influence perception of different types of visual stimuli.
Selective adaptation methods designed to produce visual aftereffects were used
to test the hypothesis that adaptation to stimuli with different mnemonic contents
should produce different types of aftereffects. Indeed, transient changes in
perception produced by mnemonic representations associated with adapting
stimuli were revealed by various visual aftereffects. The present results suggest
that processing of motion and faces, each believed to engage specialized visual
processing areas, is subject to top-down mnemonic influences.
In the first study described in Chapter 3, the principle underling viewpoint-
dependent aftereffects was applied to discover the effects of familiarity on the
perception of facial viewpoints. Adaptation to slightly rotated images of familiar
and unfamiliar faces produced temporary distortion in the perception of
viewpoint of subsequent images of faces. This result suggests that neural
networks mediating viewpoint information are involved in the representation of
these faces. However, category-specific transfer of the viewpoint-dependent
aftereffect occurred only with unfamiliar faces, providing evidence for the idea
91
Chapter 6 Concluding Remarks
that differential weights are attached to viewpoint information of the
representations of familiar and unfamiliar faces.
Representations of familiar facial identities are further explored in the
subsequent study, in which face-identity aftereffect was used to probe neural
networks underlying perception and imagery of familiar faces. Previous works
reporting overlapping neural mechanisms underlying perception and imagery
lead to the hypothesis that adaptation to both perceived and imagined familiar
faces should produce similar biases in the perception of subsequent face
identities. Indeed, face-identity aftereffect was produced after adaption to both
perceived and imagined familiar faces. Additional experiments examining the
identity information contained in these faces showed that identity boundaries of
perceived and imagined faces were equivalent. Together, these findings show
that identity information contained in imagined and perceived faces produce
similar bias in face perception in a selective adaptation study.
The influence of mnemonic processes on motion perception was also
evident, as shown in dynamic motion aftereffect produced by adaptation to static
images previously associated with unidirectional motion. This pattern of
behavioral outcome is consistent with the altered stimulus-selectivity of MT
neurons produced by associative learning reported in the previous
neurophysiological investigation (Schlack & Albright, 2007). Similar bias in
motion perception was produced after adaptation to static images of arrows
without any explicit associative learning in the experimental context, suggesting
92
Chapter 6 Concluding Remarks
that pre-existing connotations about the arrow images could also prime
mechanisms mediating motion perception. The dynamic motion aftereffects
produced in the absence of real dynamic stimuli show that top-down influences
resulting from associative learning affect the perception of motion.
L imitations
In the study investigating the effects of facial familiarity on the perception
of viewpoints, it was concluded that the stimulus-specific transfer of the
aftereffect occurred only with unfamiliar faces, because the viewpoint perception
was near chance when the adapting and test faces were of different identities.
However, it is entirely possible that this disruption in viewpoint perception is
limited to the orientation shown in the test faces. Presentation of test faces with
additional degree of orientation may help to uncover the true extent of viewpoint
perception in that experimental condition.
The decreased magnitude of the aftereffect produced after adaptation to
the imagined faces described in Chapter 3 was attributed to the difficulty
associated with imagining a stimulus in a coherent, consistent manner, which
may have led to weaker neural activations during the adaptation period. The
equivalent difference thresholds for both real and imagined faces found in the
subsequent experiment ruled out the possibility that the decreased magnitude of
93
Chapter 6 Concluding Remarks
the aftereffect was due to the difference in the identity information contained in
these faces. However, this is only one of many alternative explanations that may
exist. For example, the time course of neural activity elicited by imagined and
perceived faces may be completely different, which may result in qualitative
differences in perception of subsequent faces if the presentation duration of
adapting face was varied.
While viewing static images previously associated with motion during
adaptation in the experiment described in the previous chapter, participants
were asked to mentally recall the direction of associated motion. Therefore, it is
possible that the imagery of moving dots may have contributed to the effect.
Nonetheless, continuous, strong visual inputs have been suggested to interrupt
imagery (Pearson & Brascamp, 2008; Ryu, Borrmann, & Chaudhuri, 2008).
During the adaptation period, a stable image remained present and participants
were asked to fixate on the center of the image. Since stable images remained
present during the adaptation period, imagery alone cannot account for the
subsequent perceptual bias reported here. Therefore, it is highly unlikely that the
reported perceptual bias was solely driven from motion imagery. The only
instances in which mental imagery may have contributed to the performance
were the delayed conditions in Experiment 2, in which a blank screen with a
small fixation cross appeared during the delay period. However, the magnitude
of the motion aftereffect was decreased, not increased, after the delay.
94
Chapter 6 Concluding Remarks
Possible neural mechanisms mediating mnemonic influence on perception of complex images
The results reported in Chapters 2 and 3 suggest close interaction
between visual processing and mnemonic representation of faces. One question
that arises from the present finding is whether this type of interaction shown at
the behavioral level can also be found in the neural structures mediating
perception and memory. It is well known that face-selective cells are located in
the high-level visual processing areas in the inferior temporal cortex. The type of
learning that allows the formation of conscious representations of faces is
mediated by structures in the medial temporal cortex. While there are reports
suggesting that the influence of mnemonic processes on perception is achieved
through interactive signals amongst putative structures mediating perceptual
and mnemonic functions (Miyashita, Kameyama, Hasegawa, & Fukushima,
1998; Sugiura, Shah, Zilles, & Fink, 2005), other studies have revealed that
there may be additional brain regions that act as a mediator between the
memory-related and perception-related structures (Bussey & Saksida, 2007;
Murray, Bussey, & Saksida, 2007; Suzuki & Amaral, 1994; Suzuki, Zola-Morgan,
Squire, & Amaral, 1993).
The perirhinal cortex, located at the junction of medial temporal cortex
and the inferior temporal cortex, has been suggested to be the structure that
relays mnemonic signals to vision-related areas. This area displays extensive
bilateral connections to inferior temporal and medial temporal areas, allowing
95
Chapter 6 Concluding Remarks
easy exchange of signals from both perception-related and memory-related
areas (Suzuki & Amaral, 1994). Furthermore, it has been shown that damage to
this area caused impairments in tasks requiring memory and visual
discrimination of complex images (Bussey & Saksida, 2002, 2005). In fact,
according to the Perceptual-Mnemonic/Feature Conjunction (PMFC) model, the
perirhinal cortex is crucial in binding and discrimination of complex visual
features, which require both perceptual and mnemonic functions (Bussey &
Saksida, 2005; Lee et al., 2005). Therefore, the perirhinal cortex may be
involved in mediating mnemonic influences on the activity of high-level visual
areas that selectively process complex images.
Mnemonic influences and perception of simple stimulus features
Dynamic motion aftereffects produced by static images previously
associated with motion showed that mnemonic processes exert influences on
the processing of motion. This leads one to question whether the observed
mnemonic influence can be extended to the processing of other simple stimulus
features. Indeed, Bulthoff and colleagues showed that internal representations
of familiar objects could alter the perception of 3-D structures that are incoherent
to the objects (Bulthoff, Bulthoff, & Sinha, 1998). They also showed that
anomalous stereo-depth cues did not interfere with object recognition,
96
Chapter 6 Concluding Remarks
suggesting that top-down influences stemming from representations of a familiar
object can completely override the incoming depth information of that object.
The finding that familiarity could actually supersede incoming depth
information in producing conscious percepts is rather intriguing because it raises
further questions on the extent to which mnemonic processes could influence
visual processing of different stimulus features. For example, the strength of the
mnemonic influence exerted on the perception of motion appears to be weaker,
because the magnitude of the dynamic motion aftereffect produced by static
images was never greater than that produced by real motion. Whether this
discrepancy is due to the increased task demand of imagining, as opposed to
merely perceiving visual stimuli, or due to differential extents to which
processing of different visual features is amenable to top-down influence is a
subject of further investigation.
Processing of stimulus orientation is believed to mainly occur in the
primary visual cortex (V1). While several studies show that the activity of V1
neurons are subject to top-down influences of attention (Crist et al., 2001; Li &
Gilbert, 2002; Li et al., 2004), it is yet to be investigated whether memory-related
signals can also affect the activity of these neurons. This investigation may be of
particular interest, because perception resulting from the activity of the primary
cortex is often categorized as “early vision”, which is believed to be
encapsulated from other cognitive influences beside attention (Pylyshyn, 1999).
In the context of the current research, a further experiment reporting biased
97
Chapter 6 Concluding Remarks
perception of orientation following adaptation to complex images previously
associated with line segments of various orientations would provide evidence for
mnemonic influences on early vision.
Perception as results of Interactions amongst different visual areas
Cortical visual processing starts in the primary visual cortex and is
believed to occur in a hierarchical manner. This hierarchy is reflected in the
extensive feed-forward, or ascending connections found in the pathways
responsible for bottom-up processing of visual stimuli (Callaway, 1998; Reid,
2001). However, just as extensive are the feedback, or descending connections
projecting from high-level to low-level visual areas. For example, V1 receives
direct projections from IT as well as projections from V2, V4 and MT (Angelucci
et al., 2002; Hupe et al., 1998; Shmuel et al., 2005). Indeed, the intensity of the
feedback connections found in different visual areas have led some researchers
to question the utility of the traditional “low” and “high” labeling of different visual
areas (Gilbert & Sigman, 2007).
Just as the feed-forward connections provide circuitry for bottom-up
processing of visual stimuli, the feedback connections may provide circuitry for
top-down processing of visual information. Furthermore, these extensive
connections suggest that perception can be influenced by dynamic interactions
98
Chapter 6 Concluding Remarks
99
between different visual areas. Indeed, the evidence of perception due to the
dynamic interaction between different visual area has been shown in the study
reported by Kovacs and colleagues (Kovacs, Papathomas, Yang, & Feher,
1996). In that study, two complex images of objects that were divided into
patches and different, complementary patches of the images were presented to
each eye to induce binocular rivalry. They found that the alternating percept is
object-based, rather than eye-based. Given that the binocular rivalry is mediated
by the activity of the primary visual cortex, the alternating percepts are likely due
to the interaction between primary visual cortex and areas in the inferior
temporal cortex that process complex images.
Research conducted in the past few decades has revealed that different
visual areas are responsible for processing different aspects of visual stimuli.
However, most studies have focused on the activity of a visual area in isolation,
with little regard to its interaction with other areas. Recent findings, including the
ones described in the current thesis, show that perception is a rather dynamic
process, mediated by the activity of multiple brain areas. Therefore, it is
necessary to further study the nature of interactions amongst different structures
within the visual system, as well as how the visual system as a whole interacts
with other cognitive mechanisms in the brain.
R eferences
Albright, T. D., Desimone, R., & Gross, C. G. (1984). Columnar organization of
directionally selective cells in visual area MT of the macaque. J
Neurophysiol, 51(1), 16-31.
Angelucci, A., Levitt, J. B., Walton, E. J., Hupe, J. M., Bullier, J., & Lund, J. S.
(2002). Circuits for local and global signal integration in primary visual
cortex. J Neurosci, 22(19), 8633-8646.
Antal, A., Varga, E. T., Nitsche, M. A., Chadaide, Z., Paulus, W., Kovacs, G., et
al. (2004). Direct current stimulation over MT+/V5 modulates motion
aftereffect in humans. Neuroreport, 15(16), 2491-2494.
Baddeley, A. (1992). Working memory. Science, 255(5044), 556-559.
Bartolomeo, P., Bachoud-Levi, A. C., De Gelder, B., Denes, G., Dalla Barba, G.,
Brugieres, P., et al. (1998). Multiple-domain dissociation between
100
impaired visual perception and preserved mental imagery in a patient with
bilateral extrastriate lesions. Neuropsychologia, 36(3), 239-249.
Behrmann, M., Moscovitch, M., & Winocur, G. (1994). Intact visual imagery and
impaired visual perception in a patient with visual agnosia. J Exp Psychol
Hum Percept Perform, 20(5), 1068-1087.
Bentin, S., Moscovitch, M., & Nirhod, O. (1998). Levels of processing and
selective attention effects on encoding in memory. Acta Psychol (Amst),
98(2-3), 311-341.
Boring, E. G. (1930). A new ambiguous figure. Americal Journal of Psychology,
42, 444-445.
Born, R. T., & Bradley, D. C. (2005). Structure and function of visual area MT.
Annu Rev Neurosci, 28, 157-189.
Brainard, D. H. (1997). The Psychophysics Toolbox. Spat Vis, 10(4), 433-436.
Broerse, J., & Crassini, B. (1980). The influence of imagery ability on color
aftereffects produced by physically present and imagined induction
stimuli. Percept Psychophys, 28(6), 560-568.
Bruce, V., & Young, A. (1986). Understanding face recognition. Br J Psychol, 77
( Pt 3), 305-327.
Bulthoff, I., Bulthoff, H., & Sinha, P. (1998). Top-down influences on
stereoscopic depth-perception. Nat Neurosci, 1(3), 254-257.
Burton, A. M., Bruce, V., & Hancock, P. J. B. (1999). From pixels to people: a
model of familiar face recognition. Cognitive Science, 23, 1-31.
101
Busey, T. A., & Loftus, G. R. (1998). Binocular information acquisition and visual
memory. J Exp Psychol Hum Percept Perform, 24(4), 1188-1214.
Bussey, T. J., & Saksida, L. M. (2002). The organization of visual object
representations: a connectionist model of effects of lesions in perirhinal
cortex. Eur J Neurosci, 15(2), 355-364.
Bussey, T. J., & Saksida, L. M. (2005). Object memory and perception in the
medial temporal lobe: an alternative approach. Curr Opin Neurobiol,
15(6), 730-737.
Bussey, T. J., & Saksida, L. M. (2007). Memory, perception, and the ventral
visual-perirhinal-hippocampal stream: thinking outside of the boxes.
Hippocampus, 17(9), 898-908.
Callaway, E. M. (1998). Local circuits in primary visual cortex of the macaque
monkey. Annu Rev Neurosci, 21, 47-74.
Craik, F. I. (2002). Levels of processing: past, present. and future? Memory,
10(5-6), 305-318.
Crist, R. E., Li, W., & Gilbert, C. D. (2001). Learning to see: experience and
attention in primary visual cortex. Nat Neurosci, 4(5), 519-525.
Dolan, R. J., Fink, G. R., Rolls, E., Booth, M., Holmes, A., Frackowiak, R. S., et
al. (1997). How the brain learns to see objects and faces in an
impoverished context. Nature, 389(6651), 596-599.
Eger, E., Schweinberger, S. R., Dolan, R. J., & Henson, R. N. (2005). Familiarity
enhances invariance of face representations in human ventral visual
cortex: fMRI evidence. Neuroimage, 26(4), 1128-1139.
102
Ellis, A. W. (1992). Cognitive mechanisms of face processing. Philos Trans R
Soc Lond B Biol Sci, 335(1273), 113-119.
Fang, F., & He, S. (2005). Viewer-centered object representation in the human
visual system revealed by viewpoint aftereffects. Neuron, 45(5), 793-800.
Farah, M. J. (1985). Psychophysical evidence for a shared representational
medium for mental images and percepts. J Exp Psychol Gen, 114(1), 91-
103.
Felleman, D. J., & Van Essen, D. C. (1991). Distributed hierarchical processing
in the primate cerebral cortex. Cereb Cortex, 1(1), 1-47.
Finke, R. A., & Schmidt, M. J. (1977). Orientation-specific color after-effects
following imagination. Journal of Experimental Psychology: Human
Perception and Performance, 3, 599-606.
Finke, R. A., & Schmidt, M. J. (1978). The quantitative measure of pattern
representation in images using orientation-specific color aftereffects.
Percept Psychophys, 23(6), 515-520.
Frisby, J. (1980). Seeing: Illusion, brain and the mind. Oxford: Oxford United
Press.
Furl, N., van Rijsbergen, N. J., Treves, A., & Dolan, R. J. (2007). Face
adaptation aftereffects reveal anterior medial temporal cortex role in high
level category representation. Neuroimage, 37(1), 300-310.
Gibson, J. J. (1933). Adaptation, after-effect and contrast in the perception of
cuved lines. Journal of Experimental Psychology, 20, 453-467.
103
Gibson, J. J., & Radner, M. (1937). Adaptation, after-effect and contrast in the
perception of tilted lines. Journal of Experimental Psychology, 20, 453-
467.
Gilbert, C. D., & Sigman, M. (2007). Brain states: top-down influences in sensory
processing. Neuron, 54(5), 677-696.
Gorno-Tempini, M. L., & Price, C. J. (2001). Identification of famous faces and
buildings: a functional neuroimaging study of semantically unique items.
Brain, 124(Pt 10), 2087-2097.
Grossman, E. D., & Blake, R. (2001). Brain activity evoked by inverted and
imagined biological motion. Vision Res, 41(10-11), 1475-1482.
Hancock, P. J., Bruce, V. V., & Burton, A. M. (2000). Recognition of unfamiliar
faces. Trends Cogn Sci, 4(9), 330-337.
Hebb, D. O. (1949). The Organization of Behavior; A neuropsychological
Theory. New York: Wiley.
Hill, H., Schyns, P. G., & Akamatsu, S. (1997). Information and viewpoint
dependence in face recognition. Cognition, 62(2), 201-222.
Hiris, E., & Blake, R. (1992). Another perspective on the visual motion
aftereffect. Proc Natl Acad Sci U S A, 89(19), 9025-9028.
Hubel, D. H., & Wiesel, T. N. (1968). Receptive fields and functional architecture
of monkey striate cortex. J Physiol, 195(1), 215-243.
104
Hupe, J. M., James, A. C., Payne, B. R., Lomber, S. G., Girard, P., & Bullier, J.
(1998). Cortical feedback improves discrimination between figure and
background by V1, V2 and V3 neurons. Nature, 394(6695), 784-787.
Ishai, A., & Sagi, D. (1995). Common mechanisms of visual imagery and
perception. Science, 268(5218), 1772-1774.
Ishai, A., Ungerleider, L. G., & Haxby, J. V. (2000). Distributed neural systems
for the generation of visual images. Neuron, 28(3), 979-990.
Ishai, A., Ungerleider, L. G., Martin, A., Schouten, J. L., & Haxby, J. V. (1999).
Distributed representation of objects in the human ventral visual pathway.
Proc Natl Acad Sci U S A, 96(16), 9379-9384.
Kanwisher, N., McDermott, J., & Chun, M. M. (1997). The fusiform face area: a
module in human extrastriate cortex specialized for face perception. J
Neurosci, 17(11), 4302-4311.
Koehler, W., & Wallach, H. (1944). Figural afterefects. An investigation of visual
processes. Proceedings of Americal Philosophical Society, 88, 269-260.
Konen, C. S., & Kastner, S. (2008). Two hierarchically organized neural systems
for object information in human visual cortex. Nat Neurosci, 11(2), 224-
231.
Korner, C., & Gilchrist, I. D. (2008). Memory processes in multiple-target visual
search. Psychol Res, 72(1), 99-105.
Kosslyn, S. M., Thompson, W. L., & Alpert, N. M. (1997). Neural systems shared
by visual imagery and visual perception: a positron emission tomography
study. Neuroimage, 6(4), 320-334.
105
Kourtzi, Z., & Kanwisher, N. (2000). Activation in human MT/MST by static
images with implied motion. J Cogn Neurosci, 12(1), 48-55.
Kovacs, I., Papathomas, T. V., Yang, M., & Feher, A. (1996). When the brain
changes its mind: interocular grouping during binocular rivalry. Proc Natl
Acad Sci U S A, 93(26), 15508-15511.
Kreiman, G., Koch, C., & Fried, I. (2000). Imagery neurons in the human brain.
Nature, 408(6810), 357-361.
Krekelberg, B., Vatakis, A., & Kourtzi, Z. (2005). Implied motion from form in the
human visual cortex. J Neurophysiol, 94(6), 4373-4386.
Lee, A. C., Bussey, T. J., Murray, E. A., Saksida, L. M., Epstein, R. A., Kapur,
N., et al. (2005). Perceptual deficits in amnesia: challenging the medial
temporal lobe 'mnemonic' view. Neuropsychologia, 43(1), 1-11.
Leopold, D. A., & Logothetis, N. K. (1999). Multistable phenomena: changing
views in perception. Trends Cogn Sci, 3(7), 254-264.
Leopold, D. A., O'Toole, A. J., Vetter, T., & Blanz, V. (2001). Prototype-
referenced shape encoding revealed by high-level aftereffects. Nat
Neurosci, 4(1), 89-94.
Leube, D. T., Erb, M., Grodd, W., Bartels, M., & Kircher, T. T. (2003). Successful
episodic memory retrieval of newly learned faces activates a left fronto-
parietal network. Brain Res Cogn Brain Res, 18(1), 97-101.
106
Li, W., & Gilbert, C. D. (2002). Global contour saliency and local colinear
interactions. J Neurophysiol, 88(5), 2846-2856.
Li, W., Piech, V., & Gilbert, C. D. (2004). Perceptual learning and top-down
influences in primary visual cortex. Nat Neurosci, 7(6), 651-657.
Lleras, A., Rensink, R. A., & Enns, J. T. (2005). Rapid resumption of interrupted
visual search. New insights on the interaction between vision and
memory. Psychol Sci, 16(9), 684-688.
Lorteije, J. A., Kenemans, J. L., Jellema, T., van der Lubbe, R. H., de Heer, F., &
van Wezel, R. J. (2006). Delayed response to animate implied motion in
human motion processing areas. J Cogn Neurosci, 18(2), 158-168.
Magnussen, S., Greenlee, M. W., Asplund, R., & Dyrnes, S. (1991). Stimulus-
specific mechanisms of visual short-term memory. Vision Res, 31(7-8),
1213-1219.
Martin, A., Haxby, J. V., Lalonde, F. M., Wiggs, C. L., & Ungerleider, L. G.
(1995). Discrete cortical regions associated with knowledge of color and
knowledge of action. Science, 270(5233), 102-105.
McCollough, C. (1965). The Conditioning of Color-Perception. Am J Psychol, 78,
362-378.
Medendorp, W. P., Tweed, D. B., & Crawford, J. D. (2003). Motion parallax is
computed in the updating of human spatial memory. J Neurosci, 23(22),
8135-8142.
107
Messinger, A., Squire, L. R., Zola, S. M., & Albright, T. D. (2001). Neuronal
representations of stimulus associations develop in the temporal lobe
during learning. Proc Natl Acad Sci U S A, 98(21), 12239-12244.
Michelon, P., & Biederman, I. (2003). Less impairment in face imagery than face
perception in early prosopagnosia. Neuropsychologia, 41(4), 421-441.
Miyashita, Y. (1988). Neuronal correlate of visual associative long-term memory
in the primate temporal cortex. Nature, 335(6193), 817-820.
Miyashita, Y., Kameyama, M., Hasegawa, I., & Fukushima, T. (1998).
Consolidation of visual associative long-term memory in the temporal
cortex of primates. Neurobiol Learn Mem, 70(1-2), 197-211.
Moradi, F., Koch, C., & Shimojo, S. (2005). Face adaptation depends on seeing
the face. Neuron, 45(1), 169-175.
Murray, E. A., Bussey, T. J., & Saksida, L. M. (2007). Visual perception and
memory: a new view of medial temporal lobe function in primates and
rodents. Annu Rev Neurosci, 30, 99-122.
Nassi, J. J., & Callaway, E. M. (2006). Multiple circuits relaying primate parallel
visual pathways to the middle temporal area. J Neurosci, 26(49), 12789-
12798.
Newsome, W. T., & Pare, E. B. (1988). A selective impairment of motion
perception following lesions of the middle temporal visual area (MT). J
Neurosci, 8(6), 2201-2211.
108
O'Craven, K. M., & Kanwisher, N. (2000). Mental imagery of faces and places
activates corresponding stiimulus-specific brain regions. J Cogn
Neurosci, 12(6), 1013-1023.
O'Toole, A. J., Deffenbacher, K. A., Valentin, D., & Abdi, H. (1994). Structural
aspects of face recognition and the other-race effect. Mem Cognit, 22(2),
208-224.
Over, R., & Broerse, J. (1972). Imagined lines fail to induce contour masking.
Psychonomic Science, 29, 203-204.
Paller, K. A., Ranganath, C., Gonsalves, B., LaBar, K. S., Parrish, T. B.,
Gitelman, D. R., et al. (2003). Neural correlates of person recognition.
Learn Mem, 10(4), 253-260.
Parkkonen, L., Andersson, J., Hamalainen, M., & Hari, R. (2008). Early visual
brain areas reflect the percept of an ambiguous scene. Proc Natl Acad
Sci U S A, 105(51), 20500-20504.
Pearson, J., Clifford, C. W., & Tong, F. (2008). The functional impact of mental
imagery on conscious perception. Curr Biol, 18(13), 982-986.
Pourtois, G., Schwartz, S., Seghier, M. L., Lazeyras, F., & Vuilleumier, P.
(2005). View-independent coding of face identity in frontal and temporal
cortices is modulated by familiarity: an event-related fMRI study.
Neuroimage, 24(4), 1214-1224.
Puce, A., Allison, T., & McCarthy, G. (1999). Electrophysiological studies of
human face perception. III: Effects of top-down processing on face-
specific potentials. Cereb Cortex, 9(5), 445-458.
109
Pylyshyn, Z. (1999). Is vision continuous with cognition? The case for cognitive
impenetrability of visual perception. Behav Brain Sci, 22(3), 341-365;
discussion 366-423.
Quiroga, R. Q., Reddy, L., Kreiman, G., Koch, C., & Fried, I. (2005). Invariant
visual representation by single neurons in the human brain. Nature,
435(7045), 1102-1107.
Ramachandran, V. S., & Braddick, O. (1973). Orientation-specific learning in
stereopsis. Perception, 2(3), 371-376.
Ramachandran, V. S., Ruskin, D., Cobb, S., Rogers-Ramachandran, D., &
Tyler, C. W. (1994). On the perception of illusory contours. Vision Res,
34(23), 3145-3152.
Rees, G. (2001). Attention, adaptation, and the motion aftereffect. Neuron,
32(1), 6-8.
Reid, R. C. (2001). Divergence and reconvergence: multielectrode analysis of
feedforward connections in the visual system. Prog Brain Res, 130, 141-
154.
Ryu, J. J., Borrmann, K., & Chaudhuri, A. (2008). Imagine Jane and identify
John: face identity aftereffects induced by imagined faces. PLoS ONE,
3(5), e2195.
Schacter, D. L., Norman, K. A., & Koutstaal, W. (1998). The cognitive
neuroscience of constructive memory. Annu Rev Psychol, 49, 289-318.
110
Schlack, A., & Albright, T. D. (2007). Remembering visual motion: neural
correlates of associative plasticity and motion recall in cortical area MT.
Neuron, 53(6), 881-890.
Schoups, A., Vogels, R., Qian, N., & Orban, G. (2001). Practising orientation
identification improves orientation coding in V1 neurons. Nature,
412(6846), 549-553.
Shmuel, A., Korman, M., Sterkin, A., Harel, M., Ullman, S., Malach, R., et al.
(2005). Retinotopic axis specificity and selective clustering of feedback
projections from V2 to V1 in the owl monkey. J Neurosci, 25(8), 2117-
2131.
Shulman, G. L., Corbetta, M., Buckner, R. L., Raichle, M. E., Fiez, J. A., Miezin,
F. M., et al. (1997). Top-down modulation of early sensory cortex. Cereb
Cortex, 7(3), 193-206.
Singer, G., & Sheehan, P. W. (1965). The effect of demand characteristics on
the figural aftereffect with real and imaged inducing figures. Am J
Psychol, 78, 96-101.
Slotnick, S. D., & Schacter, D. L. (2004). A sensory signature that distinguishes
true from false memories. Nat Neurosci, 7(6), 664-672.
Snodgrass, J. G., & Feenan, K. (1990). Priming effects in picture fragment
completion: support for the perceptual closure hypothesis. J Exp Psychol
Gen, 119(3), 276-296.
111
Snodgrass, J. G., & Hirshman, E. (1994). Dissociations among implicit and
explicit memory tasks: the role of stimulus similarity. J Exp Psychol Learn
Mem Cogn, 20(1), 150-160.
Sperling, R. A., Bates, J. F., Cocchiarella, A. J., Schacter, D. L., Rosen, B. R., &
Albert, M. S. (2001). Encoding novel face-name associations: a functional
MRI study. Hum Brain Mapp, 14(3), 129-139.
Sugiura, M., Shah, N. J., Zilles, K., & Fink, G. R. (2005). Cortical representations
of personally familiar objects and places: functional organization of the
human posterior cingulate cortex. J Cogn Neurosci, 17(2), 183-198.
Suzuki, W. A., & Amaral, D. G. (1994). Topographic organization of the
reciprocal connections between the monkey entorhinal cortex and the
perirhinal and parahippocampal cortices. J Neurosci, 14(3 Pt 2), 1856-
1877.
Suzuki, W. A., Zola-Morgan, S., Squire, L. R., & Amaral, D. G. (1993). Lesions
of the perirhinal and parahippocampal cortices in the monkey produce
long-lasting memory impairment in the visual and tactual modalities. J
Neurosci, 13(6), 2430-2451.
Tikhonov, A., Handel, B., Haarmeier, T., Lutzenberger, W., & Thier, P. (2007).
Gamma oscillations underlying the visual motion aftereffect. Neuroimage,
38(4), 708-719.
Tong, F., Meng, M., & Blake, R. (2006). Neural bases of binocular rivalry.
Trends Cogn Sci, 10(11), 502-511.
112
Treisman, A. M., & Gelade, G. (1980). A feature-integration theory of attention.
Cogn Psychol, 12(1), 97-136.
Tulving, E. (1987). Multiple memory systems and consciousness. Hum
Neurobiol, 6(2), 67-80.
Valentine, T. (1991). A unified account of the effects of distinctiveness,
inversion, and race in face recognition. Q J Exp Psychol A, 43(2), 161-
204.
Valentine, T., and Bruce, V. (1986). Recognising familiar faces: The role of
distinctiveness and familiarity. Canadian Journal of Psychology, 40, 300-
305.
Wagner, A. D., Koutstaal, W., & Schacter, D. L. (1999). When encoding yields
remembering: insights from event-related neuroimaging. Philos Trans R
Soc Lond B Biol Sci, 354(1387), 1307-1324.
Wallentin, M., Lund, T. E., Ostergaard, S., Ostergaard, L., & Roepstorff, A.
(2005). Motion verb sentences activate left posterior middle temporal
cortex despite static context. Neuroreport, 16(6), 649-652.
Wang, Q., Cavanagh, P., & Green, M. (1994). Familiarity and pop-out in visual
search. Percept Psychophys, 56(5), 495-500.
Watson, T. L., & Clifford, C. W. (2003). Pulling faces: an investigation of the
face-distortion aftereffect. Perception, 32(9), 1109-1116.
Webster, M. A., Kaping, D., Mizokami, Y., & Duhamel, P. (2004). Adaptation to
natural facial categories. Nature, 428(6982), 557-561.
113
114
Wenderoth, P., & Johnstone, S. (1987). Possible neural substrates for
orientation analysis and perception. Perception, 16(6), 693-709.
Westheimer, G., & Truong, T. T. (1988). Target crowding in foveal and
peripheral stereoacuity. Am J Optom Physiol Opt, 65(5), 395-399.
Winawer, J., Huk, A. C., & Boroditsky, L. (2008). A motion aftereffect from still
photographs depicting motion. Psychol Sci, 19(3), 276-283.
Yoshida, T. (1978). Orientation detectors in the human visual system and figural
aftereffects. Perception, 7(6), 717-723.
Zhang, H., Liu, J., Huber, D. E., Rieth, C. A., Tian, J., & Lee, K. (2008).
Detecting faces in pure noise images: a functional MRI study on top-down
perception. Neuroreport, 19(2), 229-233.
Zohary, E., Celebrini, S., Britten, K. H., & Newsome, W. T. (1994). Neuronal
plasticity that underlies improvement in perceptual performance. Science,
263(5151), 1289-1292.