How cortical neurons help us see: visualrecognition in the human brain

Julie Blumberg, Gabriel Kreiman

J Clin Invest. 2010;120(9):3054-3063. https://doi.org/10.1172/JCI42161.

Through a series of complex transformations, the pixel-like input to the retina is convertedinto rich visual perceptions that constitute an integral part of visual recognition. Multiplevisual problems arise due to damage or developmental abnormalities in the cortex of thebrain. Here, we provide an overview of how visual information is processed along theventral visual cortex in the human brain. We discuss how neurophysiological recordings inmacaque monkeys and in humans can help us understand the computations performed byvisual cortex.

Review

Find the latest version:

http://jci.me/42161-pdf

Review series

3054 TheJournalofClinicalInvestigation      http://www.jci.org      Volume 120      Number 9      September 2010

How cortical neurons help us see: visual recognition in the human brain

Julie Blumberg1,2 and Gabriel Kreiman1,3

1Department of Ophthalmology, Children’s Hospital, Harvard Medical School, Boston, Massachusetts, USA. 2Epilepsy Center, University Hospital Freiburg, Freiburg, Germany. 3Center for Brain Science, Harvard University, Boston, Massachusetts, USA.

Throughaseriesofcomplextransformations,thepixel-likeinputtotheretinaisconvertedintorichvisualper-ceptionsthatconstituteanintegralpartofvisualrecognition.Multiplevisualproblemsariseduetodamageordevelopmentalabnormalitiesinthecortexofthebrain.Here,weprovideanoverviewofhowvisualinformationisprocessedalongtheventralvisualcortexinthehumanbrain.Wediscusshowneurophysiologicalrecordingsinmacaquemonkeysandinhumanscanhelpusunderstandthecomputationsperformedbyvisualcortex.

IntroductionMore is known about the functions and deficits of the eye than about the rest of the visual system. However, most visual process-ing occurs in the cortex of the brain. What we see is a heavily trans-formed version of the input provided by photons impinging on the photoreceptors. By and large, that transformation happens in the vast mass of neocortex devoted to visual processing in the primate brain, the visual cortex (Figure 1). Combining systematic studies of the neurophysiology and neuroanatomy of the visual system in human and non-human animal models is providing new under-standing of some of the computations performed in the visual cortex. We hope that this emerging knowledge will play a critical role in the future to help the large number of subjects with visual impairment due to cerebral cortex damage.

Even though many visual deficits that are presented clinically relate to the eyes, the retina mediates only the initial steps in pro-cessing visual information. There are abundant reports in the liter-ature of cases that involve visual deficits that are caused by damage to cerebral cortex. The appearance of visual problems after cortical damage is not surprising, given that a large part of the cerebral cortex serves visual processing functions (1–5).

Among the many functions of vision, object recognition is arguably one of the most crucial. Visual object recognition is essential for most everyday tasks, including reading, navigation, and face identification. In a small fraction of a second (approxi-mately 150 ms), we can recognize complex shapes and categorize objects and scenes (3, 6–10). Visual object recognition depends on combining two key properties: visual selectivity and the robust-ness of recognition to object transformations. Visual selectivity refers to the ability to discriminate among similar objects. A key to vision is that we can successfully recognize objects despite transformations in size, position, rotation, and illumination, among others (Figure 2). Given that an object can cast an infi-nite number of projections on the retina, our ability to recognize objects in a manner that is robust to object transformations is likely to have played a key role in the evolution of the visual sys-tem. This robustness to image transformations constitutes one of the major challenges for computer-based visual recognition, because most of the image pixels can change dramatically while the content remains the same.

Here, we provide an overview of the processing steps along the ventral visual cortex in the human brain, focusing on what is known (and what is not known) about the mechanisms respon-sible for visual recognition. Because it is difficult to directly access human visual cortex through invasive methodologies, much more is known about the macaque monkey visual cortex than about the human visual cortex. Throughout this Review, we discuss data generated from the analysis of both the macaque monkey and the human visual system. We hope that our article provides an over-view of a small fraction of the studies of visual cortex. The goal here is not to be comprehensive, as there is insufficient space for this; rather, we hope that this article will arouse readers’ curiosity, and we encourage them to peruse more literature.

Neuroanatomy of the visual systemWe start by providing an overview of neural connectivity in the primate visual system based on anatomical studies in macaque monkeys (Figure 1; for more details, see refs. 1 and 2). Vision starts when photons impinge on photoreceptors in the retina (for an overview of the retinal circuitry, see refs. 2, 11, 12). Information flows through the retinal circuit from the photoreceptors to retinal ganglion cells. These are the cells that provide the retinal output. Retinal ganglion cells project through axons in the optic nerve to a part of the thalamus called the lateral geniculate nucleus (LGN) (2). Neurons in the LGN project to the primary visual cortex (V1), which is located in the occipital neocortex. Like other parts of neo-cortex, V1 is composed of six layers, many of which can in turn be subdivided into sublayers (13–16). Neurons in V1 are organized into columns, approximately perpendicular to the cortical surface, where neurons share similar functional properties (16, 17). V1 is the first stage of visual processing, where information from the right and left eyes is combined (Figure 1).

Perhaps due to its relative accessibility, much more is known about V1 than about any other part of the visual cortex. Yet there are a large number of different visual areas in the neocortex out-side of V1; these are collectively known as the “extrastriate visual cortex.”  From  V1,  two  main  pathways  can  be  distinguished: a dorsal stream and a ventral stream (1, 16, 18, 19). The dorsal stream includes areas known as area medial temporal (area MT, also known as area V5) and medial superior temporal (MST) and is predominantly  involved  in processing spatial  information, including detecting object motion, position, and stereovision, i.e., depth perception; the dorsal stream is often referred to as 

Conflictofinterest: The authors have declared that no conflict of interest exists.

Citationforthisarticle: J Clin Invest. 2010;120(9):3054–3063. doi:10.1172/JCI42161.

review series

TheJournalofClinicalInvestigation      http://www.jci.org      Volume 120      Number 9      September 2010  3055

the “where/how” pathway (20, 21). The ventral stream includes areas such as V2, V4, and the inferior temporal cortex (ITC) and is predominantly involved in discriminating colors and shapes; the ventral stream is often referred to as the “what” pathway  (3, 22–24). To a first approximation, we can think about the visual cortex as two parallel and hierarchical cascades (Figure 1). How-ever, this is only a coarse approximation. There are also numerous interconnections between these two pathways. The V1 to V2, to V4, to ITC pathway is often thought of as a feed-forward pathway (whereby information flows “up” the visual hierarchy). In addi-tion to these forward connections, there are backward connec-tions at every stage of the visual system; the only exception to this rule is the retina: there are no known back-projections from the LGN to the retina. These back-projections are likely to play a key role in top-down influences in visual perception. Thus, the ITC projects back to V4; V4 projects back to V2; and so on. There are also numerous internal recurrent connections within each area. Furthermore, “bypass connections” link, for example, V1 directly to V4 and the ITC directly back to V2.

The last purely visual stages in the “what” and “where/how” streams project to multiple nonvisual areas of the cortex. Among them, two important targets of these projections are areas of the prefrontal  cortex and the medial temporal lobe, including the hippocampus and surrounding structures. The prefrontal cortex is likely to play a key role in the moment-to-moment and task-dependent interpreta-tion of visual input to orchestrate cognitive decisions (25, 26), while the hippocampus and surrounding structures play a key role in the consolidation of information into long-term memories (27, 28).

The description in this section provides a simplified overview of the neuroanatomy of the visual system. The system is highly complex (with billions of interconnected neurons), and at this stage we probably know only a small fraction of the connections in neocortex. There has been much excitement recently in the field of connectomics, in which researchers are aiming to develop high-throughput tools to map connections at high resolution within a small part of neocortex.

Visual deficits due to cerebral cortex lesionsThere are multiple clinical cases where the patient does not have any apparent deficit in eye function and yet manifests profound visual deficits. In several cases, these deficits can be ascribed to damage to specific areas within the neocortex. The study of these patients has provided key insights about visual recognition by providing a correlate between brain areas and functions. We pro-vide an overview of some of these studies, highlighting what we have learned about the visual cortex. We start the discussion by describing the functional consequences of damage to early visual cortex. Lesion and neurophysiological studies in cats, and sub-sequently in monkeys, led the way to uncover a multiplicity of visual areas outside V1 (1, 20, 22, 29–31). We summarize below some of these visual deficits caused by damage to extrastriate visual areas (e.g., refs. 22, 32–35), dividing the discussion into damage to dorsal visual cortex and damage to ventral visual cor-tex. Although the cases involving damage to ventral visual cortex are particularly relevant for our discussion on visual recognition, we describe some studies involving dorsal stream damage because 

Figure 1Schematic illustration of the basic architec-ture in the primate visual system. On the left, we show a schematic diagram of the human brain indicating the approximate location of several brain areas discussed in the text. On the right, we provide a schematic description of some of the main pathways involved in pro-cessing visual information in cortex. VP, ven-tral stream; DP, dorsal stream; RGC, retinal ganglion cells; CIT, central ITC; AIT, anterior ITC; MST, medial superior temporal cortex; POR, post-rolandic area; PR, pre-rolandic area. Solid lines indicate forward projections; dashed lines indicate back-projections; and the curved lines represent recurrent connec-tions within a given area. As emphasized in the text, this diagram is a substantial simplifi-cation of the actual connectivity in the primate visual system. Many important visual areas and connections are not represented in this diagram; for more details, see ref. 1.

review series

3056 TheJournalofClinicalInvestigation      http://www.jci.org      Volume 120      Number 9      September 2010

they vividly illustrate the relationship between location and dam-age and also because of the multiple interconnections between the dorsal and ventral processing streams. Many of these conditions are described in single case studies, and it is generally difficult to rigorously quantify their prevalence; some studies suggest that the combination of visual deficits due to cortical damage is even more prevalent than the combination of visual deficits that can be ascribed to the eyes (32).

Deficits associated with lesions in early visual cortex. The fundamental role of V1 in vision was appreciated early in the last century. Stud-ies in cats, dogs, and monkeys that involved complete lesion of the occipital cortex left animals with little if any visual function (for a historical review, see ref. 31). After World War I, the study of subjects who suffered limited occipital lobe lesions due to bullet wounds revealed localized scotomas (i.e., islands of impaired visual acuity) in the part of the visual field contralateral to the wound (36, 37). The more severe condition hemianopia refers to loss of vision in one-half of the field of vision. Effects similar to those caused by bullet wounds in the occipital lobe are often encoun-tered through vascular damage, tumors, and trauma.

A few case studies have discussed a phenomenon referred to as “blindsight” (38, 39). As the name suggests, some subjects with lesions in the occipital cortex are still capable of certain visual behavior within the scotoma. For example, subjects may be able to tell light from dark, or even to detect the direction of a mov-ing object. Several possibilities were proposed to account for these observations, including anatomical routes that bypass V1 and the presence of small intact islands in V1 despite the lesions. Although the phenomenon has been clearly demonstrated, the range of visual behaviors preserved within the scotoma is rather limited, 

and the subjects’ capacity for fine visual discrimination within it is clearly lost. V1 is absolutely necessary for most visual functions. The profound visual deficits caused by V1 lesions in both animals and humans, combined with the challenges in examining visual behavior in animals, led several prominent investigators to argue that V1 is not only necessary but also sufficient for visual percep-tion. In an interesting historical overview, Gross cites several strik-ing demonstrations of this narrow-minded perception that we now know to be completely wrong (31).

Deficits associated with lesions in dorsal visual cortex. One of the condi-tions in which the extrastriate visual areas of the dorsal stream are damaged is akinetopsia, which involves a specific inability to dis-criminate visual motion (40, 41). Zihl et al. described a patient who suffered a bilateral cerebral vascular lesion, outside V1, and who had a specific impairment in her ability to see objects in motion (41). The lesion included the posterior portion of the middle tem-poral gyrus, the retrorolandic area, and the temporoparietal and occipital white matter. It is quite tempting to ascribe the deficits in this subject to area MT, as several investigators have described neu-rons that are sensitive to the direction of motion in that region of the macaque monkey visual cortex (21). As in other human lesion studies, although the lesion in this subject does include the likely human homolog of area MT (40), making a precise connection is challenging due to the relatively large region of cortex impaired in this patient. It is interesting to note that electrical stimulation through large grid electrodes of area MT of the human brain can induce direction-specific motion blindness, i.e., blindness to motion in one direction while perception of motion in other directions is completely normal (42). Dutton described the opposite defect in a woman who could not see immobile objects following bioccipital infarctions (32). While the dorsal pathway was apparently intact, she was able to navigate independently only when she moved her head or eyes to see the environment. These studies suggest that the ability to discriminate specific aspects of the visual image (motion or lack thereof) can be correlated with the presence of intact cortex within relatively circumscribed parts of visual cortex.

Damage to the parietal cortex is often associated with differ-ent variations of neglect syndrome. While this is not necessarily a strictly visual deficit, it is often manifest as partial blindness. Individuals with this condition are unable to perceive part of the environment unless they pay direct attention to the neglected area. The most common form is left hemispatial neglect (i.e., an inability to perceive objects in the left side of the visual field, caused by posterior parietal lobe lesions in the contralateral right hemisphere) (35, 43–46). A syndrome that is often associated with hemineglect is Balint syndrome, which is characterized by the triad of simultanagnosia, optic ataxia, and ocular apraxia. Simul-tanagnosia refers to the inability to identify several objects in a scene at once. Patients perceive the world in fragments and are 

Figure 2Robustness of the human visual system to image transformations. In spite of large transformations of the original image (top left) in terms of color, noise, scale, blurring, and inversion, recognition remains rela-tively straightforward. Careful psychophysics studies have quantified and documented the robustness of the human visual system to image transformations. Achieving specificity while maintaining tolerance to object transformations is one of the key challenges for computer vision and constitutes one of the hallmarks of object recognition in primates. Original photo taken by Julie Blumberg.

review series

TheJournalofClinicalInvestigation      http://www.jci.org      Volume 120      Number 9      September 2010  3057

unable to form the whole picture. They typically show difficulty in recognizing a toy in a cluttered box or a person in a crowd, but they do not have a problem identifying the person or toy when it is isolated. Playing team sports or coping in busy environments (e.g., a supermarket or a busy playground) can also be challenging. Optic ataxia and ocular apraxia are conditions in which the visual input to the eyes cannot be used for visually guided movements and coordination of eye movements, respectively. Patients with Balint syndrome are also characterized by an impaired ability to estimate distances. In six patients with Balint syndrome show-ing similar visual deficits, bilateral lesions in the splenium and angular and supramarginal gyri were observed. The etiology of Balint syndrome is thought to be most frequently bilateral pari-etal damage, although lesion localization is difficult due to the small number of cases (32, 47, 48).

Deficits associated with lesions in ventral visual cortex. Several pro-found visual deficits have also been described as occurring as a result of damage along the ventral visual stream. These conditions are particularly relevant to understanding how ventral cortex contributes to visual recognition. One such condition, known as achromatopsia, involves a specific inability to perceive colors (30). This cortical color blindness is distinct and dissociable from reti-nal color blindness, which can be linked to missing or abnormal color pigments in the retinal cones. Subjects with achromatopsia typically have relatively large lesions in the inferior ventromedial sector of the occipital lobe, including the lingual and fusiform gyri. Depending on the extent of the lesion, subjects may also dis-play a scotoma within their visual field and more specific visual recognition deficits (35). These subjects show poor performance in hue discrimination tasks but perform well in tasks that involve verbal instructions such as answering the question “What color are bananas?” emphasizing the visual nature of the deficit.

An interesting dissociation between the dorsal and ventral streams was reported in a patient with profound ventral cortex damage, particularly within the temporal lobe, by Goodale and Milner (34). This subject, who had suffered from carbon monoxide poisoning, had severe impairment in object shape recognition. Yet despite her inability to recognize objects, she showed a rather remarkable abil-ity to interact with many objects. She showed an appropriate reach response toward objects whose shapes she could not describe. She also showed correct behavioral performance in visuomotor tasks. For example, in one of the experiments, the subject was asked to indicate the orientation of a slit and performed essentially at chance levels. Subsequently, she was asked to push an envelope through the slit. Surprisingly, the subject was almost as good as controls in this visually guided motor task despite the fact that she was unable to describe the orientation of the slit. Goodale and Milner proposed that the dorsal pathway is particularly engaged in “vision for action,” the immediate use of visual information to carry out specific visual-ly guided behaviors. In contrast to this action mode, they proposed that awareness about an object requires activity in the ventral stream and the temporal lobe in particular.

One of the earliest demonstrations that V1 could not be the entire story in terms of visual recognition and that there must be  extrastriate  visual  function  was  the  study  of  Kluver-Bucy syndrome (49). After bilateral removal of the temporal lobe in macaque monkeys, a variety of behavioral effects, including loss of visual discrimination, but also increased tameness, hypersexu-ality, and altered eating habits, were initially reported (49). The work was subsequently refined by inducing smaller lesions that 

were able to dissociate different aspects of Kluver-Bucy syndrome. These circumscribed lesions showed that the ITC plays a critical role in visual recognition (50–53). Bilateral removal of the ITC in macaque monkeys leads to impairment in learning visual shapes, as well as deficits in retaining information about visual discrimi-nations that were learned before the lesions. The deficits are long lasting, and the severity of the deficit is typically correlated with task difficulty. In other words, monkeys can still perform “easy” visual discrimination tasks after bilateral ITC lesions. The behav-ioral deficits are restricted to the visual domain and do not affect discrimination based on tactile, olfactory, or auditory inputs. The deficits after ITC lesions are restricted to the visual domain. The other behavioral abnormalities in Kluver-Bucy syndrome were not present when the lesions were restricted to the ITC. This observa-tion underscores the importance of studying spatially restricted lesions (whenever possible) to properly interpret behavioral defi-cits and highlights the multiple confounding factors that arise when considering lesions that involve multiple brain locations. In the same way that Kluver-Bucy syndrome could be fractionated by more detailed and circumscribed lesions, it is quite likely that in future studies, more specific lesions within the ITC will further fractionate the object recognition deficits prevalent after bilateral ITC ablation. The lesion studies suggest that the ITC plays an important and necessary role in fine object discrimination.

In humans, to our knowledge, there is no evidence for a global object recognition deficit as a result of brain damage. Temporal lobe lesions typically lead to different forms of agnosias, an inability to recognize certain types of objects (22, 54). The variety of behav-ioral manifestations in the cases of agnosia described in the litera-ture can be to a first approximation linked to the variety of brain regions damaged. Because these are not controlled lesion studies, as in the animal cases, the exact circuits that are affected vary from one patient to another. Perhaps the most widely discussed form of agnosia (although itself a rare condition) is prosopagnosia, a deficit in recognizing faces even though vision in general and recognition of other objects is not impaired (55–60). This condition is often caused by a stroke that affects the right posterior artery, which leads to obstruction of the fusiform and lingual gyri. Other etiologies for prosopagnosia have also been described (e.g., refs. 61–63).

The exact interpretation of the nature of the deficit associated with prosopagnosia has been a matter of debate in the neuroscience community. While some researchers emphasize that it is a deficit in recognizing the special category of face stimuli (55, 58, 64), other investigators consider this to be a specific instance of a more gen-eral impairment in recognizing specific exemplars that look alike within a category (65). In addition to acquired prosopagnosia, a few studies have described cases of apparent congenital prosopagnosia. The causes of congenital prosopagnosia are not clear. In these cases, patients sometimes may not even be aware of their deficit, as they eas-ily recognize people by their clothes, voices, gait, and other clues.

Other forms of agnosia have also been described. For example, some studies have reported subjects that  fail  to discriminate between animate and inanimate objects (66). Other reports have emphasized specific types of impairment for certain object cat-egories (35, 67–71). Topographic agnosia refers to the inability to navigate (32). The extent to which these cases of agnosia are restricted to the visual modality remains unclear. To show that a form of agnosia is purely visual, it is necessary to test recogni-tion using other cues, and this has not been done in all studies. Thus, it is conceivable that at least some of these cases involve a 

review series

3058 TheJournalofClinicalInvestigation      http://www.jci.org      Volume 120      Number 9      September 2010

more generic failure related to information retrieval or semantic interpretation that is linked to but goes beyond pure visual rec-ognition. There is also debate about how to define such seman-tic categories (e.g., consider a definition of “inanimate”). Visual form agnosias have been reported following disseminated lesions or medial temporal stroke (34, 35, 72).

Impaired reading ability is also a common deficit, particularly in children. There is a very large variety of conditions that manifest as reading difficulties, only a few of which directly relate to cortical visual impairment (32, 73). Hemianopia, alexia (typically a discon-nection between the right occipital lobe and the language centers), simultanagnosia, and temporal lobe dyslexia have all been associ-ated with poor reading.

Neurophysiological studies of the visual cortexIn order  to understand the  function of  the neuronal circuits that process visual information, it is necessary to examine the activity of individual neurons in response to visual stimuli. The main method of recording the activity of individual neurons in awake behaving animals or humans is to use microwire electrodes (microelectrodes) that are implanted extracellularly and monitor the extracellular voltage at millisecond temporal resolution and neuronal spatial resolution. Due to technical limitations and the invasive nature of microwire recordings, little is known about the activity of individual neurons in the human cortex. A recent review summarized the properties of neurons in the human temporal lobe upon presentation of visual stimuli (ref. 5; see also refs. 7, 74).

Primary visual cortex. More work has been done examining the activity of single neurons in the cat and monkey brains compared with the human brain (5). Neurophysiological recordings in reti-nal ganglion cells and LGN neurons have revealed that neuronal responses are constrained to a relatively small patch of the visual field known as the receptive field (75) (Figure 3). The receptive field of a neuron is functionally defined as the region of the visual field that triggers activity above baseline levels. There is a consid-erable degree of variability in the size of the receptive field among different neurons. In particular, neurons with receptive fields in the fovea (the most sensitive part of the retina) have very small receptive fields, typically well below one degree of visual angle, whereas neurons with receptive fields in the periphery can have receptive field sizes up to one degree of visual angle or more. For 

orientation, one degree of visual angle approximately corresponds to the size of the thumb at arm’s length. In addition to the size, receptive fields typically have an important substructure that indicates how light patterns influence the response of the neuron. The receptive fields of retinal ganglion cells and LGN cells are typ-ically characterized by a center-surround architecture. Two main types of cells have been described: “on-center” cells enhance their activity when stimulated in the center of their receptive fields and are inhibited by illumination of the surround, while “off-center” cells show the opposite responses (2).

Hubel and Wiesel pioneered the investigation of V1 through microwire recordings and opened the way for examining visual cortex neurophysiology (76). Initially working in cats, and subse-quently in monkeys, they discovered that neurons in V1 typically respond to bars shown at a specific orientation within their recep-tive fields (17, 76, 77). The receptive field size for V1 neurons is typ-ically below one degree of visual angle. Their responses are tuned to the orientation of the bar; this type of response is similar to the type of operations used in computer vision to extract the edges of an image (78, 79) (Figure 4). This has led to the notion that one of the initial stages in processing visual information in the cor-tex is related to the extraction of edges in the image. Neurons in V1 are retinotopically organized, arranged within columns where neurons share similar response preferences. Hubel and Wiesel also proposed a model of how the orientation tuning in V1 could arise by combining multiple on-center LGN cells in a feed-forward fash-ion, whereby the centers of the LGN cells follow the orientation preferred by the V1 neuron. Neurons in V1 are also activated by motion, and some neurons are sensitive to color (80).

Although V1 remains by far the most studied part of visual cor-tex, multiple investigators have examined the responses of neurons in extrastriate visual areas. Overall, less is known about the prop-erties of extrastriate neurons. Partly, this can be ascribed to the difficulty in examining the vast and high-dimensional visual space with limited recording time. Until recently, most neurophysiology experiments involved inserting and removing a single electrode, holding a recording for time periods of hours at best. More recent-ly, several investigators have started to explore the possibility of using chronic recording technologies that may enable recording for prolonged periods of time. Given the limits in recording time and the vast number of possible stimuli, investigators often use a battery of visual stimuli that are presumed to be of interest to the neurons under study. A couple of general observations can be drawn about the responses in extrastriate visual cortex. As we ascend through the visual hierarchy (along both the dorsal and the ventral streams), there is a progressive increase in receptive field size, in response latencies, in the complexity of feature preferences, and in the degree of tolerance to image transformations. Addition-

Figure 3Schematic definition of receptive fields. Neurons throughout the visual system respond only to local patches of the visual field. This schematic diagram shows an experimental protocol in which the subject’s eyes are fixated on the center X while investigators show circles at differ-ent positions (at each moment in time, there is only one circle on the screen). Above each circle, the ticks indicate the activity of a hypotheti-cal neuron. This neuron fires vigorously when the circle appears in the lower left position, defining its receptive field (arrow). Note that there is spontaneous activity in other locations, but only one location elicits a vigorous response. 0.25 deg, 0.25 degree of visual angle.

review series

TheJournalofClinicalInvestigation      http://www.jci.org      Volume 120      Number 9      September 2010  3059

ally, neurons in higher visual areas typically show stronger modu-lation by attention compared with neurons in earlier visual areas (81, 82). That is, the activity of a neuron in response to a stimulus is substantially enhanced if the subject is paying attention to the receptive field of the neuron compared with the response to the same stimulus if the subject is paying attention elsewhere.

Neurophysiology along the dorsal stream. In the dorsal stream, V1 projects to area MT. Neurons in area MT show tuning to the direc-tion of motion within their receptive fields, that is, they respond preferentially to certain directions of motion (21). MT neurons are also sensitive to disparity between the two eyes (see ref. 21). Electrical stimulation of groups of MT neurons has been shown to bias the perception of monkeys toward the preferred motion direc-tion of the neuron (83). Lesions in area MT lead to motion percep-tion deficits in monkeys (21). Combining the neurophysiological, electrical stimulation, and lesion findings in the macaque, one can speculate that area MT may play an important role in the condi-tion known as akinetopsia mentioned in the previous section.

Neurophysiology along the ventral stream. Investigators have described  neurons in the ventral stream that respond to a variety of com-plex shapes in the ITC (22–24, 31). ITC neurons often respond vigorously to color, orientation, texture, direction of movement, and shape. Neurons in the posterior ITC (PIT) show a coarse reti-notopic organization and an almost complete representation of the contralateral visual field. The receptive field sizes are typically larger than the ones found in V4 neurons. The receptive fields in anterior parts of the ITC are often large, but there is a wide range of estimates of the sizes of receptive fields in the literature, ranging from two degrees (84) to several tens of degrees (85, 86). Most ITC receptive fields include the fovea, i.e., the central pit in the retina, which shows maximal visual acuity.

Investigators have often found strong responses in ITC neurons elicited by all sorts of different stimuli. For example, several inves-tigators have shown that ITC neurons can be stimulated when faces, hands, and other biological stimuli are presented (87–91). Figure 5 shows an example of a neurophysiological recording from a neuron in the human medial temporal lobe, which receives input from the ITC. This neuron showed a remarkable degree of selec-tivity and tolerance to transformations (92). Other investigators have used parametric shape descriptors of abstract shapes (93–95). Logothetis and Pauls trained monkeys to recognize paperclips forming different 3D shapes and subsequently found neurons that were selective for specific 3D configurations of paperclips (96). While this wide range of responses may appear puzzling at first, it is perhaps not too surprising given a simple model whereby ITC 

neurons are tuned to complex shapes. Our interpretation of the wide number of stimuli that can drive ITC neurons is that these units are sensitive to complex shapes that can be found in multi-ple different types of 2D visual stimuli, including fractal patterns, faces, and paperclips. This wide range of responses also empha-sizes that we still do not understand the key principles and tuning properties of ITC neurons. Tanaka and others have shown that there is clear topography in the ITC response map. Advancement of an electrode in a trajectory approximately tangential to cortex has revealed that neurons within a tangential penetration show similar visual preferences (86, 97–99). They argue for the presence of “columns” and higher-order structures such as “hypercolumns” in the organization of shape preferences in ITC.

While each neuron shows a preference for some shapes over oth-ers, the amount of information conveyed by individual neurons about overall shape is limited (85). Additionally, there seems to be a substantial amount of “noise” in the neuronal responses in any given image presentation. Hung et al. recorded (sequential-ly) from hundreds of neurons and used statistical classifiers to decode the activity of a population of neurons in individual pre-sentations (100). They found that a relatively small group of ITC neurons (approximately 200) could support object identification and categorization quite accurately (up to approximately 90% and approximately 70% for categorization and identification, respec-tively), with a very short latency after stimulus onset (approxi-mately 100 ms after stimulus onset). Furthermore, the population response could extrapolate across changes in object scale and posi-tion. Thus, even when each neuron conveys only noisy information about shape differences, populations of neurons can be quite pow-erful in discriminating among visual objects in individual trials.

As emphasized above, a key property of visual recognition is the capacity to recognize objects despite image transformations at the pixel level. Several studies have shown that ITC neurons show a substantial degree of tolerance to object transformations. ITC neurons show similar responses despite large changes in the size of the stimuli (96, 101, 102). Even if the absolute firing rates are affected by the stimulus size, the rank order preferences among different objects can be maintained despite stimulus size changes (101). ITC neurons also show more tolerance to changes in object position than neuronal units in earlier parts of the ventral stream (96, 101, 102) and can tolerate a certain degree of depth rotation (22). They even show tolerance to the particular cue used to define the shape (such as luminance, motion, or texture) (103).

There has been increased interest recently in the possibility of exam-ining the human brain at the neurophysiological level in cases where electrodes are implanted for clinical reasons. The most common of these cases involves the study of epileptic patients (5, 7, 74, 104).  These recordings have been performed in subjects that have forms of epilepsy resistant to pharmacological treatments. Subjects are implanted with electrodes in order to map the region of the brain responsible for the seizures and to examine cortical function for potential surgical treatment of epilepsy. This approach provides a rare opportunity to examine neurophysiological activity in the human brain at high spatial and temporal resolution. Recordings in the human hippocampus, entorhinal cortex, amygdala, and parahippocampal gyrus have shown strong responses to visual stimuli and a substantial degree of tolerance to object transforma-tions (5, 105) (Figure 5). In another study recording from these areas, investigators found neurons that showed responses to mul-tiple objects within a semantically defined object category such as 

Figure 4An edge detection algorithm applied to the image on the left produced the image on the right. Edge detection is thought to mimic one of the initial steps in processing visual information in primary visual cortex (2, 79, 136). Original photo taken by Nambi Nallasamy.

review series

3060 TheJournalofClinicalInvestigation      http://www.jci.org      Volume 120      Number 9      September 2010

faces, animals, and cars (105). While these areas receive input from the highest stages of the visual hierarchy (Figure 1), they are mul-timodal areas and do not seem to be directly involved in the visual deficits that are typically described in subjects with neocortical damage. It is also possible to examine the responses of visual areas in the human neocortex by recording intracranial field potentials, also in the context of epilepsy surgery (7, 106). These recordings have revealed selectivity to complex shapes and tolerance to object transformations (7, 106).

Recording in patients with epilepsy has provided an extreme example of tolerance to object transformations. Some neurons in the human hippocampus, parahippocampal gyrus, entorhinal cortex, and amygdala show a remarkable degree of selectivity to individual persons or landmarks. For example, one neuron in one subject showed a selective response to images in which for-mer president Bill Clinton was present (Figure 5). Remarkably, the images that elicited a response in this neuron were quite distinct in terms of their pixel content, ranging from a black-and-white drawing to color photographs with different poses and views (92). As discussed above for the macaque ITC neu-rons, we still do not have any understanding of the circuits and mechanisms that give rise to this type of selectivity or tolerance to object transformations.

How is the selectivity to oriented bars at the level of V1 trans-formed into the responses to complex shapes and tolerance to object transformations at the level of ITC? The current line of thought is that shape preferences and tolerance to object trans-formations build up gradually as we ascend the visual hierarchy (24, 85, 107, 108). Consistent with this notion, neurons in areas V2 and V4 typically show larger receptive fields and responses to more complex shapes than do neurons in V1 but smaller receptive fields and less complexity and tolerance to object transformations than neurons in the ITC (109–112). Much more work is needed to elucidate the computations that lead to shape recognition along the ventral visual pathway.

Open questions and future workTranslating visual neuroscience to the clinic. As noted above, there are a wide variety of visual deficits that arise as a result of damage to different parts of the visual cortex. It is rather tempting to link the visual deficits described above (in “Visual deficits due to cerebral cortex lesions”) and the neurophysiological findings (described in “Neurophysiological studies of the visual cortex”). However, this requires solving several challenges. As mentioned earlier, most human lesions are not well circumscribed and involve multiple brain areas as well as neurons connecting different areas. It is there-

Figure 5Example of a neurophysiological recording from a neuron in the medial temporal lobe. The figure is based on an experiment per-formed by our research group, in which the images shown to participants were similar to those shown here (see ref. 137 for the actual images and ref. 138 for more examples). The raster plots indicate the action potentials fired by the neuron in response to the image above the raster. Each line represents a separate repetition. Below the raster plots, we show a post-stimulus time histogram (PSTH) aligned to the stimulus onset, showing the average activity of the neuron as a function of time. The activity of this neuron increased in response to three different images of former president Bill Clinton and not to other faces or other stimuli. The neuron showed a remarkable degree of selectivity and tolerance to transformations (92, 137). Image credit (tiger photo): www.freedigitalphotos.net.

review series

TheJournalofClinicalInvestigation      http://www.jci.org      Volume 120      Number 9      September 2010  3061

fore challenging to establish a one-to-one map between visual defi-cits and anatomical locations (it is also not clear whether there is a one-to-one map). The neuroanatomical connectivity in the human visual cortex is also much less well established than in the macaque (and even in the macaque, we only know some of the coarse con-nectivity patterns). Additionally, most of the neurophysiological data come from primate studies, and the homology of visual areas between humans and monkeys is not yet clearly established.

Some of these visual challenges may be subtle and hard to diagnose while others are dramatic and obvious. Dutton has provided several detailed strategies for the diagnosis of visual impairment due to cor-tical factors (32, 113). Although it might seem easier to treat visual deficits originating from the eye than from the brain, physicians can still help in cases of cerebral visual impairment. A useful and practi-cally oriented review of therapeutic options can be found in ref. 113. Orientation, object recognition, and visually guided movements can be partly trained. The remaining partial function can be supported by several strategies, including enlarging text, reducing distractions, usage of good colors and contrasts, usage of specific visual cues, and other problem-specific solutions. For example, attentional deficits such as hemineglect can be tackled by clear markers such as colored stickers in the affected hemifield or approaches such as turning the plate 180° during every meal. Visual object recognition problems can be ameliorated by training recognition with nonvisual cues including labels and text. Even though most often there is no causal treatment of cerebral visual impairment, there is a lot to be done to improve the quality of life of the affected individuals. More research and effort is needed to translate the knowledge gained from the last four decades of studying visual cortex into methods to help patients who show visual impairment due to cortical damage or malfunction.

Neuroimaging. There are a large number of studies that use human neuroimaging to bypass the single-unit recordings and examine indirect correlates of human brain function in a noninvasive man-ner. This is a promising research tool given the potential to examine the human brain in a noninvasive manner. As it stands, the spatial and temporal resolution is very poor compared with that of neuro-physiological recordings. The spatial resolution is approximately 1 mm3, which probably encompasses at least tens of thousands of neu-rons as well as other cells (compare with neurophysiological record-ings that examine the activity of individual neurons). The temporal resolution is on the order of seconds, which is about 1,000 times slower than the speed of the dynamical signals that neurons use to communicate with each other in cortex. Part of the problem arises because current imaging techniques only indirectly measure neural activity. In one of the most common techniques, magnetic reso-nance imaging signals are used to infer blood oxygenation levels; blood oxygenation levels are in turn indirectly related to neural activ-ity (for a detailed review of functional imaging, see refs. 114–117).  Several studies have compared functional imaging measurements against neurophysiological recordings. Some of these studies have revealed a rather striking correlation between the two types of mea-surements (e.g., refs. 118, 119). However, other studies have suggest-ed that the relationship between these two signals is quite complex and have described several situations where functional imaging sig-nals can appear in the absence of concomitant neural activity and vice versa (e.g., refs. 115, 117, 120–122). Therefore, caution must be 

taken in interpreting human neuroimaging studies. For examples of recent neuroimaging work on human visual cortex, see refs. 18, 58, 123–131. In addition to functional studies, there is increasing enthusiasm regarding the usage of imaging techniques to examine other aspects of brain structure in the human brain. In particular, diffusion tensor imaging holds promise as a key tool to investigate connectivity in the human brain.

Computational models of visual cortex. Several investigators are devel-oping computational models of the ventral visual cortex. These computational models may play an important role in helping us quantitatively understand the complex and dynamic processes involved in visual recognition (e.g., refs. 3, 108, 132–134). In the future, one can imagine that we might be able to induce “lesions” in the models to examine the type of visual deficits caused by these lesions and to compare their output against the visual deficits that patients experience. While we still have a long way to go, combined with the neurophysiological, neurological, and imaging studies described above, this is a very promising line of investigation. In addition to these computational studies, much progress has also been made through the quantitative study of visual recognition in psychophysics (e.g., see ref. 135 for an overview).

Visual prosthetics.  Combining  the  knowledge  and  techniques described in this article, some investigators are trying to develop prosthetic devices to help the visually impaired and even blind peo-ple. The field of prosthetic devices has been particularly active in the context of paralyzed patients and motor deficits. In the motor domain, the idea is to “read out” cortical signals, decode them in real time, and use them to actuate a robotic device. In the visual domain, one idea is to use digital devices to capture light informa-tion and then “write in” this information to the subject’s brain. Investigators are thinking about neural prosthetic approaches at multiple different levels in the visual hierarchy, from the retina (retinal prosthesis) to primary visual cortex all the way to higher visual cortex (cortical prosthesis). Due to space constraints, we cannot describe these approaches in detail, but we would like to emphasize that the field of visual prosthetics is possible thanks to a combination of talent and dedication from researchers in multiple fields, including those involved in neurophysiological recordings, psychophysics, cognitive science, neurology, neurosurgery, neu-roimaging, engineering, applied mathematics, machine learning, and engineering. This is a highly active and interdisciplinary area of research that may one day transform the lives of people with severe visual impairment that cannot be treated by other means.

AcknowledgmentsWe acknowledge support from the Massachusetts Lions Foundation, the NIH (1R21EY019710, 1DP2OD006461 to G. Kreiman), and the National Science Foundation (0954570 to G. Kreiman). We also acknowledge support from the Hamburger Foundation (J. Blum-berg) and the German National Merit Foundation (J. Blumberg).

Address correspondence  to: Gabriel Kreiman, Department of Ophthalmology, Children’s Hospital, Harvard Medical School, 1 Blackfan Circle, Karp 11217, Boston, Massachusetts 02115, USA. Phone: 617.919.2530; Fax: 617.919.2772; E-mail: gabriel.kreiman@tch.harvard.edu.

  1. Felleman DJ, Van Essen DC. Distributed hierarchi-cal processing in the primate cerebral cortex. Cereb Cortex. 1991;1(1):1–47.

  2. Wandell  BA.  Foundations Of Vision.  Sunderland, 

Massachusetts, USA: Sinauer Associates Inc; 1995.  3. Serre T, Kreiman G, Kouh M, Cadieu C, Knoblich 

U, Poggio T. A quantitative theory of immediate visual recognition. Prog Brain Res. 2007;165:33–56.

  4. Deco  G,  Rolls  ET.  Computational Neuroscience of Vision. New York, New York, USA: Oxford Univer-sity Press; 2004.

  5. Kreiman G. Single neuron approaches to human 

review series

3062 TheJournalofClinicalInvestigation      http://www.jci.org      Volume 120      Number 9      September 2010

vision  and  memories.  Curr Opin Neurobiol. 2007;17(4):471–475.

  6. Potter  M,  Levy  E.  Recognition  memory  for  a rapid sequence of pictures. J Exp Psychol. 1969; 81(1):10–15.

  7. Liu H, Agam Y, Madsen JR, Kreiman G. Timing, timing, timing: fast decoding of object informa-tion from intracranial field potentials in human visual cortex. Neuron. 2009;62(2):281–290.

  8. Thorpe  S,  Fize  D,  Marlot  C.  Speed  of  pro-cessing  in  the  human  visual  system.  Nature. 1996;381(6582):520–522.

  9. vanRullen R, Thorpe S. Surfing a spike wave down the ventral stream. Vision Res. 2002;42(23):2593–2615.

  10. Schyns PG, Oliva A. Flexible, diagnosticity-driven, rather than fixed, perceptually determined scale selection in scene and face recognition. Perception. 1997;26(8):1027–1038.

  11. Cepko CL. Genomics approaches to photorecep-tor development and disease. Harvey Lect. 2001–2002;97:85–110.

  12. Kandel E, Schwartz J, Jessell T. Principles of Neural Science. 4th ed. New York, New York, USA: McGraw-Hill Medical; 2000.

  13. Callaway EM. Local circuits in primary visual cor-tex of  the macaque monkey. Annu Rev Neurosci. 1998;21:47–74.

  14. Callaway EM. Feedforward, feedback and inhibi-tory connections in primate visual cortex. Neural Netw. 2004;17(5–6):625–632.

  15. Douglas RJ, Martin KA. Neuronal circuits of the neocortex. Annu Rev Neurosci. 2004;27:419–451.

  16. Nassi JJ, Callaway EM. Parallel processing strate-gies of the primate visual system. Nat Rev Neurosci. 2009;10(5):360–372.

  17. Hubel DH, Wiesel TN. Receptive fields and func-tional  architecture  of  monkey  striate  cortex.  J Physiol. 1968;195(1):215–243.

  18. Haxby J, et al. Dissociation of object and spatial visu-al processing pathways in human extrastriate cortex. Proc Natl Acad Sci U S A. 1991;88(5):1621–1625.

  19. Mishkin M. A memory system in the monkey. Philos Trans R Soc Lond B Biol Sci. 1982;298(1089):83–95.

  20. Maunsell JHR. The brain’s visual world: represen-tation of visual targets in cerebral cortex. Science. 1995;270(5237):764–769.

  21. Born RT, Bradley DC. Structure and function of visual area MT. Annu Rev Neurosci. 2005;28:157–189.

  22. Logothetis NK, Sheinberg DL. Visual object recog-nition. Annu Rev Neurosci. 1996;19:577–621.

  23. Tanaka K. Inferotemporal cortex and object vision. Annu Rev Neurosci. 1996;19:109–139.

  24. Connor CE, Brincat SL, Pasupathy A. Transforma-tion of shape information in the ventral pathway. Curr Opin Neurobiol. 2007;17(2):140–147.

  25. Miller  EK,  Cohen  JD.  An  integrative  theory  of prefrontal  cortex  function.  Annu Rev Neurosci. 2001;24:167–202.

  26. Miller E. The prefrontal cortex and cognitive control.  Nat Rev Neurosci. 2000;1(1):59–65.

  27. Eichenbaum  H.  A  cortical-hippocampal  sys-tem  for  declarative  memory.  Nat Rev Neurosci. 2000;1(1):41–50.

  28. Squire LR, Stark CE, Clark RE. The medial tempo-ral lobe. Annu Rev Neurosci. 2004;27:279–306.

  29. Allman JM, McGuinness E. Primate visual cortex. In: Steklis H., Erwin J, eds. Comparative Primate Biol-ogy. Volume 4. Neurosciences. New York, New York, USA: John Wiley & Sons, Inc.; 1988:279–326.

  30. Zeki S. A century of cerebral achromatopsia. Brain. 1990;113(pt 6):1721–1777.

  31. Gross CG. How inferior temporal cortex became a visual area. Cereb Cortex. 1994;4(5):455–469.

  32. Dutton GN. Cognitive vision,  its disorders and differential  diagnosis  in  adults  and  children: knowing where and what things are. Eye (Lond). 2003;17(3):289–304.

  33. Gronqvist S, Flodmark O, Tornqvist K, Edlund G, 

Hellstrom A. Association between visual impair-ment and functional and morphological cerebral abnormalities in full-term children. Acta Ophthal-mol Scand. 2001;79(2):140–146.

  34. Goodale  M,  Milner  A.  Separate  visual  path-ways for perception and action. Trends Neurosci. 1992;15(1):20–25.

  35. Heilman KM, Valenstein E. Clinical Neuropsychol-ogy. New York, New York, USA: Oxford University Press; 1993.

  36. Riddoch G. Dissociation of visual perceptions due to occipital injuries with special reference to appre-ciation of movement. Brain. 1917;40:15–57.

  37. Holmes G. Disturbances of visual orientation. Br J Ophthalmol. 1918;2(10):506–16.

  38. Weiskrantz L. Blindsight revisited. Curr Opin Neuro-biol. 1996;6(2):215–220.

  39. Zeki  S,  Ffytche  DH.  The  Riddoch  syndrome: insights into the neurobiology of conscious vision. Brain. 1998;121(pt 1):25–45.

  40. Zeki S. Cerebral akinetopsia (visual motion blind-ness). A review. Brain. 1991;114(pt 2):811–824.

  41. Zihl J, von Cramon D, Mai N. Selective disturbance of movement vision after bilateral brain damage. Brain. 1983;106(pt 2):313–340.

  42. Blanke O, Landis T, Safran AB, Seeck M. Direction-specific motion blindness induced by focal stimu-lation of human extrastriate cortex. Eur J Neurosci. 2002;15(12):2043–2048.

  43. Farah MJ, Brunn JL, Wong AB, Wallace MA, Car-penter PA. Frames of reference for allocating atten-tion to space: evidence from the neglect syndrome. Neuropsychologia. 1990;28(4):335–347.

  44. Driver J, Mattingley JB. Parietal neglect and visual awareness. Nat Neurosci. 1998;1(1):17–22.

  45. Berger A, Posner MI. Pathologies of brain attention-al networks. Neurosci Biobehav Rev. 2000;24(1):3–5.

  46. Bisiach E, Luzzatti C. Unilateral neglect of repre-sentational space. Cortex. 1978;14(1):129–133.

  47. Rizzo M, Vecera SP. Psychoanatomical substrates of Balint’s syndrome. J Neurol Neurosurg Psychiatry. 2002;72(2):162–178.

  48. Holmes G. Disturbances of visual orientation. Br J Ophthalmol. 1918;2(9):449–468.

  49. Klüver H, Bucy PC. Preliminary analysis of  the functions of the temporal lobes in monkeys. Arch-Neurol Psychiatry. 1939;42(6):979–1000.

  50. Mishkin M, Pribram KH. Visual discrimination performance  following  partial  ablations  of  the temporal lobe. I. Ventral vs. lateral. J Comp Physiol Psychol. 1954;47(1):14–20.

  51. Mishkin M. Visual discrimination performance following partial ablations of the temporal lobe. II. Ventral surface vs. hippocampus. J Comp Physiol Psychol. 1954;47(3):187–193.

  52. Holmes E, Gross C. Stimulus equivalence after infe-rior temporal lesions in monkeys. Behav Neurosci.  1984;98(5):898–901.

  53. Dean P. Effects of  inferotemporal  lesions on the behavior of monkeys. Psychol Bull. 1976;83(1):41–71.

  54. De Renzi E. Disorders of visual recognition. Semin Neurol. 2000;20(4):479–485.

  55. Damasio A, Tranel D, Damasio H. Face agnosia and the neural substrates of memory. Annu Rev Neurosci. 1990;13:89–109.

  56. Sacks O. The Man Who Mistook His Wife For A Hat And Other Clinical Tales. New York, New York, USA: Touchstone Books; 1998.

  57. Milders M, Perrett D. Recent developments in the neuropsychology and physiology of face processing.  Baillieres Clin Neurol. 1993;2(2):361–384.

  58. Kanwisher N, Moscovitch M. The cognitive neu-roscience of face processing: an introduction. In: Kanwisher N, Moscovitch, eds. The Cognitive Neu-roscience of Face Processing: A Special Issue of Cognitive Neuropsychology (Special Issues of Cognitive Neuro-psychology). Florence, Kentucky, USA: Psychology Press; 2000:1–11.

  59. Buxbaum L, Glosser G, Coslett H. Impaired face and  word  recognition  without  object  agnosia.  Neuropsychologia. 1999;37(1):41–50.

  60. Humphreys G. Object recognition: the man who mis-took his dog for a cat. Curr Biol. 1996;6(7):821–824.

  61. Rossion B, Caldara R, Seghier M, Schuller AM, Lazeyras F, Mayer E. A network of occipito-tem-poral face-sensitive areas besides the right middle fusiform gyrus is necessary for normal face pro-cessing. Brain. 2003;126(pt 11):2381–2395.

  62. Steeves JK, et al. The fusiform face area is not suf-ficient for face recognition: evidence from a patient with dense prosopagnosia and no occipital face area. Neuropsychologia. 2006;44(4):594–609.

  63. Le S, et al. Seeing, since childhood, without ven-tral  stream:  a  behavioural  study.  Brain.  2002; 125(pt 1):58–74.

  64. Dixon M, Bub D, Arguin M. Semantic and visual determinants of face recognition in a prosopag-nosic patient. J Cogn Neurosci. 1998;10(3):362–376.

  65. Gauthier I, Behrmann M, Tarr M. Can face recogni-tion really be dissociated from object recognition?  J Cogn Neurosci. 1999;11(4):349–370.

  66. Gainotti G. Cognitive and anatomical  locus of lesion in a patient with a category-specific semantic impairment for living beings. Cognitive Neuropsych. 1996;13(3):357–357.

  67. Humphreys G, Riddoch M. Object agnosias. Bail-lieres Clin Neurol. 1993;2(2):339–359.

  68. Magnie M, Ferreira C, Giusiano B, Poncet M. Cat-egory specificity in object agnosia: preservation of sensorimotor experiences related to objects. Neuro-psychologia. 1999;37(1):67–74.

  69. Warrington  E,  McCarthy  R.  Multiple  meaning systems in the brain - A case for visual semantics. Neuropsychologia. 1994;32(12):1465–1473.

  70. Warrington E, McCarthy R. Categories of knowl-edge - Further fractionations and an attempted integration. Brain. 1987;110(pt 5):1273–1296.

  71. Warrington E, Shallice T. Category specific seman-tic impairments. Brain. 1984;107(pt 3):829–854.

  72. Karnath HO, Ruter J, Mandler A, Himmelbach M. The anatomy of object recognition--visual form agnosia caused by medial occipitotemporal stroke. J Neurosci. 2009;29(18):5854–5862.

  73. Rayner K. Eye movements in reading and informa-tion processing: 20 years of research. Psychol Bull. 1998;124(3):372–422.

  74. Agam  Y,  et  al.  Robust  selectivity  to  two-object images  in  human  visual  cortex.  Curr Biol. 2010;20(9):872–879.

  75. Kuffler  S.  Discharge  patterns  and  functional organization of mammalian retina. J Neurophysiol. 1953;16(1):37–68.

  76. Hubel  D,  Wiesel  T.  Receptive  fields  of  single neurons  in  the  cat’s  striate  cortex.  J Physiol. 1959;148:574–591.

  77. Hubel DH, Wiesel TN. Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J Physiol. 1962;160:106–154.

  78. Canny J. A computational approach to edge detec-tion. IEEE Trans Image Process. 1986;8(5):679–714.

  79. Horn BK. Robot Vision. Cambridge, Massachusetts, USA: The MIT Press; 1986.

  80. Livingstone  M,  Hubel  D.  Segregation  of  form, color, movement and depth: anatomy, physiology and perception. Science. 1988;240(4853):740–749.

  81. Reynolds  JH,  Chelazzi  L.  Attentional  modula-tion  of  visual  processing.  Annu Rev Neurosci. 2004;27:611–647.

  82. Desimone  R,  Duncan  J.  Neural  mechanisms of  selective visual attention.  Annu Rev Neurosci. 1995;18:193–222.

  83. Salzman C, Britten K, Newsome W. Cortical micro-stimulation  influences perceptual  judgments of motion direction. Nature. 1990;346(6280):174–177.

  84. DiCarlo  J, Maunsell  J. Anterior  inferotemporal neurons of monkeys engaged in object recognition 

review series

TheJournalofClinicalInvestigation      http://www.jci.org      Volume 120      Number 9      September 2010  3063

can be highly sensitive to object retinal position.  J Neurophysiol. 2004;89(6):3264–3278.

  85. Rolls E. Neural organization of higher visual func-tions. Curr Opin Neurobiol. 1991;1(2):274–278.

  86. Tanaka K. Neuronal mechanism of object recognition.  Science. 1993;262(5134):685–688.

  87. Rolls  ET.  Neurons  in  the  cortex  of  the  tempo-ral lobe and in the amygdala of the monkey with responses selective for faces. Hum Neurobiol. 1984; 3(4):209–222.

  88. Young MP, Yamane S. Sparse population coding of faces  in the  inferior temporal cortex. Science. 1992;256(5061):1327–1331.

  89. Perrett D, Rolls E, Caan W. Visual neurones respon-sive to faces in the monkey temporal cortex. Exp Brain Res. 1982;47(3):329–342.

  90. Desimone R. Face-selective cells in the temporal cortex of monkeys. J Cogn Neurosci. 1991;3(1):1–8.

  91. Gross C, Bender D, Rocha-Miranda C. Visual recep-tive fields of neurons in inferotemporal cortex of the monkey. Science. 1969;166(910):1303–1306.

  92. Quian  Quiroga  R,  Reddy  L,  Kreiman  G,  Koch C,  Fried  I.  Invariant  visual  representation  by single neurons in the human brain. Nature. 2005; 435(7045):1102–1107.

  93. Schwartz  E,  Desimone  R,  Albright  T,  Gross  C. Shape-recognition and inferior temporal neurons. Proc Natl Acad Sci U S A. 1983;80(18):5776–5778.

  94. Miyashita Y, Chang HS. Neuronal correlate of pic-torial short-term memory in the primate temporal cortex. Nature. 1988;331(6151):68–71.

 95. Richmond BJ, Optican LM, Spitzer H. Tempo-ral  encoding  of  two-dimensional  patterns  by single  units  in  primate  primary  visual  cortex. I.  Stimulus-response  relations.  J Neurophysiol. 1990;64(2):351–369.

  96. Logothetis  NK,  Pauls  J.  Psychophysical  and physiological evidence for viewer-centered object representations  in  the  primate.  Cereb Cortex. 1995;5(3):270–288.

  97. Kobatake E, Tanaka K. Neuronal selectivities to complex object features in the ventral visual path-way of the macaque cerebral cortex. J Neurophysiol. 1994;71(3):856–867.

  98. Gawne TJ, Richmond BJ. How independent are the messages carried by adjacent inferior temporal cor-tical neurons? J Neurosci. 1993;13(7):2758–2771.

  99. Fujita I, Tanaka K, Ito M, Cheng K. Columns for visual features of objects in monkey inferotempo-ral cortex. Nature. 1992;360(6402):343–346.

 100. Hung CP, Kreiman GK, Quiroga R, Kraskov A, Pog-gio T, DiCarlo JJ. Using ‘read-out’ of object identity to understand object coding in the macaque ante-rior inferior temporal cortex. Computational and Systems Neuroscience (COSYNE), Salt Lake City, Utah, USA, March, 2005.

 101. Ito  M,  Tamura  H,  Fujita  I,  Tanaka  K.  Size  and position  invariance  of  neuronal  responses  in monkey  inferotemporal  cortex.  J Neurophysiol. 

1995;73(1):218–226. 102. Hung C, Kreiman G, Poggio T, DiCarlo J. Fast read-

out of object identity from macaque inferior tem-poral cortex. Science. 2005;310(5749):863–866.

 103. Sary G, Vogels R, Orban GA. Cue-invariant shape selectivity of macaque inferior temporal neurons. Science. 1993;260(5110):995–997.

 104. Engel AK, Moll CK, Fried I, Ojemann GA. Invasive recordings from the human brain: clinical insights and beyond. Nat Rev Neurosci. 2005;6(1):35–47.

 105. Kreiman G, Koch C, Fried I. Category-specific visu-al responses of single neurons in the human medial temporal lobe. Nature Neurosci. 2000;3(9):946–953.

 106. Allison T, Puce A, Spencer D, McCarthy G. Elec-trophysiological  studies of human face percep-tion. I: Potentials generated in occipitotemporal cortex by face and non-face stimuli. Cereb Cortex. 1999;9(5):415–430.

 107. Serre T, Oliva A, Poggio T. A feedforward architec-ture accounts for rapid categorization. Proc Natl Acad Sci U S A. 2007;104(15):6424–6429.

 108. Deco G, Rolls ET. A neurodynamical cortical model of visual attention and invariant object recogni-tion. Vision Res. 2004;44(6):621–642.

 109. Hegde J, Van Essen DC. Strategies of shape repre-sentation in macaque visual area V2. Vis Neurosci. 2003;20(3):313–328.

 110. Hegde J, Van Essen DC. A comparative study of shape representation in macaque visual areas v2 and v4. Cereb Cortex. 2007;17(5):1100–1116.

 111. Pasupathy A, Connor CE. Population coding of shape in area V4. Nat Neurosci. 2002;5(12):1332–8.

 112. Brincat S, Connor C. Dynamic  shape synthesis in  posterior  inferior  temporal  cortex.  Neuron. 2006;49(1):17–24.

 113. McKillop E, Dutton GN. Impairment of vision in children due to damage to the brain: a practical approach. Br Ir Orthop J. 2008;5:8–14.

 114. Logothetis N. Can current fMRI techniques reveal the microarchitecture of cortex? Nature Neurosci. 2000;3(5):413.

 115. Logothetis NK. The neural basis of the blood-oxy-gen-level-dependent  functional  magnetic  reso-nance imaging signal. Philos Trans R Soc Lond B Biol Sci. 2002;357(1424):1003–1037.

 116. Logothetis NK. The underpinnings of the BOLD functional  magnetic  resonance  imaging  signal.  J Neurosci. 2003;23(10):3963–3971.

 117. Logothetis NK. What we can do and what we cannot do with fMRI. Nature. 2008;453(7197):869–878.

 118. Mukamel R, Gelbard H, Arieli A, Hasson U, Fried I, Malach R. Coupling between neuronal firing, field potentials, and FMRI in human auditory cortex. Science. 2005;309(5736):951–954.

 119. Tsao DY, Freiwald WA, Tootell RB, Livingstone MS. A cortical region consisting entirely of face-selec-tive cells. Science. 2006;311(5761):670–674.

 120. Maier A, Wilke M, Aura C, Zhu C, Ye FQ, Leopold DA. Divergence of fMRI and neural signals in V1 

during perceptual suppression in the awake mon-key. Nat Neurosci. 2008;11(10):1193–1200.

 121. Leopold  DA.  Neuroscience:  pre-emptive  blood flow. Nature. 2009;457(7228):387–388.

 122. Sirotin YB, Das A. Anticipatory haemodynamic signals  in sensory cortex not predicted by  local neuronal activity. Nature. 2009;457(7228):475–479.

 123. Hasson U, Yang E, Vallines I, Heeger DJ, Rubin N. A hierarchy of temporal receptive windows in human cortex. J Neurosci. 2008;28(10):2539–2550.

 124. Logothetis  NK,  Wandell  BA.  Interpreting  the BOLD signal. Annu Rev Physiol. 2004;66:735–769.

 125. Haxby J, Hoffman E, Gobbini M. The distributed human neural system for face perception. Trends Cogn Sci. 2000;4(6):223–233.

 126. Haxby JV, Gobbini MI, Furey ML, Ishai A, Schouten JL, Pietrini P. Distributed and overlapping repre-sentations of faces and objects in ventral temporal cortex. Science. 2001;293(5539):2425–2430.

 127. Ishai A, Ungerleider LG, Martin A, Schouten JL, Haxby JV. Distributed representation of objects in the human ventral visual pathway. Proc Natl Acad Sci U S A. 1999;96(16):9379–9384.

 128. Grill-Spector K, Malach R. The human visual cortex.  Annu Rev Neurosci. 2004;27:649–677.

 129. Epstein  R,  Kanwisher  N.  A  cortical  representa-tion  of  the  local  visual  environment.  Nature. 1998;392(6676):598–601.

 130. Kanwisher N, McDermott J, Chun MM. The fusi-form face area: a module in human extrastriate cortex specialized for face perception. J Neurosci. 1997;17(11):4302–4311.

 131. Tong  F,  Nakayama  K,  Moscovitch  M,  Weinrib O,  Kanwisher  N.  Response  properties  of  the human fusiform face area.  Cognitive Neuropsych. 2000;17(1/2/3):257–279.

 132. Foldiak P. Learning invariance from transforma-tion sequences. Neural Comput. 1991;3(2):194–200.

 133. Mel B. SEEMORE: combining color, shape and tex-ture histogramming in a neurally inspired approach to  visual  object  recognition.  Neural Comput.  1997;9(4):777.

 134. Riesenhuber  M,  Poggio  T.  Hierarchical  mod-els of object recognition in cortex. Nat Neurosci. 1999;2(11):1019–1025.

 135. Reagan D. Human Perception of Objects: Early Visual Processing of Spatial Form Defined by Luminance, Color, Texture, Motion, and Binocular Disparity. Sunderland, Massachusetts, USA: Sinauer Associates Inc; 2000.

 136. Dayan P, Abbott L. Theoretical Neuroscience: Computa-tional and Mathematical Modeling of Neural Systems. Cam-bridge, Massachusetts, USA: The MIT Press; 2001.

 137. Kreiman G. On the Neuronal Activity in the Human Brain during Visual Recognition, Imagery and Binocular Rivalry [dissertation]. Pasadena, California, USA: California Institute of Technology; 2002.

 138. Quiroga RQ, Reddy L, Kreiman G, Koch C, Fried I. Invariant visual representation by single neurons in the human brain. Nature. 2005;435(7045):1102–1107.