The Parallel Guidance of Visual AttentionJeremy M. Wolfe

In the fifth-act battle scene ofShakespeare's Henry IV, Part 1, rebelnobles are searching the battlefieldat Shrewsbury for King Henry. Thetask is made more difficult by Hen-ry's stratagem of disguising some ofhis soldiers in the royal colors. "Theking," says one rebel, "hath manymarching in his coats." Douglas, an-other rebel, replies:

Now, by my sword, I will kill all his coats;I'll murder all his wardrobe, piece by piece,until I meet the king.

Shakespeare offers a good, if sangui-nary, account of a real-world visualsearch task: a search for one targetitem in a field of distractors. Becauseof processing limitations, candidatetargets must be processed in a serial,self-terminating manner. However,not all items in the field are reason-able candidate targets. The serialprocessing of unfortunate royal sur-rogates is made more efficient byprocesses that guide search towardlikely kings and away from trees,rocks, sheep, and those soldiers for-tunate enough not to be wearing theroyal disguise.

Research with far less exciting vi-sual search tasks in the laboratorysuggests that this guided search isthe normal mechanism used by in-

Jeremy Wolfe is Associate Profes-sor of Ophthalmology at HarvardMedical School and is affiliatedwith Brigham and Women's Hos-pital. He is a visiting Associate Pro-fessor at MIT and at WellesleyCollege. Wolfe received his under-graduate degree at Princeton Uni-versity and his PhD at MIT. Ad-dress correspondence to JeremyWolfe, Center for Clinical CataractResearch, 221 Longwood Ave.,Boston, MA 021 1 5; e-mai l :wolfe@psyche.mit.edu. ;

dividuals to find a desired target in acrowded visual world. Some basicattributes of the visual scene (e.g.,colors, motion, orientation) can beextracted in parallel across the entirevisual field or, at least, over a sub-stantial piece of the visual field.Thus, in Figure la, all items can beprocessed simultaneously in asearch for a white letter. Other pro-cesses (e.g., letter or object recogni-tion) can operate over only a limitedregion of the visual field at any onetime. These spatially limited re-sources must be deployed first at onelocation, then at another, in a serialmanner.^ Under some circum-stances, all items are candidate tar-gets, and that deployment is ran-dom. For example, in Figure 1b, a"J" is easily distinguished from an"L," but search for a "T" proceedsat random from item to item until thecorrect letter is found. In most cases,however, information from the par-allel processors—the front end ofthesystem—can be used to guide thedeployment of spatially limited re-sources. In Figure 1c, only half ofthe items are candidate targets be-cause a parallel processor for colorcan guide search to the white itemsin a search for a white "T," elimi-nating the black items.

As another example, consider asearch for a red vertical targetamong green vertical and red hori-zontal distractors. This task involvesa conjunction of color and orienta-tion. Evidence suggests that we donot have parallel processors for con-junctions of features. However, thistask can be accomplished efficientlyby having a parallel "color" proces-sor activate all "red" locations whilean "orientation" processor activatesall "vertical" locations. If these ac-tivations are summed, the greatestactivation will be at the location of ared vertical item if it is present. A

spatially limited process can be de-ployed to check that location for thepresence of a red vertical item. Evenif this process of activation is noisy,it will reduce the set of candidatetargets from the set of all items to asubset. This is the essential idea be-hind the guided search model.^

In this review, the focus is on theproperties of the parallel, preatten-tive, stage of guided search. Whatinformation is available for guidingthe deployment of spatially limitedprocesses? How is this preattentiverepresentation of visual informa-tion related to other representa-tions, such as those found in electro-physiological studies of visualcortex or those involved in texturesegmentation?

WHAT IS DOING THEGUIDING? THE NATURE OF

THE PREATTENTIVEREPRESENTATION'^ n

Parallel processing appears to belimited to a relatively small set of ba-sic features. A partial list includescolor, orientation, size, motion, ste-reoscopic depth, curvature, a varietyof two-dimensional pictorial cues tothree-dimensional space, and a va-riety of surface properties like luster(shininess).'* Form is probably not asingle basic feature. Rather, it seemsto be composed of a small set of fea-tures, though the precise set is un-known and the featural status ofform properties such as line termina-tion and closure is still open to de-bate. Still, there is little debate thatthe list of preattentive, or parallel-processed, features is a limited list,and there is little evidence in thevisual search literature that prac-tice increases the number of suchfeatures.

Preattentive Features Are NotEarly-Vision (VI) Features

About 10 years ago, it was tempt-ing to think that the basic features

Published by Cambridge University Press

CURRENTpmcriONS !N PSYCHOtOCICAL SCIENCE

T

T

iT

i T

T i T H

J

J

rT

J

-in

r

TJ

T

JH

T

1

J

Tr

=, =̂

^ TT 1

T 1=0 T1 I• •

T H

Fig. 1. The three broad classes of visual search tasks. In each case, the task is to find the white "T." (a) In parallel feature search, thetime to find a target defined by a single feature does not increase as the number of distractors increases, (b) In serial search, items mustbe examined sequentially, and each additional distractor increases the search time, (c) In guided search, parallel processes restrictsubsequent serial search to a subset of the items (here, the white items).

supporting parallel visual search^were the same as the basic featuresextracted in the first steps of visualcortical processing (e.g., color, size,orientation). Though the idea was at-tractive, there are at least two rea-sons why it is probably not correct:The lists of features are not identical,and features that do appear on bothlists may have different properties.

The Two Lists Are Not the SameWork in the past decade has re-

vealed features that produce parallelvisual search results but that havenot been found to be tuning proper-ties of single cells in the primary vi-sual cortex (VI) of primates. Exam-ples include perceived shape fromshading cues, linear perspectivecues lo depth, and binocular luster(an appearance of shininess pro-duced by unequal luminance inthe two eyes).^ In addition, someproperties of VI neurons (e.g., theability to determine which eyehas been stimulated) are not featuresin visual search/ suggesting thatsome VI features are lost beforethe generation of the preattentiverepresentation.

Cortical Features andPreattentive Features HaveDifferent PropertiesEven when an attribute tike

"color" or "orientation" can be

studied with both cortical electro-physiology and the parallel visualsearch paradigm, it seems clear thatoften the same attribute name is be-ing used for two quite differentthings. An example is illustrated inFigure 2a. There are four target itemsin the figure, each tilted 20° left ofvertical. The distractors are tilted 20°to the right, 60° right, and 80° left.The task is very difficult even thoughthe minimum difference in orienta-tion between targets and distractorsis 40°. Standard estimates of the ori-entation tuning^ of cortical neurons

are around 15° to 20°, so target anddistractor lines must be stimulatingvery different populations of corticalneurons. Psychophysical orientationdiscriminations based on the outputsof these eel Is are on the order of 1 °.Yet even on a homogeneous back-ground of distractor items of one ori-entation, a target line cannot befound in parallel if the difference be-tween target and distractor orienta-tion is less than about 15°.^

Parallel processing of orientationin visual search is not merely cruderthan cortical orientation processing.

(a) Find the four lines tilted 20°

^ / / •

r

7

/ -̂

•mm

>

z1 ̂1

left of vertical

^ ^

• / ;

• / -

/ ^ -

- * -

Jl'

1

(b) Find the four steeply tilted lines (10° right)

Fig. 2. Visual search for a specific orientation, (a) Search for lines tilted 20° left ofvertical is difficult among heterogeneous distractors even though no distractor orienta-tion is closer than 40° to the target orientation, (b) With the same angular relationsbetween stimuli (all lines rotated 30° clockwise), the search becomes simple becausethe targets are categorically different from all distractors. Here, the targets (tilted 10° tothe right) are the only "steep" items.

Copyright © 1992 American Psychological Society

It is qualitatively different. This canbe seen in Figure 2b: All the angularrelationships between targets anddistractors are the same as in Figure2a, but all orientations have been ro-tated 30° clockwise. The target itemsare now oriented 10° to the right,and the distractors are now oriented50° to the right, 50° to the left, and90° (horizontal). More important,this rotation makes the targets theonly "steep" lines in the field. Nowthey can be found easily. In a seriesof visual search experiments, mycolleagues and I have shown that theparallel representation of orientationused in visual search is categorical.That is, when there are several ori-entations in a display, search is effi-cient only when the target is the solemember of an orientation category(e.g., steep, shallow, left, or right).^°

A similar point could be madeconcerning VI and preattentive pro-cessing of color information.'^ Also,visual cortex seems to have four toeight channels for size, but we findpreattentive size processing to belimited to the categories "biggest"and "smallest."

The Preattentive Representation ofVisual Information Is Sophisticatedand Task-Specific

In the preceding section, I haveargued that the early-vision featuresstudied with cortical electrophysiol-ogy in VI are different from the pre-attentive features studied with visualsearch. How are these two sets offeatures related? It seems likely thatthere is a hierarchical relationshipbetween early cortical features andthe preattentive features that canguide visual search. The latter arethe product of further parallel pro-cessing of the former.'^ Some of thatfurther processing is task-specific,and other visual tasks (e.g., texturesegmentation) make different use ofthe output of early cortical pro-cesses. As evidence in support ofthese conclusions, it can be shown

that the preattentive representation"knows" a lot, suggesting that it rep-resents a relatively late stage in vi-sual processing. At the same time,there is a lot that the representationdoes not "know," suggesting that itis usingonly a task-specific subset ofthe information available from ear-lier stages in processing.

The Preattentive Representation"Knows" a LotStudies of part-whole relations in

guided search provide evidence ofthe sophistication of the processesthat generate preattentive represen-tations. Although it is possible toguide attention to conjunctions oftwo features (e.g., a red verticalitem), guidance fails when the targetis a conjunction of two examples ofthe same feature type. Thus, in Fig-ure 3a, it is hard to find the item thatis white and black among white-and-gray and gray-and-black dis-tractors.''' The same very inefficientsearch is found for orientation X ori-entation and size X size conjunc-tions. Recently, however, we havefound a class of stimuli that produceefficient, guided searches for size Xsize and color X color conjunctions.

An example is illustrated in Figure3b, though the effect is more com-pelling in color. Here, the search isfor a white house with black win-dows. The distractors are chosen soas to make this a color X color con-junction. This search and others likeit suggest that within-featuresearches are efficient when onecolor describes the whole object (thewhite house) while the other de-scribes a part of the object (the blackwindows).

Part-whole color X color con-junctions allow for guided search.Part-part conjunctions (e.g., thehouse that is half red and half yel-low) do not.^" These results suggestthat there is separate parallel pro-cessing for the colors of whole ob-jects and the colors of parts, analo-gous to the separate processing ofthe color and orientation of items.Moreover, these results suggest thatparallel, preattentive processing iscapable of dividing the visual sceneinto items or objects with constitu-ent parts—an indication of a sub-stantial amount of prior processingin parallel.

Part-whole size X size conjunc-tions also yield efficient, guided

(a) Find the white-and-black item (b) Find the white house with biacif windows

Fig. 3. Search for a conjunction of two colors. Search is very inefficient when theconjunction is between the colors of two parts of a target (a). However, search is mucheasier when the conjunction is between the color of the whole item and the color of oneof its parts (b).

Published by Cambridge University Press

(a) Part-whole conjunction:Find the snowman with

the big body and the small checks

Target

(b) Part-part conjunction:Find the snowman with

tlie big body and the small head

Fig. 4. Search for a conjunction of two sizes. Search is efficient if the conjunction isbetween the size of the whole item and the size of one of its parts (a). Conjunctionsbetween the sizes of parts are hard to find (b).

search. Thus, in Figure 4a, it is rel-atively easy to find the snowmanwith the big body and little checkson his waistcoat among snowmenwith big bodies and big checks andsnowmen with little bodies and littlechecks. For contrast. Figure 4bshows a part-part conjunction withsimilar stimuli. It is hard to find thesnowman with a big body and asmall head among snowmen withbig bodies and big heads and snow-men with small bodies and smallheads. Interestingly, we have beenunable to find any part-whole stim-uli that support guided search fororientation X orientation stimuli. It isas if the color (or size) of a wholeitem can be represented separatelyfrom the color (or size) of part of theitem, but the orientations of partsand wholes are not separately repre-sented.^^

There Is a Lot the PreattentiveRepresentation DoesNot ''Know''Although parallel processing of

part-whole relationships is evidencethat the preattentive representation"knows" a lot, it is also clear thatmuch information is absent from the

representation. It cannot differenti-ate between a "T" and an "L," forexample (Fig. Ib). As noted, the rep-resentation does not code conjunc-tions directly, it cannot differentiatebetween lines that differ widely inorientation or patches that differwidely in color unless those differ-ences cross categorical boundaries.Visual processing is not a single,unitary stream. Many different pro-cessing tasks occur at the same time,and visual search is guided by only asubset of the available information.This subset can be thought of as arepresentation created specificallyfor the task of guiding search.^^

Visual Search andTexture SegmentationThe task-specific nature of the

preattentive representation can be il-lustrated by comparison with therepresentation that supports "effort-less" texture segmentation.^^ In Fig-ure 5a, there is little or no evidencefor effortless texture segmentation.The bulk of the texture is composedof black horizontal and white verti-cal lines, but one region containsblack verticals and white horizon-tals. That region does not "popout,"^^ or segment. However, it ispossible to do a successful guidedsearch for a black vertical or whitehorizontal target among distractorsthat are black horizontal and whitevertical. By contrast, there is cleartexture segmentation in Figure 5b.Here, the smaller region is com-posed of items that are vertical andoblique on a background of vertical-and-horizontal and horizontal-and-obtique items. As a visual searchtask, this is an orientation X orienta-tion conjunction and is very ineffi-cient. In fact, there is a lone vertical-and-oblique target in the lower rightcorner of the figure. As a visualsearch target, it does not pop out,even though a group of these itemssupports texture segmentation. Theorientation X orientation items showtexture segmentation because seg-mentation processes seem to inte-grate over multiple items, creatingan impression of average orienta-tion.^^ Such integration would be a

D

Ufl

fln

^ fl

fl

1

M

-flnD

fl B 1

1 i "il

1 . _

fl°

yfl.^fl"

E

•

flfl

nfl

fl n

—D f̂l— D

• • —

+ ̂

+ **

/ 4

Jt <•

+ -

r/̂

' ^ /f / ^

^+^•+ '^

r + ̂ '^H'»* + + '•»

'' -i- +»"'

h +• ^L

^ +

y +

Fig. 5. Visual search versus texture segmentation. Conjunctions of color and orientation(a) support parallel search without supporting texture segmentation, whereas orientationX orientation conjunctions (b) support texture segmentation but not parallel search.

Copyright © 1992 American Psychological Society

useless operation in visual search fora single item.

SUMMARY

In attempting to execute complextasks like object recognition, the vi-sual system is faced with significantlimits on its processing capacity. Fullprocessing of all locations in parallelis not possible. Some processes mayoperate on one item or, perhaps, onone limited region at a time. De-ployment of the resources requiredfor these tasks is under attentionalcontrol. To operate reasonably effi-ciently, that attention must beguided nonrandomly from item toitem or location to location. To sup-port guidance of attention, the visualsystem extracts information in paral-lel from visual input to create a rep-resentation intended for the specificpurpose of guiding attention. Thatparallel, preattentive representationsacrifices a great deal of fine-graininformation about attributes such asspecific orientations and colors. Itquickly parses input into featuralcategories, like "steep" and "shal-low" in orientation. It is not, how-ever, merely a crude or early repre-sentation of visual input. It issensitive to such comparatively lateperceptual properties as pictorialdepth cues and part-whole rela-tions. It is a representation well-suited to perform its appointedtask—to quickly direct attention to ameaningful subset of items or loca-tions in a visually rich environment.

Acknowledgments—I thank Anne Treis-man, Kyle Cave, Judith Friend, StaciaFriedman-Hiil, and two anonymous re-viewers for their useful comments ondrafts of this paper. The research was sup-ported by the National Institutes of Health(Grant No. EY05O87).

Notes

1. It may be possible to process clumps ofitems; e.g., H. Pashler, Detecting conjunctions of

color and form; Reassessing the serial search hy-pothesis, Perception and Psychophysics. 41, 191-201 (1987).

2. Authors of reviews for Current Directions areasked to keep references lo a minimum. The workdescribed here builds on and is related to a largebody of work from other labs. Interested readersshould look at the bibliographies of the publicationscited here for a more complete guide to this litera-ture. Description of our work on guided search forconiunctions and the original formulation of theguided search model are found in J.M. Wolfe, K.R.Cave, and S.L. Franzel, Guided search: An alterna:tive to the feature integration model for visualsearch, lournai of Experimental Psychology: i-lumanPerception and Performance. 15, 419-433 (1989).Interested readers should certainly look al H.E,Egeth, R.A. Virzi, and H. Garbarl, Searching forconjunctively defined targets, journal of Experimen-tal Psychology: Human Perception and Perfor-mance. 10, 32-39 11984), as well as Anne Treis-man's work. One of her recent articles on this topicis A. Treisman and S, Sato, Conjunction search re-visited, lournal of Experimental Psychology: HumanPerception and Performance. !b, 459^78 (19901.

3. The term preattentive representation is usedhere to refer exclusively to information that is ex-tracted in parallel from the visual input and is avail-able to guide the deployment of visual attention invisual search, hence the term preattentive. The termis used more broadly elsewhere. This topic is ad-dressed at the end of this review.

4. A list, complete at ihe time it was created,can be found in A, Treisman, Properties, parts, andobjects, in Handbook of Human Perception andPerformance. K.R. Boff, L. Kaufmann, and |.P.Thomas, Eds. (Wiley, New York, 19861, See alsoJ.T. Enns and R.A. Rensink, Preattentive recovery ofthree-dimensional orientation from line drawings.Psychological Review, 98. 335-351 (1991).

5. Common evidence for parallel processing invisual search is a pattern of reaction times (RTsl thatare independent of the number of items displayed(set sizel. This yields RT x set size slopes near 0ms/item. The classic pattern for a serial, self-terminating search is a slope of 20-30 ms/item ontrials when a target is present and twice that, 40-60ms/item, for target-absent trials. These conclusionsmust be taken with caution; see J.T. Townsend, Se-rial and parallel processing: Sometimes they looklike Tweedledum and Tweed led ee but they can (andshould! be distinguished. Psychological Science, I,46-54 (1990). Guided searches have shallowerslopes than serial searches, reflecting their status asserial searches through a subset of items. Thesmaller the subset, the shallower the slope. Most ofthe conclusions and assertions in this review arebased on data from search experiments of this sort.The reader is directed to the cited publications forexperimental details,

6. On shading, see V.S. Ramachandran, Per-ception of shape from shading. Nature. 331, 163-165 (1988). On linear perspective, see Enns andRensink, note4. On binocular luster, see J.M, Wolfeand S.L. Franzel, Binocularity and visual search.Perception and Psychophysics, 43. a i -93 (1988).

7. An example is eye-of-origin information,readiiy available in VI but unavailable in visualsearch; Wolfe and Franzel, note 6.

8. The orientation tuning of a cell is a measureof the range of orientations to which thai cell willrespond with at leas! some fraction (often 50%) of itsmaximum firing rate. For a discussion, see L,A.Olzak and |.P. Thomas, Seeing spatial patterns, inHandbook of Perception and Human Performance.K.R. Boff, L. Kaufmann, and J.P. Thomas, Eds.IWiiey, New York, 1986).

9. D.H. Foster and P.A. Ward, Asymmetries inoriented-line detection indicate two orthogonal fil-ters in early vision. Proceedings of the Royal Societyof London {B). 243, 75-61 (1991),

10. J.M. Wolfe, S.R. Friedman-Hill, M.I. Stew-

art, and K.M. O'Conned, The role of categorizationin visual search for orientation, journal of Experi-mental Psychology: Human Perception and Perfor-mance. 18. 34-49 (1992). This article gives refer-ences to some standard psychophysical data oncortical orientation processing. Further wrinkles onpreattentive orientation processing are found in J.M.Wolfe and S.R. Friedman-Hill, On the role of sym-metry in visual search. Psychological Science. 3.194-198 (1992).

11. M, D'Zmura, Color in visual search. VisionResearch. 31. 951-966 (19911; A.L. Nagy and R.R.Sanchez, Critical color differences determined witha visual search task, journal of the Optical Society ofAmerica A, 7. 1209-1217 (1990).

12. See, for instance, P, Cavanaugh, M. Ar-guin, and A. Treisman, Effect of surface medium onvisual search for orientation and size features, lour-nal of Experimental Psychology: Human Perceptionand Performance, 16, 479-492 (1990), This is anelegant article showing how preattentive orienlatronfeatures can be created from variations in featuressuch as color and texture.

13. The actual experiments with these color Xcolor conjunctions were done with more colorfulstimuli (e.g., target: red and green; distractors: redand blue, blue and green); J.M. Wolfe, K,P. Yu, M.I.Stewart, A.D. Shorter, S.R. Friedman-Hill, and K.R.Cave, Limitations on the parallel guidance of visualsearch: Color X color and orientation X orientationconjunctions, journal of Experimental Psychology:Human Perception and Performance. 76, 879-892(1990).

14. This is not just a complex conjunction ofsize and color. In controlled experiments, we havevaried the size of items so that tbe whole red of asmall item is no bigger than the yellow part of someother item in the same display. Guided search is stillpossible.

15. This difference between size and color onone side and orientation on the other is by no meansarbitrary. Consider that, in general, the color of apart is not constrained by the color of the whole,whereas the orientation of a part may be defined byits relationship to the primary axis whole. See, e.g.,D. Marr and FH.K. Nishahara, Representation andrecognition of the spatial organization of three-dimensional shapes. Proceedings of the Royal Soci-ety of London (6), 200. 269-294 (1977),

16. This idea is couched in conditional lan-guage for a reason. It is probably a mistake to thinkthat the preattentive representation exists as a neu-rally distinct locus in the brain. In the absence ofclear evidence, it seems more plausible to imaginethat it is a virtual map of tbe visual world, generatedfor the purposes of guiding visual attention but in-stantiated in neurons that are doing other tasks at thesame time. This is analogous to the multiple func-tions of a retinal ganglion cell that functions as partof a color system and a contour system at the sametime.

17. We run into a terminological problem herebecause preattentiVe has been used to refer to bothparallel visual search and texture segmentation; see,e.g., B, Julesz and J.R. Bergen, Textons, the funda-mental elements in preattentive vision and percep-tions of textures. Bell System TechnicaJ lournal. 62.1619-1646 (1983). Here, as discussed in note 3, Iuse the term to refer exclusively to the representa-tion that is used to guide the deployment of visualattention. The matter is discussed more extensivelyin J.M. Wolfe, "Fffortless" texture segmentation and"parallel" visual search are not two measures of thesame thing. Vision Research. 32, 757-763 (1992).

18. Treisman sometimes uses pop-out as a tech-nical term in her feature integration theory. In thecontext of guided search, pop-out is a description ofthe phenomenological experience that occurs whenvisual attention is grabbed by a particularly salientstimulus.

19. Seen in Figure 6 of Wolfe, note 17.

Published by Cambridge University Press