-MIS? 092 REVIEW OF WINOGRAD MO FLORES' UNDERSTAING COMPUTERS "IAN COGNITION: A FRYORRILE INTERPRETRTION(U) STANFORDUNIV CA KNOWLEDGE SYSTEMS LRB W J CLANCEY JUL 06

WOCLRSSIFIED KSL-96-96-49 F/O 12/5EhhnnhhmhmhhumhEEE|hh|hnhEI

- ~ ~ ~ ~ ~ ~ L lw w~wJ! PU T JJWNJ .R~7 N B WWWWWUWUMiw

L: I2P2

9.6i, 112.

J111 511111-

S 0 S S S0 0 0 S 5.S

OflC COP Report No. STAN-CS-87] 173Also numbered K.S*L-86-48

Reviewm of Winograd and Flores'Understanding Computers and Cognition:

A Favorable Interpretation

by

William .. Clancey

Department of Computer Science

Stanford Unitiversity -Stanford, CA 94305

DTIC

DRSRIUTONSTATEMEN4T A........ 28.1.

- "¢., " , . ,, .,., ,.. .. 3.., _',,.,, .-. ,., ., ,, . "%, '.;, , :-. :, : 1 2 ! .?.::c .. ,,:.,. ,',.. '., ,.

SECURITY CLASSIFICATION OF THIS PAGE 14. -."_. ' .

REPORT DOCUMENTATION PAGEla REPORT SECURITY CLASSIFICATION lb RESTRICTIVE MARKINGS /" ,

UNCLASSIFIED __%"__-"

2a SECURITY CLASSIFICATION AUTHORITY 3 DISTRIBUTION/ AVAILABILITY OF REPORT

APPROVED FOR PUBLIC RELEASE,2b. DECLASSIFICATIONi DOWNGRADING SCHEDULE DISTRIBUTION UNLIMITED .'.-

4 PERFORMING ORGANIZATION REPORT NUMBER(S) S. MONITORING ORGANIZATION REPORT NUMBER(S)

STAN-CS-87-1173 or KSL-86-48 ONR TECHNICAL REPORT # 21

6a NAME OF PERFORMING ORGANIZATION 6b. OFFICE SYMBOL 7a NAME OF MONITORING ORGANIZATION

STANFORD KNOWLEDGE SYSTEMS (If applicable) PERSONNEL AND TRAINING RESEARCH PROGRAMSLABORATORY I

6c. ADDRESS (City, State, and ZIPCode) 7b ADDRESS (City, State, and ZIP Code)COMPUTER SCIENCE DEPARTMENT OFFICE OF NAVAL RESEARCH (CODE 1142PT)701 WELCH ROAD, BUILDING C 800 NORTH QUINCY STREETPALO ALTO, CA 94304 ARLINGTON, VA 22217-5000

8a. NAME OF FUNDING i SPONSORING 8b OFFICE SYMBOL 9 PROCUREMENT INSTRUMENT IDENTIFICATION NUMBERORGANIZATION (If applicable)

N00014-85K-0305

8c. ADDRESS (City, State, and ZIP Code) 10 SOURCE OF FUNDING NUMBERS

PROGRAM PROJECT TASK WORK UNITELEMENT NO. NO, NO. ACCESSION NO

61153N RR04206 ,04C NR702-003 .'V'-11 TITLE (Include Security Classification)

REVIEW OF WINOORAD AND FLORES' UNDERSTANDING COMPUTERS AND COGNITION: A FAVORABLE 'INTERPRETATION

12 PERSONAL AUTHOR(S)WILLIAM J. CLANCEY ' .-f

13a TYPE OF REPORT 13b. TIME COVERED 14. DATE OF REPORT (Year, Month, Day) 15 PAGE COUNTTECHNICAL FROM TO_ JULY 1986 24

16 SUPPLEMENTARY NOTATIONKSI, #86-48; APPEARED IN THE JOURNAL OF ARTIFICIAL INTELLIGENCE, 31(2), 232-250,

FEBRUARY 198717 COSATI CODES 18 SUBJECT TERMS (Continue on reverse if necessary and identify by block number)

FIELD GROUP SUB-GROUP PHILOSOPHY, ARTIFICIAL INTELLIGENCE, KNOWLEDGE05 09 REPRESENTATION

19 ABSTRACT (Continue on reverse if necessary and identify by block number) 'N

Al tesearches and cognitive scientists comlnmonly believe that thinking Involvesmanipulating tepresentations. Thinking involves search, inference, and making choices. Thisis how we model reasoning and what goes on in the brain is similar. Winograd and Florespresent a radically different view. They claim that our knowledge is not represented in thebrain at all, but rather consists of an unformalized shared background, from which wearticulate representations in order to cope with new situations. In contrast, computerprograms contaiii (mnly pre-selected objects and properties, and there is no basis for movingbeyond this initial lormalization when breakdown occurs. - -

(Continued on next page)

20 DISTRIBUTION /AVAILABILITY OF ABSTRACT 21 ABSTRACT SECURITY CLASSIFICATION[2 UNCLASSIFIED/UNLIMITED I SAME AS RPT 0 DTIC USERS UNCLASSIFIED.,

22a NAME OF RESPONSIBLE INDIVIDUAL 22b TELEPHONE (Include Area Code) 22c OFFICE YBOL "

DR. SUSAN CHIPMAN (202) 696-4318 ONRI 4PDD FORM 1473,84 MAR 83 APR edition may be used until exhausted SECURITY CLASSIFICATION OF THIS PAGE

All other editions are obsolete

% % _;-2%-

5. 19. Abstract Continued

*, Winograd and Flores provide convincing arguments with examples familiar to most Atresearchers. However, they significantly understate the role of representation In mediating

,intelligent behavior, specifically in the process of reflection, when representations aregenerated prior to physical action. Furthermore, they do not consider the practical benefitsof expert systems and the extent of what can/be accomplished. Nevertheless, the book iscrisp and stimulating. It should should make A researchers more cautious about what theyare doing, more aware of the nature of formalization, and more open to alternative views.

4-

%

'm"

Review of Winograd and Flores'

Understanding Computers and Cognition:

A Favorable Interpretation

by

William J. Clancey

.5

Stanford Knowledge Systems LaboratoryDepartment of Computer Science

701 Welch Road, Building CPalo Alto, CA 94304

The studies reported here were supported (in part) by:

The Office of Naval ResearchPersonnel and Training Research ProgramsPsychological Sciences Division A,Contract No. N00014-85K-0305 Accession For

NTIS GRA&IThe Josiah Macy, Jr. Foundation DTIC TAB r"Grant No. B852005 UnatinouncudNew York City Jtirt tc'a-___ ",G

The views and conclusions contained in this document are the authors' By-'-,and should not be interpreted as necessarily representing the official Dlstributlo-'_policies, either expressed or implied of the Office of Naval Research Avallabillty Coe. _or the U.S. Government. - ...r,- Avail avdi or

Diat SpecialApproved for public release: distribution unlimited. Reproduction in , p lwhole or in part is permited for any purpose of the United States '

Government. Vs

,,-

5-

V *~ V. ,

1om IW W W W U WV ' -K -VK - IV - V W W 7V W -.--"

Understanding Computers and Cognition: A New Foundation for DesignTerry Winograd and Fernando Flores %Norword, NJ: Ablex Publishing Corporation, 1986Bibliography, indices, 207 pps.

1. Introduction"Every triumphant theory passes through three stages: first it is dismissed as

untrue; then it is rejected as contrary to religion; finally, it is accepted as dogma andeach scientist claims that he had long aipreciated the truth." (Gould, in Ever SinceDarwin, quoting embryologist von Baer)

AI researchers and cognitive scientists commonly believe that thinking in~olves manipulatingrepresentations. For example, when we speak our thoughts are translated into words. We don'tknow what the mental operations are, but we assume that they are analogous to computermodels of reasoning. There are hierarchical networks in the brain and stored associationallinks between concepts. There are propositions, implication rules, control processes, and so on.Thinking involves search, inference, and making choices. This is how we model reasoning, andwhat goes on in the brain is similar.

Winograd and Flores present a radically different view. In a nutshell, intelligence of thekind exhibited by people isn't possible by manipulating representations alone. In fact, theyclaim that our knowledge is not represented in the brain at all, at least not as stored facts andprocedures. Many readers will reject this argument as obviously wrong, so obviously wrongthey won't have to read the book to be convinced that it is just some variant of all Easternreligion or simply anti-scientific and not worth their time.

After reading the book twice and much consideration, I believe that Winograd and Flores aremostly right. We have the stuff here of Copernicus, Darwin, and Freud: At its heart, thehuman world is not what we thought. However, I believe that Winograd and Flores 4significantly understate the role of representation in mediating intelligent behavior, specificallyin the process of reflection, when representations are generated prior to physical action.Furthermore, while the book convincingly describes the limitations of formal reasoning in theextreme, the practical extent of what can be accomplished is uncertain.

In understanding a book like this, it is useful to start with the problem that the authors aretrying to address, that is, what they believe needs to be fixed. Winograd and Flores object tohow computers are described in the popular literature, how Al researchers talk aboutintelligence, and the kinds of programs Al researchers, particularly in the area of NaturalLanguage, are trying to develop. Winograd and Flores reject the commonly accepted beliefsthat expert systems or any program could be intelligent, that representations can be used tomodel intelligent behavior, and that developing autonomous agents is an effective use forcomputers.

The book is based on the idea that understanding the nature of human cognition and whatcomputers can do will enable us to use them more effectively. While many examples are givento illustrate limitations of current computer programs and to raise new possibilities, it isimportant to keep in mind that. the authors have little interest in establishing what lies withinpractical bounds, that is, of what the representational paradigm will allow. For example, whatexpert systems might ultimately be able to do is not clear. The authors' philosophical stanceplaces more value "in asking meaningful questions-ones that evoke an openness to ne% waysof being" not "in finding the 'right answers' (page 13). Without a doubt, this book raisesgood questions.

Any attempt to summarize the arguments of this book in a few paragraphs is sure to raisemany more questions than it answers. I will only present a few of the important terms anddescribe the general structure of the argument. In subsequent sections, I describe what I 0learned by reading the book and the problems that I perceive.

-% N .' .-

2

In general, Winograd and Flores approach cognition and computation in terms of what itmeans to "understand language in the way people do." Their analysis leads them to concludethat computers cannot understand natural language-not just now, never. This is because allprograms-all representations, abstractions and primitives alike-are based on pre-selectedobjects and properties. The background that motivates representations, the experience behindthe designer's analysis, has been cut out. Thus, when breakdowns occur, that is, when aninability to cope occurs because the demands for action placed upon the program are differentfrom expected, there will be no basis for moving beyond the initial formalization. Yes, thedesigner can anticipate typical breakdowns and provide for representational change, but thesewill themselves be limited and prone to breakdown. The only way out is to generate newrepresentations from outside the representational realm.

The key to this argument is that new representations spring from a shared, unforinalizedbackground. Coping with a breakdown involves articulating the basis of a representation. Ifyou don't have this background, you can't speak with commitment, that is, with an implicitpromise to clarify your meaning if questioned. Since you can't negotiate meaning, you can'tengage in language.

According to Winograd and Flores, the view that we codify and store experiences inrepresentations that exist is the brain is naive. Rather, representation is a post-hocinterpretation of history. What we articulate has meaning within a context, and what we sayhas been shaped historically by that context. But it is only formalized (represented) when wespeak. We are not translating what we have already formalized.

The question naturally arises, just what are we storing in the brain? Perhaps we do not storeanything? What is memory anyway? What are experiences? Surely we retain something. Butperhaps we are not carrying around things in our heads? Consider how much we take forgranted, in particular how our conception of objects and space shapes our conception of themind, and how little we understand.

This book is anti-illusion, not anti-technology. It is not about what compulers can't do (seeDreyfus) or shouldn't do (see Weizenbaum), but what they can do and ho%& we should usethem. It is primarily a positive statement, an insider's attempt to articulate Al practice, tounderstand what an Al researcher is doing when he writes a program. Vhe goal is tounderstand how programs relate to life, what they capture of our nature, and what they leaveout.

While the stated objective of the book is "how to design computer tools suited to human useand purposes," the authors are most interested in understanding what it means to be human(12). They believe that the rationalistic tradition, based on ideas such as internalrepresentation, search, and choice among a set of alternatives, must be replaced if we are tounderstand human thought.

2. What the book is likeUnderstanding Cot0nlmt('r. and ( o iMton Is iltelligent, measured, mid imitru t te. It

deliheratel y avoids "philosophic schol arshi-p in order to focus on ceIIIr I po t s critical tode,,eloping a new understanding. In four nt rodluctory chapters, the atithors desir be therationalistic tradition, heruieneu tic,, misensull dollaliits, and speech A |t theor. Ihe discussionis admirably crisp. In just fifty paiges, the boo()k relates subtle, unfamiliar philosophical.biologldI, and linguisti ideas to whAt Al researchers do eeryday as programmers.

While a cursor', scan ih ,w,, the huk to be full t)f jargon -throv nness, ieadiness-to-hand,

shared background, hlind ness, hre dkmw1i, C.lni it meni -these words turn ml to he useful forretnn|g the message. I.ike Freud', lrgo i (e.g.. ego, subconscious), these teiiis iti rt)duce atnew langutge for thinking ahmut familtr things (40)

I he ho()k is als( renmklhle lot ,hap. definite statements that seem so cilitrar to) iomn1 nbeief: " ie catlit t c olltit I nii hines that either e hihil tor successfully model Intelligent

o -. -_-. - - - -. . . . -- .... o. , . .............

3

beha% Jor." Ama/ingl,, this conmes from sotlmeone who gdae uS alhl her hook e niil-'d,Uldcr,M'.ndim,'( .aturtl lanrguage. A "born again" coniction might lie behind the hook's boldremarks.

Al Al reSea rcher who." reads only the section on expert ssstells arnd d(oes ot( take the tirlue toseriousl study the arguments is likel.k to be greatly alarmed hy the inflammator language:"Calling it intelligent might he useful for titose trying to get research funding..." "l hedesigners themsel es are blind to the limits.."(93). In a few places the polemic becomesobscure and-I is easily dismissed: "World as the background of obviousness is manifest in ourevery day dealings as the familiarity that pervades our situation, and every possihle utterancepresupposes this" (58). But with rare exception, the jargon and original point of view combinein clear and thotught-pro, oking observations: "In trying to understand a tradition, the firstthing we must become aware of is how it is concealed h its ohviousness" (7). Almost everypage has an idea worth underlining.

Sometimes the hook has a poetic, mystical tone: "... (We) present the main points, listeningfor their relevance to our own concerns." The authors evoke reverence for their ideas,reflecled by the hook's final sentence: ''[he transfornation we are concerned with is not a

technical one, hut a continuing evolution of how we understand our surroundings andoturselves-of how we continue becoming the beings that we are" (179). Ihis is a hook ofreligiouS philosophy, an inquiry into the origin of beliefs, values, and practices, of their nature,why they work, why they fail, and how they change. To quickly dismiss this book on technicalgrounds is to in iss Much more than the authors' conclusions about cognition.

If ou are committed to understanding the hook, I encourage reading the introduction(Chapter 1), then skipping around in whatever order appeals to you. Chapters 6 and 8 areexcellent reviews; you could start there and then go forwards for the detailed discussion. Or ifyou prefer to start with the familiar, work backwards from the final chapter (which gives agood example of program design) to Chapter 10 (on current directions li AI) and thediscussion in Chapters 7, 8, and 9 (on representation and language). liowe er, I think that thebook must he read completely to be accepted, and I found that readi rg it twice, separated bymore than a \ecar, was valuable.

3. Important ideasWell into my second reading, I realized that somebody was wrong in a big way and I kept

stopping in allazement at the possibilities. Could it he that Winograd aid llores are mnostlywrong? What a grand embarrassment! From the other perspective, it's equal 1 amaiirnrg howmany researchers will respond with staunch certainty, "lhat's nut right. ('ornputers will dowhateer we program them to (10." Now l'',e accepted the maIin argUinlenlt and h:,,e settleddown to musing over details. I still think mIany people wkill liSt shake thei i heads and go,,

about their husiness.

Ott first reading, the idea that seemed Iost importanrt waX. that the LoilerM should bethought of as a niediurln for comlUnition, l'athel than a irI autono111tus agent. A computerdoes not understand, it is exhibiting ny commitments renliotel.. It is it)t the Ottiputer thatmakes requests )r promises, hut the progrimmer. Ihe c in ptitel shmt,, m1i\ pat terns, riiyassociations, my preferences.

This view increases my sense of responsihilit. and glli'lti. It is ni (,ik after all, notsomebody else printing things on the screen. [his also leads to ;m lillee,,111 question: lhowshould I project Myself? What should I put oil the screen to reflect ti c.hoices'? But after aYear, I don't think the 'computer as a nledIum'" idea has changed Ahat I dt. Itst m theoreticalunderstanding of what I am doing. After all. I tlw;is fell emh ra,,sed mi prtd about ill\programs. I alwa s knet that the computer was just sht)1 11+ Ill\ owlconlintis t(M that offellow programnners).

()n second reading, a .ompletcly different message hit tc I eat/ed thit I dm't hale all+ ,patterns, associations, or plefelences stored in I 111in i I ll', I, surileMl,hi deplessing aiid

Iconfusitug thought. A, Witiograd aind [lures ind1iLe, lhe inlpiItLtiuri,, AIn inur +e thdin tec.'hnical,

n M4

relating to our image of what a person is, and this can influence our response to the argument.But the new conception is useful, and after awhile it starts to make sense. To illustrate this, inthe following sections I summarize the ideas that I find particularly exciting by quoting fromand paraphrasing the book. I amplify these ideas by discussing other connections and researchimplications.

3.1. All behavior proceeds from the subconscious"To exist historically means that knowledge of oneself can never be complete" (33).

Language is necessarily blind to its context because it involves a formalization based on thehistorical structure of interactions. "As carriers of a tradition, we cannot be objectiveobservers of it. Continuing work to revealing a tradition is at the same time a source ofconcealment" (179). Language crystallizes what we are, but it is always partial, biased, andmomentary. The power of language is to articulate recurrence, to identify patterns, to claimstructure, to explain. But it is always post hoc and apart from our being.

While these ideas may sound strange, most people are familiar with the idea that somebehavior (at least) proceeds subconsciously, that is, not from articulated beliefs. This is theFreudian view of the subconscious: We act without knowing our own motivations. We do notalways act rationally, by choice. Winograd and Flores take this to the extreme: All behavioris direct, without intervening representation.

In the popular understanding of psychiatric problems, subconsciously-directed behavior isassociated with illness. We associate the subconscious with unusual, unhealthy behavior becausethe subconscious only becomes important to us when a breakdown occurs: a failure of acommitment, a violated expectation, a frustrated desire. When we perceive that we are apartfrom the world or when our actions are confused, this is when we use language to articulatewhat lies behind our behavior and our discomfort. We try to spell out what is not obvious,the assumed background that is affecting our behavior or our emotions.

In the "talking cure" of psychotherapy, a person articulates the recurrent structure in hisbehavior, naming situations and responses to them. Thus, he may become aware of thestructural coupling in his life, allowing a new interpretation and new behavior. In this way,Freudian psychology has been reinterpreted as a form of hermeneutics: "The mental self is astory whose meaning is only interpretable in my life's history" (Wilber in The HolographicParadigm, 276).

3.2. We are aliays already interpretingIn one of the most powerful ideas in the book, Winograd and Flores tell us that we are

always attending, always selecting. To understand the value of this conception, consider theproblem of explaining how we happen to attend to something. Suppose that I Aalk into amuseum and see something interesting and walk over to study it. How did I know that it wasinteresting? What little clue made me decide to attend to it and to realize that it wasinteresting? What littler clue made me notice the first clue? Maybe the frame showed mewhere to look, but when did I decide to decide to notice that frame? In the miuseum I analways attending. always making interpretations. I am [ot matching preconceptions andrecognizing value, as if they pre-existed as symbols in my head.

Following our practice ii naning objects and op)erties artotiuid us, we place thirigs it] themind: memories, symhols, patterns. We say: "Something gets recogniied." There is aii objectivesomething in the world; there is a pattern being searched for in the brain: there is a matchingprocess. Instead, Wiinogrid and Flores tell us. there are just interpretations. lhere are nopreconceived representations, no matching process. Instead, there is a "pre-orientation." "Weare always already oriented to a certain lirectior of possibilities" (147).

Similarly, iii pschiatric analysis, the rdea k,' a ssmbol Is used as if it were something thatresides in the head. But to say that "X sNilholizes Y for person P" is onl\ to sa\ that Presponds to X a.s if it were Y, iI a historical iterpretation of heha\ior. we note a paIttern and

+, -iI]1_

5

explain it by this association. There need be no translation, no "symbol mapping rules."

The point is more stark than it might first appear. The aigunlent leaves no room for sayingthat representations are perhaps "compiled," and this is why we have no conscious awareness oftranslating from representations to words. Rather we are not making decisions at all: We haveno choice, we are simply acting. There are no "things stored in the brain" that we aresearching or selecting between: "... the breakdown of a representation and jump to a new onehappens independently of our will, as part of our coupling to the world we inhabit" (99).

3.3. All reasoning involves reinterpretationAnother perspective on "direct action," or what Winograd and Flores following Ileidegger call

"readiness-to-hand," is that all intelligent reasoning is reinterpretation. This is far moreadvantageous than acting according to pre-conceived representations, and only finding out laterthat they are wrong. Yes, we make mistakes because we act inappropriately, but we are notfollowing "plans." There is extreme plasticity in our behavior. Every action is aninterpretation of the current situation, based on the entire history of our interactions. In somesense every action is automatically an inductive, adjusted process.

As an example of this phenomenon, close your eyes and consider how many windows are inyour bedroom. Did you visualize the process of moving around your room? Possibly we arereactivating a motoric sequence, simulating that we are actually in the room and moving about.We replay the history and articulate what we would see. But we are not necessarilyremembering a particular walk through the room. We are constructing a coherent story, whichis implicitly a generalization of our experience because it is based on all of our experience.We chain together a sequence of impressions and pretend that they occurred together. In thisway, the chains of association are constructed freshly each time, as a reinterpretation of theunfornialized background.

A functional simulation of the cognitive system in terms of manipulated representationscannot generate the range of reinterpretation an unformalized background allows. Wiiiogradand Flores conclude, following Searle, that manipulating a representation formally is notunderstanding. Certainly there are formal games which we understand how to play. We caneven accomplish our goals by playing formal games. But as soon as the interaction changesfrom the previous history, breakdown occurs, and a reinterpretation in language is required.We engage in a dialogue that articulates the basis of a representation and adjusts it to a newsituation. To understand is to be able to make the commitment to do this reinterpretation.

3.4. We assume commitment in other peopleAs Weizenbaum pointed out about ELIZA, it is amazing that we are so convinced b so little

and that we assume so much. Even when you are told how little ELIZA, SiIRDI.U, andMYCIN know, it still strains the imagination to appreciate the magnitude of their ignorance.We are all like children preferring to believe that the fantasy is real. Indeed, we don't have to"suspend disbelief" (in the theatrical sense), this is how cognition normally works: Attributingmeaningfulness and assuming commitment go hand and hand. The situation is insidiousness:We don't normally articulate shared background, and computers don't have any. It plays rightinto our assumptions. If you use my words, I assume that you know what I mean. If y.ou sayou believe something, I assume that you are ready to convince rie.

Weizenbaum stressed the lack of responsibility of computers because they are not part of thesocial fabric. The argument here is stronger: Computers cannot be responsible because thexcannot even form commitments. When I speak with commitment I do more than just mouthwords. I do not pretend. I am ready to defend what I sa . I am comm itted. lo speak thetruth means to be \killing and able to articulate why ou believe it.

In providing explanations, we must determine why breakdown occurred. What Is 1iot Obvious,not part of the shared background? What must he articulated? In constructing explanationprograms, I have often concluded that we have riot placed enough of the burden on the personasking the question. Unless there are systematic surprises that the explainer might guess, the

%.

9. J

questioner must articulate the nature of his surprise. The explainer must then be ready toform an interpretation that is contrary to his own point of view. Howe~er, we cannotcompletely model how a system's activity will perturb the interaction between user andmachine. We can't anticipate every user's interpretations of what the machine is doing (53).Given these limitations, we might focus instead on opening ip the program's representations sothey are easily browsed, making it easy for the questioner to figure out what he needs to knowon his own.

Winograd and Flores provide an intriguing discussion of how computers might be best suitedas coordinators of commitments, "the essential dimension of collective work." Following fromtheir analysis of language, they propose that a program keep track of what we have to do,recording the status of our active commitments. Their proposal broadens our awareness fromthat of the individual working alone at a personal workstation to the social dimension of whatis being written, computed, or recorded (158). This idea is likely to have widespread appealand could significantly enhance how we use computers.

3.5. People do not carry models around in their headsCognitive models explain patterns of behavior; they are developed by scientists. It is a

strange and tremendous leap to say that these models actually exist in the heads of the peoplebeing modeled. While we commonly say that a person has knowledge, knowledge is notsomething that you can possess like anl object. Knowledge is always an individualinterpretation within a shared background. It is neither subjective, nor objective (75). To saythat someone knows something is not to say that he is in a certain state, but to explain hisbehavior over a sequence of interactions and to claim that he is predisposed to act in a similarway in similar situations (47).

Few people believe that when we ride a bicycle we are manipulating internal representationsof the handlebars and pedals, modeling their location internally, and conmputing trajectories.According to Winograd and Flores, speech is the same, a kind of skill coupled to theenvironment. While we ma) s mbolize our utterances on paper in some calculus or writtennotation, there is nothing corresponding to these notations ini the brain.

This has significant implications for understanding expert behavior. We are not modelingobjects that exist inside an expert's head. This explains what is so patently obvious when youwork with experts, namely that they have so much difficulty laying out consistent networks anddescribing relations among concepts in a principled way. If experts knew causal andsubsumption networks as discrete concepts and relations, why would we find it so difficult toextract these statements from them? The concepts are often not defined, let alone related in afixed, systematic way to one another. Experts know how to behave and they knowformalizations that model how they behave.

The evidence ii student modeling research is similar. Brown and kanl ehn found thatstudent errors in subtraction, modeled as bugs in the procedure followed h\ students, changedover time. They called this "hug migration" and sought a systematic explanation for \Ahy thebugs changed. The key is that there ne,,er was a hug in the student's head. When ou reali/ethis, you realite that you (lont have to explain why the Students decidcd that one plrocedurewas wrong and another was better. This does not mean that there is no pattern to be found.Winograd and Flores would say that there is a mechanistic argument, it just isn't based onmanipulating a represen tat ion.

l'his analysis might lead to an entr rely different teaching method: not Isolating the bug, butestablishing an appropriate couplirrg and forcing breakdown. Ilowe,,er. we still need tounderstand what articulation does to hehavior. For example, in physrcal skills, sucth as playing

the piano, attending directly to a fatlty a1cton1-actually feeling that you are making ithappen-al lows you to get a grasp on the beh i\i,,i and change it. fhe role of language Inisolating where an undesirable action oCCuS. ,t0orlaMcall\ is perhaps similar, again, thepossible value of psychoanalysis.

By this model, the most effecti',e ralririg OC mLS oi tile jolb site. hence the mrP,, lui.trorial

S

-"'" -- ' ." ' - " - .-. -- ;,"," " , '- _- , , ."I€€?. a-3 ". t, A " '-,""""".4,:€ - :,; : ,- " "

'""

'

7

Stratg of' gettinig thle students to thie workplace and niniiir g the okaSSloort hLihheWi nograd and Flores' anlal ysis p ro0ideS it Subtler understanding: lecKh I g ri st : lhi hthe history of' interaction that COnIstitutes aI backgrounVd that will lead to uiSeful mintrpietltutrISEstablishinig a situctural coupling' means experiencing this brlstor\ of, Ilirreraclron, not sbeing told %%hat you sAould do if' you had gonle through thie process. Ihle probleIII IS 1,1111lrrm.'You dl't uriderstairld me because \ou (lou't kilo" Abhele l's e been. If- aill rrmclI gem behas off lowss froMl thle uLilforinald i/ed baIckground, teaching hosA to behase by sas rig hlrt tot o canoils prosmide patterns to lolloss bs rote. 'You hase to trs it yourself arid gel thie feel of it.

Hl s, the repeaC ted teacher's tall back, "YUI kirov better sAhat I mieari Alleni \oi Iet mut there.''While thle implications I'or Inls(tuctionlal comlputing are riot clear, it Seems att least

theoreticall importat ito realize thilt there is a difference betweeni solking aI problem arndarticulating a imodel (raionliinilg thle soIlution sequence). We knew, this already, hilt Wiliogradarid Flores proside al theoreticall f'oundaion for bUidI rig upl a ilesA 1.iidlrStaridirig. so swe can

riess the iriabilits to articulaite a niodel as not a lacko of self_-kritt% ledge, bilt the riorrrl stateCof, afflairs.

.. 11-riinvering (vlVoipN fromi recurrent breakdo~inUnderstandinig arid aiiticipAirIg filure (breakdos\ri) is at the heairt of e I rrl ie er Ing.

F-Sseii Iilly, Wi iigrad amid Flores has e prov ided a theoretical background ftor uinderStanri lig hoss

progr iii rg. especially kriossledge engineering, is like Structural eiigiiieerr[i. It rograrrIIS donl'talsasdo ss hat 'se espect because the designer did riot anticipate ho"4 users ssould iteract Asith

hie Pr ograin and Initerpret its actifoils. A large part of this book concerns hoss progami-mersrist an iclmme Ihe demnirds of the env~i ronmeri aid ( p re paIr e prI-ogram II)s I(o cope witIh

ASii 11) IiUciiral erigiieerinrg. \, hat ma kes k nowvledge engi neerinrg possible is that brea kdossec Lu i. 1 heSe pat errs lead the enginieer to fom u1Llate "objectiv~e distinct ions." Fsseri all A shat

sse tae to be obhjectise truth is sshat many people hase articulated oiser a long period of timeaMid "se as1 obsIerve s e\1ect to Co11il l mte ill the futu re.

lit SSes er.cois COI uct1 i ri an a tOrIOMOUS agenlt is Munch more dlifficul t than typical engrineerinrgproblemrS. It is, like building aI bridge that will change its own structure as its interaction wsithtire enirorirrierit canges. Iis is the idea of anl autopoictic systemr: It mai'iitain1s its function01s.Itfirs theory Asas des elo)ped inl biology , arid li ving Organisms are the best examples we base.tilie idlea m)" ts 'ises Inl ctiln pUter sy sternls eiigi neerinrg and plays aii imniportan t role i (tihe design

of satellites arid planetary probes.

A cruiafl pmInt is that the organlismi adapts to rieet the demiands oif its ifnteractOios. But tIsis fitl tt aPit cess 0'i rep reserit inrg the world: 'The demands Of con tin ned a ntopoiesis shape thisstrmct iiF inn a Asas that can he viewed as a reflect ion Of a externa 't "orl d. Hu thct ti espt tiuerice is iot tine ill which the form of the world is sonnehoss mapped on to thestnitilure tof thre rtrgarrisrir'' (02).

3.7. S~ stematic domiains admiit to formal representation~S Semiit dtomairns are those Ii which there is a great regularity iii relatimtiS oser tirre andift rig people, so that there appears to be objective kniowledge (172). Iii moidel inrg initell igent

beha sittr Asitbin aI ssslemat ic dormin, we don't arid rieed n'tlnecessarlyII rideed, carl't)I representlie Irfearri rig of, termis. Rather we represent their systemaic role wsi thrin a rietsork of" requestsirid pr t tnises. Inrdeed, the operat ion oif a program does riot require that it repreSents any t hrrgit :ill, It'S ill i the eye of the beholder, who Interprets Input arid ttitpnrt Ii teris of- a

sySleriritfi( iriappllii InI Inls sorld (80). When \IYCIN says thle "mtt ''culture,'' yti inter wet itXs sMttifig tth.trs~le tli1t you krios abutu.

I IriS ial\Srs prosIes at Fascinating handle ttn the nature otf languaige, rirttdels. aridlf r ilail 11i Ithelpt" (IS irrdeitrsaid ssh\ progrars ssork as wsell as, they dtt mid sliutl oan gtt'A Igml. I bns, AVe Cirl better understand "hbat sue aire dttirig ari perhaps bossA ito Vtbou it

TIA 1) III IT (-,I cI I c 1) 1 v' I I I cII I I l II I ko t ( it, Ic 1

rese osirt \lit , t heilI IC t I I kI I I ,NlI t k l i t.Ill It teI IIN l~ h, IT ctc'rt!

Iell~e j tjItltvll N" Ie NIt icI t, !~ k 0 Ille'. It i tN It.*ti PI, !1ti. it , I i~l. N Ntte

kn l(!lt% It~ Stfit tile sn htl t .IN t4ll l ' v 1C. 1 1~ it tc I Il

l~~~~ti he 1Itslhe Il lF)O rel d f , M tel " t llig,, ,k 1l I\ ' (.I N'sfi it lltl P Ile !h l i;,lI~S f111 llil' fillt~

ctlPel~ Im)" jltImp( i l lu lt'iIe fiet !lt l' Ite Ilc h t t'l l \a- ht 11.c1al t'tlte . (c tallii\ 1-1

as I ilt slea lt, enef ate 11I1 1 1 cpI c set I itt Il es , rl l)JI( lIt, J II I d. k, c lejt I d fth I ghtwrseh'I hue %A vh I;I If I Iet )1ee t f itt fI I IS illti IT 1q, tale. ll Atlt hi e ) :5 keH I l t hatll the 11 1

prOilii 111111 thtis Owale ". Nc tI I I Ilit.,I i Ils - ,IdIc Iii. I he t c i can , cIli~il I [Ie

les lin hut 'AL' cll lie ce I'et hl hie, l: re eltali- IT IIltt~~i fe ant iiia 1e 1 Ckil

I(lvIl I, erche.nt.i' ,j

(AIlcl Ielehnt 1) ll 11 pc till Iit ci I eI t I htlait Iiit SIt Ii e Le e\t1 Ill h 11 !IlIl Ing ate, F

I lge ntIII1)1 ,I ( [egeI t I I I li lt I lllii (t heI /t!-i' .1 1!.J //'ii11h I ell Itd I tIt'- 11 1tehl II evtlan

& I IItI :111l 11ti\ Il glie, i "1 .1 11 5 (C I llilg I I I IlIv lIIII-N Sli I I l' tha tI Iis 11, 1 11 1t 1 VIge IIe.

eq I I I " t ItIII ' I CS SIH I 110 If II t'. heIII cd t III( Ift'tl 11,H1iltte t, A r th i e LI

fil d e III I l kIF I ll i ve , Ik '!I, Ik f I I IIC' 1 1 1 ,, 1 ed I . . II kI II _, I il t kI I" II II.j ,I II "F . Ih il,. k C 1) 111d il) 1)1 V ,C II I',tI I I 1, A I(I Ii-- O tfit d I"t I t. I 1,1

I l (M- ii p III.V 1 [)1 ~ - !:'It I heli-K .. r' . . . . .mo I fi !f de, I- I.i l M A (IIPt [t

9

While the book has clear implications for research in Natural Language and instructionalresearch, as I have described, it is unclear just how well computer models might eventuallyperform in systematic domains. That is, how well can we circumscribe domains to constructuseful and effective representations? Is it of any practical value to say that commitment andhence language cannot exist in a systematic domain (where the importance of unformalizedbackground is minimized)? Is the shared background of people as great as Winograd andFlores st-ggest when they exclude computers from participating in language? When hreakdownoccurs, just how well do people resolve it? In this section I consider ways in which thearguments in the book appear to be incomplete and perhaps distort the nature of cognition.

4.1. What is the mechanisin of ,nemory?To recapitulate, conceptual structures are not stored in the brain; the conceptS Of our

language do not organize our memory. There are no stored associations, no colcepttlnetwork. Instead, we act: We speak, we associate. We don't do this b) interpreting a iietoi kthat mirrors the conceptual structure in what we say. Rather, the history of our beha,,ior ma\exhibit recurrence that we can represent as such a network.

We are left with the image of some amorphous blob that speaks. How can we explainrecurrence, if there no structural predisposition to associate concepts in some way? Winogradand Flores believe that there is some mechanism behind jumps to new representations, but theyprovide nol description of "hat it might be. They make no attempt to reinterpret models ofmemory and learning according to their theory.

For example, what accounts for our tendency to remember exceptions (as described bySchank)? Winograd and Flores acknowledge that Schank's work and ideas like "defaultreasoning" are closer to the nature of cognition, but they insist that these approaches are stilllimited by the need to distinguish the relevant objects and properties before doing anyrepresentation ( 116).

Is there any e ,idence of a mechanism that generates the recurrence in our behavior? Whatabout timing experiments in\,olving discrimination and recall? Winograd and Flores claim thatthese experiments do not deal with "meaningful material" (114). But aren't hierarchicalrelations meaningful? Couldn't systematicity in abstraction, for example the patterns in levelsof "natural kinds," he explained by structural properties of memory? The problem is that an.admission of structure it] the brain corresponding to conceptual relations undermines theargument that representations do not exist in the brain.

It appear,, ohions that the way the brain works favors categorization and association ofcertain kinds From here it is but a short step to hierarchical search. Perhaps Winograd and

lore,, (and MattUrana) have got the main idea right, that we aren't examining representationsinternall,. hut the ha,,e woefully ignored the problem of explaining recurrence in memory. Itwould ha , been helpful if' the hook included anr appendix that at least acknowledged opposingarguments (such as Fodor's The language of Thought).

4.2. IIoA much background is shared?"If a Iion could talk, we could not understand him." (Wittgenstein, Philosophical

According to Winograd and Flores, language requires being able to commit to articulating ashared background. If no common ground is found, then breakdown is not resolxed. But thehook seems to understate the difficulty of resolving breakdown. Isn't the normal state ofaffairs one in which individuals (and countries) frequently do not understand each other? Weget by normally by making many assumptions and by ignoring differences. Perhaps resolvingspecific differences is riot as important as sharing the goal to "work something out."

A few social rules (part of shared background) for coping with unresolved breakdown may bemore important than having the shared background to resolve specific differences. Obviousl),not much communication could occur on the basis of just agreeing to disagree. Htowever, the

%" ,%.1-"',.,

.10

book, seems to adopt tile opposite stance ot' ideal ittig langunage, suiggesting that mlostbreakdowns are resol ed li the Specific elements of' Shared background. I he general agreement

* to adapt and cope with arnbiguit\ and unresoixed differentes 1.01.1d be more inpmul at.

The book's Idealized description of language is clear Ahenl we consider our Interactions with* animals and children. Shared background is immii ed hei e. but C01n1rn Lill icationi is possible.

It isn't necessary to be "full'. human" to engage lin language. Exeri if computers areprogrammed to be part of' the social struIcture, exeri if the are our ski'.es, i nteractioni withthem canl be COnISenISual. Aheir speech acts canl create coni htmen t, just ats muuch ats at dog canlrequest to play and theii become engaged lin a game of mock attack or chasing. ('ertamiily therewill be practical limitationis, as we max not always understand a chimpanzee, and the

*differences between at (og', barik ito pla\, to eat, or to warn ma', be too mubtle for anxone buthis master to discern. Iliowe e r, of what prac t cal x al ue IS It to so narrow I define language

*and intelligentce as to rule out thie behax or of' animals beC.-Huxe they are not 'fully hunmn"? Itis good to make people more aiae of the social di meniinons of lagae b ut Winiograd anidFlores have adopted ant al most religious point of' xmew that mlax ox erstate the nequ i rernen ts ofshared background and the extent to w'hni h breakdo\&n i," t~picall\ resol xed.

4.3. What are (lie practical limits of formizadi~tion?The book states that we are "now.' wxitnessi ng a major breakdown in the designi of computer

% ~technology" (78). No evidence for this obserxation us, gixen: just the in1verse seems to be truerWe are witnessing a miajoir recogiitiOli of ho0'A' inti I human knowledge is regular and can heusefully formalized. lin the rapid gr ox'th of expert s'.stems appl icant is, eniigIneers in particularare realizing the value of qualitati xc modeling technilques for describing recurrent Ohjects.

* properties, and relations. Possibhi' there has been aii misiniiterpreta tion of whlat cornputers are* ~doing and the nature of intelligence, but the payoff is onl the upswin mg and thle I iriits appear to

be years away, at least.

*The book provides very little basis for determni ong the practical Iitin its of forma I'lIiat ionl,particula rly for appl ications of Ar i fwcial inelhIreiwe ito science aind eng inieerinrg Perhaps by,corntinii lig to find structure wit hini st ructuire xxe cai get programls that are x cry good, and ex en

-fool miost people. Yes, they will fail somletimles, but so do people. There is, little ex iderice that- the practical limitations of formal reasoning are as serious as the hook Suggests

- Practical implications of the argument tend to return to coniclusionis we already knex', as I- indlicated in briefly considering explanation. teaching. and knox'. edge acquisition. I low exer. the

book gives us ant improxed understanding of xx hat is difficult and why we ini not1 Suceed'lihe most iminportan t change imight he a better understand inrg of w&hat we are doinrg.

* 4.4. Isn't reflection an essential part of' reasoning?''I lriiiian cognition inludLRes the use Of represetinros. but is not bitsed oii repiexemiitr on

''F' w ts do nlot ineed to haxe formalized iepreseiitatioiis III oider to actt [hle% nayt 'it timles* niar11)Irlate representatis ais one part Of suc:essfull ao t. but( it is fruitless t(, searlii) for aI

full forization of' thle pre-understairdin that uinder-lies all thought arnd I ion" (99 "I hteessenlce Of our' intelligence is our throw mness, rnot ouri reflek troll.'

I believe that this book signnfitdatl urr"der les thle rIIIor tan11C Of ICHCMl . 10oit the po int ofdistorting the nature of cognition 'lit reflection,. we airtr~ nlate our haL kgimnd li .)rder tocompare possible behax nors, atincipate conSequlenCes, arid plant) rather than uatmig inipirlsixelyFxen granting the nature of unforimralized background, ieaidiess- to-baird. arid thle immedjite

* nature of reflecton (we don't detide to reflett, [.the xallued ;Itioii III aI coiserisial (1lm:rnii isone that anticipates rairiiatioirs. Iltirriai rea~minrip Is innnr1errsel\ MOre sluLceSsful by M11

* ability to sim~ulate whA irriglit hadppen, toW islzltle possible oti(oMiS arid prlepare for tlerrvWe dto this by reflect inrg. sa ymrg whamt we expet, arid respondring to whatI we six (,\ir excellerrdescription of this inurigiiation ill-ess appears lin laxnes's Ricamcial 'ifod I

- ~~~We create repre.sentaionls h\ laiiqnrarg. ) atiiig. ".%e rina1keitepeti'rsb whr e x

I-xery representain is iii litePrettloll lHnt Isn't exeix relpreseirtMItIor tlnerel..ic tit-11rttll

11

crucial in our action? Granted that representations are not "inside" and that they are blind,once articulated don't they play a central role in intelligent behavior? We are alwaysreinterpreting old representations. We are not just speaking like birds singing. The Xarticulation is essential, it can change our behavior. Winograd and Flores fail to properlyemphasize the loop: We are always listening to ourselves. Even if representations are notdirectly generated from representations, the> ire generated in response to representations. Inparticular, imagery and silent utterances are a form of "mental representation" which is part oftle cognitive sstem. While these representations may be unnecessary for behavior, muchbehavior is mediated by them.

The weakness of the argument minimizing the centrality of representation is seen clearly inthe example of the chairman who is always directly acting (34). Winograd and Flores greatlyunderstate the importance of making observations, forming hypotheses, and consciouslychoosing a course of action. In chairing a meeting, I attend, stop myself from sayingsomething (anticipating a reaction), plan things to say, arrange a list of people to call upon,attempt to weigh alternative topics, watch the time, and suggest a revised agenda. The book

seems to over,eneralize the nature of physical skills-as provided by the example of a hammer

and how we attend to it-ill suggesting that cognitive behavior generally has the same degree ofautomaticity and lack of reflection. Granted behavior must be immediate; there is nohomnunculus inside interpreting representations. But forming representations and reinterpretingthem is where all of the action is! Cognitive behavior is strongly coupled to therepresentations it creates: as visualizations and silent utterances, they are "inside" the system asmuch as anything else.

Most of my day involves an inner conversation. Most of my awake activity is a longsequence of telling and asking statements to myself. Granted, I don't know where the questionscome from (I don't have to work at firing neurons). Granted, I don't know where the answerscome from. I just keep making requests and promises to myself. "What are all of the projectsI'm working on now? I have to call Jan. What will I do when I finish this? I'll work on thereview tomorrow. Where is my yellow pad?" Most of my life seems to involve responding tomy own language, the repre.,cntations I generate.

Winlograd and Flores appear to have gotten the emphasis wrong. In emphasizing that TELLand ASK actions do not come from interpreted representations, they ignore the crucial pointthat thinking involves the generation of representations and attending to them. We areconstantl, observers to our own thinking behavior. We are constantly responding torepresentatiois.

Most important, we tell ourselves what we might do. Then we react to this. And in ourreaction we promise ourselves that we will do something different or make a request. We donot just simply act. We are engaged in a loop of imagining action and visualizingconsequences. Yes, our words and motoric actions always proceed directly, but often notbefore intervening representational actions and sometimes not at all without them.

Iow then (do we get to the state of reflecting? What, after all, changes us? Perhaps 0reflection is built in? Being able to place ourselves in a situation so we can know how wemight behave is incredibly powerful. It means being able to simulate a structural coupling, toknow what we are apt to do. This is much more than articulating a background; it isarticulating the behavior that the background will elicit. By projecting forward in this way,.admittedly with uncertainty, we can anticipate the consequences of behavior. This anticipationthen has the potential of changing our background and resultant behavior.

By overemphasizing the direct, ready-to-hand, unreflective core of all behavior (includingreflection itself), Winograd and Flores understate the importance of representation inintelligent behavior. That an expert can act without a representation is not very interesting incomparison to how impoverished his behavior would be if representations were not availablefor solving the difficult problems.

%0

0)

12

5. Reconimendat ionsThis book should make Al researchecs more Cautious about what they are doing, more aware

of the natuire of formali/ation, and more open to alternative views. By addressing the natureof representalion and reasoning with examples familiar to most Al researchers, the book hasthe potential of being More influential than other criticisms of the field.

A scientific enterprise requires openness to blindness of all kinds. This hook explains whyblindness is ineti able and elevates Our awareness of the origin of language and howbreakdow us occur. When the Chenoh.ls and Challengers of Al occur, we can look hack at thisbook to better tindersaid wyh our programs failed. The book provides an importanttheoretical basis for the analisis of failure in knowledge engineering. Indirectly it tells us howto anal ze domains What are the recUrreilt dialogues? What breakdowns occur? What arethe expert's methkds for coping with breakdown? What are the shared sources of experience?Who can talk [to Ahomli atid w

*Eei Al reseaikher should read this book. I)esigners of interactive programs interested intheoretical aspetls ot language and inmproxiing their understanding of what they are doing willfind this book to he faScltti ng, eingrossinug, and obstinately provocative. 1'he title is apt: Ifyou ire Interested ill uiiderstiiding what computers can do, for example how you light usethem in )Oui htUsiineSS, aid ha,,e a. philosophical bent, you should definitely read this book.However, he forewarned that it points the way, rather than providing answers.

The authors s'tle that tile book is not intended to be a scholarly treatise, and it was probablya good idea to simplify the presentation in this way. However, I think the hook will mostlyappeal to researchers and academicians, aud these readers should be aware that there are otherbooks that adopt siiniir points of view. For example, I learned about "readiness-to-hand"from reading Polaii, who calls the idea "tacit knowledge" (using the same hanmmnier example).Yet, Winograd and Flores do not cite Polanyi, and Polanyi does not cite Ileidegger. Theintellectual development of the ideas is therefore obscure. I believe that Richard Rorty'sPhilosophy and the Mirror of Nature (not cited by Winograd and Flores) is a good referencefor readers who want a more complete understanding of the argument against the idea ofinternal representation. Understanding Computers and Cognition goes a long way towardsmaking philosophical works like this more accessible to Al researchers.

In conclusion, even though the the book is extremely well-written, its argulmlentS are socounterintuitive many readers are likely to remain confused and unconvinced. The book helpsresolve foundational issues of Al, but the practical implications are unclear.

One goal for writing the book was to prevent a false view of computers from distorting ourunderstanding of people. Ironically, the book's new view of cognition is a little scary, makingreasoning seem limited and out of our personal control. The earth is not in the center; man isnot in the center, and neither is his conscious mind. [he relation of responsihilit, toreflection needs to be better developed and balanced against the core of automnalicily Iblat liesbehind behavior. Oil the other hand, the book supports a hulialiiSt position, Cilliphasi/ing ourcommonality, that what we are is mostly what we do together.

Certainly, this hook might change how You think about the world. As I sqUiashed i hugemosquito the other night, I thought, "So Much for allother Sltlctlural cou)plinl."

References

Brown, J. S. and Vanl.ehn, K. 1980. Repair theory: A generati\e tIheory ofbugs in procedural skills. Cognitive Scientce, 4(4):379-415.

Clancey, W. .1. 1986. From Guidon to Neoinmycini and Ileracles.IFhe Al Magazim', August, 7(3):20-40.

Dreyfus, II. 1.. 1972. I'hat Computers (an't Do: A Criliqu' ol Artifctal Rca mi.New York: Ilarper and Row.

% % N?-V

13

Feigenbaum, E.A. and Feldman, J. 1963. Computers and Thought.Malabar: Robert E. Krieger Publishing Company, Inc.

Fodor, J. A. 1975. The Language of Thought. Cambridge: Harvard

University Press.

Gould, S. J. 1977. Ever Since Darwin. New York: W. W. Norton & Co.

Jaynes, J. 1976. The Origins of Consciousness in the Breakdown of theBicameral Mind. Boston: Houghton-Mifflin.

Polanyi, M. 1967. The Tacit Dimension. Garden City: Anchor Books.

Pribram, K. H. 1977. Languages of the Brain. Monterey: Wadsworth Publishing Co.

Rorty, R. 1979. Philosophy and the Mirror of Nature. Princeton:Princeton Univeristy Press.

Sclank, R. C. 1981. Failure-driven memory. Cognition and Brain Theory, 4(l):41-60.

Weizenbaum, J. 1976. Computer Power and Human Reason: From Judgment toCalculation. San Francisco: W.H. Freeman and Company.

Wilber, K. W. (editor) 1982. The Holographic Paradigm and Other Paradoxes.Boulder: Shambhala.

Winograd, T. 1972. Understanding Natural Language. New York: Academic Press.

Wittgenstein, L. 1958. Philosophical Investigations. New York: The Macmillan Company.

Y!

Personnel Analysis Division, Dr. Gordon H. Bower

AF/MPXA Dr. Eva L. Baker Department of Psychology

5C360, The Pentagon UCLA Center for the Study Stanford University

Washington, DC 20330 of Evaluation Stanford, CA 94306145 Moore Hall

Air Force Human Resources Lab University of California Dr. Richard BrabyAFHRL/MPD Los Angeles, CA 90024 NTSC Code 10

Brooks AFB, TX 78235 Lo. aes CA 90024 Orlando, FL 32751Dr. James D. Baker

AFOSR, Director of Automation Dr. Robert Breaux

Life Sciences Directorate Allen Corporation of America Code N-095RBolling Air Force Base 401 Wythe Street Naval Training Systems CenteWashington, DC 20332 Alexandria, VA 22314 Orlando, FL 32813Dr. Robert Ahlers Dr. Meryl S. Baker Dr. John S. Brown

Code N711 Navy Personnel R&D Center XEROX Palo Alto ResearchHuman Factors Laboratory San Diego, CA 92152-6800 Center

Naval Training Systems Center 3333 Coyote RoadOrlando, FL 32813 Code 71 Palo Alto, CA 94304

Dr. Ed Aiken Navy Personnel R & D Center Dr. Bruce BuchananNavy Personnel R&D Center San Diego, CA 92152-6800 Computer Science DepartmentSan Diego, CA 92152-6800 Mr. J. Barber Stanford, CA 94305Dr. William E. Alley HQS, Department of the Army

AFHRL/MOT DAPE-ZBR Dr. Howard BurrowsBrooks AFB, TX 78235 Washington, DC 20310 National Institute of HealthBldg. 10, Room 5C-106Capt. J. Jean Belanger Bethesda, MD 20892

Dr. Earl A. Alluisi Training Development DivisionHQ, AFHRL (AFSC) Canadian Forces Training System Dr. Patricia A. ButlerBrooks AFB, TX 78235 CFTSHQ, CFB Trenton OERI

Astra, Ontario, KOK 555 New Jersey Ave., NWDr. John R. Anderson CANADA Washington, DC 20208Department of PsychologyCarnegie-Mellon University CDR Robert J. Biersner, USN Dr. Tom CaffertyPittsburgh, PA 15213 Naval Biodynamics Laboratory Dept. of Psychology

P. 0. Box 29407 University of South CarolinaDr. Nancy S. Anderson New Orleans, LA 70189 Columbia, SC 29208Department of PsychologvUniversity of Marylind Dr. Menucha Birenbaum Dr. Robert CalfeeCollege Park, MD 20742 School of Education School of Education

Tel Aviv University Stanford UniversityDr. Steve Andriole Tel Aviv, Ramat Aviv 69978 Stanford, CA 94305George Mason University ISRAELSchool of InformationTechnology & Engineering Dr. Werner P. Birke Joanne Capper4400 University Drive Personalstammamt der Bundeswehr Center for Rt. 2arch into PracticeFairfax, VA 22030 Kolner Strasse 262 1718 Connecticut ; ,_ N.W.

D-5000 Koeln 90 Washington, DC 200u>

Dr. John Annett FEDERAL REPUBLIC OF GERMANYUniversity of Warwick Dr. Jaime CarbonellDepartment of Psychology Dr. Gautam Biswas Carnegie-Mellon UniversityCoventry CV4 7AJ Department of Computer Science Department of PsychologyENGLAND University of South Carolina Pittsburgh, PA 15213

Dr. Phipps Arabie Columbia, SC 29208 Mr. James W. Carey

University of Illinois Dr. John Black Commandant (G-PTE)

Department of Psychology College, Columbia Univ. U.S. Coast Guard603 E. Daniel St. 525 West 121st Street 2100 Second Street, S.W.

Champaign, IL 61820 New York, NY 10027 Washington, DC 20593

Technical Director, ARI Dr. Arthur S. Blaiwes Dr. Susan Carey5001 Eisenhower Avenue Code N711 Harvard Graduate School ofAlexandria, VA 22333 Naval Training Systems Center Education

Orlando, FL 32813 337 Gutman LibrarySpecial Assistant for Projec Appian Way

OASN(M&RA) Dr. Robert Blanchard Cambridge, MA 021385D800, The Pentagon Navy Personnel R&D CenterWashington, DC 20350 San Diego, CA 92152-6800 Dr. Pat Carpenter

Carnegie-Mellon UniversityDr. Michael Atwood Dr. Jeff Bonar Department of PsychologyITT - Programming Learning R&D Center Pittsburgh, PA 152131000 Oronoque Lane University of PittsburghStratford, CT 06497 Pittsburgh, PA 15260 Prof. John W. Carr III

Department of Computer ScienceDr. Patricia Baggett Dr. J. C. Boudreaux Moore School ofUniversity of Colorado Center for Manufacturing Electrical EngineeringDepartment of Psychology Engineering University of PennsylvaniaBox 345 National Bureau of Standards Philadelphia, PA 19104Boulder, CO 80309 Gaithersburg, MD 20899

q

Dr. Robert Carroll Dr. Stanley Collyer Goery DelacoteOP 01B7 Office of Naval Technology Directeur de L'informatiqueWashington, DC 20370 Code 222 Scientifique et Technique

800 N. Quincy Street CNRSLCDR Robert Carter Arlington, VA 22217-5000 15, Quai Anatole France

, Office of the Chief ;5700 Paris FRANCEof Naval Operations Dr. Lynn A. Cooper

OP-OIB Learning R&D Center Mr. Robert DentonPentagon University of Pittsburgh AFMPC/MPCYPRWashington, DC 20350-2000 3939 O'Hara Street Randolph AFB, TX 78150

Pittsburgh, PA 15213Dr. Fred ChangNavy Personnel R&D Center Dr. Meredith P. Crawford Mr. Paul DiRenzoCode 51 American Psychological Association Commandant of the Marine CorpsSan Diego, CA 92152-6800 Office of Educational Affairs Code LBC-4

1200 17th Street, N.W. Washington, DC 20380Dr. Alphonse Chapanis Washington, DC 200368415 Bellona Lane Dr. R. K. DismukesSuite 210 Dr. Hans Crombag Associate Director for Life SciencesBuxton Towers University of Leyden AFOSRBaltimore, MD 21204 Education Research Center Bolling AFB

Boerhaavelaan 2 Washington, DC 20332Dr. Davida Charney 2334 EN LeydenDepartment of Psychology The NETHERLANDS Defense TechnicalCarnegie-Mellon University D nf a cnterSchenley Park LT Judy Crookshanks Cameron Station, Bldg 5

ttsburgh, PA 15213 Chief of Naval Operations Alexandria, VA 22314OP-Il2G5 Attn: TCDr. Eugene Charniak Washington, DC 20370-2000 (12 Copies)Brown UniversityComputer Science Department Dr. Kenneth B. Cross Dr. Thomas M. DuffyProvidence, RI 02912 Anacapa Sciences, Inc. Communications Design Center

P.O. Drawer Q Carnegie-Mellon UniversityDr. Paul R. Chatelier Santa Barbara, CA 93102 Schenley ParkOUSDRE Pittsburgh, PA 15213PentagonWashington, DC 20350-2000 Dr. Mary Cross Barbara Eason

Department of Education Military Educator's

Dr. Michelene Chi Adult Literacy Initiative Resource NetworkLearning R & D Center Room 4145 InterAmerica Research AssociatesUniversity of Pittsburgh 400 Maryland Avenue, SW 1555 Wilson Blvd3939 O'Hara Street Washington, DC 20202 Arlington, VA 22209Pittsburgh, PA 15213

Mr. Raymond E. Christal CTB/McGraw-Hill Library CNATRA N301AFHRL/MOE 2500 Garden Road Naval Air StationBrooks AFB, TX 78235 Monterey, CA 93940 Corpus Christi, TX -8419

Dr. Yee-Yeen Chu Dr. John EllisPerceptronics, Inc. CAPT P. Michael Curran Navy Personnel R&D Center21111 Erwin Street Office of Naval Research San Diego, CA 92252!oodland Hills, CA 91367-371 800 N. Quincy St.

Code 125 Dr. Richard Elsterr. William Clancey Arlington, VA 22217-5000 Deputy Assistant Secretarytanford University of the Navy (Manpower)nowledge Systems Laboratory Dr. Cary Czichon OASN (M&RA)01 Welch Road, Bldg. C Intelligent Instructional Sy Department of the Navy'alo Alto, CA 94304 Texas Instruments AI Lab Washington, DC 20150-1000

P.O. Box 660245hief of Naval Education Dallas, TX 75266

and Training Dr. Susan EmbretsonLiaison Office Bryan Dallman 'niversity of KansasAir Force Humdn Resource Lab AFHRL. LRT Psycholoqy DepartmentOperations Training Division Lowry AFB, CO 80230 426 FraserWilliams AFB, AZ 85224 Lawrence, KS 66045LT John Deatu)I

7ssistant Chief of Staff ONR Code 125for Research, Development, 800 N. Quincy Street Dr. Randy EngleTest, and Evaluation Arlinqton, VA 22217-5000 Department of Psychology

Naval Education and 1University of South CarolinaTraining Command (N-5) Dr. Natalie Dehn Columbia, SC 29208

NAS Pensacola, FL 32508 Department of Computer andInformation Science Lt. Col Pich Entlich

Dr. Allan M. Collins ITniversity of Oregon HQ, Department of the ArmyBolt Beranek & Newman, Inc. Eugene, OR 97403 ix-SA(DACS-DPM)50 Moulton Street Washington, DC 20310Cambridge, MA 02138 Dr. Gerald F. DeJong

Artificial Intelligence Grou Dr. Susan EpsteinDr. John J. Collins Coordinated Science Labor to Hunter "ollegeDirector, Field Research lUniversity of Illinois 144 S. Mountain Avenue

Office, Orlando Urbana, IL b1HO Montclair, NJ <042NPRDC Liaison OfficerNTSC Orlando, FL 32813

Dr. William Epstein Dr. Carl H. FrederiksenUniversity of Wisconsin McGill University Dr. Joseph Goguen

W. J. Brogden Psychology Bid 3700 McTavish Street Computer Science Laboratory1202 W. Johnson Street Montreal, Quebec H3A 1Y2 SRI International

Madison, WI 53706 CANADA 333 Ravenswood AvenueMenlo Park, CA 94025

ERIC Facility-Acquisitions Dr. John R. Frederiksen4833 Rugby Avenue Bolt Beranek & Newman Dr. Sherrie GottBethesda, MD 20014 50 Moulton Street AFHRL/MODJ

Cambridge, MA 02138 Brooks AFB, TX 78235

Dr. K. Anders EricssonUniversity of Colorado Dr. Norman Frederiksen Jordan Grafman, Ph.D.Department of Psychology Educational Testing Service 2021 Lyttonsville Road

Boulder, CO 80309 Princeton, NJ 08541 Silver Spring, YD 20910

Dr. Edward Esty Dr. Alfred R. Fregly Dr. Richard H. Granger

Department of Education, OEP AFOSR/NL Department of Computer Science

Room 717D Bolling AFB, DC 20332 University of California, Irvine

1200 19th St., NW Irvine, CA 92717Washington, DC 20208 Dr. Bob Frey

Commandant (G-P-I/2) Dr. Wayne GrayDr. Beatrice J. Farr USCG HQ Army Research InstituteArmy Research Institute Washington, DC 20593 5001 Eisenhower Avenue5001 Eisenhower Avenue Alexandria, VA 22333Alexandria, VA 22333 Dr. Alinda Friedman

Department of Psychology Dr. James G. GreenoDr. Marshall J. Farr University of Alberta University of California2520 North Vernon Street Edmonton, Alberta Berkeley, CA 94720Arlington, VA 22207 CANADA T6G 2E9

H. William GreenupDr. Pat Federico Julie A. Gadsden Education Advisor (E031)Code 511 Information Technology Education Center, MCDECNPRDC Applications Division Quantico, VA 22134San Diego, CA 92152-6800 Admiralty Research Establish

Portsdown, Portsmouth P06 4A Dipl. Pad. Michael W. HabonDr. Jerome A. Feldman UNITED KINGDOM Universitat DusseldorfUniversity of Rochester ErziehungswissenschaftlichesComputer Science Department Dr. R. Edward Geiselman Universitatsstr. 1Rochester, NY 14627 Department of Psychology D-4000 Dusseldorf I

University of California WEST GERMANYDr. Paul Feltovich Los Angeles, CA 90024

Southern Illinois University Prof. Edward HaertelSchool of Medicine Dr. Michael Genesereth School of EducationMedical Education Department Stanford University Stanford UniversityP.O. Box 3926 Computer Science Department Stanford, CA 94305Springfield, IL 62708 Stanford, CA 94305

Dr. edr GennerDr. Henry M. Halff

Mr. Wallace Feurzeig Dr. Dedre Gentner Halff Resources, Inc.Educational Technology University of Illinois 4918 33rd Road, North8olt Beranek & Newman Department of Psychology Arlington, VA 2220710 Moulton St. 603 E. Daniel St.Cambridge, MA 02238 Champaign, IL 61820

Dr. Ronald K. HambletonDr. Craig I. Fields Dr. Robert Glaser Prof. ot Education & PsycholARPA Learning Research University of Massachusetts

1400 Wilson Blvd. & Development Center at AmherstArlington, VA 22209 University of Pittsburgh Hills House

3939 O'Hara Street Amherst, MA OI003Dr. Gerhard Fischer Pittsburgh, PA 15260University of Colorado Dr. Cheryl HamelDepartment of Computer Science Dr. Arthur M. lenberg NTSCBoulder, CO 80309 University or wt. consin Orlando, FL 32813

W. J. Brogden Psychology Bldg.

J. D. Flptcher 1202 W. Johnson Street Dr. Bruce W. Hamill9931 (' t 'a Street Madison, WI 53706 Johns Hopkins UniversityVienna , 22180 Applied Physics Laboratory

Dr. Marvin D. Glock Johns Hopkins RoadDr. Linda Flower 13 Stone Hall Laurel, MD 20 07Carnegie-Mellon University Cornell University

Department of English Ithaca, NY 14853 Dr. Ray HannapelPittsburgh, PA 15213 Scientific and Engineering

Dr. Gene L. Gloye Personnel and EducationDr. Kenneth D. Forbus Office of Naval Research National Science FoundationUniversit/ of Illinois Detachment Washington, DC 20550Department of Computer Science 1030 E. Green Sreet1304 West Springfield Avenue Pasadena, CA 91106-2485Urbana, IL 61801 Stevan Harnad

Dr. Sam Glucksberg Editor, The Behavioral and

Dr. Barbara A. Fox Department of Psychology Brain Sciences

University of Colorado Princeton University 20 Nassau Street, Suite 240Department of Linguistics Princeton, NJ 08540 Princeton, NJ 08540

Boulder, CO 80309

Dr. James Howard Dr. Richard JohnsonMr. William Hartung Dept. of Psychology Boise State UniversityPEAM Product Manager Human Performance Laboratory Simplot/MicronArmy Research Institute Catholic University of Technology Center5001 Eisenhower Avenue America Boise, ID 83725Alexandria, VA 22333 Washington, DC 20064

CDR Tom JonesDr. Wayne Harvey Dr. Steven Hunka ONR Code 125SRI International Department of Education 800 N. Quincy Street333 Ravenswood Ave. University of Alberta Arlington, VA 22217-5000Room B-$324 Edmonton, AlbertaMenlo Park, CA 94025 CANADA Dr. Douglas A. Jones

Dr. Reid Hastie P.O. Box 6640Northwestern University Dr. Earl Hunt Lawrenceville

Department of Psychology Department of Psychology NJ 08648Evanston, IL 60201 University of Washington

Seattle, WA 98105 Col. Dominique Jouslin de NoProf. John R. Hayes Etat-Major de l'Armee de TerCarnegie-Mellon University Dr. Ed Hutchins Centre de Relations Humaines

e Department of Psychology Intelligent Systems Group 3 Avenue Octave Greard. Schenley Park Institute for 75007 Paris

Pittsburgh, PA 15213 Cognitive Science (C-015) FRANCEUCSD

Dr. Barbara Hayes-Roth La Jolla, CA 92093 Dr. Marcel JustDepartment of Computer Science Carnegie-Mellon UniversityStanford University Dr. Dillon Inouye Department of PsychologyStanford, CA 95305 WICAT Education Institute Schenley ParkDr. Frederick Hayes-Roth UT 84057 Pittsburgh, PA 15213

Teknowledge Dr. Alice Isen525 University Ave. Del irtment of Psychology Dr. Milton S. KatzPalo Alto, CA 94301 UnLv. frsity of Maryland Army Research Institute

Catonsville, MD 21228 5001 Eisenhower AvenueDr. Joan I. Heller Alexandria, VA 22333505 Haddon Road Dr. Zachary Jacobson

' Oakland, CA 94606 Bureau of Management Consulting Dr. Scott Kelso' 365 Laurier Avenue West Haskins Laboratories,Dr. James Hendler Ottawa, Ontario KlA 0S5 270 Crown Street

New Haven, CT06510Dept. of Computer Science CANADA New Haven, CT 06510

University of MarylandCollege Park, MD 20742 Dr. Robert Jannarone Dr. Dennis Kibler

Department of Psychology University of CaliforniaUniversity of South Carolina Department of Information

Dr. Jim Hollan Columbia, SC 29208 and Computer ScienceIntelligent Systems Group Irvine, CA 92717Institute for Dr. Claude Janvier

Cognitive Science (C-015) Directeur, CIRADE Dr. David KierasUCSD Universite' du Quebec a Montreal University of MichiganLa Jolla, CA 92093 Montreal, Quebec H3C 3P8 Technical Communication

CANADA College of EngineeringDr. Aelissa Holland 1223 E. Engineering BuildingArmy Research Institute for COL Dennis W. Jarvi Ann Arbor, MI 48109

Behavioral and Social Sci CommanderP 5001 Eisenhower Avenue AFHRL Dr. Peter Kincaid

Alexandria, VA 22333 Brooks AFB, TX 78235-5601 Training Analysis& Evaluation Group

Dr. Robert W. Holt Dr. Robin Jeffries Department of the NavyDepartment of Psychology Hewlett-Packard Laboratories Orlando, FL 32813George Mason University P.O. Box 104904400 University Drive Palo Alto, CA 94303-0971 Dr. Walter KintschFairfax, VA 22030 Department of Psychology

Dr. Robert Jernigan University of ColoradoDr. Keith Holyoak Decision Resource Systems Campus Box 345University of Michigan 5595 vantage Point Road Boulder, CO 80302Human Performance Center Columbia, MD 21044330 Packard Road Dr. Paula KirkAnn Arbor, MI 48109 Oakridge Associated Universities

Margaret Jerome University Programs Division% Prof. Lutz F. Hornke c/o Dr. Peter Chandler P.O. Box 117

Institut fur Psychologie 83, The Drive Oakridge, TN 37831-0117RWTH Aachen Hove

Jaegerstrasse 17/19 Sussex Dr. David KlahrD-5100 Aachen UNITED KINGDOM Carnegie-Mellon UniversityWEST GERMANY Department of Psychology

Dr. Joseph E. Johnson Schenley Park

Mr. Dick Hoshaw Assistant Dean for schenhe PAOP-135 Graduate Studies Pittsburgh, PA 15213Arlington Annex College of Science and MatheRoom 2834 University of South CarolinaWashington, DC 20350 Columbia, SC 29208

:. .. ,. - .. ., ,. -. .. .. . . .. .., , . . ..

Dr. Mazie Knerr Dr. Jim LevinProgram Manager University of California Dr. Richard E. MayerTraining Research Division Laboratory for Comparative Department of PsychologyHumRRO Human Cognition University of California1100 S. Washington D003A Santa Barbara, CA 93106

Alexandria, VA 22314 La Jolla, CA 92093

Dr, Janet L. Kolodner Dr. John LevineGeorgia Institute of Technology Learning R&D Center Dr. James McBride

School of Information University of Pittsburgh Psychological Corporation& Computer Science Pittsburgh, PA 15260 c/o Harcourt, Brace,

Atlanta, GA 30332 Javanovich Inc.1250 West 6th Street

Dr. Stephen Kosslyn Dr. Michael Levine San Diego, CA 92101Harvard University Educational Psychology1236 William James Hall 210 Education Bldg. Dr. David J. McGuinness33 Kirkland St. University of Illinois Gallaudet CollegeCambridge, MA 02138 Champaign, IL 61801 800 Florida Avenue, N.E.

Washington, DC 20002Dr. Kenneth Kotovsky Dr. Charles LewisDepartment of Psychology Faculteit Sociale Wetenschappen Dr. Kathleen McKeownCommunity College of Rijksuniversiteit Groningen Columbia University

Allegheny County Oude Boteringestraat 23 Department of Computer Science800 Allegheny Avenue 9712GC Groningen New York, NY 10027Pittsburgh, PA 15233 The NETHERLANDS

Dr. Joe McLachlanDr. David H. Krantz Dr. Clayton Lewis Navy Personnel R&D Center2 Washington Square Village University of Colorado San Diego, CA 92152-6800Apt. 0 15J Department of Computer ScienceNew York, NY 10012 Campus Box 430 Dr. James McMichael

Boulder, CO 80309 Assistant for MPT Research,

Dr. Benjamin Kuipers Development, and StudiesUniversity of Texas at Austi LibraryDepartment of Computer Scien Naval War College OP 01D7T.S. Painter Hall 3.28 Newport, RI 02940 Washington, DC 20370Austin, Texas 78712 Dr. Barbara Means

Library Human ResourcesDr. David R. Lambert Naval Training Systems Cente Research OrganizationNaval Ocean Systems Center Orlando, FL 32813 Re00 South WashingtonCode 441T Alexandria, VA 22314271 Catalina Boulevard Dr. Charlotte LindeSan Diego, CA 92152-6800 SRI International Dr. Douglas L. Medin

333 Ravenswood Avenue Department of Psychology

Dr. Pat Langley Menlo Park, CA 94025 University of IllinoisUniversity of California 603 E. Daniel StreetDepartment of Information Dr. Marcia C. Linn Champaign, IL 61820

and Computer Science Lawrence Hall of ScienceIrvine, CA 92717 University of California Dr. Arthur Melmed

M. Diane Langston Berkeley, CA 94720 U. S. Department of Education

Communications Design Centei Dr. Robert Linn 724gBrown

Carnegie-Mellon University College of EducationSchenley Park University of Illinois Dr. Jose MestrePittsburgh, PA 15213 Urbana, IL 61801 Department of Physics

Hasbrouck LaboratoryDr. Jill Larkin Dr. Don Lyon University of MassachusettsCarnegie-Mellon University P. 0. Box 44 Amherst, MA 01003Department of Psychology Higley, AZ 85236Pittsburgh, PA 15213

Dr. Jane Malin Dr. Al MeyrowitzDr. Robert Lawler Mail Code SR 111 Office of Naval ResearchInformation Sciences, FRL NASA Johnson Space Center Code 1133GTE Laboratories, Inc. Houston, TX 77058 800 N. Quincy40 Sylvan Road Arlington, VA 22217-5000

Waltham, MA 02254 Dr. William L. MaloyDr. Ryszard S. Michalski

Dr. Paul E. Lehner Chief of Naval Education University of IllinoisPAR Technology Corp. and Training Department of Computer Scien7926 Jones Branch Drive Naval Air Station 1304 West Springfield AvenuESuite 170 Pensacola, FL 32508 Urbana, IL 61801McLean, VA 22102

Dr. Sandra P. Marshall Prof. D. MichieDr. Alan M. Lesgold Dept. of Psychology The Turing InstituteLearning R&D Center San Diego State University T6 North Hanover StreetUniversity of Pittsburgh San Diego, CA 92182 Glasgow GH 2AD, Scotland

Pittsburgh, PA 15260 UNITED KINGDOMDr. Manton M. MatthewsDr. Alan Leshner Department of Computer Scien Dr. George A. Miller

Deputy Division Director University of South Carolina Dere of PclogyBehavioral and Neural Sciences Columbia, SC 29208 Department of Psychology

National Science Foundation Green Hall

1800 G Street Princeton University

Washington, DC 20550 Princeton, NJ 08540

U' ~ % ~ '~' ~ '.~S *'~~*" %. **

Leadership Management Educat Office of Naval Research,Dr. James R. Miller and Training Project Offi Code 1133MCC Naval Medical command 800 N. Quincy Street9430 Research Blvd. Code 05C Arlington, VA 22217-5000Echelon Building 01, Suite 2 Washington, DC 20372Austin, TX 78759 Office of Naval Research,

Technical Director, Code 1142Dr. Mark Miller Navy Health Research Ctr. 800 N. Quincy St.Computer*Thought Corporation P.O. Box 85122 Arlington, VA 22217-5000

1721 West Plano Parkway San Diego, CA 92138

Plano, TX 75075 Office of Naval Research,

Dr. T. Niblett Code 1142EPDr. Andrew R. Molnar The Turing Institute 800 N. Quincy StreetScientific and Engineering 36 North Hanover Street Arlington, VA 22217-5000

Personnel and Education Glasgow Gl 2AD, ScotlandNational Science Foundation UNITED KINGDOM office of Naval Research,

Washington, DC 20550 Code 1142PT

Dr. William Montague D Richard E. Nisbett 800 N. Quincy StreetDC Codem 1n Dr. RArlington, VA 22217-5000

NPRDC Code 13 University of Michigan (6 Copies)San Diego, CA 92152-6800 Institute for Social Researc 6

Room 5261 Director, Technology Programs,Dr. Tom Moran Ann Arbor, MI 48109 office of Naval Research

Xerox PARC Code 12

3333 Coyote Hill Road Dr. Mary Jo Nissen 800 North Quincy Street

Palo Alto, CA 94304 University of Minnesota Arlington, VA 22217-5000

Dr. Melvyn Moy N218 Elliott Hall 5"Minneapolis, MN 55455

Navy Personnel R & D Center Special Assistant for MarineSan Diego, CA 92152-6800 Dr. Donald A. Norman Corps Matters,

Institute for Cognitive Scie Code MCUniversity of California ONR Code 00MC,

Dr. Allen Munro La Joita o CAlifonia800 N. Quincy St.

Behavioral Technology La Jolla, CA 92093 Arlington, VA 22217-5000

Laboratories - USC DiLector, Training Laboratol Assistant for Long Range

1845 S. Elena Ave., 4th Floor NPRDC (Code 05) A eirent n

Redondo Beach, CA 90277 Saii Diego, CA 92152-6800 Requirements,CNO Executive Panel ,'

DDirector, irector, Manpower and Perso OP 00K

Decision Support Laboratory, 2000 North Beauregard Street

Systems Division, NMPC NPRDC (Code 06) Alexandria, VA 22311

Naval Military Personnel Command San Diego, CA 92152-6800

N-164 Assistant for MPT Research,

Washington, DC 20370 Director, Human Factors Development and Studies

& Organizational Systems OP 0187

Director, NPRDC (Code 07) Washington, DC 20370

N-Distribution Department, PC San Diego, CA 92152-6800Wangtn D 030Dr. Judith Orasanu

Washington, DC 20370 Fleet Support Office, Army Research Institute

NPRDC (Code 301) 5001 Eisenhower Avenue

Director, San Diego, CA 92152-6800 Alexandria, VA 22333

Overseas Duty SupportProgram, NMPC Dr. Jesse Orlansky

N-62 Library, Institute for Defense Analys

Washington, DC 20370 Code P2OL 1801 N. Beauregard St.

San Diego, CA 92152-6800 Alexandria, VA 22311

Head, HRM Operations Branch,

NMPC Commanding Officer, Prof. Seymour Papert

N-62F Naval Research Laboratory 20C-109

Washington, DC 20370 CodeMassachusetts InstituteCode2627of Technology

Assistant for Evaluation, Washington, DC 20390 Cambridge, MA 02139

Analysis, and MIS, NMPC Dr. Harold F. O'Neil, Jr.

N- 6C CRR .PreW~hiNgton, DCC20370 School of Education - WPH 801 COP R. T. ParletteWishington, DC 20370 Department of Educational Chief of Naval Operations

Psycholooy & Technology OP-li2GSmec. Asst. for Research, Experi- University of Southern California Washington, DC 203702000

mental & Academic Programs, Los Angeles, CA 90089-0031NTTC (Code 016) Dr. James Paulson

NS Memphis (75) Dr. Stellan Ohlsson Department of Psychology

Millington, TN 38054 Learning R & D Center Portland State University

University of Pittsburgh P.O. Box 751

Rrector, 3939 O'Hara Street Portland, OR 97207

Research & Analysis Div., Pittsburgh, PA 15213NAVCRUITCOM Code 22 Dr. Douglas Pearse

)15 Wilson Blvd. Director, Research Programs, DCIEM 0A'linqton, VA 22203 Office of Naval Research Box 2000

Sai OiegooCA9238 w, Ontario800 North Quincy StreetTchnical Director Arlington, VA 22217-5000 CANADA

MNy" Health Research CenterP.). Box 85122Sat Diego, CA 92138

Dr. Virginia E. Pendergrass Dr. Mary C. Potter Dr. Donald RubinCode 711 Department of Psychology Statistics DepartmentNaval Training Systems Center MIT (E-10-032) Science Center, Room 608Orlando, FL 32813-7100 Cambridge, MA 02139 1 Oxford Street

Harvard UniversityDr. Joseph Psotka Cambridge, MA 02138

Dr. Robert Penn ATTN: PERI-ICNPRDC Army Research Institute Dr. David RumelhartSan Diego, CA 92152-6800 5001 Eisenhower Ave. Center for Human

Alexandria, VA 22333 Information ProcessingDr. Nancy Pennington Univ. of CaliforniaUniversity of Chicago Dr. Roy Rada La Jolla, CA 92093Graduate School of Business National Library of Medicine1101 E. 58th St. Bethesda, MD 20894 MS. Riitta RuctsalainenChicago, IL 60637 General Headquarters

Military Assistant for Training and Dr. Lynne Reder Training SectionDepartment of Psychology Military Psychology office

personnel Technology, Carnegie-Mellon University PL 919OUSD (R & E) Schenley Park SF-00101 Helsinki 10, FINLA;

Room 3D129, The Pentagon Pittsburgh, PA 15213Washington, DC 20301-3080 Dr. Michael J. Samet

Dr. Ray Perez Brian Reiser Perceptronics. IncARI (PERI-II) Department of Psychology Woodland Hills, CA 9.364

5001 Eisenhower Avenue Princeton UniversityAlexandria, VA 2233 Princeton, NJ 08544 Dr. Roger Schank

Yale UniversityDr. David N. perkins Computer Science DepartmentEducational Technology Center Dr. James A. Reggia

University of Maryland P.O. Box 2158337 Gutman Library School of Medicine New Haven, CT 06520Cambridge, MA 02138 Department of Neurology

22 South Greene Street Dr. W. L. ScherlisLCDR Frank C. Petho, MiSC, USN Baltimore, MD 21201 Dept. of Computer Science

Carnegie-Mellon U'niversityCNATRA Code N36, Bldg. 1 CDR Karen Reider Schenley ParkNAS Pittsburgh, PA 15213Corpus Clyristi, TX 78419 Naval School of Health Scienc":s

National Naval Medical CenterMr.BrteSceilbh

Administrative Sciences Department, Bldg. 141 Mrs. Birgitte Schneidelbach

Naval postgraduate ScIool Washington, DC 20814 Forsvarets Center for LedersMontery, CA93940Christianshavns Voldgade9

Monterey CA 93940Dr. Fred Reif 1424 Kobenhavn K

Department of Computer Science, Physics Department* Naval postgraduate School University of California Dr. Walter Schneicler

Monterey, CA 93940 Berkeley, CA 94720 Learning R&D CenterSeniversity of Pittsburgh

Department of Operations Research, Dr. Lauren Resnick 393ivera StreetNaval Postgraduate School Learning R & D Center 3939 O'Hara Street

Monterey, CA 93940 University of Pittsburgh Pittsburlh, PA 152603939 O'Hara Street

Dr. Tjeerd P~omp Pittsburgh, PA 15213 Dr. Alan H. SchoenfeldUniversity of California

Twente University of Technol Dr. Mary S. Riley Department of EducationDepartment of Education Program in Cognitive Science Berkeley, CA 420P.O. Box 217 Center for Human Information7500 AE ENSCHEDE Processing Dr. Janet Schofield

SUniversity of California Learning R&D CenterLa Jolla, CA 92093 University of Pittsbur~h

;e THE NETHERLANDSWillam RzzoPittsburgh, PA 152 60,'¢ William Rizzo

Dr. Martha Polson Code 712 Dr. Marc SebrechtsDepartment of Psychology Naval Training Systems Center Department of Psycholoqy

*'' Campus Box 346 Orlando, FL 32813 wesleyan UniversityUniversity of Colorado Middletown, CT 064>Boulder, CO 80309 Dr. Linda G. Roberts

Science, Education, and Dr. Judith Segal, Dr. Peter Poison Transportation Program OERI* University of Colorado Office of Technology Assessment 555 New Jersey Ave., NW

Department of Psychology Congress of the United States Washington, DC 20208. Boulder, CO 80309 Washington, DC 20510

Dr. Robert J. SeidelDr. Steven E. Poltrock Dr. Andrew M. Rose US Army Research InstitteMCC American Institutes 5001 Eisenhower Ave.9430 Research Blvd. for Research Alexandria, VA 22111Echelon Bldg #1 1055 Thomas Jefferson St., NWAustin, TX 78759-6509 Washington, DC 20007 Dr. Ramsay W. Selden

Assessment CenterDr. Harry E. Pople Dr. William B. Rouse CCSSO

" University of Pittsburgh Search Technology, Inc. Suite 3'9

Decision Systems Laboratory 25-b Technology Park/Atlanta 400 N. Capitol, ;W1360 Scaife Hall Norcross, GA 30092 Washington, DC 22IPittsburgh, PA 15261

Dr. W. Steve Seliman Dr. Linda B. SmithOASD(MRA&L) Department of Psychology Navy Personnel R&D Center, ASD(TeP& ao Indiana University San Diego, CA 92152-68002B269 The Pentagon Bloomington, IN 47405Washington, DC 20301 Bomntf~I 70

Dr. Alfred F. Smode Dr. John Tangney

Dr. Sylvia A. S. Shafto Senior Scientist AFOSR/NL

OERI Codie 0'A Boiling AFB, DC 20332

555 New Jersey Ave., NW Naval Training Systems Center Dr. Kikumi TatsuokaWashington, DC 20208 Orlando, FL 32813 CEFL

Dr. T. B. Sheridan Dr. Richard E. Snow 252 Engineering Research

Dept. of Mechanical Engineering Department of Psychology Laboratory

MIT Stanford University Urbana, IL 61801

Cambridge, MA 02139 Stanford, CA 94306 Dr. Martin M. TaylorDCI.M

Dr. Ben Shneiderman Dr. Elliot Soloway Box 2000

Dept. of Computer Science Yale University Downsvew, Ontario

University of Maryland Computer Science Department Dv

College Park, MD 20742 P.O. Box 2158 CANADA

Dr. Ted Shortliffe Haven, CT 06520 Dr. Perry W. ThorndykeDr. ed Sortlffe MC CorporationComputer Science Department Dr. Richard Sorensen Coprain

* Stanford University ~a: esne & etrCentral Engineering LabsStantordCA 9430t Navy Personnel R&D Center 1185 Coleman Avenue, Box 580Stanford, CA 94305 San Diego, CA 92152-6800 Santa Clara, CA 95052

Dr. Lee i.lfman Dr. Kathryn T. Spoehr Major Jack ThorpeStanford University Brown University DARPA1040 Cathcart Way Department of Psychology 1400 Wilson Blvd.Stanford, CA 94305 Providence, RI 02912 10lon vd.

Arlington, VA 22209

Dr. Randall Shumaker James J. Staszewski Dr. Douglas TowneNaval Research Laboratory Research Associate Behavioral Technology Labs

Jode 10 Carnegie-Melion University 1845 5. Elena Ave.

4555 Overl , k Avenue, S.W. Department of Psychology Redondo Beach, CA 90277

Washington, DC 20375-5000 Schenlev' ParkPittsbu:Th, PA 15213 Dr. Amos Tversky

Dr. Robert S. Siegler Stanford UniversityCarnegie-Mellon University Dr. Yirian Stearns Dept. of PsychologyDepartment of Psychology SRI International Stanford, CA 94305

Schenley Pirk 333 Ravenswood Ave.Pittsburih, PA 15213 Room B-$324 Dr. James Tweeddale

Menlo Park, CA 94025 Techrtcal Directorpr. Herbert A. PSmon Navy Personnel R&D Center'partent of Psychology Dr. Robert Sternberg San Diego, CA 92152-6800-arnegie-Mellon University Department of Psychology

Schenley Park Yale University Dr. Paul Twohigpittsburgh, PA 15213 Box 11A, Yale Station Army Research Institute

New Haven, CT 06520 5001 Eisenhower Avenue

:.TCC. Rubert Simpson Alexandria, VA 22333

tefense Advanced Research Dr. Albert StevensProects Administration Bolt Beranek & Newman, Inc. Headquarters, U. S. Marine Corps

14:&,i _3ison Blvd. 10 Moulton St. Code MPI-20

A int,.] , '.A 22209 Cambridge, MA 02238 Washington, DC 20380

'r ' a M SL"'It"s Dr. Paul J. Sticha.,:.rvytirl rehnclogy Senior Staff Scientist Dr. Kurt Van Lehn

I _te ms AL ,a Training Research Division Department of Psychology

<HI HumRRO Carnegie-Mellon University

1 Eisenh'w-r Avenue 1100 5. Washington Schenley Park* ,-[ex~ninic VA 22333 Alexandria, A 22314 Pittsburgh, PA 15213

" H Wal.ice Sinaiko Dr. Thomas Sticht Dr. Beth Warren''iAnpower Psearch I i:y Personnel R&D Center Bo.t Peranek & Newman, Inc.

" . , A~visory Services ,i<n Diego, CA 92152-6800 Camb ,'i .!A 02138i tnhson ian InstitutionCab h, -018

0 1N rth Pitt Street Dr. David StoneAlexanJra VA 22314 KAJ Software, Inc. Dr. David -. Weiss

'420 East Shea Blvd. NC60 Elliott HallDr. Derek Sleeman Suite 161 University of MinnesotaStarfor] Tniversity Phoenix, AZ 85028 75 E. River RoadSchool of Education Minneapolis, MN 55455

tnirford. -A 4305 'Jr Michael S3uan, PD 303•.v l rraininq Systems Cente Roger Weissinger-Baylon

Dr. :harles F. Smith "-.de N51, Comptroller Department of Administrative

;orth Carolina State Univers .rlando FL 32813 Sciences

:epartment of Statistics Naval Postgraduate SchoolRaleigh, NC 2-695-8703 Dr. Nariharan Swaminathan Monterey, CA 93940

Dr. Edward E. Smith LLboratory of Psychometric a Dr. Donald WeitzmanDr. eard & Nman, Evaluation Research MITRE

S Bolt Beranek & Newman Inc. School of Elucation 1820 Dolley Madison Blvd.a50 Moulton Street 'niversity of Massachusetts MacLean, VA 22102

*" Cambridge, MA 02138 Amherst, MA 01003

[ ,%"% = S ' i". .%.]. i" "]' "" ,]" "" l lil 1 " "" i % " I ." " 1[ , % %.-- .•-- . r:- - - [i"[ % • _% L%=%--i '"[ . . ." i, ]

Dr. Keith T. Wescourt Dr. Joseph L. YoungFMC Corporation Memory & CognitiveCentral Engineering Labs Processes1185 Coleman Ave., Box 580 National Science FoundationSanta Clara, CA 95052 W&.nington, DC 20550

Dr. Douglas Wetzel Dr. Steven ZornetzerCode 12 Office of Naval ResearchNavy Personnel R&D Center Code 1140San Diego, CA 92152-6800 800 N. Quincy St.

Arlington, VA 22217-5000Dr. Barbara WhiteBolt Beranek & Newman, Inc. Dr. Michael J. Zyda10 Moulton Street Naval Postgraduate SchoolCambridge, MA 02238 Code 52CK

Monterey, CA 93943-5100LCDR Cory de~root WhiteheadChief ot Naval OperationsOP-112G1Washington, DC] 20370-2000

Dr. Michael WilliamsIntelliCorp1975 El Camino Real WestMountain View, CA 94040-2216

Dr. Hilda WingArmy Research Institute5001 Eisenhower Ave.Alexandria, VA 22333

A. E. WinterbauerResearch AssociateElectronics DivisionDenver Research InstituteUniversity Park

* Denver, CO 80208-0454

Dr. Ro-bert A. WisherU.S. Army Institute for the

Behavioral and Social Sciences%001 Eisenhower AvenueAlexandria, VA 22333

EDr. Martin F. WiskoffNavy Pe-rsonnel R & D Centersan D[,1o, CA 92152-6800Dr. M-erlin C. Wittrock

;,ra]uate School of EducationTr-LALos Ariieles, CA )()24

Mr. John H. WolteNavy Personnel R&D CenterSan Diego, CA 92152-6800

Dr. Wallace Wulfeck, IIINavy Personnel R&D CenterSan DLiego, CA 92152-t800

Dr. Joe YasatikeAFHRL, LRTLowry AFB, CO 80230

Dr. Masoud YazdaniDept. of Computer ScienceUniversity of ExeterExeter EX4 4QLDevon, ENGLAND

Major Frank Yohannan, USMCHeadquarters, Marine Corps(Code MPI--O)Washington, DC 20380

Mr. earl YorkSystem Development Foundation181 Lytton AvenueSuite 210Palo Alto, CA '44301

1'0J '

.- .

J. %.'

% J-

:'."."5

_ . _':. .+ + _-

-

O

: ... -',,::- -"-' -"."*.'+"''-."+'"'-''- ". +' "+"_-

'2'.''",';' " _'

r,',,'". , ,'. .'.,". .%--- .'.'.'...+.'.--.---AM-,'''.-5 '+.+-.- ''" "'-'".-'- ... -. ,- ; .'",...-.,,,

,,,- ,..' .. .-, ...-. .

. • . - .. . -+.,,,- .- • .. . . .. - . • •, -..- .-.. + . .,,..

.'. . '.,y,% ",,.

--'. + - +o+ + + __++ + +.. +."4 ' m ," " +- + .+ •+-•- " -"-- .... .+,_% % , +% +S•m *I C + •- -r -" I -