social studies of sciencemechanism.ucsd.edu/teaching/f12/cs200/readings/alac-2008... · 2012. 8....

http://sss.sagepub.com/Social Studies of Science

http://sss.sagepub.com/content/38/4/483The online version of this article can be found at:

DOI: 10.1177/0306312708089715

2008 38: 483Social Studies of ScienceMorana AlacLaboratory

Working with Brain Scans : Digital Images and Gestural Interaction in fMRI

Published by:

http://www.sagepublications.com

can be found at:Social Studies of ScienceAdditional services and information for

http://sss.sagepub.com/cgi/alertsEmail Alerts:

http://sss.sagepub.com/subscriptionsSubscriptions:

http://www.sagepub.com/journalsReprints.navReprints:

http://www.sagepub.com/journalsPermissions.navPermissions:

http://sss.sagepub.com/content/38/4/483.refs.htmlCitations:

What is This?

- Aug 8, 2008Version of Record >>

at UNIV CALIFORNIA SAN DIEGO on August 21, 2012sss.sagepub.comDownloaded from

http://sss.sagepub.com/http://sss.sagepub.com/content/38/4/483http://www.sagepublications.comhttp://sss.sagepub.com/cgi/alertshttp://sss.sagepub.com/subscriptionshttp://www.sagepub.com/journalsReprints.navhttp://www.sagepub.com/journalsPermissions.navhttp://sss.sagepub.com/content/38/4/483.refs.htmlhttp://sss.sagepub.com/content/38/4/483.full.pdfhttp://online.sagepub.com/site/sphelp/vorhelp.xhtmlhttp://sss.sagepub.com/

ABSTRACT A significant part of functional magnetic resonance imaging (fMRI)practice in neuroscience is spent in front of computer screens. To investigate the brainneuroscientists work with digital images. This paper recovers practical dealings withbrain scans in fMRI laboratories to focus on the achievement of seeing in the digitalrealm. While looking at brain images, neuroscientists gesture and manipulate digitaldisplays to manage and make sense of their experimental data. Their gesturalengagements are seen as dynamical phenomenal objects enacted at the junctionbetween the digital world of technology and the world of embodied action.

Keywords digital images, fMRI, gesture, laboratory studies, multimodal semioticinteraction

Working with Brain Scans:

Digital Images and Gestural Interaction infMRI Laboratory

Morana Alac

Magnetic resonance imaging (MRI) technology is a key modern digitalimaging system used for scientific and medical purposes. The goal of MRIis to provide detailed images of the anatomical structure of internal bodyparts, such as the brain. This technique uses radiofrequency, magneticfields, and computers to create images based on the varying local environ-ments of water molecules in the body. During an MRI scanning session,hydrogen protons in brain tissues are induced to emit a signal that isdetected by the computer, where the signals, represented as numericaldata, are converted into visual representations of the brain. A new dimen-sion in the acquisition of physiological and biochemical information withMRI is mapping human brain function, or functional MRI (fMRI). fMRIdetects the local changes in magnetic field properties occurring in the brainas a result of changes in blood oxygenation. The measures of the changeare translated into visual representations of the activity in the specific brainareas activated during the scanning session.

These ‘visual representations of what is not visual’ are the concern ofthe present paper. But rather than discussing their numeric versus visualcharacter, the focus is on the specific ways in which such ‘pictures of num-bers’ (Tufte, 1983) feature in work and collaboration. By recalling Michael

Social Studies of Science 38/4 (August 2008) 483–508© SSS and SAGE Publications (Los Angeles, London, New Delhi and Singapore)ISSN 0306-3127 DOI: 10.1177/0306312708089715www.sagepublications.com


http://sss.sagepub.com/

Lynch’s (1991) discussion of optical and digital ‘topical contextures’, thispaper describes how brain scans, despite their visual power, become ‘visi-ble’ (Goodwin, 1994) for fMRI practitioners, not only through visual perception, but also through the involvement of hands.

Important recent projects in Science and Technology Studies havefocused on social and cultural aspects of brain mapping (for example,Beaulieu, 2002, 2004; Alac, 2004; Dumit, 2004; Joyce, 2005; Prasad, 2005a,b). This research has pointed out that a comprehensive understanding ofMRI and positron emission tomography (PET) images requires an accountof the environment of production and reception of such images. While suc-cessfully engaging with questions about objectivity, standardization and gen-eralization, the research remains somewhat remote from the details ofindividual actions in fMRI practice, and thus leaves questions of ‘digitality’(Lynch, 1991), materiality and embodiment unexplored. We do not learnmuch about the material status that ‘digital brains’ may conserve or acquireduring specific instances of laboratory work and interaction. To address thesequestions, while staying attentive to discursive constructions, the presentpaper examines the intersection between scientists and fMRI technology.

By focusing on the ordinary methods that scientists use in daily situa-tions of research, the paper aims to take forward work initiated by ‘labora-tory studies’ more than two decades ago (Latour & Woolgar, 1979;Knorr-Cetina, 1981; Lynch, 1985, 1993; Traweek, 1988). The goal is an‘examination of the methodical way in which observations are experiencedand organized so that sense can be made of them’ (Latour & Woolgar, 1979:37). Moreover, in addition to the study of talk and graphical representations(for example, Lynch & Woolgar, 1990), the present paper is concerned withgestural acts (for example, Goodwin 2000b). I look at how the gesture participates in the dynamic, embodied, and often non-representationalenactments that feature in the details of situated action (Suchman, 1987,2000) and characterize the work with digital images.

Where the Interaction is

This paper is based on an ethnographic study of two research laboratoriesthat investigate human cognitive processes through the employment offMRI. Initially, the focus of the study was on documenting activities at thefMRI center where scientists conduct their scanning sessions. The atten-tion was on the ways in which fMRI technology features in collaborationamong scientists from multiple laboratories. As the project progressed, myfocus gradually moved from the fMRI center to the individual laboratories– work environments where fMRI data are analyzed. The actions in the lab-oratories highlighted the intersection between scientists and digital screens.

During the study, I observed and recorded working sessions, carriedout semi-structured interviews, and gathered documents that ranged fromscientific reports to email correspondences and architectural plans. Toattend to the interface between the body and technology (Ihde, 2002), mylong-term participant observation was combined with an interest in specific

484 Social Studies of Science 38/4



occurrences of multimodal semiotic modalities, such as talk, gesture, gaze,and body orientation (for example, Goodwin, 2000a) recorded on digitalvideo, and subsequently discussed with the scientists. While conductingthe study, I was a doctoral student in the Department of Cognitive Scienceand hence a member of the research field. Because of this position, theeveryday activities of a doctoral student – attending talks, taking classes,presenting and discussing research results – were not easily discernablefrom my ethnographic work.

I shall examine two excerpts from the interactional and practical activ-ity in one of the laboratories. The excerpts are drawn from a corpus of 18hours of video recordings collected during approximately 2 years of field-work. Both instances feature interactions between two neuroscientistsinvolved in the identification of an fMRI image artifact.

My attention to ongoing, local practices brings the complex characterof ‘brain scans’ to the forefront. I argue that brain scans feature in every-day laboratory work as sites of interaction.

fMRI Laboratories

The laboratories in which my fieldwork took place – ‘Laboratory I’ and‘Laboratory II’ – study human visual perception, mainly through fMRItechniques. Their primary interest is to localize visual processes in specificbrain regions, and to determine ways in which visual stimuli are processedthere. Dissatisfaction with commercial ‘off-the-shelf’ data analysis tools ledeach laboratory to design its own software. Although the two softwaresuites are different, they both generate ‘inflated’ and ‘flattened’ corticalmaps, and thus allow for more precise identification and location of brainactivation. While the use of such software programs requires specific com-petencies, the members of the two laboratories emphasize the higher degreeof creativity and control over experimental data that such software allows.

During my ethnographic study the number of active members in bothlaboratories fluctuated from five to ten, with a PhD student (PH) fromLaboratory I collaborating with Laboratory II on a project regarding herdoctoral dissertation. As she initiated the collaboration, the two laborato-ries had a long-awaited opportunity to closely compare the two softwareprograms. The design of this kind of software and the related methodologicalimprovements are the way for laboratories to become obligatory points of pas-sage in the research field (Latour, 1988). The comparison between the twosoftware programs, while allowing for some competition between the labora-tories, strengthened the possibility of the laboratories’ impact on the field,largely dominated by the commercial software. To accomplish the compari-son, the PH and the postdoctoral student (PD), who introduced the PH tothe software designed in Laboratory II, acted as ‘linkages’ between the twolaboratories. While the contact between the laboratory members remainsusually confined to scientific conferences and other professional meetings(members may meet each other when collecting their experimental data atthe fMRI center or when serving on various university committees), the PH

Alac: Working With Brain Scans 485



and the PD – two young female researchers – crossed the laboratoryboundaries to encounter the members of the other laboratory in their workenvironment.

This paper focuses on two short videotaped excerpts from thoseencounters. Both excerpts come from an early stage of fMRI data analysisin Laboratory I where ‘functional’ images – low-resolution images that rep-resent cognitive processes – and ‘structural’ images – high-resolutionimages that reveal the anatomy of the brain – have to be aligned with eachother.1 This instance of collaborative work and interaction took placebetween the PD visiting from Laboratory II and the principal investigator(PI) of Laboratory I.2 In order to carry out the software comparison, thetwo practitioners used software from Laboratory I to process experimentaldata previously analyzed with software used in Laboratory II.

The processing of the same data set with the two software programs wasof interest to the practitioners inasmuch as it promised to highlight the dif-ferences and potential advantages of one software program over the other.The process was of interest to me inasmuch as it promised to recover theways in which the collaborative seeing of digital images may be achieved.

Seeing the Brain Image Artifact

During the moments of collaborative work reported here, the PD and PIare seated in front of a computer in Laboratory I. A small functional imageis displayed at the center of the screen (Figure 1, upper left-hand side). Theuse of mouse commands allows the PI to alternate the display of functionaland structural images (Figure 1, upper left- and right-hand sides). Whilerapidly switching between the images, the PI notices a ‘shearing’ artifact.The artifact is considered to be an intrusion that needs to be removed fromthe data. In brain-mappers’ jargon the image showing the characteristicdistortion is said to be ‘sheared’ (from what is called ‘shear strain’ in clas-sical mechanics), while the activities dedicated to its reduction are called‘shearing correction’.3 The elimination and purification of data from arti-facts is one of the central aspects of laboratory practice (see, for example,Latour & Woolgar, 1979; Lynch, 1985).

The artifact detected by the PI regards the functional scan. To picturethe distortion, two functional images in Figure 1 – upper and lower left-hand side – should be compared. The upper-left portion of the figure fea-tures a brain image with an artifact, while the representation in the lowerleft-hand side shows how a corrected image that represents another layer ofthe brain may look. As the figure illustrates, the cross sign in the upper-leftfield is not located on the edge of the brain representation, as is the case forthe rest of the images. The discrepancy between the images indicates thecharacteristic type of distortion. Importantly, the distortion in the singlefunctional image displayed on the computer screen indicates that the func-tional scan representing the entire volume of the brain is characterized by it.The practitioners believe that this type of distortion, rather than pertainingto the experimental subject’s brain or to her/his actions during the scanning




session, is relative to the workings of the scanning technology. To deal withthe artifact, practitioners direct a group of computational processes to alignthe functional scan to the structural scan of the same experimental subject(which is not considered to be distorted).

The defining mark of the fMRI culture is its interest in the anatomicalspecialization of brain regions for processing of different types of informa-tion. To create ‘brain maps’ fMRI researchers project measures of cogni-tive behavior on the spatial representations of the human brain. LaboratoryI and Laboratory II study the visual cortex mapped into complex regionswhose different parts are considered to be dedicated to processing of spe-cific visual information. Despite being one of the best-studied parts of thehuman brain, there are still controversies and disagreements about the exis-tence, general layout, and the extent of some of the areas of the visual cor-tex. Thus, the proper alignment of functional data with the correspondinganatomical scans is imperative.4

Importantly, the shearing artifact that the PI notices was not detectedwhen the data were analyzed in the other laboratory. Thus, the PI demon-strates to the PD how to identify an image that can be classified as ‘sheared’,and shows how its correction is achieved by using the software designed in hislaboratory.5 As the excerpts from the interaction suggest, the two aspects ofthe activity are intertwined: the correction itself functions as a part of the clas-sification process, and the classification – oriented towards the practicalactions – depends on such processes for its own articulation. An important


FIGURE 1Functional and structural images as they appear on the computer screen

FUNCTIONAL

SHEARED

CORRECTED

STRUCTURAL



issue, then, is to decipher what those practical actions entail when the ‘stuff’that the scientists are dealing with is digital. As Lynch (1991: 62) proposes ina study of digital image processing in astronomy, in this context I also will‘emphasize implications of digitality that do not turn on the correctness of thevisual psychology or epistemology developed in its terms … [and] will arguethat digitality is an embodied relation with its own forms of practical efficacy’.

Certainly, the practitioners’ actions concern the web of relations thatgo beyond the digital screen. The fact that the shearing artifact was noticedin the PI’s laboratory, while it had been overlooked in the PD’s laboratory,has to do with a chain of historical events and disciplinary-specific prac-tices. Those practices, heavily invested in enhanced ways of looking at thehuman body (for example, Cartwright, 1995), shape and are shaped by thedesign, development, and use of instruments and technology. For example,it is significant that Laboratory II’s ‘predecessor’ laboratory had conductedresearch in human psychology rather than physiology and that the softwaredeveloped there was originally designed for function localization ratherthan brain mapping. Software designed for function localization enablesthe user to compute parts of the process automatically, rather than allow-ing her/him to engage with some of the components of the data manipula-tion process. Frequently, the user looks only at what neuroscientists call‘regions of interest’ rather than inspecting images of the whole brain. It isto be expected that this type of software and the associated techniques forviewing and analyzing fMRI data can lead researchers to overlook certainkinds of image artifacts.

However, local practices with digital images are not simply ‘shaped by’and do not merely ‘reflect’ the dynamics of the encompassing socio-technicalnetwork. Rather, they are directly involved in the articulation of scientific evidence. The PI proudly reported that the described sessions of collaborativedata analysis led Laboratory II to modify its software so that the inadequaciesin the brain images can be ‘more easily spotted’. In other words, the momentsof local interaction between the two scientists produced consequences for theconfigurations of technology and migration of techniques from one laboratoryto the other, thus generating potential effects on the future appearance offMRI representations.

Techniques for Seeing

The first excerpt opens with the PI using mouse commands to manipulatethe computer screen so that serially organized images can be compared.When he notices that the data were not corrected for shearing inLaboratory II, he sets out to display the distortion for others to see. Whiledoing so he ratifies the rational character of the procedures developed andused in his laboratory. In addition to reporting the practitioners’ talk, thetranscript details aspects of the bodily conduct as well as the effects of thecomputer screen manipulation. The way in which the acts of seeing aremanaged through the temporal and spatial coordination of gesture, talk,and the manipulation of the digital screen is intriguing.




EXCERPT 1The practitioners’ utterances are transcribed following transcription con-ventions from Jefferson (2004):

= Equal signs indicate no interval between the end of a prior andstart of a next piece of talk.

(0.0) Numbers in brackets indicate elapsed time in tenths of seconds.(.) A dot in parentheses indicates a brief interval within or between

utterances.::: Colons indicate that the prior syllable is prolonged. The longer

the colon row, the longer the prolongation.– A dash indicates the sharp cut-off of the prior word or sound.(guess) Parentheses indicate that transcriber is not sure about the words

contained therein.(( )) Double parentheses contain transcriber’s descriptions..,? Punctuation markers are used to indicate ‘the usual’ intonation.

The action line, charter above the talk, follows transcription conventionfrom Schegloff (1984):

o indicates onset movement that ends up as gesture.a indicates acme of gesture, or point of maximum extension.h indicates previously noted occurrence held.t indicates thrust or peak of energy animating gesture.r indicates beginning or retraction of limb involved in gesture.hm indicates that the limb involved in gesture reaches ‘home position’ or

position from which it departed for gesture.p indicates point..... Dots indicate extension in time of previously marked action.

The transcript below the talk line indicates the temporal change ofthe digital display:

/ / / Oblique signs indicate the display of functional images.

/////// Gray square shapes indicate the display of structural images.

1 PI So (you) usually (0.1) at this point (0.5)/ / / / / / / / / / / / / / / / / //////// / / /// / // //

2 I usually do a little bit of shearing (0.1)// / / / / / /////////////////// / / / / / / / / / /////////

3 to help get the::(.)/ / / / ////// / / ////// /

((The following gesture is a movement of both hands placed as if holdinga round object. As the left hand moves up, the right hand moves down.Lexical affiliate is ‘sheared’.))

4 o…………...you can see/ / / / / / / / / /




5 …………......a...h……………………………….it’s sort of (0.5)((the PI looks toward the PD)/ / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / /


FIGURE 2 FIGURE 3

6 .……………PD (Interesting)

/ / / / / / / / / /7 .……………………………………………………….

PI ((the PI turns back toward the screen)) it’s sort of =/ / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / ///

FIGURE 4

8 ……...PD =Yeah

//////// /9 ……………………...........................

PI it’s actually sh– it’s actually sheared/ / / //////////// / / / / / / /////////// / / / / / /

((The following gesture is a movement of the left hand placed onto thescreen with the thumb and index finger an inch or so apart moving alongthe border of the brain image. Lexical affiliate is ‘going up this way’.))



11 ...r.hmPD Right ((the PD moves her upper body back, and then returns to

the initial position))/ / //// / /

The PI offsets up the instruction for seeing by saying: ‘So (you) usually (0.1) atthis point (0.5)=I usually do a little bit of shearing (0.1)’. By saying ‘a little bitof shearing’ he refers to the procedure of ‘shearing correction’. His usage of‘usually’ and ‘at this point’ evokes and manages the sequential order of the lab-oratory procedures, while the deictic expression ‘at this point’ also relies uponco-present recipients to ‘follow’ the unfolding sequence in the midst of its sce-nic details. Notice the interchangeable use of ‘I’ and ‘you’ in ‘so (you) usually(0.1) at this point (0.5)’ (line 1), and ‘I usually do a little bit of shearing (0.1)’(line 2), suggesting a reflexive coupling between ‘what is usually done’, ‘whatshould be done’ and ‘what is being done in the current moment of practice’.Similarly, the PI’s usage of ‘you’ in line 3 – ‘you can see’ – does more than rep-resent the PI’s belief that the PD can already see the distortion in the image; itprepares the terrain for a collaborative management of the visual scene. The PDis invited to participate with the PI in an enactment of ‘professional vision’(Goodwin, 1994). In and through the techniques used in his laboratory, she willprogressively be subsumed under the generic ‘you’ that can see what he says.

One of the significant aspects of the PI’s activity is the rapid alterna-tion of the images on the computer screen. Over the course of the experi-ment, series of brain images are recorded in which each image represents abrain slice. The PI uses mouse commands to rapidly alternate the imagesin a series comparing functional with corresponding structural images. The transcript indicates the display of functional images with series ofobliques, while the presence of structural images is represented with gray


Figure 5

10 o….p….a…………….h..So – it’s going up this way////////////////////////////////// / / /



rectangular shapes. When the PI detects a particular non-alignment amongimages, he refers to the shearing artifact. The artifact is thus relative to theimage series. Likewise, identifying the artifact – ostensively governed bymethodological prescriptions – requires a skill that involves a fine coordi-nation of eyes and hands.

The opening of the excerpt is characterized by an active search throughthe image series (lines 1–3). When an appropriate functional image is iden-tified (line 3), the PI utters ‘you can see it’s sort of’ (line 4–5). The PI thenplaces both of his hands in the space located between the two practitionersand the computer screen. He uses his hands to enclose a portion of the voidspace as if holding a round object (Figure 2). He then moves his left handup while his right hand moves down (Figure 3). Throughout the activityhis two hands conserve the shape in which they were placed at the begin-ning of the gestural enactment.

In line 5 the PI turns towards his addressee, and in line 6 the PD signalsher understanding and the co-participation in the activity. But while turningtoward the PD, the PI’s hand remains in the position assumed during the pre-vious gesture (Figure 4). The steady hand position is significant, as it links theenactment carried out in line 5 with its lexical affiliate – ‘sheared’ – which ispronounced only in line 9. As described by Emanuel Schegloff (1984: 275), ‘… the gesture – both its onset and its acme or thrust – precedes the lexicalcomponent it depicts’. Thus, by ‘projecting’ aspects of possible later produc-tions, the gesture makes them available to analysis by a recipient before theiractual occurrence (Schegloff, 1984: 267).

To understand the organization of the discursive action, however, oneneeds to take into account the tight temporal coordination between the PI’stalk and gesture with his alteration of the images on the computer screen.In line 7, while keeping his left hand in the ‘frozen’ gesture, the PI reiniti-ates to search through the image series. He utters the lexical affiliate of the‘shearing gesture’ only when an appropriate functional image is displayedon the screen (line 9). At this point, the PI remobilizes his gesturing handto enact another shearing (line 10). The second gesture is located in animmediate vicinity of the computer screen. Therefore, holding the gesturethrough lines 5–9 also links the gestural enactment in line 5 with its subse-quent elaboration in line 10.

The linkage between the already executed and the future enactmentstarts with an indexical component (line 10). When the PI directs his lefthand toward the computer screen, he briefly points with his index finger tothe upper-left portion of the image (Figure 5). After pointing to the image,the PI performs the second ‘shearing gesture’. He places his thumb ontothe screen, and, while holding his thumb and index finger an inch or soapart, he carefully moves his hand across the screen (Figure 6). During theexecution of the gesture, he keeps his right hand on the computer mouseto alter the images on the screen and orchestrate the entire performance(see, for example, Whalen et al., 2002). The PI’s talk, gestures, and hisengagement with the computer are tightly organized through their mutualreference. The organization not only concerns the coordination betweengesture and talk, but it also coordinates the practical activity at hand.




Gesture, talk and the manipulation of the digital screen function togetheras techniques for managing perception.

Coordination of Digital Screens with Bodies in Action

Particularly salient elements of the interaction are the two gestures exe-cuted in lines 5 and 10. The gestures, together with practitioners’ talk,gaze, and body orientation turn the physical space occupied by the practi-tioners into a field of meaning production. In the context of laboratorypractice, the multiple ‘semiotic fields’ (Goodwin, 2000b), such as the fieldof the digital screen and the one inhabited by material bodies, are super-imposed and intertwined. The way in which the images are aligned with thegestures, body orientation, gaze, and talk suggests an action-oriented, pub-licly available, and intersubjective character or seeing.

The first gesture – the gesture enacted in line 5 – is performed at a rel-ative distance, but in front of the brain image. Whereas the digital imageprovides a rich substrate for the action and allows for the potentially numer-ous interpretive paths, the gesture functions as an ‘eidetic’ mark that ‘bringsinto relief the essential, synthetic, constant, veridical, and universally pres-ent aspects of the thing “itself”’ (Lynch, 1990: 163). At the same time thepractitioners’ orientations toward the screen, their talk, the alignment withthe digital images, and the general context of the practice constrain theinterpretation of the gesture. Even though the gestural enactment does nottake place in synchrony with its lexical affiliate, the PI’s orientation towardthe screen displaying a functional image, and his talk (his saying ‘you cansee’ in line 4, for example) – embedded in the context of the practical prob-lem solving – indicate that the gesture is about the feature in the images.

The same is true for the gesture performed in line 10 (Figure 5): thegesture’s meaning is relative to the ongoing activity and the laboratory set-ting in which it is lodged. Here, however, the visibility of the shearing isgenerated through a physical coupling between the gestural enactment andthe digital images. The gesture marks the salient feature in the images bytouching and moving across the computer screen. However, its enactmentdoes more than reveal features of the images to the PD. The gesture is alsoimplicated by the PI’s coordination and comparison of the images.

The gesture first touches the screen just as the functional image is replacedby the structural one. During the gestural movement across the screen, the dis-play is alternated again, and the functional image appears. Through the activity,the gesturing hand links the appearance of the cross (a sign that marks the corre-sponding spot in the series of brain representations – see Figure 1) in the struc-tural image with the corresponding sign in the functional image. This allows thePI to ‘enact a memory trace’ across multiple images and single out the exact loca-tion of the misalignment. The articulation of visibility through the coordinationof hands, eyes, keyboard device, and the digital screen is part of the local act ofproblem-solving. The PD ‘learns to see the shearing’ not only by looking at theimages, but also by participating in the articulation of the space that surroundsthe two practitioners. The process of learning and understanding is one of par-ticipation: it situates the practitioners within the visual field, not outside it.




Seeing Dynamic Phenomenal Objects

The digitality of images reflects their dynamic character – not only in thenature of the images, but also in the ways that they are dealt with. Theobserved coordination of the bodies and technology has a salient temporaldimension. PI’s rapid alteration of the digital images generates the appear-ance of motion so that discrepancies among the images can be understoodin terms of a unified whole. The apparent motion is coupled with the PI’sutterance: ‘it’s going up this way’ (line 10). The utterance is an expressionof what proponents of Cognitive Semantics call ‘fictive motion’ (for exam-ple, Talmy, 2000): it organizes perception in terms of motion and change,even though the described entity – the fMRI image – is static.

Importantly, the PI also enacts motion in the space of material bodies inaction. His gesturing hands in line 5 do not directly ‘represent’ a distortedround object. Instead, they enact a process of transformation of a roundobject. Likewise, in line 10 they do more than index a feature in the image:the gestures participate in the interpretive act as an embodied enactment ofthe process of change. The dynamic quality of the act provides an account ofhow the distortion came about: the gestures generate explanations of the dis-tortion in terms of a three-dimensional (3D) object that is vertically sheared.

The practitioners I studied know that the ‘shearing’ of the image is aconsequence of an abstract computational transformation. Even so, theirsemiotic enactments indicate concrete, physical transformations. In otherwords, their practical accounts of the experimental data are given in termsof objects that can be seen, experienced and dealt with, rather than in termsof causal and abstract explanations.

It should be noticed also that the dynamic embodied account of how thedistortion came about implies that the process may be reversed by applyingvertical shearing in the opposite direction. The enactment not only performswhat happened in the past, it also evokes future practical actions. The nextsection will illustrate in detail the action of applying shearing in the oppositedirection. In fact, this is a process through which the practitioners understandthe work that needs to be done to correct the distortion.

Inasmuch as they are about future practical actions, the enactments bringto mind some aspects of Heidegger’s distinction between things ‘present-at-hand’ (Vorhandene) and things ‘ready-to-hand’ (Zuhandene). The two practi-tioners do not represent sheared features as objective properties that can bepassively contemplated. Rather, they enact those features as manipulablethings that ‘subordinate themselves to the manifold assignments of the “in-order-to”’ (Heidegger, 1962 [1927]: 98). As lines 5 and 10 indicate, the rele-vant feature of the fMRI scans is first made manifest through the activity thatmimics a direct engagement. The sheared feature becomes visible as a thingencountered through ‘circumspection’ – a purposeful way of acting withoutthe necessity of having a purpose in mind (see Dreyfus, 1991: 73). However,rather than allowing the images to fade away through such an interaction,6 theenactment of the sheared object generates visibility of the images.




Making the Work Visible

After making the artifact publicly available through enactment of the ‘in-order-to’ structures that show themselves only in practices, the purifica-tion of the experimental data needs to be explicated in a step-by-stepfashion. The practitioners deal with fMRI images – the digital structuresthat generate the effects of manipulability – to access the data by makingthe work visible. In other words, the features of fMRI images becomeperceivable in terms of the work, structured by the disciplinary expecta-tions and the routine practices that have developed for the accomplish-ment of the task, and made publically available through the interactionbetween the digital images and gestures.

EXCERPT 2

((The following gesture consists of two short clockwise motions articulatedaround wrist. The hand, positioned next to the computer screen, is formedas if holding a round object)).

12 o…..……………t….t…..PI You rotate (into the into)


FIGURE 7

13 h.………………..so that you know (.)

((The following gesture first indexes a left portion of the brain image andthen indicates up.))

14 o..………………p……………………….p(so that) the expanded part is sort of=

((The following gesture is performed as if drawing two axes. Lexical affili-ate is ‘axes’.))




FIGURE 8

FIGURE 9

15 o..a…………a…...in one of the axes

((The following gesture is a grasping motion. Lexical Affiliate is ‘squish’.))

16 o………..t…….(.) and you squish

((The following gesture is an abrupt opening of the hand. Lexical affiliateis ‘un-squish’.))

18 o.t…………r….hm(.)and un-squish ((turns toward the PD))

((The following gesture is a counterclockwise motion articulated aroundwrist. Lexical affiliate is ‘rotate back’.))

17 o……………….t…………...and then (you) rotate back



19 so: rotate 30 degrees, shrink vertically, rotate back, stretch vertically((turns towards the PD and smiles.))

20 PD: Uh-huh.

The organization of the activity described here is shaped by the way inwhich the software program designed in Laboratory I works. Because of theprogram’s limitations, the fMRI image series cannot be automatically cor-rected. Instead, a sequence of transformations is required to achieve thecorrection announced by the gestures in lines 5 and 10 (Excerpt 1).

Figure 11 shows a detail of the computer screen when the shearing arti-fact was being corrected. The correction requires the user to interact with


FIGURE 10

FIGURE 11Detail of the computer screen during the shearing correction (proportions have beenmodified to enhance the visibility of the menu)



However, like the brain images on the screen, the figure is a partial,two-dimensional (2D) representation of the experimental data.

Despite the appearance of the 2D representations on the computerscreen, the practitioners need to work with 3D data. They need to performthe changes on something that is more akin to a series of 2D images thanto a single 2D representation. The procedure requires significant expertise.The practitioners need to infer the shape of the 3D object before and afterit is digitally modified while looking at a 2D fMRI image. The design of thecomputer interface allows them to do so with the scroll bars and labels,which evoke a sense of physically manipulating 3D objects – for example,with the command ‘rotate brain’. Note that these features only evokeobjects and actions; the environment in which the interaction between theuser and the experimental data takes place is a 2D digital space.

Similar to the computer commands and labels are PI’s linguisticexpressions – ‘rotate’ (line 12), ‘squish’ (line 16), ‘rotate back’ (line 17),‘un-squish’ (line 18), ‘shrink vertically’ and ‘stretch vertically’ (line 19) –


Figure 12Schematized representation of the four stages of shearing correction

the scroll bars via mouse commands to direct mathematical transforma-tions on the digital data. The transformations of the digital data are indi-cated in the visual format: in response to the manipulations of thecommand menu, the brain images appear modified.

Once the shearing artifact is detected in the images, four consecutive com-mands need to be selected. First, the scroll bar labeled ‘ROTATE BRAIN(deg)’ needs to be ‘moved’ towards the right. When this is done, the verticalscroll bar labeled ‘SCALE BRAIN (percent)’ has to be moved down. The thirdcommand is the movement of the scroll bar ‘ROTATE BRAIN (deg)’ towardsthe left. And finally the horizontal scroll bar ‘SCALE BRAIN (percent)’ has tobe moved towards the right. The commands’ function is illustrated in Figure12. To make the distortion more easily perceivable, the Figure represents thebrain image in a schematized rectangular rather than a round form.



which evoke a sense of physically manipulating objects. But to engage 3Ddata in an immediate manner the practitioners have another resource athand: the space of gestures and embodied semiotic interaction.

In line 12, the PI places his right hand on the computer screen androtates it clockwise (Figure 7). Next, in lines 14–15, he explains that therotation has to position the brain image so that its expanded part is locatedin a vertical position (‘on one of the axes’). He points briefly toward thebrain representation and then upward (line 14). The enactment is followedby a gesture of an ephemeral representation of the two Cartesian axes (line15). This position is crucial for the next step of the transformation (‘squish-ing’), in which a vertical force is enacted. The gestures in lines 16–18 mimethe acts of ‘squishing’ (Figure 8), ‘rotating back’ (Figure 9), and ‘un-squishing’ (Figure 10). After performing the four-step transformation, thePI turns towards the PD to check that she has been following of the action.The PI’s bodily movement and the PD’s affirmation (‘Uh-huh’, line 20)mark the completion of the meaning-making unit.

An analogous activity was recorded in the laboratory 2 months earlier,during another instance of data analysis. The PI and two graduate studentswere seated in front of the computer screen when the PI noticed a distor-tion in the fMRI image. While orienting toward the computer screen, thePI described how the distortion should be corrected:

EXCERPT 3

((The following gesture is a counterclockwise motion of the right handarticulated around wrist. The hand, positioned in the immediate vicinity ofthe screen, is formed as if holding a round object. Lexical affiliate is ‘rotate30 degrees this way’.))

21 o……….t…………………………….PI (.)If you rotate 30 degrees this way

((The following gesture is a motion articulate with two hands as if holdingan object and slightly pushing it from both sides. Lexical affiliate is‘squash’.))

22 o ………..t……...and then squash


FIGURE 13



((The following gesture is an abrupt opening of the right hand. Lexicalaffiliate is ‘un-squash’.))

24 o…..t………….and un-squash.

The semiotic action performed here is quite similar to the one in theprevious excerpt, except for the fact that the PI in that instance persist-ently gestures with one rather than both hands, and starts the enact-ment with a clockwise rotation. The rituality of the performance revealsthe tight connection between semiotic enactments and the world ofinstruments and other technologies. The semiotic enactment is contin-gent on the design of the computer screen and the ‘usual’ laboratoryactivities.

At the same time the ritual performance highlights the pervasiveness ofthe gestural enactment in the management of perception. The PI reachestoward the computer screen and acts as if he were holding an object andmoving it towards the left/right (lines 21 and 23). Similarly, he enacts theaction of ‘squashing’ the image as if he were squeezing a 3D object (line 22).By enacting the physical manipulation on the fMRI data, the gesture par-ticipates in the production of visibility by making the encounter with thephenomenal object possible. Whereas in lines 5 and 10 (Excerpt 1) the twoshearing gestures participate in the enactment of phenomenal objects, sothat the future work on such objects can be grasped, here the gestures par-ticipate in the performance of the exact steps in such work. Thus, the inter-action renders the future work not only graspable, but directly perceivableand publicly shared.


FIGURE 14

((The following gesture is a clockwise motion articulated with both handsas if holding a round object. Lexical affiliate is ‘un-rotate’.))

23 o………….t…………and then un-rotate



Blurring the Boundary Between the Digital World and the World ofPractice

Despite the PI and PD’s orientation toward the computer screen, noteverything is available there. As mentioned earlier, the PI’s command selec-tion has consequences not only for the brain slice displayed on the screen,but also for the data representing the whole brain. How do the practition-ers deal with what is invisible on the screen? The interaction between thepractitioners and the digital screen recovers complex acts of coordination,delegation, and selection. These acts are involved in an enactment ofhybrid phenomenal objects.

One could speculate that the problem of invisibility would be solved bydesigning a visual display that would reveal at once all the richness of thedata – ‘it is only a matter of time and technology’. Regarding this point, thePI remarked that if in fact all the layers of the 3D data were displayed(while discussing this point he evoked an image of a ‘translucent cabbage’)their richness would overwhelm the viewer, who would then be unable todeal with the scene. Instead, including the gestural action in the processpresents the viewer with a form that evokes the 3D character of the data,while leaving the potentially overwhelming detail concealed.7

When talking about ‘gestural action’ one should, however, not assumethat the single semiotic modality carries the meaning on its own. The ges-ture is rather poor when considered in isolation. The gesturing hand isshaped as if holding a 3D object, but without its interaction with the othersemiotic resources, the enacted object remains generic: there is nothingintrinsic to the gesture that defines the object as a brain. Similarly, theimages are completely static: they simply exhibit assumed positions beforeand after their computational manipulation. The linguistic labels displayedon the screen and referenced in the PI’s utterances describe motion, butnothing is moving on the screen. The richness of the process, instead,comes from the interaction.

In this regard, the specific location and dynamic qualities of the ges-ture are significant. Their unfolding takes place in close proximity to thecomputer screen. The gesturing hand, accurately positioned around thevisual contours in the brain-slice representation, touches the digital screenwhile it enacts ‘rotation’, ‘squishing’, and ‘squeezing’ of a round object.Moreover, the sense of time and movement is generated through the ges-turing hand and the manipulation of the screen. The process is accompa-nied by the PI’s verbal labels of the movements as ‘rotation’, ‘squishing’,and ‘squeezing’.

Bringing together gesture, talk, and the structure in the environment(what Goodwin [1995, 2000b] calls ‘environmentally coupled gestures’) isnot simply a cumulative process. The interaction entails acts of selectionand delegation. Although the practitioners know that the shearing correc-tion needs to be accomplished through the mathematical manipulation, theabstract computational processes, which are not accessible through directinspection but actually performed on the digital data, are largely delegated




to the machine. While they consider the numerical data and the mathe-matical processes to be the central and omnipresent in their practice (seeBeaulieu, 2002), the practitioners do not access them directly while work-ing with ‘digital brains’. Rather, those invisible layers are indexed by thePI’s gestural enactments, tightly coordinated with the digital images andconceived as performances of practical actions. The enactments – whichdepend on the digital images – generate a sense of physical engagement,even though the digital images cannot be physically manipulated – nothinglike the ordinary sense of the words ‘rotation’, ‘squishing’, and ‘squashing’is performed on the data.

This type of activity suggests that the practitioners deal with hybridstructures. Instead of referring to numerical data, digital images, or physi-cal objects, they engage with multifaceted phenomena that change throughtime.8 In other words, the experience allows the practitioners to grasp thenature of the distortion and its correction by encountering something thatis simultaneously material and digital, concrete and abstract, human andmachine, 3D and 2D, action and object, present and future.9

Hybridization of Digital and Physical as Practical Engagement withExperimental Data

It has often been pointed out that the involvement with the digital screenstransposes the user into an alternative realm – a realm of fiction. To engagewith the digital data in an immediate manner, the user needs to ‘projecther/himself on the other side of the screen or pass through the screen to enterthe virtual world of fiction’ (for example, Morse, 1998; Lister et al., 2003; seealso Turkle, 1995).10 By analogy, it could be claimed that the fMRI practi-tioners’ semiotic accounts of enacted hybrid objects are imaginary, fictive, ormetaphoric renderings instantiated in the public space of action.

Several recent studies have dealt with the problem of imagination as apublic process accomplished through the coordination of participants’ con-duct and the material world of their practice (Nishizaka, 2000, 2003;Suchman, 2000; Murphy, 2004, 2005). In the examples they provide, theparticipants treat what is imagined and enacted in the interaction space – aloading dock (Murphy, 2004, 2005), submarines’ routes in a computergame (Nishizaka, 2003), or a highway (Suchman, 2000) – as somethingthat has potential for future existence: the loading dock could be con-structed, the submarines could be visualized as taking an alternative routeon the computer screen, the highway could be lowered and the earth couldbe removed.

In the shearing example, however, the public performance is not con-ceived as imaginary, nor is it seen as something that could exist outside itslocal enactment. In fact, we may not even be able to imagine or ‘see in frontof our mind’s eye’11 something that is simultaneously 2D and 3D, abstractand concrete, digital and material.

At the same time, and despite the hybrid character of their publiclyenacted objects, the practitioners treat them as real and ordinary. While




tacitly ‘asking’ practitioners to partially delegate their work to the machine,the production and understanding of multifaceted phenomenal objectsprovide practitioners with a sense of direct engagement with their experi-mental data. The negotiation and hybridization between the digital and thephysical is a practical way to understand and deal with the world.

Discussion

There is a substantial body of literature that deals with vision as a situatedand interactionally organized phenomenon (for example, Goodwin, 1994,2000c, 2003; Heath & Hindmarsh, 2000; Hindmarsh & Heath, 2000;Nishizaka, 2000; Suchman, 2000; Ueno, 2000). Following this line ofresearch, the present work discusses ‘seeing’ as a process situated at the inter-section between instruments and technology, practices, settings, and thepractitioners’ embodied accounts. It describes how the local management ofseeing invokes previous dealings and cumulative practical know-how, andhow it receives its rhetorical force by reference to the ‘usual’ procedures oflaboratory members. These procedures are nested in larger, historicallyevolving, socio-technical networks whose specifications are bound to thelocally instantiated assemblages of instruments, embodied techniques, andeveryday discourses in the laboratory (see, for example, Lynch, 1993).

The specific focus of the paper, however, is on the achievement of see-ing in the digital realm. The excerpts we examined recover the complexi-ties in the practical dealings with brain scans. ‘Seeing’ of brain images isabout engagement that requires coordination of scientists and digital tech-nology in the context of fMRI practice. The PI’s embodied accounts ofshearing not only function as indexical signs or transient inscriptionsimposed on the computer screen to categorize and make the features ofimages visible. They also instruct the PD on how to spot the image artifactby enacting what should be done, can be done, and usually is done in thePI’s laboratory.

These ‘acts of seeing’ explore a border between the virtual and thephysical, 2D and 3D, abstract and concrete, past, present and future: theygenerate emergent properties and distributions of agency. The enactedphenomenal objects provide practitioners with a sensation of rotating,squishing, or squashing something, even though nothing is being rotated,squished or squashed. The PI and PD talk about, see and experience around object that changes and is being changed through time while theyinteract with menus on a computer screen – to display 2D brain scans andto direct invisible mathematical transformations on them.

Understanding the artifact and its correction by enacting a hybridobject that moved and needs to be rotated, squished, and squashed is a partof practical problem-solving in the PI’s laboratory as well as a componentof his ‘powerful seeing’. Not only does the PI’s reading of the images pre-vail through moments of local interaction: it also participates in propaga-tion of the embodied techniques and the methods of organizing the toolsand instruments practiced by his research group. As pointed out by the PI,




the interaction with the PD left traces in practices for managing scientificfacts beyond the walls of his laboratory. But would this have been possiblewithout the PD’s willingness to cross the laboratory boundaries and par-ticipate in the act of collaborative seeing?

Concluding Remarks

Jonathan Crary (1990), among others, has pointed out that the introduc-tion of computer-generated imagery leads to something he calls ‘abstrac-tion of the visual’. He proposes that digital images, unlike traditionalanalog media such as film, photography, and television, are not relative anylonger to a point of view ‘located in the real space’:

Computer-aided design, synthetic holography, flight simulators, computeranimation, robotic image recognition, ray tracing, texture mapping, motioncontrol, virtual environment helmets, magnetic resonance imaging, andmultispectral sensors are only a few of the techniques that are relocatingvision to a plane severed from a human observer. … Most of the historicallyimportant functions of the human eye are being supplanted by practices inwhich visual images no longer have any reference to the position of anobserver in a ‘real’, optically perceived world. (Crary, 1990: 1–2)

While one might think that this ‘abstraction of the visual’ and the differencebetween new digital and traditional analog media necessarily entail a dis-embodiment of the digital image user, interactions in the MRI laboratoryreveal a somewhat different state of affairs. During the moments of workand interaction, digital brain images are coupled with practitioners’ bodilyorientations, gestures, and talk. Practitioners’ read the experimental data interms of embodied actions that take place at a level comparable to physi-cal, real-world engagement. Rather than estranging them from a directengagement with objects, reading digital images enables them to re-enter aworld of culturally meaningful embodied actions.

In this sense, ‘seeing’ of fMRI images is an embodied process achievedthrough a coordination of ‘visual’ information with the world of meaning-ful actions and practical problem-solving. In other words, the visibility isnot only relative to what goess on inside the practitioner’s head or to whatis present on the screen. Seeing is tied to actions that arise out of experi-ences with the manipulation of objects and everyday practical dealings.

Research in human–computer interaction (HCI) has pointed out thatbecause of their aptness for understanding, the products of scientific visual-ization – including fMRI images – provide the human user with a significantpower in handling abstract data.12 Rather than dealing with an enormousamount of numbers, digital images allow scientists to solve the abstract taskin the visual realm. Just as important, fMRI images are ‘visual’ and ‘visible’versions of what previously was not visual, inasmuch as they allow the prac-titioners to deal with the invisible in a practical manner. The excerpts I havepresented indicate how the laboratory setting and the human encounterswithin it provide practitioners not only with an abstract sense of vision, but




also with a way of seeing digital images that involves the hands as well as theeyes. Such acts of seeing concern the enactments of the dynamic and mul-timodal objects in the practice of fMRI.

NotesI would like to thank the following people for their contribution to this paper: LucySuchman, Monika Buscher, Lisa Cartwright, Gilles Fauconnier, Charles Goodwin, EdwinHutchins, Michael Lynch, Maurizio Marchetti, David Mindell, Ayse Saygin, Marty Sereno,three anonymous referees, and the participants in the ethnographic study.

1. The fMRI images that we usually encounter in research journals, popular media, orthe Internet frequently feature brain representations in black and white with colorfulspots on them – usually in blue, green, yellow, and red. These images are a result of amerging process between structural and functional images.

2. The PI was also one of the main designers of the software used in his laboratory.3. To picture the distortion one can consult its schematized rendering in the upper

portion of Figure 12.4. During the early stage of data analysis described here, the two practitioners analyze

scans pertaining to one experimental subject. Even though the members of the twolaboratories publish fMRI images that represent the brain of a single individual, amore frequent practice is to collect data from a series of subjects and publish imagesshowing averaged data. When fMRI data are averaged across subjects, thepractitioners align the data to a ‘common space’. Usually they align the structuralscans to a standard template (one example is the ‘Talairach’s coordinate system of thehuman brain where the representations of the single brain are projected, via theregistration procedure, to ‘Talairach brain’; see Talairach & Tournoux, 1988) and thencopy the parameters to the functional scan to translate it in the new space (withcommercial software it is also possible to directly co-register the functional scans tothe common space). Thus, the reduction of the shearing artifact in the early stages ofdata analysis, even though not directly related, is in fact implicated in fMRI imageseven when those images represent ‘averaged brains’.

5. Kelly Joyce pointed out that in order to see artifacts in MRI images radiologists needto ‘ … train their sight over time’ through ‘ … interaction with other physicians, texts,and machines’ (Joyce, 2005: 449).

6. The concept of ‘ready-to-hand’ is frequently referred to in the literature on interactionwith technology. See, for example, Dourish (2001: 109) and Suchman (1987: 53).

7. This certainly does not deny that an expert has no need to resort to such publicresources. But it does point out the public grounding of expertise.

8. The importance of hybrid semantic structures in human understanding has beenpointed out by Conceptual Integration Theory (Coulson, 2000; Fauconnier & Turner,2003). Rather than accounting for stable knowledge structures represented in long-term memory, the theory identifies systematic projections of language, imagery, andinferential form to model the dynamic evolution of speakers’ on-line representations(Coulson & Oakley, 2000). The semiotic enactments of shearing artifact share somefeatures with the process of conceptual integration. The hybrid semiotic construct is atthe same time about the brain image that indexes the experimental data, and theembodied multimodal performances that make such data and their manipulationspublicly available. Mutual reference and integration provides access to the problemsolution in terms of ordinary actions. The practitioners deal with the experimentaldata and their correction by performing round objects that share some visual featureswith the digital image and are being rotated, squished, and squashed, even thoughneither the digital data nor the fMRI images can be subjected to those operation. Theprocess of hybridization is largely generated in the social space of action. Rather thanbeing exclusively mental phenomena internal to single individuals, as it is largely thecase in the usual applications of the conceptual integration theory, important elements




of the production of visibility involve integrations between multiple semiotic fieldsgenerated through the use of gesture, digital images, and body orientation as featuresof the practical problem solving. For a related account, in which only one of thesemantic spaces is a material phenomenon, see Hutchins (2005).

9. Ochs et al. (1996) describe somewhat similar phenomenon – a blending between thescientists and the objects of their study – as ‘referentially ambiguous entities’.

10. For an example of the popular rendering of the idea, see the 1982 Walt DisneyProductions science fiction film Tron directed by Steven Lisberger.

11. Nishizaka (2003) contains a comprehensive discussion and critique of the idea ofmental image.

12. Neuroscientists themselves often point out that the largest part of the human visualcortex is dedicated to processing visual information.

ReferencesAlac, Morana (2004) ‘Negotiating Pictures of Numbers’, Social Epistemology 18(2): 199–214.Beaulieu, Anne (2002) ‘Images are Not the (Only) Truth: Brain Mapping, Visual

Knowledge, and Iconoclasm’, Science, Technology, & Human Values 27: 53–86.Beaulieu, Anne (2004) ‘From Brain Bank to Database: The Informational Turn in the

Study of the Brain’, Studies in History and Philosophy of Biological and BiomedicalSciences 35: 367–90.

Cartwright, Lisa (1995) Screening the Body: Tracing Medicine’s Visual Culture (Minneapolis,MN: University of Minnesota Press).

Coulson, Seana (2000) Semantic Leaps: Frame-shifting and Conceptual Blending in MeaningConstruction (New York: Cambridge University Press).

Coulson, Seana & Todd Oakley (2000) ‘Blending Basics’, Cognitive Linguistics 11(3–4):175–96.

Crary, Jonathan (1990) Techniques of the Observer (Cambridge, MA: MIT Press).Dourish, Paul (2001) Where the Action is: The Foundations of Embodied Interaction

(Cambridge, MA: MIT Press).Dreyfus, Hubert L. (1991) Being-in-the-world: A Commentary on Heidegger’s Being and Time,

Division I (Cambridge, MA: MIT Press).Dumit, Joseph (2004) Picturing Personhood: Brain Scans and Biomedical Identity (Princeton,

NJ: Princeton University Press).Fauconnier, Gilles & Mark Turner (2002) The Way We Think: Conceptual Blending and the

Mind’s Hidden Complexities (New York: Basic Books).Goodwin, Charles (1994) ‘Professional Vision’, American Anthropologist 96(3): 606–33.Goodwin, Charles (1995) ‘Seeing in Depth’, Social Studies of Science 25: 237–74.Goodwin, Charles (2000a) ‘Practices of Seeing, Visual Analysis: An Ethnomethodological

Approach’, in T. van Leeuwen & C. Jewitt (eds), Handbook of Visual Analysis (London:Sage Publications): 157–82.

Goodwin, Charles (2000b) ‘Action and Embodiment within Situated Human Interaction’,Journal of Pragmatics 32: 1489–1522.

Goodwin, Charles (2000c) ‘Practices of Color Classification’, Mind, Culture and Activity7(1–2): 19–36.

Goodwin, Charles (2003) ‘The Semiotic Body in its Environment‘, in J. Coupland & R. Gwyn (eds), Discourses of the Body (New York: Palgrave/Macmillan).

Heath, Christian & Jon Hindmarsh (2000) ‘Configuring Action in Objects: From MutualSpace to Media Space’, Mind, Culture and Activity 7(1–2): 81–104.

Heidegger, Martin (1962 [1927]) Being and Time (New York: Harper).Hindmarsh, Jon & Christian Heath (2000) ‘Embodied Reference: A Study of Deixis in

Workplace Interaction’, Journal of Pragmatics 32: 1855–78.Hutchins, Edwin (2005) ‘Material Anchors for Conceptual Blends’, Journal of Pragmatics

37(10): 1555–77.Ihde, Don (2002) Bodies in Technology (Minneapolis, MN: University of Minnesota Press).




Jefferson, Gail (1984) ‘Glossary of Transcript Symbols with an Introduction’, in G.H.Lerner (ed.), Conversation Analysis: Studies from the First Generation (New York: JohnBenjamins Publishing Company): 13–31.

Joyce, Kelly (2005) ‘Appealing Images: Magnetic Resonance Imaging and the Production ofAuthoritative Knowledge’, Social Studies of Science 35(3): 437–62.

Knorr-Cetina, Karin (1981) The Manufacture of Knowledge: An Essay on the Constructivistand Contextual Nature of Science (Oxford: Pergamon Press).

Latour, Bruno (1988) The Pasteurization of France (Cambridge, MA: Harvard University Press).Latour, Bruno & Steve Woolgar (1979) Laboratory Life: The Social Construction of Scientific

Facts (London & Beverly Hills, CA: Sage Publications).Lister, Martin, Jon Dovey, Seth Giddings, Iain Grant & Kieran Kelly (2003) New Media:

A Critical Introduction (New York and London: Routledge).Lynch, Michael (1985) Art and Artifact in Laboratory Science: A Study of Shop Work and

Shop Talk in a Research Laboratory (London: Routledge & Kegan Paul).Lynch, Michael (1990) ‘The Externalized Retina: Selection and Mathematization in the

Visual Documentation of Objects in the Life Sciences’, in M. Lynch & S. Woolgar(eds), Representation in Scientific Practice (Cambridge, MA: MIT Press): 153–86.

Lynch, Michael (1991) ‘Laboratory Space and the Technological Complex: AnInvestigation of Topical Contextures’, Science in Context 4(1): 81–109.

Lynch, Michael (1993) Scientific Practice and Ordinary Action: Ethnomethodology and SocialStudies of Science (New York: Cambridge University Press).

Lynch, Michael & Steve Woolgar (eds) (1990) Representation in Scientific Practice(Cambridge, MA: MIT Press).

Murphy, Keith M. (2004) ‘Imagination as Joint Activity: The Case of ArchitecturalInteraction’, Mind, Culture, and Activity 11(4): 267–78.

Murphy, Keith M. (2005) ‘Collaborative Imagining: The Interactive Use of Gestures, Talk,and Graphic Representation in Architectural Practice’, Semiotica 156 (1–4): 113–45.

Nishizaka, Aug (2000) ‘Seeing what one Sees: Perception, Emotion, and Activity’, Mind,Culture and Activity 7(1–2): 105–23.

Nishizaka, Aug (2003) ‘Imagination in Action’, Theory & Psychology 13(2): 177–207.Ochs, Elinor, Patrick Gonzales & Sally Jacoby (1996) ‘“When I Come Down I’m in a

Domain State”: Talk, Gesture, and Graphic Representation in the InterpretativeActivity of Physicists’, in E. Ochs, E. Schegloff & S. Thomson (eds), Interaction andGrammar (Cambridge: Cambridge University Press): 328–69.

Prasad, Amit (2005a) ‘Scientific Culture in the “Other” Theatre of “Modern Science”: AnAnalysis of the Culture of Magnetic Resonance Imaging Research in India’, SocialStudies of Science 35(6): 463–89.

Prasad, Amit (2005b) ‘Making Images/Making Bodies: Visibilizing and Disciplining throughMagnetic Resonance Imaging’, Science, Technology, & Human Values, 30(2): 291–316.

Schegloff, Emanuel A. (1984) ‘On Some Gestures Relation to Talk’, in J.M. Atkinson &J.C. Heritage (eds.), Structures of Social Action: Studies in Conversation Analysis(Cambridge: Cambridge University Press): 266–96.

Suchman, Lucy (1987) Plans and Situated Actions: The Problem of Human–MachineCommunication (New York: Cambridge University Press).

Suchman, Lucy (2000) ‘Embodied Practices of Engineering Work’, Mind, Culture andActivity 7(1–2): 4–18.

Talairach, Jean & Pierre Tournoux (1988) Co-planar Stereotaxic Atlas of the Human Brain(New York: Thieme Medical Publishers).

Talmy, Leonard (2000) Toward a Cognitive Semantics (Cambridge, MA.: MIT Press).Traweek, Sharon (1988) Beamtimes and Lifetimes: The World of High Energy Physics

(Cambridge, MA: Harvard University Press).Tufte, Edward R. (1983) The Visual Display of Quantitative Information (Cheshire, CT:

Graphics Press).Turkle, Sherry (1995) Life on the Screen: Identity in the Age of the Internet (New York: Simon

and Schuster).




Ueno, Naoki (2000) ‘Ecologies of Inscriptions: Technologies of Making the SocialOrganization of Work and the Mass Production of Machine Parts Visible inCollaborative Activity’, Mind, Culture and Activity 7(1–2): 59–80.

Whalen, Jack, Marilyn Whalen & Kathryn Henderson (2002) ‘ImprovisationalChoreography in Teleservice Work’, British Journal of Sociology 53(2): 239–58.

Morana Alac is Assistant Professor of Communication at the University ofCalifornia San Diego, where she is also on the faculty of the Program inScience Studies. She holds doctorate degrees in Semiotics from the Universityof Bologna and in Cognitive Science from the University of California SanDiego. Her prior publications on brain imaging practice appeared in SocialEpistemology, Journal of Culture and Cognition, and Semiotica.

Address: University of California San Diego, Department of Communication,9500 Gilman Drive 0503, La Jolla, CA 92093-0503, USA; email: [email protected]



social studies of sciencemechanism.ucsd.edu/teaching/f12/cs200/readings/alac-2008... · 2012. 8....

Documents