ethnography using - chalmers · experimented with ethnography in design [2, 10, 11]. we have...

82 July 1997/Vol. 40, No. 7 COMMUNICATIONS OF THE ACM

Ethnography originates from anthropologywhere anthropologists spend extendedperiods of time with primitive societiesmaking detailed observations of theirpractices.

In a design context the aim of ethnography is todevelop a thorough understanding of current workpractices as a basis for the design of computer support.A major point in ethnographically inspiredapproaches is that work is a socially organized activitywhere the actual behavior differs from how it isdescribed by those who do it. This implies thatdetailed studies of work must include observations aswell as interviews [for example 1, 4, 12]. Blomberg etal. [1] characterize ethnography with four principlesand three main techniques: It takes place in naturalsettings; it is based on the principle of holism, that is,particular behaviors must be understood in the respec-tive context; it develops descriptive understanding incontrast to prescriptive; and it is grounded in a mem-ber’s point-of-view. They use observation, interview,and video analyses as main techniques.

Using ethnography in the design of computer-basedsystems has become increasingly prominent, especiallywithin the research communities of computer-sup-ported cooperative work (CSCW), but also within par-ticipatory design, and human-computer interaction.

Plowman et al. [9] recently reviewed all studiesusing ethnography published within the CSCW lit-erature. In this review, three issues (of particular con-cern to us) are raised. First, the dominant approachis sociologists conducting the ethnographic studiesand informing computer scientists of their findings,such as in debriefing meetings [for example 5, 6]).Second, reports on concrete consequences of a spe-cific design due to such an approach are typicallyabsent. Third, a “need to consider developing hybridand tailored forms of ethnography that can play dif-ferent practical roles in the various phases of design”is argued [9 p. 321].

As computer scientists, we have adopted andexperimented with ethnography in design [2, 10,11]. We have developed a method for participatorydesign where ethnography is an embedded part ofthe overall design activities [8]. Participatory designrefers to an approach where users play an active part.Users and designers engage in mutual learning activ-ities in order to understand users’ current work andgenerate coherent visions for change [3].

We believe that practitioners can benefit fromusing ethnography in contextual design (particularlywhen designing systems in a specific organizationalcontext), but they must be aware of the conditionsneeded for such an approach.

UsingEthnographyIn Contextual Design

This article presents a case from our research inthe form of a design project for the Editorial Boardof a Film Board (detailed in [10]). The project wasconducted in two parts. Traditional techniques likemeetings, interviews, document analysis, rich pic-tures, and mock-ups were used in Part One leadingto a first design proposal. In Part Two, experimentswith ethnographic techniques like observation andvideorecording were applied and the effect was eval-uated in light of the first design proposal.

Here, we present the organization and describethe Editorial Board design project. We spent approx-imately 14-person weeks over a period of 10 monthson the project because it also served as a research pro-ject. Had it been a real-life consulting job, our esti-mate would be approximately 10-person weeks.

The OrganizationThe Film Board is a public organization in NorthernEurope under a Ministry of Cultural Affairs. Theorganization has approximately 50 employees and abudget of $7.5 million a year.

The Film Board’s main function is to promoteinformation, education, and artistic and culturalactivities by producing and buying films along withdistributing such films to educational institutions,associations, and individuals.

The film categories of The Film Board includecultural and social conditions, such as docu-mentaries, portraits, and debate films; education;and art, such as experimental video art.

The production of films involves funding and sup-porting directors and producers, and to some extent,managing the production. This is conducted by theEditorial Board. Nine people work here: three edi-tors who consider applications (about 800 per year)and decide which productions should be funded(about 100 per year); one production manager incharge of financing all productions; three secretaries;a consultant specialized in buying and managing thetranslating of foreign films and videos; and one tech-nician. The overall production of films—from theproducers’ ideas, to the distribution to the con-sumers—comprises an editorial process, a produc-tion process, and distribution as depicted in Figure 1.

The Editorial Board handles all applications forfilms and videos productions, decides which ones tosupport, negotiates contracts, and manages the pro-duction of films and videos. Three editors (hired onlyfor 2-4 years to secure a broad selection of produc-tions) are in charge of this task. They are responsiblefor four different areas of productions (16mm. film;video; film and video for children; foreign film and

COMMUNICATIONS OF THE ACM July 1997/Vol. 40, No. 7 83

An impressive design approach for determining user needs incorporates keen observations of workers

as they go about their daily routines.

MC

KLO

SE

video), each of which has its own budget. Decidingwhich productions to support, and coordinating dif-ferent productions with various Film Board depart-ments, is done at the weekly production meeting. Atthis meeting all employees from the Editorial Board,along with the president and the managers from theorder receiving and marketing departments partici-pate. Here, information regarding applications, sta-tus, and actions for each production currently in

progress is exchanged, and the necessary decisionsand coordination are made.

The editors primarily take care of the applicationsand production in regard to content. They are sup-ported by the production manager who is responsi-ble of the overall budget, and by three secretarieswho handle most of the administrative follow up:fielding calls, informing relevant parties, receivingand filing applications, and all succeeding data thatconcerns the productions (budget, funds, expendi-tures, technical data, correspondence, among others).

In recent years video has been introduced as anadditional medium besides the traditional 16mmfilm. This has raised the number of productions fromabout 25 per year to nearly 100 and the number ofapplications from about 100 to about 800. Theorganization was not geared for this. All work in theEditorial Board was paper-based, except for word-processing chores.

The secretaries especially felt the increased num-ber of applications and productions was an over-whelming and cumbersome administrative burden;the paperwork and manual updating of all the paperfiles engrossed most of their time. This left littletime for their previous skilled and qualitative sup-port to the editors, producers, and directors of thefilms and videos. The secretaries wanted computersupport for recording all the information on eachproduction, and for the financial management of theproductions. One key issue was to get rid of repeatedwork within and outside the department. Anotherissue was to keep track of the current status of theproductions for cooperative purposes.

The Design Project (Part One)To get an initial overview we conducted unstruc-tured interviews of all nine employees individually.Each interview lasted 1-2 hours. In parallel, we didthorough document analysis of written materials(leaflets, booklets, production plans, minutes fromvarious meetings, the Act for the institution) alongwith studying the different paper-forms in use.

Since the editors saw no relationship between theirproblems and computers and had no idea of theirneeds besides word processing, a second round ofinterviews was performed with the secretaries. Theseinterviews were conducted as dialogues, where thesecretaries often showed how they carried out specifictasks. Often, specific design ideas emerged duringthese interviews. Thus, the interviews establishedmutual learning situations where relevant structuresof the secretaries’ current work were developed on thebasis on their concrete experiences [7]. The outcomewas drawings that captured aspects of their current


Figure 1. The overall production of films in The Film Board.

Directors and

Producers

Application: Idea for production or manuscripts

Marketing takes care of the premiere

Order Receiving distributes film and video

Editor reviews first version/cut

Shooting and cutting film

Consideration by editor

Support or refuse

Negotiate contract

Consumers

Editor reviews final cut and mix

Editorial process

Production Process

Distribution

work as well as design visions.Our preliminary design was drawn as rich pictures

and mock-ups on flip-overs and presented at a themeeting with the Editorial Board and its technologycommittee. We had identified and sketched two sys-tems: A production-management system that assuresall data on a production was recorded only once in acentral database; and a financial part of the produc-tion-management system that supports a generalview of the budget and money spent on all produc-tions currently in progress.

The systems were evaluated as very appropriate byall Board employees and by the technology commit-tee. If the project had not been part of our researchagenda, and a preplanned Part Two, the next appro-priate step might have been refin-ing and prototyping the designproposal.

The Design Project (Part Two)The design proposal from PartOne primarily supported the sec-retaries, the production manager,and the consultant. Their jobs areto support the editors. The edi-tors handle The Film Board’smain function in relation to thefilm producers. This function iscomplex and somewhat invisibleto the rest of the organization,and, as stated earlier, the editorshad no better ideas for computersupport except as word proces-sors. On this basis we decided tofocus our experiments withethnography towards the workpractices of the editors.

To obtain a thorough under-standing of the editors responsibilities, we observedthem in their daily routines. This was done simplyby following them for several days at their office andin the field. We observed them having meetingswith applicants; negotiating new productions withdirectors and producers; reviewing versions withdirectors, producers, photographers, and cutters; andparticipating in the weekly production-meeting.The editors themselves helped by suggesting daysand times where we should join them. This assuredthat we observed the variety of different tasksinvolved in their job.

We videotaped some of our observations. Themain use of the videotapes was to replay them anddiscuss what happened at the tapes. This was done at

our University’s lab; Editorial Board employees didnot participate.

The observations and reviews of the videos poseda great deal of questions that we subsequently fol-lowed up by interviews with all employees in theEditorial Board. The focus was on the cooperativeaspects of the work both internally in The FilmBoard and externally with applicants, directors, andproducers from the film and video industry. Most ofthese interviews were audiotaped and roughly tran-scribed.

The observations revealed a complex cooperativepattern in the lifecycle of a production, involving allemployees in the Editorial Board. In order to obtaina coherent picture, we organized a series of wall-

graph sessions where the workinvolving various peopleand competencies weredescribed coherently in thesense it detailed the flow ofa production from theemployees’ point of view.We asked them to writedown all activities and func-tions and who was in chargeof them (on the upper partof the wall graph), andrelated data and information(on the lower part of thewall graph). Everything waswritten on one piece ofpaper with an applicationreceived in one end, and thefilm or video discharged andtaken out of distribution inthe other. Each participantused his or her own colorwriting on the wall graph.The wall graph sessions

were important for all to realize the complex, coop-erative work involved in the life span of a produc-tion. It formed a coherent picture of the cooperativeaspects of their work. Hence, the wall-graph servedas a reference in the succeeding discussions concern-ing possible computer support.

In order to revise and refine the design, we con-ducted two sessions with the secretaries discussingdetailed functions and data in the system, screen lay-out, and so on. This was followed by a visit to aninstitution using a standard system supporting reg-istration and file/project management. This visit wassucceeded with a demonstration and discussion atthe computer company which offered the system.

Finally, we wrote a design report and had prepara-


Practitionerscan benefitfrom using

ethnography incontextualdesign, but

they must be aware of theconditions

needed for suchan approach.

tory meetings with the president and the productionmanager before we presented the final report to theEditorial Board and the technology committee. Thereport suggested a revised version of the two systemsoriginally suggested.

Effects from Using Ethnographyin the Design ProjectAnalyzing the work in Part Two led to two essentialfindings which, again, had several impacts on thefinal design proposal:

We realized there was a difference in how a production wasperceived by the secretaries and by the editors. To the sec-retaries, a production startswhen the editor decides to fundit (from “negotiate contract,” asshown in Figure 1). Besides cor-respondence, they mainly takecare of a production from thepoint where the contract wasmade and hence a productionbecomes relevant (in terms ofbeing cumbersome/problematicand, therefore, considered a can-didate for computer support)after an editor has decided tofund it.

To the editors, the main con-siderations and decisions occurbefore it reaches this status.

We realized there was a power-struggle between the productionmanager and the editors. The edi-tors are responsible for decidingwhich projects to fund and byhow much. The productionmanager is responsible for theoverall budget, including con-siderations about whether the total budget for a pro-duction looks sound and realistic. The main concernof the production manager, who holds a permanentjob, was the total amount of productions the organi-zation could handle simultaneously, as well as ensur-ing that each production was sufficiently fundedfrom the very beginning. The production managerwants fewer productions to be funded with moremoney.

The editors, hired for just 2 or 4 years, want asmany of their preferred applications as possible tobecome productions. Since they are recruited fromthe film milieu to which they usually return, theyalso had to take into account their reputation in that

milieu, thus preferring to give many producers anddirectors a possibility to produce films. In informaltalks this was referred to in terms like “unavoidableincestuous relations,” unavoidable due to the size ofthe film and video industry in question.

The first finding led to the following impacts:

• Support of the editorial process, where our firstdesign mainly supported the production process(see Figure 1).

• Allow all applications (also those refused) to berecorded. This provided support for the editors,such as allowing a new editor to check if a similarapplication had been considered by a predecessor.

The first design did notconsider applicationsrefused by the editors.

• Record data about “whohas funded which kinds ofproductions” giving editorssupport in fundraisingactivities.

• Involve the Registry Office(the department that han-dles incoming mail and thecentral files), allowing theproduction (or the applica-tion as its status is at thistime) to be recorded whenthe first mail is received.The first design did notinvolve the RegistryOffice.

• Require the design to beportable, as the editors arefrequently “out of thehouse.” The first designdid not take this issue intoaccount.

The difference in perception of what constitutes aproduction between the secretaries and the editorswas found through observation and elaborated in thewall-graph sessions. The difference was harmoniousin the sense the functionality needed for the editorscould easily be added to those functions needed bythe secretaries.

The second finding divided the financial systeminto two parts—a private part for the editors and apublic part for all employees in the Editorial Board.

Financial support of productions considered bythe editors should be strictly confidential. None ofthe editors’ personal calculations (about which pro-ductions they were considering to fund and with


This projectdemonstrateshow multipleviewpoints onwork practices

may be harmonic or

problematic interms of

consequencesfor different

design options.

how much) should be public unless made so by theeditor managing the production. If this had not beenincluded in the design, the editors simply would nothave used the system for this complex task, and thefinancial part of the production would only havebeen supported by the system after the final decisionto fund it had been taken.

The difference in viewpoints between the produc-tion manager and the editors was found through ourobservations and elaborated in succeeding inter-views. For example, one of our videotapes shows theproduction manager trying to ensure a production inquestion was sufficiently funded, eager to increasethe support, while the editor was reluctant to do so,because it would be difficult for the editor to supportother projects currently under consideration.

This difference was more problematic and chal-lenged our role as neutral experts. It was not possibleto allow the editors to keep the current amount offunding for productions under consideration tothemselves and, at the same time, to allow theproduction manager, who had the responsibility forthe total budget, to have access to the same data. Werealized this contradiction was crucial to the designof the financial system. Either the system is open toall (and that means supporting the production man-ager, as in our first design), or it allows the editors towork with their budget in confidence.

We brought up this conflict by proposing aredesigned system supporting the editors in budget-ing the productions, and allowing the productionmanager access to the data only when the editors havedecided to make them public. Therefore, this wouldpotentially reduce the production manager’s influ-ence. The situation was tricky since few of the editors,now and in the past, had been good at estimating pro-duction costs. The job of production manager wascreated for that reason. The current editors agreed tothe need for such a job, however they expressed con-cerns that the production manager implicitly wouldtake over part of their responsibility.

The production manager ended up agreeing tothis proposal. It was, however, quite controversial forsome time. Indeed, at one point, it led the produc-tion manager to suggest to the president that ourdetailed analysis of their work should be brought toan end.

Lessons LearnedThis project demonstrates how multiple viewpointson work practices may be harmonic or problematicin terms of consequences for different design options.Using ethnography did result in specific changes ofour first design proposal. To some extent, that was a

surprising result, as both we and the users found thefirst design proposal very appropriate. Thus, it servesas a concrete example of how an understandingdeveloped by using ethnography may challenge animmediate understanding developed mainly throughmeetings, interviews, and document analyses [2].

The first design proposal did not offer muchdirect support to the editors. They did not have anyideas as to what kind of IS support they needed andthey accepted the first design proposal. However, ourobservations in the second part of the projectrevealed that editors could benefit from a redesignedproduction management system as well. Also, thepublic access to the financial data in the first designproposal was redesigned as it could have lead to a sit-uation where parts of the system would not havebeen used as intended.

Observations, in general, had the effect of generat-ing immediate questions for later interviews and pro-vided us with an experience of their work whichformed a qualitative input to succeeding interviews.The point is to be present when things happen andnot only to have things referred after they have hap-pened. The observations unveiled and illuminated theamount and complexity of the work performed by theeditors, such as before an idea for a production reachesthe process of negotiation of the contract and theirstruggles with fundraising. Such concrete experienceswith their work provided a substance and richnessthat developed the interviews into mutual dialoguesand discussions. It was through such additional andsubstantial discussions that the conflict with the pro-duction manager was conceptualized.

Time was also an issue. The fact that Part Two ofthe design project was performed during a period ofapproximately three months is also significant. Itgave us time for developing the insight into the edi-tors work, thinking through different design possi-bilities, and discussing and reflecting designproposals against current work practices.

Summarizing these experiences, two lessons mightbe learned from the project: Firstly, designers mayhave to observe users while they are involved in theireveryday activities. Observations may be necessary inestablishing a mutual learning process with users,aiming towards a shared understanding of the currentwork practice and in developing realistic visions offuture use of computers. Secondly, using ethnographymay unveil users’ multiple viewpoints on the currentwork as well as on future use of computers. Multipleviewpoints might be harmonious or problematic interms of the possibilities of integrating them in acoherent system. In the case of conflicting view-points, leading to different design solutions, design-


ers should bring up the conflict and its consequencesin terms of different design proposals.

Conditions for Using Ethnographyin Contextual DesignSince the design was conducted as part of a researchproject, a relevant question is what conditionsshould be considered for using ethnography in com-mercial design projects? Would a consultant in acommercial situation be provided with the possibil-ity, time, and resources to conduct such types ofstudies and analyses? Ethnographic approaches arerather unknown within information systems designin industrial settings. For an organization to investa relatively large amount of time and resourcesdoing observations of current work practices, thefollowing preconditions should be required at thevery least:

• The designers and the user organization musthave a positive attitude towards investing neededresources, and these resources must be available.When using ethnography, you may not know inadvance what effects it will have on the finaldesign. Investing resources in such an approachrequires that the organization has its own positiveexperiences with it, is provided with experiencesfrom others, such as in the form of convincingexamples, and/or has the resources to do it as anexperiment.

• Using ethnography in contextual design meansgetting very close to the current work practices ofthe users in question. In order for the users toaccept and participate, they must be confidentwith the overall purpose of such an approach.Therefore, a requirement might be that the pur-pose is to support the existing workforce ratherthan attempts to de-skill users’ work and/orattempts to reduce or replace their function andcompetence.

• The designers must have the competencies toconduct such an approach and to handle the (pos-sible conflicting) situations that such an approachmay imply. Alternatively, external help might beneeded.

• The designers and the user organization must beable to identify potential domains in terms ofwork practices, where applying resource demand-ing ethnographic techniques seems appropriate inrelation to systems design. The Film Board’s over-all function in relation to the directors and pro-ducers of the film milieu was a guiding factor inthis case. But the need for general guidelines forsuch a zooming method remains. [4].

Acknowledgements

We are especially thankful for useful comments onearlier versions of this article from the Design Stud-ies Group at Xerox PARC, the CORPS Group at UCIrvine, and the reviewers.

References

1. Blomberg, J., Giacomi, J., Mosher, A., and Swenton-Hall, P. Ethno-graphic field methods and their relation to design. Participatory Design:Principles and Practices. D. Schuler and A. Namioka, Eds. Lawrence Erl-baum, London, 1993, pp. 123–155.

2. Bødker, K. and Kensing, F. Design in an organisational context—Anexperiment. In Scandinavian J. Info. Syst. 6, 1 (1994), pp. 47–68.

3. Communications of the ACM. Special Issue: Participatory Design 36, 4(June 1993).

4. Goguen, J.A. and Linde, C. Techniques for requirements elicitation.In Proceedings of the IEEE International Symposium on Requirements Engi-neering. (Jan. 1993, Los Alamitos, Calif.). IEEE Computer Society Press,New York.

5. Hughes, J.A., Randall, D., and Shapiro, D. From ethnographic recordto system design: Some experiences from the field. Computer SupportedCooperative Work (CSCW). Kluwer, Dordrecht, The Netherlands, 1993,pp. 123-141.

6. Hughes, J.A., King, V., Rodden, T., and Andersen, H. Moving outfrom the control room: Ethnography in systems design. In Proceedingsof the Conference on Computer Supported Cooperative Work. (Oct. 1994,Chapel Hill, N.C.). ACM, New York, pp. 429–439.

7. Kensing, F., and Munk-Madsen, A. Participatory Design; Structure inthe Toolbox. Commun. ACM 4, 36, 1993, 78–85.

8. Kensing, F. Simonsen, J., and Bødker, K. MUST—A method for par-ticipatory design. In Proceedings of the 4th Biennial Conference on Partic-ipatory Design. (Nov. 1996, Boston). Computer Professionals for SocialResponsibility, Palo Alto, Calif. 1996, pp. 129–140.

9. Plowman, L., Rogers, Y., and Ramage, M. What are workplace stud-ies for? In Proceedings of the 4th European Conference on Computer-SupportedCooperative Work. (Sept. 1995, Stockholm). H. Marmolin, Y. Sundblad,and K. Schmidt, Eds. Kluwer, Dordrecht, The Netherlands, pp. 309-324.

10. Simonsen, J. Designing systems in an organizational context: An ex-plorative study of theoretical, methodological, and organizational is-sues from action research in three design projects. Writings in ComputerScience. Roskilde University, Denmark, 1994.

11. Simonsen, J. and Kensing. F. Take users seriously, but take a deeperlook: Organizational and technical effects from designing with anethnographically inspired approach. In Proceedings of the 3rd BiennialConference on Participatory Design. (Oct. 1994, Chapel Hill, N.C.). Com-puter Professionals for Social Responsibility, Palo Alto, Calif.

12. Suchman, L.A. Plans and Situated Actions: The Problem of Human-Ma-chine Communication. Cambridge University Press, Cambridge, Mass.,1987.

Jesper Simonsen ([email protected]) is an assistant professor inthe computer science department at Roskilde University, Denmark.Finn Kensing ([email protected]) is an associate professor in thecomputer science department at Roskilde University, Denmark.

Part of this study was made possible by financial support from the actual organization,the Danish Research Academy, and the Danish Natural Science Research Council.Authors were on leave from Roskilde University at CSLI/Stanford University andXerox PARC for the major part of the time this article was produced.

Permission to make digital/hard copy of part or all of this work for personal or class-room use is granted without fee provided that copies are not made or distributed forprofit or commercial advantage, the copyright notice, the title of the publication andits date appear, and notice is given that copying is by permission of ACM, Inc. Tocopy otherwise, to republish, to post on servers, or to redistribute to lists requiresprior specific permission and/or a fee.

© ACM 0002-0782/97/0700 $3.50c


ethnography using - chalmers · experimented with ethnography in design [2, 10, 11]. we have...

Documents