ethnography on and in software engineering

Ethnography on and in Software Engineering

© deSouza, Dittrich, Sharp

Practical Reflections


Roadmap

• History• Action Research in IS• Cooperative Method Development


Empirical Research - qualitative or quantitative - aims at understanding

BUTSoftware Engineering aims doing things better


Software process evolution at the SEL

• Do small scale experiments first.• Do bigger scale experiments in a more

realistic environment.• Try to implement it in practice.

© Basili, V. Greeen, S. (1994). Software Process Evolution at the SEL. IEEE Software, 58-66.See also: V. R. Basili, G. Caldieri, H.D. Rombach, “Experience Factory”, Encyclopedia of Software Engineering Vol.2 Editor J. Marciniak, John Wiley and Sons Inc., 1994, pp 528-532.


Action Research

The research needed for social practice can best be characterized as research for social management or so-cial engineering. It is a type of action-research, a compar-ative research on the conditions and effects of various forms of social action, and research leading to social action. Research that produces nothing but books will not suffice.

© K. Lewin, "Frontiers in Group Dynamics," Human Relations 1947 (1) 2, pp. 143-153.


Kurt Lewin’s approach and ‘participatory action research’


Participatory Action Research

• The participants are part of the (self-)reflective enquiry.

• Interventions and changes are designed together with the participants.

“Action research is not a ‘method’ or a ‘procedure’ for research but a series of commitments to observe and problematise through practice a series of principles for conducting social enquiry.”

© McTaggart, R. (1996). Issues for participatory action researchers. In O. Zuber-Skerritt (ed.),New Directions in Action Research. 242-255.


Checkland’s Action Research Cycle

© Checkland P, Holwell S (1998) Action research: its nature and validity. Syst Pract Action Res 11:9-21


Action Research in and on Software Engineering

First experiences:• We entered the organisation as software engineers. The

practitioners expected recommendations!• It became important to whom we reported. We had to

answer for our action, even as we only observed.(Suchman, L, 1995. “Making Work Visible” Comm. Of the ACM 38, 9, 56-64)

• Who are our members/partners, and what do we base our recommendations on?

Solution: Making the own role explicit!


Helen’s Practical Considerations• Sifting the important from the irrelevant

– In some senses nothing is irrelevant, but need to focus– Have a set of questions or themes to focus on– Refine focus as you go through.

• Avoiding judgment– Non-judgmental, treat everything as ‘strange’. – Treat the collaborator as the expert. – Maintaining this perspective is not easy.

• What am I doing here?– It is impossible to suspend who you are, your training, your experiences

and your expectations. – Be aware of them and question your interpretations – Easy to feel uncertain of what you are looking for.

• Communicating our findings– Our collaborators had expected us to be judgmental and comparative.


Finding a Path between Understanding, Intervention and Method development

• We cannot study Software Engineering as software engineering researchers in the same way as social scientists can.

• We will be part of the internal and external politics of ‘how should software be developed’.

• That means that our choices regarding perspective, cooperation, and improvement will decide on what we are allowed to see.

• Methodological decisions and ethical considerations and decisions influence each other. (But that is probably not new…)

© Dittrich Y (2002) Doing empirical research in software engineering-finding a path between understanding, intervention and method development. In: Dittrich Y et al. (eds.) Social thinking-software practice. MIT Press, 243-262


Lars Mathiassen’s Reflective Systems Development

‘But when designing and organising research projects based on collaboration with practitioners the challenge is not so much which methods to choose. Rather it is to find practical ways to combine qualitatively different research approaches to support the diverse, and partly contradictory goals involved in such an effort.’

©Mathiassen L (1998) Reflective systems development. Scand J Inf Sys 10(1 and 2):67-118


Reflective Action Research…

… focuses on the relation between research and practice.

… provides a rather loose framework for empirical research that is addressing improvement of the same practice it is evaluating.

… gives a good overview over different ‘research interests’ and how they relate to the overall framework.


Cooperative Method Development

1. Action research consisting:– Understanding– Deliberating Change– Implementing and Evaluating Improvements

2. Ethnomethodological and Ethnographical Inspired fieldwork.

3. Focusing on Shop Floor Software Development Practices.

4. Taking the Practitioners Perspective when improving the practice.

5. Deliberating change with the Practitioners involved.

© Y. Dittrich et al. Co-operative Method Development - Combining qualitative empirical research with method, technique and process improvement. Journal for Empirical Software Engineering. 13 (2008), 231-260.


Case I: Design for Change together with the IT unit of a Telecom provider

participants: a telecommunication provider, a small software developer and the university .

case: a business application to administrate contracts and compute payments based on the contracts and triggered by certain events.

context: rapidly changing business practices require a flexible adaptable system.

methods and tools: a flexible meta-modelling database system allowing to manipulate the data model during operation.

main results: – applicability of technical solution depends not only on the functional

specification but also on use context, development context, and technical context of the application.

– How use-oriented development can take place


CMD applied Phase 1: Understanding Practice

– investigating understand the problems with the old system– participatory observation of the project

Phase 2: Deliberating change– workshops

• about software flexibility, • to evaluate different solutions, • about how the co-operation between developers and users actually

took place– building prototypes to demonstrate possible solutions

Phase 3: Implementing and Evaluating Improvements– participating in the project.– design evaluation


Case I: Method lessons learned

• Technical possibilities were confronted with real world requirements. Contextual influences on the selection of technology became visible.

(Guidelines 3 and 4)• Prototypes and thus Design played a major role in the research

process. (Guideline 1 and 5)• We experimented with mapping technologies to visualise de facto

development processes. (Guideline 2)


Case II: End-User Tailoring together with the same IT unit

participants: the same telecommunication provider as previously and the university .

case: End User Tailoring of IT-Infrastructure context: adaptable business applications require a flexible

communication with other databases.methods and tools: Metamodelling techniques.main results:

– End-Users can be enabled to tailor infrastructures when provided with a well designed application

– To provide for long term sustainability of tailorable system, EU tailoring has to be interlaced with software evolution. Cooperation between users, tailors and developers is important.


CMD appliedPhase 1: Understanding Practice

– Observing domain experts preparing data input for the contract handler

Phase 2: Deliberating change– Discussing the design of a prototype– Implementing a case based prototype– Evaluating the prototype as a case based prototype

Phase 3: Implementing and Evaluating Improvements– Observing the development project that implemented aspects of the prototype as part of a revision of the contract handler.


Case II: Method lessons learned

• We used Design Research as a complementary framework as technical innovation was involved. (Guideline 1)

• We included users in the observation and deliberation, as they were tightly cooperating with the developers. (Guideline 3 and 4)


Case III: Agile Development for e-Government

participants: – 2 small SE companies, one developing a booking system for sports

places, one developing custom application supporting web based communication for municipalities and citizens,

– several municipalities, – researchers from software engineering, human work science,

telecommunicationcase: development to support municipal agency (e-Government) context: designing e-government application involves designing

municipal services as well as designing the tools to support themmethods and tools:main results: Agile development useful to cope with continuous

change and experimental application domains.

This is about 2 similar projects


CMD appliedPhase 1: Understanding Practice

– Observing the development in 2 small se companies, Phase 2: Deliberating change

– Both companies were happy with their exisiting development practices.

Phase 3: Implementing and Evaluating Improvements– No improvements were implemented


Case III: Method lessons learned

• The empirical studies helped us to understand why the development was sustainable. (Guideline 2)

• As the participants were satisfied with todays practices, we did not implement any changes. (Guideline 3 and 4)

• We continued the research on the observed agile development practices with an interview based study, experimenting with complementary methods exploring genralisability of results. (Guideline 2)


Case IV: Interaction Design for UI framework

development for mobile devices participants: the Interaction Design group of UIQ developing a GUI

framework for mobile applications based on the Symbian OS, researchers from software engineering

case: lacking communication between, Interaction Designers and software engineers

context: simultaneous development of customer specific applications and a complex UI framework

methods and tools: Personas, representations of archetypical users.main results:

– The organisation of the business domain influenced the applicability of methods.

– The problems the method introduction should have addressed were solved by the introduction of a new interaction paradigm, reorganisation, and an attitude change.


CMD appliedPhase 1a: Understanding Practice

– Participatory observation of interaction design group– Interviews with other actors in the development process

Phase 2a: Deliberating change– Summarising research on Personas–Workshops on the Personas method organised together with the Interaction Design group for different actors in the company– ’Qualitative experiments’ with student projects.

Phase 3a: Implementing and Evaluating Improvements– No implementation took place, the reasons were elicitated through interviews

Phase 1b: Understanding Practice– New empirical study on the communication problems. They did not exist any more. The developments leading to this situation were documented.

Phase 2a: Deliberating changeNo change needed


Case IV: Method lessons learned

• Ethnography had been an adequate instrument to understand the influence on socio-political factors and method implementation. (Guideline 2)

• We complemented the deliberation of improvements with ‘qualitative experiments’ using student projects (Guideline 5)

• Cooperative writing as a form of ‘member checking’ (Guideline 2)


Summing up: CMD revisited

1. Action research can be complemented with Design Research, either as supporting the deliberation phase, or as a additional support to guide deliberation and implementation & evaluation.

2. Complementing with techniques that make software development visible. Co-authoring as an alternative method for member checking.

3. Focus on shop floor development practice rendered contextual influences visible.

4. Taking practitioner’s point of vies countered research bias.5. Supporting the deliberation of rather expensive changes with other

emprical methods, e.g. with ‘Qualitative Experiments’.

ethnography on and in software engineering

Documents