How can qualitative data be re-used?

Louise CortiESDS Qualidata

Secondary analysis of qualitative data: using Atlas-ti to explore archived sources

24 June 2004

Re-using data• Archived qualitative data are a rich and unique,

yet too often unexploited, source of research material.

• They offer information that can be re-analysed, reworked, and compared with contemporary data.

• In time, too, archived research materials can prove to be a significant part of our cultural heritage and become resources for historical as well as contemporary research.

• What then are the methodological, ethical and theoretical considerations relating to the secondary analysis of qualitative data?

Culture of re-use

• well-established tradition in social science of reanalysing quantitative data

• no logical intellectual reason why this should not be so for qualitative data

• however, among qualitative researchers no similar research culture

• lack of discussion of the issues involved in literature on the benefits and limitations of such approaches

• more now published, but more needed …!

What data?• often a diversity of methods and tools rather than a

single one are encompassed

• types of data collected vary with the aims of the study and the nature of the sample

• samples are most often small, but may rise to 500 or more informants

• as we have seen data include interviews, group discussions, fieldwork diaries and observation notes, personal documents, photographs etc.

• created in a variety of formats: digital, paper (typed and hand-written), audio, video and photographic

How then?

How people are re-using data is somewhat dependent on:

• what is ‘publicly’ available to suit research needs

• access and restriction policies• data formats – audio-visual is richer• amount of contextual documentation that

is available • time to get to know data well• analytic skills

Secondary analysis potential

• description

• comparative research, restudy or follow-up

study• augment data you collect e.g. expand sample

size

• re-analysis or secondary analysis

• verification

• research design and methodological

advancement

• teaching and learning

Description

• describing the contemporary and historical attributes, attitudes and behaviour of individuals, societies, groups or organisations

• data created now, will in time become a unique historical resource

• providing alternative sources (the people’s voice etc.) to the public record that will be deposited in archives

Comparative research, replication or restudy

• of original research

• to compare with other data sources

• to provide comparison over time or between social groups or regions etc.

• to follow up original sample

• verification - substantiating results, although we have yet to see any evidence of re-use for this purpose (might be useful in a teaching context though)

Re-analysis • secondary analysis

• asking new questions of the data and making different interpretations to the original researcher

• approaching the data in ways that weren't originally addressed, such as using data for investigating different themes or topics of study

• the more in-depth the material, the more possible this becomes

Research design and methodological advancement

• designing a new study or developing a methodology or research tool by studying sampling methods, data collection and fieldwork strategies and topic guides

• although researchers often publish a section on methods used, researchers' own fieldwork diaries can offer much insight into the history and development of the research

• encourage researchers to reflect on the researcher’s own experiences

Teaching and learning

• older 'classic' studies and more contemporary focused sets of transcripts can provide unique case material for teaching and learning in both research methods and substantive areas across a range of social science disciplines

• ESDS Qualidata can advise teachers and students on many aspects of using data resources in lectures and for self-study– providing a number of teaching datasets and

associated learning materials – training workshops and online materials

• We are always looking for partnerships with academics and researchers to create new resources!

Qualitative data re-use examples

• classic restudies include Rowntree's three surveys of poverty in York, and Llewellyn Smith's repeat of Booth's survey in London.

• also the two successive community studies of Banbury

• anthropological example - the controversial restudy and reinterpretation by Oscar Lewis of Redfield's Tepotzlan in Mexico

• material from Paul Thompson's national study of 'Family Life and Work Experience before 1918', has been re-used by over 100 researchers and students, providing the basis of a series of books and articles

‘Difficulties’ in re-using data • practice of secondary analysis of qualitative

data is not a commonplace research activity

• Why reluctance to draw on material created by other researchers?

– Is it that it is a problem of the implicit nature of qualitative data collection and analysis?

– Is it a question of lack of time to get fully acquainted with research materials created by someone else?

– How constraining is informed consent?

– Is there an insecurity about the exposure of one’s own research practice?

The evidence

Discussions with many qualitative researchers suggest that:

• views are by no means homogenous

• when asked what, if any, barriers existed to further exploitation of data by a secondary analyst, responses varied from

– overt support for sharing one's own data to

– vehement displeasure at the thought of being asked to share a 'possession' considered to be of personal value

Ethical and consent considerations

• questions of confidentiality and agreements made at the time of fieldwork

• archived data should always conform to ethical and legal guidelines with respect to the preservation of anonymity when this has been requested by informants or guaranteed to them

• achieve this by various strategies – see ESDS Qualidata web pages:

• anonymising the original data • restricting access/vetting• obtaining legal undertakings to protect

respondents’ anonymity • seeking views of respondents

Representation, coverage and context of the research and fieldwork

• In the process of analysing and coding data, researchers use their own personal knowledge and experiences as tools to make sense of the material, which may be indefinable and cannot be easily documented.

• Researcher’s own reflexivity enhances the raw data gathered and stimulates the formulation of new hypotheses in the field.

• Can data be effectively used by someone who has not been involved in the original study?

• How much of the jigsaw can be missing yet leave the puzzle still worth attempting?

Yes…but…• the loss of context in archived data should not be seen

as an insurmountable barrier to re-use

• there already exist common and accepted instances where research data is used in a 'second hand' sense by investigators themselves:

– PIs writing up the final analysis may not have been directly engaged in fieldwork, having employed research staff to collect data

– those working in research teams sharing one's own experiences of the fieldwork and its context are essential, but never total

– in both instances, THE analysers or authors must rely on fieldworkers and co-workers documenting detailed notes about the project and communicating them, and having access to original data e.g. audio materials

Unfamiliarity with the method

• because most qualitative researchers are unused to consulting data collected by other people, their concept of 'secondary analysis' is still typically associated with the 'number crunching' of survey datasets

• by contrast the positive experiences of researchers who have gone back to their own data or have embarked upon new investigations on archived data testify how fruitful the effort can be

Fielding and Fielding 2000 Akerstrom and Jacobsson 2004

Lack of infrastructure for data-sharing

• one of the barriers to re-using data in the past, and indeed in many countries still, has been the lack of an infrastructure to enable access to the rich research data collected in the academic community

• the growing establishment of national data-sharing policies is helping to secure data

• stock of data archived still rests on the willingness to share by the original investigators…

Threat of misinterpretation• concerns arise from fear of selective and opportunistic

interpretation in re-analysis

• researchers conducting longitudinal programmes feel particularly vulnerable due to the potentially negative impact of bad publicity on sample attrition

• concern surrounding the mis-use of politically-sensitive occupational or environmental data by pressure groups, industry, or the press

• straightforward misinterpretation by other academics is not often a problem - much more typical, and usually after a considerable time lag, is a serious critique of the original study

• But this is precisely how intellectual understanding advances! paradoxically, at the same time it may give the original study new attention and esteem

Threat to intellectual property rights

• some researchers have voiced concern over the loss of their control over data or their intellectual property rights when data become publicly archived

• some wish to complete working on the data before it can be offered to the wider community

• in the medical field, keeping data in-house and restricting access is thus one way of preserving intellectual capital

• other researchers consider their data as private property e.g. anthropologists may have built a career around studying one particular remote region, and the data generated over the course of a professional career will be seen as unique, as a stock of intellectual capital that he or she can exploit through their lifetime

ESDS Qualidata:enabling re-use

• enhanced user guides and digital samplers

• exemplars and case studies of re-use

• online access to qualitative data

• user support and training activities to support secondary analysis of qualitative data

Enhanced user guides and digital samplers

providing a better understanding of the study and research methods

• enhanced users guides – detailed notes on study methodology and re-use; 'behind-the-scenes' interviews with depositors; FAQs

• thematic pages – combining interviews

• digital samplers of classic sociology collections

Exemplars and case studies of re-use

providing guidance on data resources and how to re-use them

• overview of ways of re-using data

• case studies of re-use including reflections and commentary

• full bibliography of re-use articles

• 'packaged' training resources

• user support and training programme

Online access to qualitative data

• new emphasis on providing direct access to collection content– supports more powerful resource discovery

– greater scope for searching and browsing content of data (supplementary to higher level study-related metadata)

– since users can search and explore content directly… can retrieve data immediately

• providing access to qualitative data via common interface (ESDS Qualidata Online)

• supporting tools for searching, retrieval, and analysis across different datasets

Web pages on re-use

ESDS Qualidata

www.esds.ac.uk/qualidata/support

Email: qualidata@esds.ac.uk

AND…PLEASE TELL US ABOUT ANY

RE-USE YOU DO!!!