+

Mixing science and intuition: the process of synthesising data from a longitudinal mixed methods study of volunteering

Rose Lindsey and Liz Metcalfe, University of SouthamptonThird Sector Research CentreESRC grant no. ES/K003550/1

+Presentation aim

To explore the challenges encountered when combining longitudinal qualitative and quantitative secondary data to study volunteering across time

Key challenge: What do we mean when we talk about synthesising, integration, combining, mixing, interweaving, blending, merging…? (Bryman, 2008)

Do we think our methods of combining have actually worked?

+Presentation outlinePart 1: Designing the projectIntroduction to Continuity and Change projectDiscussion of the methodological challenges faced within the mixed-method research design

Bringing different data sources and findings into dialogue

Part 2: Challenges in practiceExploring the analytical challenges faced when putting the design into practice:

Working across methodological paradigmsUnderstanding the effect of, and working across, time

Design versus practice

+Part 1: The Continuity and Change Project Aim: To explore individual attitudes and behaviours

towards volunteering, and individual views on the role and responsibility of the state towards provision for social need, across a period of thirty years.

Design: Concurrent use of longitudinal mixed-methods to analyse secondary data

Time-frame: 1981-2012, encompassing different periods of economic adversity and prosperity

Project website: http://longitudinalvolunteering.wordpress.com

+Choice of secondary data sets

The Mass Observation Project Aim: to capture experiences,

thoughts and opinions of individuals

A national panel of volunteers writing in response to themed questions or ‘directives’ (1981 to present day)

Longitudinal data following the same people through thirty years of their life-course

British Household Panel Survey/Understanding Society Aim: to understand individuals’

and households’ social and economic change

A national panel of the British population and volunteers (1991 to 2012)

Longitudinal data British Social Attitudes Survey

Aim: to track people‘s changing social, political and moral attitudes

A national survey of the British population (1983 to 2012)

Cross-sectional data

Qualitative data Quantitative data

+Why did we use mixed-methods?Enhancing strengths and offsetting weaknesses

Study strengths Study weaknessesQualitative

•Provides depth and nuance relating to the complex reasons why people behave in a certain way, or hold particular viewpoints

•Offers potential insights into how and why perspectives change or continue over time

• Enables insights into the connection between the life-course and routes into volunteering

• Not representative of the population

•Too much data

Quantitative

•Representative of volunteers within the population

•Can formally test how volunteering behaviour and attitudes change over time

•Offers potential insights into contextual re external events affect on change or continuity over time

•Insight into motivations and barriers are limited

•Limitations re understanding how individual time and the life-course affect volunteering

Our research design aimed to potentially ‘offset’ the respective weaknesses of these two analytical methodologies by taking

advantage of their joint strengths to provide a ‘complete[ness]’, and ‘comprehensive’ picture (Bryman, 2008,

p.91)

+Multi-layered picture of volunteering behaviour

Contextual: social, economic and political events over time

Behaviour and attitude analysis for volunteers within the population

In-depth analysis of individual volunteers

Sam

ple

size

dec

reas

es

Focu

s on

indi

vidu

als

incr

ease

s Contextual: social,

economic and political events over

time

In-depth analysis of individual volunteers

+Bringing secondary data sources, analyses and findings into dialogueWe aimed for three types of mixed-method dialogue:

across the lifetime of the project, described by Tashakkori and Teddlie (2008, p.104) as a ‘continuous feedback loop’, to enable an iterative research process;some direct comparisons between qualitative and quantitative analyses where there was a fit between the data; combining substantive findings so that the sum of our joint knowledge claims would be greater than our individual findings

Qualitative data

Quantitative data

Project beginning Project end

Substantive

findings

+Study design challenges-sample fit The Mass Observation

Project (MOP) 15 directives (sets of

questions) were selected N=38 2 samples were taken to

provide a range of ages and occupations

Sample 1, n=20 were writers from 1981 to 2012

Sample 2, n=18 were younger and wrote for shorter periods of time

Sample restricted to available volunteering data (every other year between 1996 to 2011)

N=2067

The Mass Observation Project British Household Panel Survey/Understanding Society

British Social Attitudes Survey

Questions of volunteering only asked a limited number of times

Number of people each year mean (sd) 3393 (711.7)

+How the three datasets complement each other, temporally and thematically

+Part 2: Challenges in practice

Three main challenges were present throughout the project:

Working across methodological paradigms

Understanding the effect of, and working across time

Putting the design into practice

+Working and communicating across methodological paradigmsWorking across methodologies we encountered some challenges:

Differences in terminology

Forming definitions

Timings/speed of analysis

Methodological standpoints: differences in the types of questions that are being addressed

Conceptions of time

+How time fits togetherThe way that these multiple perceptions of time interact and intersect (or not) was at the heart of the mixed methods effort for our research project.

the flow of personal biographical, narrative, retrospective, life-course, individual time

chronological time, moving from one year to the next

contextual public/collective time related to chronological time

Marriage

Children

Retirement

Double-dip recession

Recession

Recession

Hi, I’m Sarah

1981 1984 1987 1990 1993 1996 1999 2002 2005 2008 2011 2013

+Multi-layered picture of volunteering behaviour

Biographical time

Chronological time

Contextual timeChanges in social, political and moral attitudes over time

Behaviour and attitude analysis for volunteers within the population

In-depth volunteer analysis

Sam

ple

size

dec

reas

es

Focu

s on

indi

vidu

als

incr

ease

s

+Design versus practice

Longitudinal mixed-methods are more complicated than a single methodological approach

Over-estimation of mixed methods: It has not been possible to answer all of the designed research questions with the data chosen, the fit of the samples and the timing of the analysis

Did we achieve our mixed method dialogue?

Paradigm, background, and terminology differences make maintaining a mixed-method dialogue difficult

How time fits together: in practice time does not relate directly between different methodologies

Has the project benefited from using mixed-methods?

+References

Bryman, C. (2008) ‘Why do Researchers Integrate/Mesh/Blend/Mix/Merge/Fuse Quantitative and Qualitative research?’, in M.M. Bergman (ed.) Advances in Mixed-Methods Research, London: Sage. pp 87-100.

Tashakkori, A. and Teddlie, C. B. (2008) Quality of Inferences in Mixed Methods Research: Calling for an Integrative Framework in in M.M. Bergman (ed.) Advances in Mixed-Methods Research, London: Sage. pp.101-119

+Thank you for listening, any questions?

Contact details: R.Lindsey@soton.ac.uk E.Metcalfe@soton.ac.uk

Project website:

http://longitudinalvolunteering.wordpress.com/

+

Back-up slides

+The challenges of analysing secondary data over timeQuantitative:Variations in data collection process were difficult to uncoverThe questions that were asked limits the data availableData collected is set within the present time, only part of the life-course is recorded

Qualitative:Inconsistent descriptions of the life-course at different time-pointsLack of awareness of the what is happening within the present timeAccuracy of retrospective writings

+Study design challenges

The mixed-method design framed the study, and influenced how well the data sources fitted together. Compromises around the following choices needed to be made:

Choice of secondary data sources Choice of timing of analyses Choice of samples and how these substantively

fit together Choice of samples and how these fit together

across time (thematic and temporal bunching)

+Concurrent mixed method designOur research design aimed to potentially ‘offset’ the respective

weaknesses of these two analytical methodologies by taking advantage of their joint strengths to provide a ‘complete[ness]’,

and ‘comprehensive’ picture (Bryman, 2008, p.91)

Qualitative data

Quantitative data

Project beginning

Project end

+Cross-sectional or Longitudinal?Synchronic or Diachronic?

1981 20131984 1987 1990 1993 1996 1999 2002 2005 2008 2011

Marriage

Children

Retirement

Double-dip recession

Recession

Recession

The length of chronological time being researched affects our perceptions and understandings of behaviour and attitudesLongitudinal/diachronic: following a person through timeCross-sectional/synchronic: A certain point in time

Do not volunteer

Volunteered

Increasing age

Hi, I’m Sarah