From graph paper to digital research: our Framework journey

Kandy Woodfield Director of Learning NatCen Social Research

©NatCen Social Research 2014

Overview

NatCen Social Research – who we are and what we do

Qualitative research in applied social policy research

Framework and NatCen’s approach to QR

Big journeys begin with small steps - our Framework journey

The impact of QDAS on analysis

2

We’re here because social research has the power to make life better

4

The areas we specialise in

Research at NatCen

Deliberative

Understanding the context

“(Funders) have certain requirements of the research they commission… that evidence is systematically generated and analysed, with interpretations that are well-founded and defensible and able to support wider inference. It also means that emphasis is placed on research findings which are accessibly presented and sufficiently focused to inform policy planning and implementation.”

(Ritchie et al, 2014)

Shifting sands in applied social policy research

Long struggle for credibility, by 2012: 61% of commissioned studies included qualitative elements - 18 used only/mainly qualitative methods A further 35 used a combination of qual/quant

Almost routine use of qualitative approaches in evaluation (Jago, 2014 forthcoming)

7

The black box…

Our approach

Applied vs. purely theoretical

Draws on varied traditions to meet requirements of clients

Characterised by:

9

Critical or subtle realism (e.g. Robson, 2002; Hammersley,1992)

Interpretivism Pragmatism Use of Framework for qual data management

Reflexivity Rigour Inferential status

Learning curves with QDAS

10

The Tippex years

1980s

Late 1990s

2008

2011

11

Aim of analysis - analytical outputs

Categories of things (mapping, description, thematic analysis) • Reasons for gambling • Sources of debt advice • Attitudes to the environment

Categories of people or processes (typologies) •Types of gambler •Types of school programmes •Grant application processes

Why people do or think what they do (explanatory analysis) •Factors influencing how people feel about income in retirement •What helps achieve positive outcomes from an employment programme

Theory development

What is Framework?

Framework - a tool not a method

In substantive tradition, concerned with: capturing and interpreting meanings in the data focusing on what the text says

“Data are treated as windows on the participants’ social world, referring to and representing feelings, perceptions and events.” (Ritchie et al, 2014)

Framework meant to assist in the ordering of data to facilitate interpretation

15

16

Objectives of data management

Primary objectives

Re-order tangled discourse

Make data accessible

Secondary objectives

Reduce data volume

Prioritise questions

Key features

Case and theme based approach (to enable

cross & within case analysis) Reduces data through summarisation &

synthesis Matrix display of summarised data Retains links to original data Output allows comprehensive and transparent

data analysis

Framework - case and theme based

Name

1.1 Nature of crime 1.2 Setting /locale

1.3 Impact on day to day life

John

John

Ellen

Paul

Chart 2: Making contact with Police

Chart 3: Experiences of police help and support

Chart 4: Impact of engagement with Police

Chart 1: Exp. of crime

How do you choose your

themes?

What goes in the cells?

Value of Framework matrix output

Data displayed in descriptive ‘chunks’ aids enquiry-focussed intepretation

Matrix preserves context - aids search for explanation

Links btwn cognitive processes & visual display

Data organised, but reduced

Encourages display of diversity

Great for teamwork

Systematic

Comprehensive

Transparent

Analytical process is ...

Limitations

Time and labour intensive

Danger of becoming process rather than outcome focused (as with coding)

Need to be reflexive and critical – not forcing square pegs into round hole

Not great for exploring narratives or linguistics on it’s own

Outline

25

In 2006 we tried again Commissioned Cordsoft & experienced project

manager Followed a rigorous approach to PM and

development Specified what we wanted to the software to

do, how and why

In 2008 we launched our own software

Developing software

Huge respect for software developers in this field

Development forced us to reappraise what Framework was and was for

What do we do and why?

What would we like to be able to do better?

Adopt a more methodologically flexible approach now vs tightly controlled

Development work made us reappraise where Framework sits alongside other tools and raised the quality of our methodological discussions

29

Benefits of using software Practical – formatting and reliable handling of volumes of data

Positions Framework alongside other approaches to data management

Eases tension between abstraction to get perspective and remaining close to the data

Analysts given back freedom to choose whatever approach suits their research study – plurality

Aids creativity by supporting iteration up and down the analytical ladder from descriptive to theory building

Keeps pace with technological change i.e. social media data

What impact?

30

I'm a fan of pen and paper; getting post-its out &moving

them around; sharing opinions etc. This isn't really something that you're able to

do when working in any CAQDAS but that I feel can

give an added benefit to thinking creatively in qual

analysis.

It is not a substitute

for thinking!

With NVivo I don't have to stick so rigidly to the model, but can adapt different techniques

depending on the needs of a particular project. I think there's a lot of functionality

people aren’t using.

Framework can make you more creative in the interpretative stage of analysis because it

allows you more time to analyse, and it allows you to compare

cases effectively while making sure not to get lost in the details.

Key challenges

Time needed – learning and development Need champions and experts to support skills development and growth

Getting people beyond the core functionality limited uses of advanced features like visualization, queries to develop creativity in the search for explanations and understanding

Encouraging creative and intelligent use of the tools available Horizon scanning

31

Learning curves

32

The Tippex years

Where next?