Evaluating the Headstart programme

Dr Jessica Deighton

Evaluation team Manchester team • Professor Neil Humphrey • Lawrence Wo • Kirstin Kerr • Michael Wigelsworth • Ann Lendrum

CommonRoom team • Kate Martin • Naomi Gilchrist • Amy Feltham

Advisors • Professor Peter Fonagy • Dr Chih Hoong Sin • Professor Angie Hart • Dr Nick Midgley

EBPU team • Dr Jess Deighton (lead) • Dr Miranda Wolpert • Jeni Page • Isobel Fleming • Praveetha Patalay • Julian Edbrooke-Child Fieldwork team • Mia Eisenstadt • Charlotte Payne • Ritika Chokhani • Sarah Clark • Stephanie Henwood

Evaluation Framework

How will partnerships achieve outcomes?

Evaluation Framework

3 main questions:

• What have partnerships done to increase resilience?

• Has resilience increased among children at partnership sites?

• How do these change relate to what partnerships have done?

What is the source that feeds these questions?

Evaluation Framework: Overview

What is the source that feeds these questions?

• Activity and process data

• Common measurement framework

• Qualitative fieldwork

• Fact finding

• Local evaluation information

An evidence pipeline

• Joining all strands of activity

• Quantitative and qualitative

• Local and national

An evidence pipeline

What have partnerships done to increase resilience? (Including taking account of local context)

– Activity and process data (TIDieRs, change Logs, project docs etc.) – Jan – July 2015

– Local evaluation summaries from each partnership

– Fact finding around support for children not in school

An evidence pipeline

Has resilience increased among children at partnership sites?

An evidence pipeline: Qualitative fieldwork

2 stages 1. Scoping (Feb to Jul):

– 2 site visits to each partnership – shadowing partnership staff/interventions – Drawing on relevant documents – Interviews with partnership staff, delivery staff,

and other stakeholders – Build a plan for stage 2

2. In-depth case study (Sept 2015-Jun 2016) – Evaluating one key intervention – 3 site visits to each partnership – Bespoke evaluation based on toolkit

An evidence pipeline: Common Measurement Framework

• 3 outcomes – Young people are better able to

cope in difficult circumstances and do well in school and in life.

– Building resilience helps to prevent the onset of common mental health problems.

– Learning from different approaches contributes to an evidence base for service re-design and for investment in prevention.

An evidence pipeline: Common Measurement Framework

4 Measures Measure Example questions

General well-being The Child Outcome Rating Scale

Me (How am I doing) Family (How are things in my family)

Mental Health Me and My School

I am unhappy I lose my temper I am calm

Resilience Student resilience survey

I can work out my problems I know where to go for help when I have a problems

QALY EQ-5DY

Doing usual activities I have no problems doing my usual activities I have some problems doing my usual activities I have a lot of problems doing my usual activities

An evidence pipeline: Common Measurement Framework

• Local evaluation findings

– Quantitative and qualitative

– If shared by April 2016

An evidence pipeline

How do these change relate to what partnerships have done?

– Linking CMF data with qualitative work and activity data

– Also local evaluations if shared

The evidence pipeline

Local evaluation summaries

– current

Activity and process data

– current-July 2015

Out of school facts

– Feb-Jul 2015 (with potential follow-up)

National quantitative (CMF) – Feb-May 2015 – Jan-April 2016 Local evaluation – By April 2016 Qualitative fieldwork – Scoping: Feb-

July 2015 – In depth: Sept

2015-May 2016