action for prisoners’ families seminar 25 june 2014 quality services: ticking all the boxes

Action for Prisoners’ Families Seminar 25 June 2014

Quality services: ticking all the boxes

Evaluation and family relationship measures

Anna Kazimirski

v

EVALUATION AND FAMILY RELATIONSHIP MEASURESANNE KAZIMIRSKI, NPC

Action for Prisoners’ Families – 25th June 2014

TRANSFORMING THE UK CHARITY SECTOR

4

NPC works at the nexus between charities and

funders

Charity

SectorFunder

Increasing the impact of charities

eg, impact-focused theories of change

Strengthening the partnership

Eg, collaboration towards shared

goals

Increasing the impact of funders

eg, effective commissioning

ConsultancyThink tank

WHY MEASURING IMPACT IS IMPORTANT

5

The Brandon Centre provides counselling and psychotherapy to

young people between the ages of 12 and 21

Influences the debate on

“what works”Improved services

Raises profile

Motivates staff

Taken from: Rickey, B, Lumley, T and Ni Ogain, E . (2011) A Journey to Greater Impact New Philanthropy Capital.

Helps secure funding

COMPONENTS OF NPC’S APPROACH TO MEASURING SOCIAL IMPACT

6

Map your theory of change

Prioritise what you measure

Choose your level of

evidence

Select your sources and

tools

Effective measurement framework developed

Strategic visionLeadership

Case for impact measurement

THEORY OF CHANGE

Links activities intermediate outcomes final outcomes

clarifies what the activities aim to achieve and how

provides a structure for identifying what can be measured

provides the case for why achieving intermediate outcomes is important

7

A conceptual map of how activities lead to outcomes

EXAMPLE THEORY OF CHANGE

8

Family therapy

Levels of conflict within families

reduced Observable

reduction in problem behaviours during

process

Offenders (and families) feel

empowered to take responsibility and

maintain gains

Strengthened family ties & relationships

Contribution to reduced reoffending

Reintegration into (non criminal) social & family

groups

OFFENDERS’ FAMILY RELATIONSHIPS

9

• Commissioned by NOMS to develop toolkit to measure intermediate outcomes of work to improve family & peer relationships of offenders

• Project going through peer review, to be published later this year

• Parallel project: RAND Europe, with ARCS (UK) and University of Glamorgan, focusing on arts and mentoring work with offenders

10

PROJECT PROCESS

Evidence review – identifying outcomes and toolkits

Consultation with providers & commissioners to select outcomes to focus on

Development and piloting of toolkit

Consultation with providers & commissioners to inform guidance

Analysis and reporting

DEVELOPMENT OF RELATIONSHIPS TOOLKIT

• Desistance theory:

• Link between creation/ investment in family relationships / reduction in peer pressure with reduced offending

• Consultation:

• Outcomes differed depending on the type of intervention & beneficiary

• Needed to reach a balance between being prescriptive and allowing for flexibility

• We researched existing scales and evaluations in each outcome area and prioritised:

• good validity and reliability; recently developed; brevity.

12

Toolkit: 25 standardised scales, to be used in paper questionnaires for offenders to complete.

For each outcome area: short general scales & longer specialised scales.

TOOLKIT STRUCTURE

13

Child relationships

Wider family relationships

RAND personal

development outcomes

Peer relationships

 Partner

relationships

Reduced reoffending

+

+ +

+

++

Parenting

OUTCOME AREAS

Partner relationships

Increased satisfaction with partner relationship, reduced conflict and improved commitment

Child relationships

Quality of relationships with children, children’s well-being and behaviour

Wider family relationships

Family functioning, resilience, conflict and communication, satisfaction with relationships in the family and quality of relationships

Parenting

Improved satisfaction with the parent-child relationship and parenting skills

Peer relationships

Satisfaction with peer relationships and reduction of negative peer relationships

14

CHARACTERISTICS OF TOOLKIT

• The questions can be used as monitoring or as part of an evaluation with a ‘before and after’ (pre/post) research design, with a comparison group where possible

15

Before measure

After measure

Programme / service

Before measure

After measure

“Intervention” group

“Comparison” group

RESOURCES

http://www.thinknpc.org/

NPC’s Wellbeing measure: http://www.well-beingmeasure.com

NPC’s papers on

How to approach evaluation: http://www.thinknpc.org/publications/npcs-four-pillar-approach/

Theory of Change: http://www.thinknpc.org/publications/theory-of-change/

How to communicate your results: http://www.thinknpc.org/publications/talking-about-results/

Profile of six charities who radically improved their approach to impact measurement: http://www.thinknpc.org/publications/a-journey-to-greater-impact/

http://inspiringimpact.org/

Code of Good Impact Practice

Online impact marketplace

anne.kazimirski@thinknpc.org

020 7620 4855

Twitter: @npcthinks, @AnneKazimirski

16

Action for Prisoners’ Families Seminar 25 June 2014

Quality services: ticking all the boxes

www.prisonersfamilies.org.uk @Prisonerfamily