Evaluation of New Care Models Overall approach Charles Tallack,

Head of Operational Research and Evaluation, NHSE

1

Evaluation to support the aims of the programme../1

Widespread adoption of

new models of care which

have positive impact

Models of care that improve outcomes

and efficiency

Models of care that improve outcomes

and efficiency

Disseminated and taken-up

2

Improved health &

wellbeing, care & quality and efficiency,

at scale

Evaluation to support the aims of the programme../2

Well designed models

Based on sound

theory of change

Evidence- and best-practice-

based

Rapid cycle evaluation and improvement

Evaluation shows improvement in outcomes

and efficiency

Against local

metrics

Against national outcome metrics

Models of care that improve outcomes

and efficiency

Well implemented models

3

Evaluation to support the aims of the programme../3

Knowledge of ‘active ingredients’

Role of context

Interventions used and part

played

Codification and transmission of knowledge

Knowledge of implementation

Models of care that improve outcomes

and efficiency

Disseminated and taken-up

4

Evaluation to understand the elements of the new care models, how they are implemented, what effects they have, for whom, how and why

To develop and describe new care models, for implementation across the country, which will improve health outcomes and efficiency

NCM aim

Evaluation aim

Evaluation phases and questions to be answered

Evaluation needed across the life-cycle

During planning and set up (pre April 2016)

During implementation and model development

After implementation is complete

5

Evaluation phases and questions to be answered

During planning and set up (pre April 2016)

During implementation and model development

After implementation is

complete

• What is the context?

• How was model developed?

• What interventions are planned?

• What enablers are needed?

• What impacts are expected?

• Is implementation as planned? Barriers?

• What is working and not working, and why?

• What are the early results: outcomes, numbers of people affected, changed processes?

• What is the final model?

• What have been the outcomes and impacts?

• What are the key elements of success? 6

Evaluation to understand the elements of the new care models, how they are implemented, what effects they have, for whom, how and why

To develop and describe new care models, for implementation across the country, which will improve health outcomes and efficiency

Monitoring of impact comparing national outcome metrics with counterfactual

Monitoring of progress against outcome, output and process metrics

Interviews, qualitative and quantitative research to establish the “how” and “why”

NCM aim

Evaluation aim

Evaluation phases and questions to be answered

Evaluation activities

Evaluation needed across the life-cycle

During planning and set up (pre April 2016)

During implementation and model development

After implementation is complete

7

Local and national evaluation need to be complementary

National Local

• Monitoring and reporting against core metrics for each vanguard

• Establishment of matched controls for key interventions

• Specification of common elements of local evaluations

• Analysis and synthesis of local data and intelligence

• National “summative” evaluation

• Data collection (e.g. through surveys of patient experience, staff experience)

• Fact finding (e.g. where are interventions being implemented)

• Qualitative and quantitative research and analysis

• Progress tracking, monitoring and reporting against locally defined metrics

8

Proposed oversight and joint working groups

Evaluation oversight group Vanguard reps + ALB NCM rep +

patient rep + clinical rep + independent evaluation expert

Patient experience

measurement

Core metrics and national

reporting

Staff experience measurement

Local evaluation support & approach

Patient experience measurement

Quality of life for patients with

LTCs

9

Better health and care for Sunderland

The power of evaluation!

The story of our care homes pilot evaluation and the post

code saga!

Penny Davison,

Senior Commissioning Manager, NHS Sunderland CCG

Care home pilot and the post code saga! • Delivered Enhanced care in care homes from December 2013 – March 2015

• Carried out evaluation throughout the pilot.

• Learned significant amount along the way!

• How evaluation helped us in new model design

Reduction in emergency admissions and A&E

attendances Significant reduction in

out of hours callouts and GP callouts

Communication has vastly improved

Improvements in clinical effectiveness and patient

safety

Reduction in admissions for foot ulcers

An increase in the number of people supported to die in their preferred place

Improvements in patient experience and satisfaction

Pharmacy contribution – improved patient

safety and saving money

Clinicians feel better equipped to care for the frail and elderly

Outcomes

Power of evaluation! During the pilot

• Focus on innovation

• Create a learning environment

• Gain momentum and support

• Understand our key ingredients of success

Rolling Out the pilot to rest of Sunderland

• Build a model for the rest of Sunderland

• Apply lessons to people at home

Local Evaluation • On going formative evaluation

• Logic model - for planning and project management

• Programme level Outcomes

• Understand our key ingredients of success