THE GOOD, THE BAD AND THE EVIDENCE

Siân Jones, Head of Evidence & Effectiveness, Avon Primary Care Research Collaborative

Alison Turner, Head of Evidence Analysis, Midlands & Lancashire Commissioning Support Unit

@EvidentlySian @ali_pals

@EvidentlySian @ali_pals

Today’s session – what’s the story?

• NHS commissioning – an introduction• What do we mean by ‘evidence’ in commissioning• How decisions are made• Why it isn’t easy – the challenges• Discussion - real life example• Renewed impetus for evidence informed commissioning –

what next?

@EvidentlySian @ali_pals

Who are we?

@EvidentlySian @ali_pals

Commissioning – the what • A complex process of procuring health servicesthrough

• assessment of population health need• planning of services• purchasing services (limited budget) from providers• monitoring

• Ongoing activity• Diversity and complexity of services

@EvidentlySian @ali_pals

Commissioning is about…

• Doing the right things• Making the right choices and investment decisions and getting the best balance of services

• Doing things rightGetting the best value for money and achieving the desired outcomes

Beecham J, 2006 Beyond boundaries, citizen-centred local services for Wales, Cardiff: Welsh Assembly Government

@EvidentlySian @ali_pals

Commissioning – the who• Clinical Commissioning Groups (CCGs)

• Clinically led statutory bodies• Health & Social Care Act 2012• 209 CCGs across England• 2/3 NHS England budget: £71.9bn 2016/17

• Commissioning Support Units (CSUs) • finance, HR, contracting, data etc

• Public Health (Local Authority)

• NHS England – primary care; specialised services

http://www.kingsfund.org.uk/projects/nhs-65/alternative-guide-new-nhs-england

@EvidentlySian @ali_pals

Example questions in commissioning of services

• Which models of service delivery are the most appropriate?

• How should changes to services be implemented?

• What impact do new models of care have on relationships

between users and professionals?

• How can different health care professionals work in teams to

provide more integrated care to users?

@EvidentlySian @ali_pals

Evidence based medicine“The integration of best research evidence with clinical

expertise and patient’s values.”

Sackett DL et al; Churchill Livingstone,2000

EBM

Evidence informed commissioning

Strategic decision making – populationsSpring, B. & Hitchcock, K. (2009) Evidence-based practice in psychology. In I.B. Weiner & W.E. Craighead (Eds.) Corsini’s Encyclopedia of Psychology, 4th edition (pp. 603-607). New York:Wiley

Population needs/

demands

Finite resources

@EvidentlySian @ali_pals

Evidence and value based healthcarePopulation

needs/ demands

Finite resources

VALUE• that patient gains from own care• that whole population gains from investment in their healthcare

IMPROVE PRODUCTIVITY• all work differently & smarter• reshape services

Better value for every £ spent

To achieve more for lessTo create more value for patients

Need the best available evidence

@EvidentlySian @ali_pals

The danger is…‘There’s lots of commissioning work going on, but we don’t have much evidence about what works, not much analysis, and little real change in services.’

• Ineffective commissioning which is: • Not evidence–based• Has very little impact on service configuration• Does not engage stakeholders• Does not meet the future needs of the population

@EvidentlySian @ali_pals

‘Journey of evidence into commissioning’ Mobilising Knowledge

• Use of evidence is NOT simple• Not a linear process: from supply demand

Much more complex…

• Healthcare commissioning ‘most challenging context for use of evidence and best practice’ (Checkland et al 2011)

• Evidence is not ignored – incorporated into practice via ‘mindlines’ (Gabbay & Le May 2004)

Checkland http://www.netscc.ac.uk/hsdr/files/project/SDO_FR_08-1808-240_V01.pdfGabbay & le May http://www.ncbi.nlm.nih.gov/pmc/articles/PMC524553/

@EvidentlySian @ali_pals

What is “evidence” in this context?

ExpertiseEvidence from local

context

Critically evaluated research

Perspectives of those impacted

Decision

Experiential

Theoretical

Empirical

Adapted from: Briner, R. B., et al. (2009) Evidence-Based Management: Concept Cleanup Time? Academy of Management Perspectives, 19-32.

Adapted from Williams, I. and Glasby, J. (2010) Making ‘what works’ work: The use of knowledge in UK health and social care decision-making. Policy and Society, 29, 95-102.

@EvidentlySian @ali_pals

Why is evidence important?• Austerity in the NHS

• Increasing demand• Increasing financial pressures• “Wicked problems”

• Expectations• 5 Year Forward View• Scale and pace of change• Pressure to be innovative

• A complex environment• Complexity, volatility, uncertainty, ambiguity‡

• “Zone of productive distress”‡

• Tensions and trade-offs‡Ghate, D., et al. (2013) Systems leadership: exceptional leadership for exceptional times: synthesis paper. The Virtual Staff College.

@EvidentlySian @ali_pals

Variable use of evidenceOveruse Underuse Misuse/ variationOver use of health service interventions where they are not particularly effective

Health service interventions known to be effective but are not applied appropriately

Use of health service interventions for which the evidence base is unclear or leads to wide variation

Measurement of patient satisfaction using poorly conceptualised instruments

Lifestyle and behavioural interventions to improve health

Shifting services to out of hospital settings to save money

Adapted from Walshe, K. and Rundall, T. G. (2001) Evidence-based Management: From Theory to Practice in Health Care. Milbank Quarterly, 79, 429-457

@EvidentlySian @ali_pals

Exercise: scenariosPick one of the scenarios and list the types of evidence which might be needed in each case:

• What types of evidence might offer useful insights?

• Where would you find these types of evidence?

• Are there types of evidence which may not be appropriate for this scenario?

@EvidentlySian @ali_pals

The reality: making decisions• Decisions made by groups rather than

individuals

• A need to integrate multiple sources and types of evidence, to consider evidence “in its totality"

• “Variance in value”‡ with preferences towards practice-based and patient-generated evidence

• Context is important to help understand how evidence may apply locally

“People can’t argue with concrete evidence about what actually someone has experienced”

Bjorn Larsson, CC BY-SA 3.0https://commons.wikimedia.org/wiki/File:Jigsaw_puzzle_01_by_Scouten.jpg

‡ Weber, E. P. and Khademian, A.M. (2008) Wicked Problems, Knowledge Challenges, and Collaborative Capacity Builders in Network Settings. Public Administration Review, March/April, 334-349.

@EvidentlySian @ali_pals

The reality : making decisions• Evidence typically used to build case for change, inform design of

solutions, build consensus, challenge assumptions

• Whilst evidence is used to support design of change, little to suggest this is sustained through lifecycle of the programme

• Knowledge mobilisation important to “fail fast, learn fast”

• Time pressures often result in a “satisficing” strategy (stopping when they feel the information is “good enough“)

Highways Agency CC BY 2.0https://en.wikipedia.org/wiki/Gravelly_Hill_Interchange

@EvidentlySian @ali_pals

Evidence and complexity• Complexity science increasingly informing large scale change:

• Advocates small scale pilots and experiments with “minimal specifications”

• Iterative Plan Do Study Act cycles

• What does this mean for how we work as evidence producers and knowledge brokers?

@EvidentlySian @ali_pals

Barriers to using evidence• Evidence:

• hard to use• gaps and uncertainties• relevance• timeliness

• Individual• skills• confidence

• Organisational• capacity• culture

• Wider system• fragmented support

Phil Roeder, CC BY 2.0https://commons.wikimedia.org/wiki/File:Hurdles_(Scenes_from_a_Track_Meet).jpg

@EvidentlySian @ali_pals

Enablers?• Helping decision makers to apply evidence locally by being more

explicit about the context of research and evaluation studies

• Balancing rigour and timeliness by identifying which questions warrant rapid review or in-depth research

• Embedding evidence and knowledge mobilisation throughout the lifecycle of commissioning/transformation

• Researchers and practitioners working together to prioritise research questions using methods which support iterative change

• More collaboration in evidence support to reduce duplication

@EvidentlySian @ali_pals

Case study • Business case – CCG planning round 2015/16• Priority programme area: diabetes• Long Term Conditions Steering Group• Business case template – includes reference to evidence• 12 month trial

@EvidentlySian @ali_pals

Exercise: business case

1. What surprises you?2. What sources of evidence have been used to support

the business case? 3. What is your view of the ‘case for change’ section? 4. Has a clear issue/question been identified that the

evidence will support?5. Do you feel confident that this service model provides

an effective solution to the issue/question?

@EvidentlySian @ali_pals

Feedback from group work

• What could change or be done better?

@EvidentlySian @ali_pals

Local observations – business cases

Variable quality

Reliance on ‘data’

Last minute

Need timely expert input

Targeted on solution

What is the problem?

Little challenge at senior level

@EvidentlySian @ali_pals

It’s all happening!• Renewed emphasis on research and evidence! • NHS R&D Forum

• Primary care workstream – guide for commissioners• NHS England R&D – Research & Innovation

• Insight tool• Infographic (research landscape)• E-repository• Community of practice

• Alliance for Useful Evidence and partners• The Commissioning Elf – National Elf Service

@EvidentlySian @ali_pals

Next steps• Manifesto for evidence informed commissioning• What Works Global Summit• Alliance for Useful Evidence• Evidently Commissioning blog • Network• Connect with us……

Sian.Jones@bristolccg.nhs.ukalison.turner14@nhs.net

https://evidentlycommissioning.wordpress.com/