measurement for improvement

Measurement for Improvement

Patient Safety Team

• What are your expectations of the next three hours?

Aims and objectives

Aims• To have a greater knowledge and practical

understanding of Data Collection and Analysis

Objectives• Understand how to do ‘real’ analysis• Use data to improve decision-making and identify

real improvements

Introduction

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Distributions – “How long do cars last?”

68.26%

95.44%

99.73%

“Measurement is for improvement not judgement.”

D. Berwick

Measurement for improvement5

“You can’t fatten a cow by weighing it” (Palestinian proverb)

Improvement is not about measurement, but……..

How do we know if a change is an improvement?

“If you can’t measure it, you can’t improve it”

Measurement for improvement6

Model for improvement

Measurement throughout the project cycle

Project Identification

Getting a baseline

Did project make a

difference

Will project sustain

Evaluating worth of the

project

7 Repeat steps 4-6

Seven steps to measurement

1 Decide Aim

2 Choose Measures

3 Define Measures

4 Collect Data

5 Analyse and Present

6 Review Measures

Step 2 – Choose Measures

Step 2 – What to measure?

Patient experience and outcomes

Patient satisfaction

survey

% patients complication

free in recoveryPain score Average time

patient starved

Safety and reliability Clinical incidents ReadmissionsExceptions from

time out checklist

% correct equipment to

Efficiency and value Turnaround time % theatre utilisation Cancellations

Delays(Late starts &

finishes)

Leadership and high performing teams Staff survey Training and

development Staff turnover Staff absence

Driver DiagramsSchematic view of a system on the left we depict outcomeAs we move right we drill down into the network of causes that drive the outcome, from ‘primary’ to ‘secondary’ drivers.

Aim: An improved system

PrimaryDriver

Secondary

Driver

Secondary

Driver

Secondary

Driver

Secondary

Driver

Secondary

Driver

Driver Diagrams

On the right we depict ideas for system changes that might ultimately impact the outcome. Diagram represents our theory about how to modify the system to change the outcome.

Aim: An improved system

PrimaryDriver

Change

Secondary

Driver

Secondary

Driver

Secondary

Driver

Secondary

Driver

Secondary

Driver

Aim: 2 stones lighter!

Energy Out

Energy In

Walk daily commute

Stairs not lift

Exercise

Reduce alcohol intake

Eat Less

Driver Diagrams – weight loss example

Pedometer

Gym work out 3 days

Squash weekends

No pub weekdays

Take packed lunch

Low fat meals

Defect FreeSurgery

Avoid Mistakes

AvoidComplications

AvoidDelays

Driver Intervention

Conduct team brief

Conduct team debrief

Conduct time out

Produce accurate lists

Implement SSI bundle

Implement VTE bundle

Have correct kit to hand

Ensure staff adequately trained

O1 Overall glitch countO2 Never eventsO3 Number of surgical site infections

P1 % lists with Team BriefP2 % lists with Team DebriefP3 % compliance with SSI bundle

Driver Diagramsclinical example

Exercise: Create your driver diagram

Thinking about your project• Start to create a driver diagram and choose measures

• Complete the driver diagram to link Aims with Measures

You have 20 minutes

Step 2 – Choose Measures

An important note:

As well as clinical and quality measures – you may need to consider what financial measures are required for your project? You may need initial and ongoing funding?

Your success in gaining access to funds will be helped if you have completed a project financial justification or return on project investment analysis

Covered in more detail later in the presentation …

•Measurement Definitions

Step 3 – Define Measures

• An operational definition is a description, in quantifiable terms, of what to measure and the steps to follow to measure it consistently

• Are we measuring the same thing?

Exercise

• M&M Activity

RepeatabilityCan you, who created the definition, understand it and repeat it? Also known as test-retest error, used as an estimate of short-term variation

ReproducibilityAfter repeatability, try seeing if

the definition that you have created can be reproduced by other people?

Advice on creating measurement definitions

•The Measures Checklist

•Why important?•Who owns?•Definitions?•Goals?

Step 4 – Collect DataPractical considerations:5 W’s and 1H

Step 4 – Collect DataDecisions, decisions… • What - All patients or a sample?• Who – took the data? (what role?)• When – When was the data taken - real time or

retrospective?• Where is the data from?• How – was the data taken - hospital system or

audit?

• Turn the data into a different unit (hours into days)

Step 5 – Analyse and Present

We will now focus in more detail on methods of presenting and analysing our chosen measures….

•Understanding •and dealing with

• variation in analysis

The Airplane GameUsing the paper provided – make a paper planeYou have 5 minutes

• When instructed – throw your planes!

• What happened?

• Why are they not flying the same distance?

Exercise – The plane game

Fishbone diagram

Aeroplanes fly different

distances

Problem

Equipment People Procedures

Measurement (+Diagnostic)

Communication Materials

Causes

Types of paper e.g.

card, tracing paper,

No clear instructions provided

Some tables had scissors, rulers to help

Skills / ideas

Throwing styles

“We live in a world filled with variation –and yet there is very little recognition or understanding of variation”

William Scherkenbach

“Data should always be presented in such a way that preserves the evidence in the data…”

Walter Shewhart

Variation

What do people understand by the word variation?

What does this data tell us?

Patients treated in April

2008 2009

Patients treated

300April 2008 April 2009

This Month Last Month

Given two different numbers, one will always be bigger than the other!

What action is appropriate?

CommonCause

Stable in time and therefore relatively predictable

Paper selection

Persons technique

Design of the plane

Special Cause

Irregular in time and therefore unpredictable

Water spill

Can we classify variation?

It is important to distinguish between the two types of variation because they require different approaches to deal with them

Can we classify variation?

“There are different improvement strategies depending of which type of variation is present (common cause or special cause), so it is important for a team to know the difference.”

M.L. George

Step 5 – Analyse and present

Presentation

Dot plotIndividual value Plot

Interval Plot

Bar Chart

RunChart

Pareto

Scatter Plot

Tally Chart

Pie Chart

Radar Chart

Area Chart

Run Char

Plotting the dots - example Run ChartNumber of calls to outreach team (weekly)November 2007 to June 2008

1st Nov 15th Nov 29th Nov 13th Dec 27th Dec 10th Jan 24th Jan 7th Feb 21st Feb 6th Mar 20th Mar 3rd Apr 17th Apr 1st May

Calls per week Median

The Myth of Trends

Upward trend ? Downward trend ?

Downturn ?Setback ?

Turnaround ?Rebound?

Static ?Flatline ?

Downward trend

Upward trend

Looking for a trend

7 points all in upward direction

7 points all in downward direction

Looking for a trend

7 points above centre line 7 points below centre line

Below centre

Above centre

Exercise 7Creating your own chart

• Using graph paper, ruler and a pencil:

1. Draw and label the axis2. Plot the dots (daily or weekly data is the best)3. Work out the median and plot it4. Add a title (with dates)5. Add a legend6. Analyse it!

• You have 20 minutes

100908070605040302010

% Compliance with hand hygiene (weekly) April – Sept 2008

Week1 Apr 15 298 22 6 May13 20 273 Jun10 17 241 Jul 8 15 22 295 Aug12 19 262 Sep9 16

% Compliance with hand hygiene Medianx

Your chart… should look like this?

Discussion …

• How could you apply these charts to your projects?

•What is SPC?

Statistical Process Control(SPC) charts…

• …use the pattern of events in the past to predict with some degree of certainty where future events should fall

• …distinguish between the natural/common cause • variation and special cause variation• …enable you to look for problems when they are there, not

when they are not• …can motivate staff to improve practice thereby reducing

adverse events and minimising variation

Statistical Process Control(SPC) Charts:

Performance Report – Number of Admissions

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

No Admissions Median Lower Limit (66.5) Upper Limit (222.4)

Two ways to improve a process

If uncontrolled (special) variation

Process is unstable and unpredictable

Variation caused by factors outside process

External cause should be identified and sorted

If controlled (common) variation

Process is stable and predictable

Variation is inherent to the process

Therefore the process must be changed

The improvement process

Performance Report

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Act Plan

DoStudy

Special causes present -

unpredictable

Process predictable

(within control limits)

Process improvement

Three dangers to beware of …

1. Reacting to special cause variation by changing the process

2. Ignoring special cause variation by assuming “it’s part of the process”

3. Comparing more than one process

•Interpreting •Charts

Point above UCL

Special causes

Point below LCL

Rule 1 - Any point outside one of the control limits

7 Points above centre line

Special causes

Rule 2 - A run of seven points all above or all below the centre line, or all increasing or all decreasing

7 Points below centre line

7 Points upward direction

Special causes

Rule 2 - A run of seven points all above or all below the centre line, or all increasing or all decreasing

7 Points downward directionX

Cyclic pattern

Special causes

Rule 3 - Any “unusual” pattern or trends within the control limits

Trend pattern

Less than 2/3 of all the points fall in this zone

Special causes

Rule 4

More than 2/3 of all the points fall in this zone

Process out of control

• These rules are important!• They tell us if the process is stable or unstable• They tell us if common or special cause • variation is present

Remember the rules!• Outside control limits• Run of 7 or more consecutive points• Patterns• Rule of thirds

Exercise 8Interpreting SPC charts

•Apply the SPC rules to the charts in the handout

• Are there rules present?• Is the chart in control?

Step 6 – Review Measures

“It is a waste of time collecting and analysing your data if you don't take action on the results

measurement for improvement

Healthcare

better measurement for performance improvement in low‐ and

dissolution measurement system: current state and...

measurement for improvement: successful measurement...

measurement for quality improvement for board members and

performance improvement from measurement · pdf...

measurement for improvement

measurement for improvement 18 march 2008 mike davidge

measurement for quality improvement

performance improvement from measurement action

phc performance measurement for improvement monitoring

improvement of disability measurement and statistics

practical software measurement: measuring for process...

improvement of mechanical loss measurement system of ... ·...

key stage 2- presented by dr wahyudi for further improvement...

webinar introducing measurement for improvement

the powers and pitfalls of measurement for improvement ·...

measurement for improvement curriculum - hse.ie · the...

wide area measurement applications for improvement … area...

measurement for improvement. hello & welcome: what are you...

productivity measurement and improvement - diva...