learning event for commissioners - using data to support system improvement - 21 january 2016

7/25/2019 Learning Event for Commissioners - Using Data to Support System Improvement - 21 January 2016

1/163

Using Data to Support

System Improvement

21 January 2016

0900 1330

London Law Society

Learning Event for Commissioners

Robert Lloyd, PhD

Vice President

Institute for Healthcare Improvement


2/163

2015 Institute for Healthcare Improvement/R. C. Lloyd

2

Consider the following issues

The focus on measurement will only increase in health and social

services.

The role of measurement: Is it for the patient, the family or the care giver?

For staff? For the public, politicians or for researchers? Who is thecustomer of the measurement system?

Ultimately, measurement should be for those receiving the output of our

processes.

Financial measures, for example, usually have been for someone else notthe patient or family.

How do we open a new mind set and dialogue on measurement since

historically much of the measurement for health and social services has

been required and done by external groups and used for passing

judgement?

So, why do we need a dialogue on

Using Data to Support Heal th Systems Improvement?


3/163


3

A few more things to think about

If we trust the data but it is lagged by several quarters or a year or

more, how do we use it for improvement?

How can we develop measurement systems that reflect currentperformancerather than being aggregate by quarter or year?

During the last 5 years we have seen a new perspective emerging.

The data collected nationally are expected to drive improvement

at the sites of care. How do we make this happen? Can it

happen?

Improvement can only happen if the people who produce the

actual work own the measures and the data not someone

removed from the work.

So, why do we need a dialogue on

Using Data to Support Heal th Systems Improvement?


4/163

Discussion Questions for Today

Question 1What is the difference between a Commissioningprocess that is focused on QA and one that is focused onQI? How do we strike a balance between assurance andimprovement?

Question 2How do analyse data from a QI perspective and whatquestions should we ask about the results?

Question 3How can Commissioners support providers in buildingcapacity and capability for improvement?

4


5/163

We want to know what you think is thedefinition of quality.

Use the sticky notes on your table.

Fill in the following statement:

Qual ity is ___________________.

Place your note(s) on the designated

flipchart.

What is Quality?


6/163


Quality isa combination of value and outcome in the eyes of the consumer

a product or service delivered with 100% satisfaction the first time, every time

a product or service that provides an expected valuea product that lasts, for the best price

a satisfied customer

a very good product or service - one you would want again

above standard results or outcomes

an excellent product or service delivered by professional, friendly,knowledgeable people in a timely manner at the appropriate time

an unending struggle for excellence

accurate results to health care consumers

anticipation and fulfillment of needs

A vision which provides growth and satisfaction for the customer or consumer of

our service

attentive and excellent patient care

attention to detail, timeliness, competence

being the best, best of the best!

being present for every experience

best result possible in a given category


7/163

Quality is meeting and

exceeding the customersneeds and expectations and

then continuing to improve.W. Edwards Deming

What is Quality?


8/163

On the use of Statistical Analysis

in assessing Quality in Health Care

These statistics will enable usto ascertain what diseases andages press most heavily on theresources of particularhospitals.

They (i.e., the statistics) willshow subscribers how theirmoney is being spent, whatamount of good is really being

done with it, or whether themoney is doing mischief ratherthan good. Florence Nightingale

(1820-1910)

8


9/163

Health Care Quality Improvement

A broad range of activities of varying degrees of

complexity and methodological and statistical

rigor through which health care providers

develop, implement, and assess small-scale

interventions and identify those that work well

and implement them more broadly in order toimprove clinical practice.

The Ethics of Improving Health Care Quality & Safety: A Hastings Center/AHRQ

Project, Mary Ann Baily, PhD, Associate for Ethics & Health Policy, The HastingsCenter, Garrison, New York, October, 2004

9


10/163

X

Is life this simple?

Patient encounter

with physician

A healthy and productive

member of society

Lets start by thinking about the

Messiness of Life

Y

If it was this simple we wouldnt need to be

here discussing improvement!


11/163

Life looks more like this

X3

X2

X1

X5

X4Y

There are numerous direct effects between the independent

variables (the Xs) and the dependent variable (Y).

Time 1 Time 3Time 2

Patient Assessment

Score (could be

health outcomes,

functional status or

satisfaction)

IndependentVariables

Coordination of Care

Current

health

status

Age

Gender

Communication


12/163

In this case, there are numerous direct and indirect effects between the

independent variables and the dependent variable. For example, X1 and X4both have direct effects on Y plus there is an indirect effect due to the

interaction of X1 and X4 conjointly on Y.

Y

Well, actually, it looks like this!

X3

X2

X1

X5

X4

Time 1 Time 3Time 2

R3

R2

R1

R5

R4

RY

R = residuals or error terms representing the

effects of variables not included in the model.

Coordination of care

Age

Gender

CommunicationCurrent health

status

Patient Assessment

Score (could be

health outcomes,

functional status orsatisfaction)


13/163

Quality is about improving

Complex Problems! But13

Some problems are so

complex that you have to be

h igh ly intell igen t and wel lin formed jus t to be undecided

about them.--Laurence J. Peter


14/163

Walter

Shewhart(18911967) Joseph Juran

(1904 - 2008)W. Edwards

Deming

(1900 - 1993)

The Quality Pioneers


15/163

" Both pure and app l ied science have

gradual ly pushed fur ther and fur ther the

requ irements fo r accuracy and precis ion .

However, app l ied science, is even more

exact ing than pure science in certain

matters of accuracy and prec is ion ."

Dr. Walter Shewhart

A li d S i i t


16/163

Applied Science requires two

types of knowledge

SOI

Knowledge

Subject Matter

Knowledge

Science of Improvement (SOI)

Knowledge:The interplay of thetheories of systems, variation,

knowledge, and psychology.

Subject Matter Knowledge:Knowledge basic to the things wedo in life. Professional knowledge.Knowledge of work processes.


17/163

Knowledge for Improvement

SOI

Knowledge

Subject Matter

Knowledge

Improvement:Learn to combine subject matterknowledge and SOI knowledge in creative ways todevelop effective changes for improvement.

Improvement


18/163

Y

Improving the messiness of

life requires applied science.

X3

X2

X1

X5

X4

Time 1

Time 3

Time 2

R3

R2

R1

R5

R4

RY


19/163

Exercise

Assessing the Messiness of Life!

Do you think Commissioners and providers regularly view issues as

being rather messy and complex or do they see them as simple problems

that should be resolved quickly and easily (i.e., X causes Y)?

List a few of these messy problems that you are currently addressing and

why they are this way.

On a scale of 1-10, how messy is each of these problems? (1= not verymessy to 10 = extremely messy).

Do you have current measuresfor these messy problems that allow you

to determine just how complex and challenging each problem is?

If you have measures, do you feel that they are valid, reliable and

appropriate?


20/163

Exercise

Assessing the Messiness of Life!

What is the topic of thisMessy Problem?

How Messy is this

Problem? Select anumber 1 -10 with1 = not very messy

10 = extremely messy

List the current measures

you have for this MessyProblem?

Do you have baseline data on

these measures?

Do you feel that these

measures are valid,reliable and appropriate?


21/163


21

The Challenge

QA QI


22/163

QualityBetter

Old Way

(Quality Assurance)

QualityBetter Worse

New Way

(Quality Improvement)

Action taken

on all

occurrences

Reject

defectives

The Challenge:

Moving from the Old Way to the New Way

Source: Robert Lloyd, Ph.D., 2012

Requirement,Specification or

Threshold

Noaction

taken

here

Worse

Th S i tifi M th d id th f d ti f ll


23/163

23

Theoretical

Concepts

(ideas & hypotheses)

Interpretation

of the Results

(asking why?)

Information

for DecisionMaking

Data

Analysis and

Output

Select &

DefineIndicators

Data

Collection(plans & methods)

Deductive Phase

(general to specific)

Inductive Phase

(specific to general)

Source: R. Lloyd Quality Health Care, 2004, p. 153.

Theory

and

Prediction

The Scientific Method provides the foundation for all

Quality Improvement models and approaches


24/163

Source: Moen, R. and Norman, C. Circling Back: Clearing up Myths about the Deming

Cycle and Seeing How it Keeps Evolving, Quality ProgressNovember, 2010:22-28.

Understanding the Timeline is Critical

API Model for

Improvement

(1996)


25/163


Quality Models & Approaches

Across the Years

Human Factors/Ergonomics (Ancient Greece initially thenrefined in 1857 and then again in 1949)

The International Federation of the National Standardizing

Associations (ISA) (1926)

International Organization for Standardization (ISO) (1947)

Toyota Production System (1950s)

Six Sigma (Motorola, 1980s)

Baldrige Criteria (1987)

European Foundation for Quality Management (EFQM)

(1988)

Model for Improvement (1996)


26/163

Adding Six Sigma & Lean to the Timeline

Bill Smith (1986)

Motorola

Six SigmaMikel Harry (1988)

Motorola- MAIC

Forrest Breyfogle 111

(1992)- Integration

Michael George

(1991)- Integration

F.Taylor-The Principles of

Scientific Management

(1911)

Toyoda Family

Kiichiro Toyoda

Sakichi Tooda

Taiichi Ohno 1950-1980

Toyota Production System

Reference: Wortman 2001

Womack & Jones

Scoville & Little Comparing

Lean and Quality

Improvement (2014)

S th A di f f th


27/163

See the Appendices for further

details on the history of QI

Evolution of Quality Management over time

Age of Craftsman

Age of Mass Production

Age of Quality Management

Evolution of Quality Management (1850-1974)

Evolution of Quality Management (1978-2014)

Fourth Generation Management (Dr. Brian Joiner)

Evolution of Quality Management in Healthcare

What is Lean?


28/163

Institute for Healthcare Improvement, 2004

The choice of a quality system, approachor model should be driven by the

objectives of the organization, its culture

and its products or services!

The decision should NOT be driven by

how popular a particular approach is or

even if it has been used successfully inother settings.

In short

The Key: Constancy of Purpose!


29/163


29The Quality Improvement Journey for IHI(blending Jurans and Demings approaches)

Jurans

Quality

Trilogy

QualityPlanning

Quality

Improvement

Quality

ControlDemings System

of Profound

Knowledge


30/163

The Juran Trilogy

The Juran Trilogy consists of three types ofactivities:

Quality Planning,

Quality Control (or Quality Assurance)

Quality Improvement

Quali ty Planning: Setting aims

Selecting improvement projects

Selecting team and providing resources

30


31/163

Juran on Quality Control

Quality Control (QC): Quality control is theregulatory process through which we measure

actual quality performance, compare it with

quality goals, and act on the difference

(Juran, 1988)

This is usually done by operations (e.g.,

clinicians and managers) with support from a QCDepartment.

31


32/163

The Juran Trilogy Journey32

Demings Lens of


33/163

33

Appreciation

of a system

UnderstandingVariation

Theory

ofKnowledge

Human

Behaviour

Demings Lens of

Profound Knowledge

QI

The system of profoundknowledge provides alens. It provides a newmap of theory by whichto understand and

optimise ourorganisations.(Deming, Out of the Crisis)

It provides an

opportunity fordialogue and learning!


34/163

34

Appreciation for a System Interdependence, dynamism of the parts

The world is not deterministic

Direct, indirect and interactive variables

The system must have an aim

The whole is greater than sum of the parts

Understanding Variation Variation is to be expected!

Common or special causes of variation

Data for judgement or improvement?

Ranking, tampering & performance management Potential sampling errors

Theory of Knowledge What theories drive thesystem?

Can we predict?

Learning from theory and

experience

Operational definitions(what does a concept

mean?)

PDSAs for learning and

improvement

Human Behavior Interaction between people

Intrinsic versus extrinsic

motivation

Beliefs, values & assumptions

What is the Will to change?

What insi tsmight be obtained by looking

through the Lens of Profound Knowledge

Exercise


35/163

Apply the Lens of Profound Knowledge to an improvementproject.

This is best accomplished with an improvement team.

Use the PK Worksheet(next page) to record yourresponses. Remember that there are no right or wrong

responses.

Engage in a dialogue on PK (not a debate, a discussion or

idle chit-chat but rather a true dialogue about the theoriesand assumptions surrounding the project and the degree to

which it is messy.

Share the results of this exercise with others to obtain their

thoughts and input.

Exercise

Profound Knowledge


36/163

36

Profound Knowledge Worksheet

Appreciation for a System

Human Behaviour

Theory of Knowledge

Understanding Variation


37/163


37

Can you help providers start

to apply Profound

Knowledge to their messy

problems?


38/163

38

Is applicable to all types oforganizations.

Provides a framework for the

application of improvement

methods guided by theory.

Emphasizes and encourages the

iterative learning process of

deductive and inductive thinking.

Allows project plans to adapt as

learning occurs.

1996 API* added three basicquestions to supplement the PDSA Cycle.

The PDSA Cycle is used to develop, test, and implement changes.

*API = Associates in Process Improvement


39/163

Langley, J. et al. The Improvement Guide. Jossey-Bass Publishers, 2009.

The IHI Approach

When you

combine

the 3

questionswith the

the Model

forImprovement.

PDSA cycle,you get


40/163

Di l


41/163

Dialogue

Science of Improvement

What is your current level of knowledge about theScience of Improvement (SOI)?

Could you explain to a provider how the SOI can

help them to achieve better performance?

Are you and your colleagues prepared to engage ina dialogue with providers on how to move from a QAperspective to a QI perspective?

What structures and process can be established tosupport providers in their quality journeys?


42/163

Improvement?(improving the effectiveness or

efficiency of a process)

Accountab i l i ty

or Judgement?(making comparisons;

no change focus)

Research?(testing theory and building

new knowledge; efficacy)

The answer to this question will guide your entire

quality measurement journey!

Why are you measuring?

The Three Faces of


43/163

The Three Faces of

Performance Measurement

Aspect Improvement

Accountability

(Judgement) ResearchAim Improvement of care

(efficiency & effectiveness)

Comparison, choice,

reassurance, motivation for

change

Build new theories and

knowledge

(efficacy)

Methods:

Test ObservabilityTest observable

No test, evaluate current

performance Test blinded or controlled

Bias Accept consistent bias Measure and adjust to

reduce bias

Design to eliminate bias

Sample Size Just enough data, smallsequential samples

Obtain 100% of available,

relevant data

Just in case data

Flexibility of

Hypothesis

Flexible hypotheses, changes

as learning takes place No hypothesis

Fixed hypothesis

(null hypothesis)

Testing Strategy Sequential tests No tests One large test

Determining if achange is animprovement

Analytic Statistics

(statistical process control)

Run & Control charts

No change focus

(maybe compute a percent

change or rank order the

results)

Enumerative Statistics

(t-test, F-test,

chi square,

p-values)

Confidentiality ofthe data

Data used only by those

involved with improvement

Data available for public

consumption and review

Research subjects identities

protected

Adapted from: Lief Solberg, Gordon Mosser and Sharon McDonald,Journal on

Quality Improvement vol. 23, no. 3, (March 1997), 135-147.


44/163


Source: Provost, Murray & Britto (2010)

Example of Data for Judgement


45/163


Slide #45Slide #45

How Is the Error Rate Doing?



46/163


Slide #46Slide #46

How is Perfect Care Doing?


So how do you view the Three Faces


47/163


So, how do you view the Three Faces

of Performance Measurement?

Or,

As As a

Impro

vement

Jud

gment

Re

search

Integrating the


48/163


The three faces of performancemeasurement should not be seen as

mutually exclusive silos. This is not an

either/or situation.

All three areas must be understood asa system. Individuals need to build

skills in all three areas.

Organizations need translatorswho

and be able to speak the language ofeach approach.

The problem is that individuals identify

with one of the approaches and

dismiss the value of the other two.

Integrating the

Three Faces of Performance Measurement

Dialogue


49/163


Dialogue

Why are you measuring?

How much of your organizations energy is aimed at

improvement, accountability and/or research?

Does one form of performance measurement dominate

your journey?

Is your organization building silos or a Rubik's cube when it

comes to data collection and measurement?

Do you think the three approaches can be integrated or arethey in fact separate and distinct silos?

How many translators exist within your organization? Are

people being developed for this role?

Now how would you design a study to


50/163

2015 Institute for Healthcare Improvement/R. C. Lloyd50

Now, how would you design a study to

improve performance?

Li fe is fu l l of

opt ions!

E ti A l ti St di d


51/163

Enumerative versus Analytic Studies and

Related Statistical Techniques

The teaching of pure statistical theory in universities, including

the theory of probability and related subjects is almost

everywhere excellent. Application to enumerative studies is

mostly correct, but application to analytic problems is deceptive

and misleading.

Analysis of variance, t-test, confidence intervals, and other

statistical techniques taught in books, however interesting, are

inappropriate because they provide no basis for prediction and

because they bury the information contained in the order ofproduction. Most if not all computer packages for analysis of

data, as they are called, provide flagrant examples of

inefficiency.Dr. Deming, Out of the Crisis, page 132.


52/163

Deming classified studies into two types depending on the type of actionthat will be taken:

Enumerative Studiesones in which action will be taken on the entireuniverse. The aim of an enumerative study is estimation of some aspect of the

universe. Action will be taken on the universe based on this estimate through the

sampling frame. The U.S. Census is a classic example of an enumerative study.

Analytic Studiesones in which action will be taken on a cause system toimprove performance of a product, process, or system in the future. The aim of an

analytic study is prediction that one of several alternatives will be superior to the

others in the future.

In an analytic study, the focus is on the cause system. There is no

identifiable universe, as there is in an enumerative study, and, therefore, no

frame.

Source: Quali ty Improvement Through Planned Experimentat ion by R. Moen, T. Nolan and L.

Provost, McGraw-Hill, New York, 1999, 2nd edition.

Enumerative versus Analytic Studies


53/163

On Probability as Basis for Action

W. E. Deming, The American

Stat ist ic ian, November 1975, vol. 29,

No. 4. Pages 146-152.

53

It is possible, in an

enumerative problem, to

reduce errors of sampl ingto any specif ied level. In

contrast, in an analyt ic

problem, i t is imposs ib le

to compute the r isk of

mak ing a wrong

decision.


54/163

Enumerative and Analytic Studies

Enumerative: a Pond Analytic: a River

Fixed population-universe, frameRandom sampling

Probability based

Purpose:

- determine how much variation in a sample

- apply learning to the sample

(should not extrapolate)

- reject or do not reject sampled population

Hypothesis, statistical tests (t-test,F-test, chi square, p-values)

No fixed populationPopulation-ongoing stream of data

Also uses judgment sampling

Not totally based on probability

Purpose:

-how much variation, what type

-take action on underlying process to

Improve future outcome of process

Run charts or Shewhart control charts

Pull one sample from

this spot, walk away

and make a conclusion

about the total pond!

But, how do you pull

a sample from a

moving process?


55/163

Descriptive Studysummarize all the fish in one barrel by type.

Enumerative Studytake a sample from one barrel as a point estimate(audit) of the fish and generalize to all barrels on the boats deck.

Analytic Studyunderstand the process that places fish in one barrel by

studying previous and future barrels. Whyare these fish in this barrel?

Different Types of Studies

The approach toresearch and the

statistical methodsused should be based

on the question(s)being asked.


56/163

Does th is purp ose sound l ike i t w i l l be

an enumerative or analyt ic study?

56

Case Study: The Chicago Tr ibuneMonday, September 19, 2011

The purpose of the study, whichrepresents the most

com prehensive exam inat ion o f

rai lroad pedestr ian fatal i t ies in

no rtheastern Il l inois, was to

determ ine the facto rs leading to

the inc idents and recommend

solut ions the researchers said.


57/163

57

Variables in the study

Train type (Metra, Amtrak or Freight)

Number of pedestrian deaths by age

Number of pedestrian deaths by gender

Pedestrian death rate by Metra route

Pedestrian deaths (count) and rate by municipality

Percentage of deaths by season

The Chicago Tr ibuneMonday, September 19, 2011

Th Chi T ib


58/163

58

The Chicago TribuneMonday, September 19, 2011


59/163

59

Fatal rail pedestrian

inc idents are occurr ing

at an average of about

one every 10 days in

the Chicago area, the

study said.Last

week, there were two,

both on Thursday.

The Chicago TribuneMonday, September 19, 2011

Now what do you th ink?

Is th is an enumerat ive or analy t ic s tudy?

Enumerative Studies frequently


60/163

Enumerative Studies frequently

suffer from 20-20 Hindsight!

Managing a process on the basis of monthly

(or quarter ly) averages is l ike trying to d rive a

car by looking in the rear view mirror.

D. Wheeler

UnderstandingVariation, 1993.

Dialogue 61


61/163

Dialogue

Enumerative and Analytic Studies

When you consider the use of data in the CommissioningProcess, do you think it is designed around an Enumerative

or an Analytic approach?

If it is more aligned more with an Enumerative approach,how will this lead to improving care processes and

outcomes?

If you think the use of data in the Commissioning Process

is more aligned with an Analytic approach, then what are

you doing to convey this approach to providers?

61

Read more about Enumerative and


62/163

In the spring of 2010 the BMJ sponsored the Vin McLoughl in Sympos ium on the

Epistemology of Improvin g Heal th Care. The papers that grew out of this symposium

are freely available online under the BMJ journals unlock scheme:

http://qualitysafety.bmj.com/site/about/unlocked.xhtml

Read more about Enumerative and

Analytic Studies

BMJ Qual i ty & Safety

April 2011 Vol. 20, No Suppl. 1

Epis temology(from Greek epistm), meaning"knowledge, science", and (logos), meaning "study

of" is the branch of philosophy concerned with the

nature and scope (limitations) of knowledge.

It addresses the questions:

What is knowledge?

How is knowledge acquired?

How do we know what we know?

M t f th 2 d ti
http://qualitysafety.bmj.com/site/about/unlocked.xhtmlhttp://qualitysafety.bmj.com/site/about/unlocked.xhtml


63/163

Langley, G. et al, The Improvement Guide, API, 2009

Measurement focuses on the 2ndquestion

But, do you know the Milestones


64/163

, y

in the Quality Measurement Journey (QMJ)?

Milestones in the


65/163

65

AIM (How good? By when?)Concept

Measure

Operational Definitions

Data Collection Plan

Data Collection

Analysis ACTION

Source: R. Lloyd. Quality Health Care: A Guide to Developing and

Using Indicators. Jones and Bartlett Publishers, 2004.

Quality Measurement Journey

Milestones in the


66/163

AIMreduce patient falls by 37% by the end of the year

Conceptreduce patient falls

MeasuresInpatient falls rate (falls per 1000 patient days)

Operational Definitions - # falls/inpatient days

Data Collection Planweekly; no sampling; all IP units

Data Collectionunit collects the data

Analysis control chart (u-chart) ACTION


Milestones in the


67/163

67

AIM(How good? By when?)

Concept

Measure

Operational Definitions


Data Collection

Analysis ACTION

Source: R. Lloyd. Quality Health Care: A Guide to Developing and

Using Indicators. Jones and Bartlett Publishers, 2004.



68/163

NHS Mental Health Dashboard

But remember to build a


69/163

69

Look

your

at

system

as a cascade!

of measures

But remember to build a

Cascading System of Measures

A Cascading Approach to Measurement


70/163

Percent compliancewith all bundles

Percent

compliance

with

Pathology

investigation

s bundle

Percent

compliance

with Cardiac

investigation

s bundle

Percent

compliance

with

Physical

observation

s bundle

Complication

rates

+ +

Percent service userson antipsychotics with

baseline investigations

M CRO

MESO

MICRO


71/163

Copyright 2013 Institute for Healthcare Improvement/R. Lloyd71

AIM(Why are you measuring?)

Concept

MeasureOperational Definitions


Data CollectionAnalysis ACTION

The Quality Measurement Journey


72/163

72

You have performance data!

Now, what do you

do with it?

U d t di i ti t ll


73/163

73

If I had to reduce

my message for

management to justa few words, Id say

i t al l had to do w ith

reducing variation.W. Edwards Demin g

Understanding variation conceptually

Th P bl !


74/163

74

The Problem!

Aggregated data presented in tabular

formats or with summary statistics,

will not help you measure the impactof process improvement efforts.

Aggregated data can only lead to

judgment, not to improvement.

Average Percent of Patients who Fall


75/163

75

Average Percent of Patients who FallStatic View of Before and After the Implementation of a New Protocol

PercentofPat

ients

whoFall

Time 1 Time 2

3.8

5.2

5.0%

4.0%

WOW!A sign i f icant drop

from 5% to 4%

Conclusion -The protocol was a success!

A 20% drop in the average mortality!

Protocol implemented here

Average Percent of Patients who Fall


76/163

76

24 Months

1.0

9.0

Now what do you conclude about the

impact of the protocol?

5.0

UCL= 6.0

LCL = 2.0

CL = 4.0

Protocol implemented here

Average Percent of Patients who FallDynamic View of Before and After the Implementation of a New Protocol

PercentofPa

tients

whoFall

If you dont understand the variation that


77/163

77

y

lives in your data, you will be tempted to ...

Deny the data (It doesnt fit my view of reality!)

See trends where there are no trends

Try to explain natural variation as special events

Blame and give credit to people for things over

which they have no control

Distort the process that produced the data

Kill the messenger!

D C b ll' I i ht Di t ti


78/163

Dr. Campbell's Insight on Distortion

P78

The more any quantitative social

indicator is used for social decision-

making, the more subject it will be to

corruption pressures and the more apt itwill be to distort and corrupt the social

processes it is intended to monitor.

"Campbell's Law" fromAssessing the Impact of Planned Social

Change, 1976

http://www.sciencedirect.com/science/article/pii/014971897990048X

https://www.globalhivmeinfo.org/CapacityBuilding/Occasional

%20Papers/08%20Assessing%20the%20Impact%20of%20Planned%20Social%20Change.pdf

Donald T. Campbell,

Ph.D., social

psychologist

(1916-1996)

D D i C l f F
https://www.globalhivmeinfo.org/CapacityBuilding/Occasional%20Papers/08%20Assessing%20the%20Impact%20of%20Planned%20Social%20Change.pdfhttps://www.globalhivmeinfo.org/CapacityBuilding/Occasional%20Papers/08%20Assessing%20the%20Impact%20of%20Planned%20Social%20Change.pdfhttps://www.globalhivmeinfo.org/CapacityBuilding/Occasional%20Papers/08%20Assessing%20the%20Impact%20of%20Planned%20Social%20Change.pdfhttps://www.globalhivmeinfo.org/CapacityBuilding/Occasional%20Papers/08%20Assessing%20the%20Impact%20of%20Planned%20Social%20Change.pdfhttps://www.globalhivmeinfo.org/CapacityBuilding/Occasional%20Papers/08%20Assessing%20the%20Impact%20of%20Planned%20Social%20Change.pdfhttps://www.globalhivmeinfo.org/CapacityBuilding/Occasional%20Papers/08%20Assessing%20the%20Impact%20of%20Planned%20Social%20Change.pdf


79/163

Dr. Demings Cycle of Fear

Source: William Scherkenbach. The Deming Route to Quality and Productivity. Ceep Press, Washington, DC, 1990, page 71.

K il l the

MessengerIncreased

Fear

Filtered

Informat ion

Micro-

management


80/163

A phenomenon will

be said to be

contro l led when,

through the use ofpast experience, we

can predict , at least

w i th in l im i ts , how the

phenomenon may beexpected to vary in

the futureW. Shewhart. Economic Control of

Quality of Manufactured Product, 1931

Dr. Walter A Shewhart

What is the variation in one system


81/163

over time?Walter A. Shewhart - early 1920s, Bell Laboratories

81

time

UCL

Every process displays variation:

Controlled variationstable, consistent pattern of variation

chance, constant causes

Special cause variationassignable

pattern changes over time

LCL

Static View

StaticVie

w

Dynamic View

Types of Variation


82/163

82

Common Cause Variation Is inherent in the design of the

process

Is due to regular, natural or ordinarycauses

Affects all the outcomes of a process

Results in a stable process that ispredictable

Also known as random orunassignable variation

Special Cause Variation Is due to irregular or unnatural

causes that are not inherent in the

design of the process

Affect some, but not necessarilyall aspects of the process

Results in an unstable process

that is not predictable

Also known as non-random or

assignable variation

Types of Variation

P i t V i ti i t !


83/163

Point Variation exists!

Common Causedoes not mean Good Variation. It only

means that the process is stableand predictable. For

example, if a patients systolic blood pressure averaged

around 165 and was usually between 160 and 170 mmHg,this might be stable and predictable but completely

unacceptable.

Similarly Special Cause variation should not be viewed as

Bad Variation. You could have a special cause thatrepresents a very good result (e.g., a low turnaround time),

which you would want to emulate. Special Cause merely

means that the process is unstableand unpredictable.

Appropriate Management Response to


84/163

84

pp p g p

Common & Special Causes of Variation

Type of variation

Right Choice

Wrong Choice

Consequences of

making the wrong

choice

Is the process stable?

YES NO

Only Common

If not at targetchange the process

Treat normal variation as a

special cause (tampering)

Increased

variat ion!

Special + Common

Change the process

Wasted

resources!( t ime, effort, mo rale,

money)

Investigate the origin ofthe special cause

Source: Carey, R. and Lloyd, R. Measuring Quality Improvement in Healthcare: A Guide to Statistical Process

Control Applications. ASQ Press, Milwaukee, WI, 2001, page 153.

2 Questions


85/163

85

2 Questions

1. Is the process s table?

If so , it is p red ic table.

2. Is the process capable?

The chart w i l l tel l you i f the process is

stable and predic table.

You have to decide if the outpu t of the process is capable ofmeeting th e target or goal you have set!

(NOTE: we wil l talk abou t sett ing targets and goals sho rt ly)

Attributes of a Leader Who


86/163

Understands Variation

Leaders understand the different ways that variation isviewed.

They explain changes in terms of common causes and

special causes.

They use graphical methods to learn from data and

expect others to consider variation in their decisions

and actions.

They understand the concept of stable and unstableprocesses and the potential losses due to tampering.

Capability of a process or system is understood before

changes are attempted.

Dialogue


87/163

Copyright 2013 Institute for Healthcare Improvement/R. Lloyd

Select several measures you review on a regular

basis.

Do you and other CCG members as well as

providers evaluate these measures according the

criteria for common and special causes of

variation?

If not, what criteria do you use to determine ifdata are improving or getting worse?

Do these methods allow you to understand the

variation inherent in the data?

12/95

2/96

4/96

6/96

8/96

10/96

12/96

2/97

4/97

6/97

8/97

10/97

12/97

2/98

4/98

6/98

8/98

10/98

12/98

2/99

4/99

6/99

m ont h

Percent

C-sections

0. 0

5. 0

10. 0

15. 0

20. 0

25. 0

30. 0

35. 0

UCL = 2 7 . 7 0 1 8

CL=18. 0246

L CL = 8 . 3 4 7 3

nt of Cesa rean Sections Performed Dec 95 - Jun

W eek

Num

ber

of

M

edications

Errors

per

1000

Patient

0 . 0

2 . 5

5 . 0

7 . 5

1 0 . 0

1 2 . 5

1 5 . 0

1 7 . 5

2 0 . 0

2 2 . 5

UCL = 1 3 . 3 9 4 6 1

CL =4 . 4 2 0 4 8

L CL = 0 . 0 0 0 0 0

Medication ErrorRate

DialogueCommon and Special Causes of Variation

Conclusions


88/163


1. The same data can show different patterns of variation

dependent on how much of it you present and how you

statistically analyse and display the data.

2. Data presented over time (i.e., plotting the data by day,week or month) is the only way you will ever be able to

improve any aspect of quality or safety!

3. Avoid using aggregated data and enumerative statistics if

you are serious about improving quality and safety!

4. A leaders job is to understand patterns of variation and

ask why!

Understanding Variation

Understand variation statistically


89/163


89

STATIC VIEW

Descriptive StatisticsMean, Median & Mode

Minimum/Maximum/RangeStandard Deviation

Bar graphs/Pie charts

DYNAMIC VIEWRun Chart

Control Chart

(plot data over time)

Statistical Process Control (SPC)

Rateper100EDP

atients

Unplanned Returnsto Ed w/in72 Hours

M41.78

17

A43.89

26

M39.86

13

J40.03

16

J38.01

24

A43.43

27

S39.21

19

O41.90

14

N41.78

33

D43.00

20

J39.66

17

F40.03

22

M48.21

29

A43.89

17

M39.86

36

J36.21

19

J41.78

22

A43.89

24

S31.45

22

Month

ED/100

Returns

u chart

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

0.0

0.2

0.4

0.6

0.8

1.0

1.2

UCL=0.88

Mean=0.54

LCL=0.19

Understand variation statistically

How do we analyze variation for


90/163

90

quality improvement?

With Stat ist ical Process Con tro l (SPC) charts!

Runand Con trol Chartsare the best

tools to determine:

1. The variation that lives in the process

2. If our improvement strategies have had thedesired effect.

Three Uses of SPC Charts


91/163

Process Improvement: Isolated Femur Fractures

0

200

400

600

800

1000

1200

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64Sequential Patients

MinutesED

toOR

per

Patient

Holding the Gain: Isolated Femur Fractures

0

200

400

600

800

1000

1200


MinutesEDtoOR

per

Patien

t

3. Determine if we are holding the gains

Current Process Performance: Isolated Femur Fractures

0

200

400

600

800

1000

1200


MinutesED

toOR

per

Patient

Three Uses of SPC Charts

2. Determine if a change is an

improvement

1. Make process performance visible

Plotting dataover time to

understand the

variation!

How do we analyze variation

t ti ti ll f lit i t?


92/163


92

statistically for quality improvement?

Measure

Time

Mea

sure

Time

A Run Chart:

is a time series plot of data

The centerline is the Median

4 Run Chart rules are used to determine

if there are random or non-random

patterns in the data

A Control Chart:

is a time series plot of data

The centerline is the Mean

Added features include Upper and lowercontrol Limits (UCL & LCL)

5 Control Chart rules are used to

determine if the data reflect common or

special causes of variation

Run Chart

Control Chart


93/163


Lets start fitting the pieces

together

The Goal: To build information and learning for improvement.

Organisation Name Region April 14

Dementia

Diagnosis Rate

May 14

Dementia

Diagnosis

Rate

June 14

Dementia

Diagnosis

Rate


94/163

Rate Rate

NHS Barking & Dagenham CCG NE 55.10 54.58 55.33

NHS Harrow CCG NW 38.14 38.37 38.76

NHS Redbridge CCG NE 49.95 49.71 50.03

NHS Sutton CCG South 45.13 45.18 46.40

NHS Havering CCG NE 45.47 46.17 47.11

NHS Richmond CCG South 52.85 52.31 53.40NHS Kingston CCG South 42.53 41.98 41.99

NHS Croydon CCG South 46.50 46.50 46.73

NHS Camden CCG NE 64.88 65.27 65.21

NHS Hillingdon CCG NW 42.84 41.38 42.88

NHS Bexley CCG South 50.04 50.18 50.91

NHS Enfield CCG NE 49.49 49.08 50.10

NHS Greenwich CCG South 54.80 54.64 55.33

NHS Bromley CCG South 44.89 44.98 45.21

NHS Lewisham CCG South 53.52 53.62 54.50

NHS Wandsworth CCG South 56.12 56.17 56.86

LONDON AREA TEAM LAT 54.94 54.90 55.49

NHS West London (K&C & QPP) CCG NW 57.35 57.41 56.05

NHS City and Hackney CCG NE 68.78 68.53 68.54

NHS Newham CCG NE 63.87 63.68 63.82

NHS Merton CCG South 49.88 49.46 50.52

NHS Southwark CCG South 58.57 55.74 56.33

NHS Waltham Forest CCG Ne 54.29 54.48 54.69

NHS Barnet CCG NE 57.53 57.65 57.50

NHS Hammersmith and Fulham CCG NW 57.03 57.20 60.32

NHS Hounslow CCG NW 54.26 53.77 53.73

NHS Central London (Westminster) CCG NW 59.15 59.59 61.10

NHS Brent CCG NW 54.37 55.23 55.86

NHS Haringey CCG NE 53.92 53.57 55.72

NHS Tower Hamlets CCG NE 66.62 66.97 66.89

NHS Ealing CCG NW 54.19 54.28 54.94

NHS Lambeth CCG South 55.50 57.50 57.71

NHS Islington CCG NE 69.88 70.41 70.27

Organisation Name Region April 14 Dementia

Diagnosis Rate

May 14

Dementia

Diagnosis

Rate

June 14

Dementia

Diagnosis

Rate

July 14

Dementia

Diagnosis

Rate

August 14 Dementia

Diagnosis Rate

September 14

Dementia

Diagnosis Rate

October 14

Dementia

Diagnosis Rate

November 14

Dementia

Diagnosis Rate

December 14

Dementia

Diagnosis Rate

January 15

Dementia

Diagnosis Rate

February 15

Dementia

Diagnosis Rate

March 15

Dementia

Diagnosis Rate


95/163


Dementia Diagnosis Rates for 32 NHS CCGs, April 2014-March 2015

NHS Barking & Dagenham CCG NE 55.10 54.58 55.33 55.57 54.31 56.25 59.47 61.69 62.77 62.84 63.07 63.96

NHS Harrow CCG NW 38.14 38.37 38.76 37.97 37.44 40.09 40.24 42.30 43.14 39.35 43.29 50.30

NHS Redbridge CCG NE 49.95 49.71 50.03 49.21 48.18 48.98 49.30 53.45 55.71 56.05 57.38 59.62

NHS Sutton CCG South 45.13 45.18 46.40 45.51 44.13 47.94 54.31 54.63 56.68 55.82 56.21 55.56

NHS Havering CCG NE 45.47 46.17 47.11 46.42 46.35 47.67 48.20 49.67 49.87 50.15 51.14 51.61

NHS Richmond CCG South 52.85 52.31 53.40 51.40 50.76 53.07 52.06 54.83 55.82 58.04 60.20 63.60

NHS Kingston CCG South 42.53 41.98 41.99 41.82 39.27 41.12 40.62 42.82 48.17 49.30 51.28 51.92

NHS Croydon CCG South 46.50 46.50 46.73 46.66 46.18 46.51 46.28 47.51 48.78 50.33 51.43 51.83

NHS Camden CCG NE 64.88 65.27 65.21 63.84 62.56 65.02 66.57 67.39 67.00 67.45 67.00 68.73

NHS Hillingdon CCG NW 42.84 41.38 42.88 41.62 41.75 42.37 42.99 43.95 47.09 48.72 52.40 54.23

NHS Bexley CCG South 50.04 50.18 50.91 49.86 51.11 50.41 50.38 51.87 52.63 53.65 55.41 57.56

NHS Enfield CCG NE 49.49 49.08 50.10 48.14 49.03 51.91 52.29 52.51 53.78 55.68 56.44 59.73

NHS Greenwich CCG South 54.80 54.64 55.33 55.60 55.77 56.84 56.12 57.78 59.72 59.88 62.95 69.33

NHS Bromley CCG South 44.89 44.98 45.21 43.81 43.46 44.94 48.07 48.22 49.51 49.99 52.30 57.56

NHS Lewisham CCG South 53.52 53.62 54.50 53.77 54.28 52.96 53.33 52.61 52.94 53.17 58.36 61.52

NHS Wandsworth CCG South 56.12 56.17 56.86 56.03 54.87 55.95 55.78 56.48 55.92 56.37 58.62 58.61

LONDON AREA TEAMLAT 54.94 54.90 55.49 54.72 54.51 55.62 56.35 57.79 58.87 60.33 62.60 65.79

NHS West London (K&C & QPP) CCG NW 57.35 57.41 56.05 55.77 53.71 57.91 61.53 63.26 64.69 65.23 68.57 73.06

NHS City and Hackney CCG NE 68.78 68.53 68.54 66.51 66.17 67.83 68.83 67.96 68.22 68.54 69.41 70.22

NHS Newham CCG NE 63.87 63.68 63.82 64.14 62.66 63.85 63.71 63.93 64.77 65.81 65.68 68.35

NHS Merton CCG South 49.88 49.46 50.52 49.75 49.48 51.86 51.30 52.39 53.52 55.80 57.52 66.45

NHS Southwark CCG South 58.57 55.74 56.33 55.66 58.04 57.16 58.52 63.19 63.47 64.39 67.49 68.54

NHS Waltham Forest CCG Ne 54.29 54.48 54.69 53.99 53.25 54.09 53.77 56.48 56.52 62.97 66.36 70.31

NHS Barnet CCG NE 57.53 57.65 57.50 57.47 56.60 57.57 57.78 57.96 58.52 62.64 64.30 67.70

NHS Hammersmith and Fulham CCG NW 57.03 57.20 60.32 60.41 60.17 62.23 61.47 60.11 60.49 62.94 65.63 68.18

NHS Hounslow CCG NW 54.26 53.77 53.73 53.43 52.84 54.26 54.73 54.25 55.18 57.55 61.99 69.68

NHS Central London (Westminster) CCG NW 59.15 59.59 61.10 59.67 59.97 62.60 62.17 63.25 63.38 64.76 69.88 71.68

NHS Brent CCG NW 54.37 55.23 55.86 55.80 55.05 55.89 56.58 58.87 59.58 66.06 68.97 70.70

NHS Haringey CCG NE 53.92 53.57 55.72 54.85 53.21 54.16 53.48 54.30 55.31 56.94 61.17 64.23

NHS Tower Hamlets CCG NE 66.62 66.97 66.89 66.86 67.54 66.52 66.71 66.45 66.14 71.40 71.93 73.09

NHS Ealing CCG NW 54.19 54.28 54.94 54.49 56.45 54.80 55.13 57.21 57.60 57.91 60.14 62.98

NHS Lambeth CCG South 55.50 57.50 57.71 57.71 57.53 58.18 62.70 63.80 64.74 64.99 65.28 64.30

NHS Islington CCG NE 69.88 70.41 70.27 69.39 67.82 69.03 68.85 69.08 71.27 72.91 74.70 77.83

How do we improve performance of the system with this data?

One optionEnumerative Summariesith C ti N

96


96/163


with Comparative Norms

What do welearn from

these bar

graphs?

How does a

provider use

these graphs

to improve?

Enumerative SummariesStratification b Tr st

97


97/163


Stratification by Trust

How can

these

charts

be usedto

improve

the

systemof care?

Whatconclusions

can we draw

from these

graphs?


98/163


NHS Mental Health Dashboard:

The beginning of a bridge betweenEnumerative and Analytic studies

80.00London Area Team - I Chart

B t l t l k t th d t f

Created by Forid Alom, ELFT


99/163

UCL

LCL

35.00

40.00

45.00

50.00

55.00

60.00

65.00

70.00

75.00

Apr-14

M

ay-14

J

un-14

Jul-14

A

ug-14

S

ep-14

Oct-14

N

ov-14

D

ec-14

J

an-15

F

eb-15

M

ar-15

Demen

tiaDiagnosisRate

Mean = 57.6

But now, lets look at the data from

an Analytic Approach:

32 CCGs (London Team)

All London Area Teams Dementia Diagnosis Rate

April 2014-March 2015

A Trend: 6 or more consecutive data

point increasing (or decreasing)

80.00I Chart of selected 18 CCG'sCreated by Forid Alom, ELFT


100/163

UCL

LCL

35.00

40.00

45.00

50.00

55.00

60.00

65.00

70.00

75.00

A

pr-14

May-14

Jun-14

J

ul-14

Aug-14

Sep-14

O

ct-14

Nov-14

Dec-14

Jan-15

Feb-15

M

ar-15

De

mentiaDiagnosisRate

A Trend: 6 or more consecutive data

point increasing (or decreasing)

18 London Area Teams Dementia Diagnosis Rate

April 2014-March 2015

Mean = 53.5

Looking at Data from an

Analytic Approach:18 CCGs

70

75

80

osisRate

NHS Barking &Dagenham CCG - I

Chart

NHS Harrow CCG- I Chart

NHS RedbridgeCCG - I Chart

NHS Sutton CCG -I Chart

NHS HaveringCCG - I Chart

NHS RichmondCCG - I Chart


101/163

UCL

LCL

35

40

45

50

55

60

65

Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

Mar-15

DementiaDiagno

UCL

LCL

Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

Mar-15

UCL

LCL

Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

Mar-15

UCL

LCL

Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

Mar-15

UCL

LCL

Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

Mar-15

UCL

LCL

Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

Mar-15

UCL

LCL

35

40

45

50

55

60

65

70

75

80

Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

Mar-15

DementiaD

iagnosisRate

NHS Kingston CCG - IChart

UCL

LCL

Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

Mar-15

NHS CroydonCCG - I Chart

UCL

LCL

Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

Mar-15

NHS Camden CCG- I Chart

UCL

LCL

Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

Mar-15

NHS HillingdonCCG - I Chart

UCL

LCL

Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

Mar-15

NHS Bexley CCG -I Chart

UCL

LCL

Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

Mar-15

NHS Enfield CCG -I Chart

UCL

LCL

35

40

45

50

55

60

65

70

75

80

Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

Mar-15

DementiaDiagnosisRate

NHS Greenwich CCG - IChart

UCL

LCL

Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

Mar-15

NHS Bromley CCG- I Chart

UCL

LCL

Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

Mar-15

NHS LewishamCCG - I Chart

UCL

LCL

Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

Mar-15

NHS WandsworthCCG - I Chart

UCL

LCL

Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

Mar-15

NHS West London(K&C & QPP) CCG

- I Chart

UCL

LCL

Apr-

14

May-

14

Jun-

14

Jul-

14

Aug-

14

Sep-

14

Oct-

14

Nov-

14

Dec-

14

Jan-

15

Feb-

15

Mar-

15

NHS City andHackney CCG - I

Chart

Dashboard of 18 London Area Teams Dementia Diagnosis Rates, April 2014-March 2015


Exercise

Understanding Variation across 18 CCGs

102


102/163


Understanding Variation across 18 CCGs

For these 18 selected CCGs:

What do we learn from these 18 charts?

Are all 18 CCGs performing the same?

Do all 18 charts match the overall performance pattern shownon the aggregated chart?

Do these 18 CCGs exhibit common or special causes of

variation?

What will it take to get these 18 CCGs performing as asystem?

Should each CCGs improvement strategy be the same?

Are any of the CCGs demonstrating excellent performance?

65

70

75

80

osisRate

NHS Barking &Dagenham CCG - I

Chart

NHS Harrow CCG- I Chart

NHS RedbridgeCCG - I Chart

NHS Sutton CCG -I Chart

NHS HaveringCCG - I Chart

NHS RichmondCCG - I Chart


103/163

UCL

LCL

35

40

45

50

55

60

65

Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

Mar-15

DementiaDiagno

UCL

LCL

Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

Mar-15

UCL

LCL

Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

Mar-15

UCL

LCL

Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

Mar-15

UCL

LCL

Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

Mar-15

UCL

LCL

Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

Mar-15

UCL

LCL

35

40

45

50

55

60

65

70

75

80

Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

Mar-15

DementiaD

iagnosisRate

NHS Kingston CCG - IChart

UCL

LCL

Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

Mar-15

NHS CroydonCCG - I Chart

UCL

LCL

Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

Mar-15

NHS Camden CCG- I Chart

UCL

LCL

Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

Mar-15

NHS HillingdonCCG - I Chart

UCL

LCL

Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

Mar-15

NHS Bexley CCG -I Chart

UCL

LCL

Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

Mar-15

NHS Enfield CCG -I Chart

UCL

LCL

35

40

45

50

55

60

65

70

75

80

Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

Mar-15

DementiaDiagnosisRate

NHS Greenwich CCG - IChart

UCL

LCL

Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

Mar-15

NHS Bromley CCG- I Chart

UCL

LCL

Apr-14

May-14

Jun-14

Jul-

14

Aug-

14

Sep-

14

Oct-14

Nov-

14

Dec-

14

Jan-15

Feb-15

Mar-15

NHS LewishamCCG - I Chart

UCL

LCL

Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

Mar-15

NHS WandsworthCCG - I Chart

UCL

LCL

Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

Mar-15

NHS West London(K&C & QPP) CCG

- I Chart

UCL

LCL

Apr-

14

May-

14

Jun-

14

Jul-

14

Aug-

14

Sep-

14

Oct-

14

Nov-

14

Dec-

14

Jan-

15

Feb-

15

Mar-

15

NHS City andHackney CCG - I

Chart

Dashboard of 18 London Area Teams Dementia Diagnosis Rates, April 2014-March 2015


S l k d t th t f


104/163


Finally, we developed a dashboard of the 18

CCGs performance over time on control

charts.

Then, we looked at the aggregate performance

for a segment of the system (18 CCGs)

So, weve looked at the aggregate performance

for the entire system (all 32 CCGs in the

London area).



105/163


I knowwhat can

a CCG do toimprove system

performance? What can a CCG do to support

system improvement?

106


106/163


system improvement? Use the Commissioning data and the related findings to identify

opportunities for provider improvement. Help providers to take responsibility for their data.

Understand the factors that drive a particular measure.

Look at data as a time series not in the aggregate or with summary

statistics.

Work with providers to set up improvement teams to work on improving

the measures.

Stress that providers need to identify a dedicated group of QI advisors and

coaches who can support the improvement teams in their work.

Build capacity and capability for improvement thinking and practice

throughout the system (from the Board and Non-Execs through senior

management, middle management and front-line staff)

Create a process to review progress of the improvement teams.

Be transparent with data and results.


107/163

ReducingHarm

Physicalviolence

Medicationerrors

Falls

Pressureulcers

Restraints

It starts with

having a

strategic focus!

Right care,right place,right time

Improvingpatient and

carerexperience

Reliable deliveryof evidence-based care

Reducing delaysand

inefficiencies inthe system

Improved accessto services at

the rightlocation

A Driver Diagram with Aim, Primary

and Secondary Drivers


108/163


It thenrequires

identifying

the factors

that drive theoutcomes!

AIM

Primary

DriversSecondary Drivers


109/163


A plan for building capacity and capability for

the science of improvement is also essential

Estimated number needed to train = 5000

Needs = introduction to quality

improvement, identifying problems, change

ideas, testing and measuring change

Pocket QI commenced in October2015. Aim to reach 200 people by

Dec 2016.

All staff receive intro to QI at

induction


110/163

Experts by experience

All staff

Staff involved in or

leading QI projects

QI coaches

Board


Needs = deeper understanding of

improvement methodology, measurement

and using data, leading teams in QI


Needs = deeper understanding of

improvement methodology, understanding

variation, coaching teams and individuals

Needs = setting direction and big goals,

executive leadership, oversight of

improvement, being a champion,

understanding variation to lead

Estimated number needed to train = 11Needs = deep statistical process control,

deep improvement methods, effective plans

for implementation & spread

induction

500 people have undertaken the

ISIA so far. Wave 5 = Luton/Beds

(Sept 2016Feb 2017)

30 QI coaches graduating in

January 2016. To identify and train

second cohort in mid-late 2016

Most Executives will have

undertaken the ISIA.

Annual Board session with IHI &

regular Board development

discussions on QI

Currently have 3 improvementadvisors, with 1.5 wte deployed to

QI. To increase to 8 IAs in 2016/17

(6 wte).

Internalexperts (QI

team)

Bespoke QI learning sessions for

service users and carers. Over 50

attended in 2015. Build into recovery

college syllabus, along with

confidence-building, presentation

skills etc.

Needs = introduction to quality

improvement, how to get involved in

improving a service, practical skills in

confidence-building, presentation,

contributing ideas, support structure for

service user involvement

Then it is time to lay out your


111/163

Quality Dashboard(organisation-level view)

Then it is time to lay out your


ELFT Quality Dashboards


112/163

Safety


113/163

Finally, build the ability to track individual teams


114/163

ACCESS TO SERVICES

COLLABORATIVEDASHBOARDDecember 2015

Finally, build the ability to track individual teams

December 2015 1- Baseline data

UCL

70Average waiting time from referral to 1st face to face appt (Collaborative, 9/11 teams) - X-bar Chart


115/163

ACCESSTOSERVICESCOLLABORATIVE

60.7

52.2LCL

40

45

50

55

60

65

Jan-14

Feb-14

M

ar-14

A

pr-14

May-14

Jun-14

J

ul-14

Aug-14

Sep-14

O

ct-14

Nov-14

Dec-14

Jan-15

Feb-15

M

ar-15

A

pr-15

May-15

Jun-15

J

ul-15

Aug-15

Sep-15

O

ct-15

Nov-15

AverageWaiting

Time/Days

1021.8

1211.0

UCL

LCL

800

900

1000

1100

1200

1300

1400

Jan-14

F

eb-14

M

ar-14

A

pr-14

M

ay-14

Jun-14

Jul-14

Aug-14

Sep-14

O

ct-14

N

ov-14

D

ec-14

Jan-15

F

eb-15

M

ar-15

A

pr-15

M

ay-15

Jun-15

Jul-15

Aug-15

Sep-15

O

ct-15

N

ov-15

No.ofReferrals

No. of referrals received (Collaborative, 9/11 teams) - C Chart

32.50%

25.52%

UCL

LCL

18%

23%

28%

33%

38%

Jan-14

Feb-14

M

ar-14

A

pr-14

May-14

Jun-14

J

ul-14

Aug-14

Sep-14

O

ct-14

Nov-14

Dec-14

Jan-15

Feb-15

M

ar-15

A

pr-15

May-15

Jun-15

J

ul-15

Aug-15

Sep-15

O

ct-15

Nov-15

DNA/

%

% of 1st face to face appt DNAs (Collaborative, 9/11 teams) - P Chart

Where would the average be for

all this data?

Psychological Therapy Service (City and Hackney, Newham & Tower Hamlets)December 2015

125

Average waiting time from referral to 1st face to face appt (PTS) - X-bar Chart

4- Baseline data


116/163

104.0

88.9

UCL

LCL

65

75

85

95

105

115

Jan

-14

Feb

-14

Mar-14

Apr-14

May

-14

Jun

-14

Ju

l-14

Aug

-14

Sep

-14

Oct-14

Nov

-14

Dec

-14

Jan

-15

Feb

-15

Mar-15

Apr-15

May

-15

Jun

-15

Ju

l-15

Aug

-15

Sep

-15

Oct-15

Nov

-15

AverageWaitingTime/Days

211.7

UCL

LCL

100

150

200

250

300

Jan-14

Feb-14

M

ar-14

A

pr-14

May-14

Jun-14

J

ul-14

Aug-14

Sep-14

O

ct-14

Nov-14

Dec-14

Jan-15

Feb-15

M

ar-15

A

pr-15

May-15

Jun-15

J

ul-15

Aug-15

Sep-15

O

ct-15

Nov-15

No.ofRe

ferrals

No. of referrals received (PTS) - I Chart

29.75%

UCL

LCL

10%

15%

20%

25%

30%

35%

40%

45%

Jan-14

Feb-14

M

ar-14

A

pr-14

M

ay-14

Jun-14

Jul-14

Aug-14

Sep-14

O

ct-14

N

ov-14

D

ec-14

Jan-15

Feb-15

M

ar-15

A

pr-15

M

ay-15

Jun-15

Jul-15

Aug-15

Sep-15

O

ct-15

N

ov-15

DNA/%

% of 1st face to face appt DNAs (PTS) - P Chart

SERVICE

LEVEL

QI0043 & QI0175Newham Psychological Therapy ServiceDecember 2015

140

Average waiting time from referral to 1st face to face appt (NH PTS) - X-bar Chart

5- Baseline data


117/163

85.4

56.6

UCL

LCL

20

40

60

80

100

120

Jan-14

Feb-14

M

ar-14

A

pr-14

M

ay-14

Jun-14

Jul-14

Aug-14

Sep-14

O

ct-14

N

ov-14

D

ec-14

Jan-15

Feb-15

M

ar-15

A

pr-15

M

ay-15

Jun-15

Jul-15

Aug-15

Sep-15

O

ct-15

N

ov-15


58.4

UCL

LCL

0

20

40

60

80

100

J

an-14

F

eb-14

M

ar-14

A

pr-14

M

ay-14

J

un-14

Jul-14

A

ug-14

S

ep-14

O

ct-14

N

ov-14

D

ec-14

J

an-15

F

eb-15

M

ar-15

A

pr-15

M

ay-15

J

un-15

Jul-15

A

ug-15

S

ep-15

O

ct-15

N

ov-15

No.ofReferrals

No. of referrals received (NH PTS) - I Chart

32.73%

22.91%

UCL

LCL

0%

10%

20%

30%

40%

50%

60%

J

an-14

F

eb-14

M

ar-14

A

pr-14

M

ay-14

J

un-14

Jul-14

A

ug-14

S

ep-14

O

ct-14

N

ov-14

D

ec-14

J

an-15

F

eb-15

M

ar-15

A

pr-15

M

ay-15

J

un-15

Jul-15

A

ug-15

S

ep-15

O

ct-15

N

ov-15

DN

A/%

% of 1st face to face appt DNAs (NH PTS) - P Chart

TEAMLEVEL

QI0104Newham Memory ServiceDecember 2015

45

50Average waiting time from referral to 1st face to face appt (NH Memory Service) - X-bar Chart

8- Baseline data


118/163

28.5

UCL

LCL

5

10

15

20

25

30

35

40

Jan-14

Feb-14

M

ar-14

A

pr-14

M

ay-14

Jun-14

Jul-14

Aug-14

Sep-14

O

ct-14

N

ov-14

D

ec-14

Jan-15

Feb-15

M

ar-15

A

pr-15

M

ay-15

Jun-15

Jul-15

Aug-15

Sep-15

O

ct-15

N

ov-15


124.6

UCL

LCL

30

50

70

90

110

130

150

170

190

210

Jan-14

Feb-14

Mar-14

Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

Mar-15

Apr-15

May-15

Jun-15

Jul-15

Aug-15

Sep-15

Oct-15

Nov-15

No.ofRefe

rrals

No. of referrals received (NH Memory Service) - I Chart

17.20%

UCL

LCL

0%

5%

10%

15%

20%

25%

30%

35%

Jan-14

Feb-14

Mar-14

Apr-14

M

ay-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

Mar-15

Apr-15

M

ay-15

Jun-15

Jul-15

Aug-15

Sep-15

Oct-15

Nov-15

DNA/%

% of 1st face to face appt DNAs (NH Memory Service) - P Chart

TEAMLEVEL


119/163

All 4 acute admissions wards

in Tower Hamlets started

working on violence

reduction at the end of 2014

7/25/2019 Learning Event for Co

learning event for commissioners - using data to support system improvement - 21 january 2016

Documents