advances in application of the science of improvement: past, present, and future lloyd p. provost...

API- 2014 1

Advances in applicationof the Science of Improvement:Past, present, and futureLloyd P. ProvostAssociates in Process ImprovementSenior Fellow, Institute for Healthcare Improvementlprovost@apiweb.org

API- 2014 2

Advances in application of the Science of Improvement:Past, present, and future

PastA brief history of the Science of Improvement

PresentApplying the Science of Improvement today in healthcare

FutureSome conjectures and hopes for the future of the Science of Improvement

API- 2014 3

Walter Shewhart

(1891 – 1967) Joseph Juran(1904 - 2008)

W. Edwards Deming(1900 - 1993)

The Pioneers of Quality Improvement

API- 2014 4

Appreciation of a system

Understanding Variation

Theory of Knowledge Psychology

Values

Deming’s System of Profound Knowledge

The Foundation of the Science of Improvement

Systems

Understanding Variation

Theory of Knowledge

Psychology

History of the System of Profound Knowledge

5

VARIATION

Walter Shewhart

R A Fisher

W Edwards Deming

PSYCHOLOGY

Eric Trist

Abraham Maslow

Kurt Lewin

Fredrick Herzberg

Elton Mayo

Douglas McGregor

KNOWLEDGE

Chris Argyris

John Dewey

C. I. Lewis

SYSTEMS

Russell Ackoff

Ludwig von Bertalanffy

Jay Forrester

Peter Senge

API- 2014

Deming’s System of Profound Knowledge

"One need not be eminent in any part of profound knowledge in order to understand it and to apply it. The various segments of the system of profound knowledge cannot be separated. They interact with each other. For example knowledge about psychology is incomplete without knowledge of variation.“

W. E. Deming, The New Economics, 1994

Two Types of Knowledge

Subject Matter Knowledge

Subject Matter Knowledge: professional knowledge, basic to the things we do in life

Profound Knowledge: The interaction of the theories of systems, variation, knowledge, and psychology.

Profound Knowledge

API- 2014

API- 2014 8

Making Improvements

Profound Knowledge

Subject Matter Knowledge

Increased capability to

makeimprovements

Learn to combine subject matter knowledge and profound knowledge in creative ways to develop effective changes for improvement

Intrinsic versus Extrinsic

Motivation

9API- 2014

PsychologyThe Human Side of the Science of Improvement

Shewhart’s Theory of Variation (1931)

10API- 2014

Common Causes—those causes inherent in the system over time, affect everyone working in the system, and affect all outcomes of the system

Special Causes—those causes not part of the system all the time or do not affect everyone, but arise because of specific circumstances

Berwick, Donald M., Controlling Variation in Health Care: A Consultation with Walter Shewhart, Medical Care, December 1991, Vol. 29, No 12, pp 1212-1225.

API- 2014 11

W. Edwards Deming - Teaching, 1951Photo from The Deming Institute

Appreciation of a System

API- 2014 12

Change in a system:First-order Change: A change that occurs within a given

system. The system remains the same.

Second-order Change: A change whose occurrence changes the system (discontinuity, logical jump).

Change Concepts in the Grouping

Eliminate Waste 11Improve Work Flow 12Optimize Inventory 4Change the Work Environment 11Producer/Customer Interface 8Focus on Time 5Focus on Variation 8Mistake Proofing 4Focus on a Product/Service 9

72

ConceptAn opportunity to create

a new connection

Thoughtprocess

Specificidea B

Specificidea A

13

BreakthroughResults

Theories, hunches,& best practices

A P

S D

Evidence & Data

A P

S D

A P

S D

A P

S D

Develop a change

Test a change

Test new conditions

Implement a change

Theory of Knowledge

API- 2014

14

Gold Standard for Evidence in QI:

“Satisfactory prediction of the results of tests conducted over a wide range of conditions is the means to increase the degree of belief that the change will result in improvement.”

The Improvement Guide, 2nd Edition, 2009

API- 2014

“Predictionis the problem”

– W. Edwards Deming

API- 2014 15

Advances in Application of the Science of Improvement:Past, present, and future

PastA brief history of the Science of Improvement

PresentApplying the Science of Improvement today in healthcare

FutureSome conjectures and hopes for the future of the Science of Improvement

API- 2014 16

The National Demonstration Project on Quality Improvement in Health Care (NDP)

• 20 Hospitals• 21 Quality Improvement Experts• September 1986 to June 1987• Initial and Summary Conferences• Founding of IHI

API- 2014 17

Health Care Quality Improvement: Key characteristics

o Multiple experimental cycles (PDSA’s) for quick feedback and learning – (not just one study design)

o The initial intervention (changes to the system) are adapted and modified as study progresses

o Measuring over time (improvement is temporal)o Graphical analysis and presentation of data (SPC)o Learning from special causes and sub-group analysis are

key strategieso Involvement of local expertise in conducting project

(passion and possible bias)

API- 2014 18

o Multi-factor study designs to learn from complex systems with non-linear and dynamic cause and effect relationships

o Contextual factors (background variables or confounders in research) are sometimes the major focus of the QI study

o Building reliability of the interventions can be a major part of the effort

o Sustainability of the changes is a consideration from the beginning of the project

Health Care Quality Improvement: Key characteristics (continued)

Greg Ogrinc, Kaveh G Shojania, “Building knowledge, asking questions”, BMJ Qual Saf 2014;23:265–267.

19

Selection

Measurement

Confounding

Chance

Conclusion

Sample

Sample

Internal Validity

Traditional Enumerative Statistics Apply

Environment in an Enumerative Study

???

Perspective: Enumerative and Analytic Studies

20

Clinical EpidemiologyFletcher, Fletcher, Wagner

Sample

Environment in an Analytic Study (QI)

Prediction is the Issue. Need different analytical

methods

Sample

Provost, Lloyd P. “Analytical studies: a framework for quality improvement design and analysis”, The British Medical Journal Quality and Safety, 2011, 20 (Supplement 1):i92-i96.

21

Research and Improvement in Health Care

“The story could end here happily with two great streams of endeavor merging into a framework for conjoint action: improving clinical evidence and improving the process of care. Instead, the two endeavors are often in unhappy tension...”

Berwick, Donald M The Science of Improvement, JAMA. 2008;299(10):1182-1184

1950-1980’s Formal Evaluation of

Medical Practice,

Randomized Control Trials,

and Levels of Evidence

1980’s

Industrial Q

uality

Improvement Methods

applied to Health

care

Processes, I

OM Reports

22

Another Possibility

LinearCause-and

-Effect Relationships

ComplexNon-Linear

Chaotic Systems

“Rigorous” Learning

Poor Learning

Learning from Experience

Randomized Controlled Trials

RCT’s

RCTs to Study Health Care Systems

What Methods ofLearning Go Here?

Adapted from Donald M. Berwick, MD – Eating Soup with a Fork, IHI Forum, 2008

API- 2014 23http://www.nytimes.com/2014/02/03/health/effort-to-test-health-policies-is-criticized-for-study-tactics.html?_r=0

API- 2014 24

Advances in Application of the Science of Improvement:Past, present, and future

PastA brief history of the Science of Improvement

Present Applying the Science of Improvement today in healthcare

Future Some conjectures and hopes for the future of the Science of Improvement

25

PREFACEDeveloping a continuously learning health

care system is critical for the future of health care, as well as for the future physical and financial health of the nation.

There is no simple path forward; rather, actions need to be taken by every stakeholder if this vision is to become a reality.

Mark D. Smith, Chair

26

Five Methods to Support a System of Learning for Health Care

Moen, Nolan, ProvostMcGraw-Hill 2013, p. 346-347

1. Recognition and investigation of special causes using Shewhart charts.

2. Study of informative cases.

3. Observational studies of relationships between factors and responses.

4. Natural experiments (with factorial thinking).

5. Planned Experiments including the use of replication blocking, randomization, and experimental patterns.

API- 2014

Context!

API- 2014

28

“Orchestrated” Testing: Coordinate sophisticated PDSA testing in a

collaborative or network using Factorial DesignsAPI- 2014

API- 2014 29

Time Series Analysis as the Standard for QI Studies

Webster, P. et al, “Using quality improvement to accelerate highly active antiretroviral treatment coverage in South Africa”, BMJ Quality and Safety, 2012;21:315e324.

API- 2014 30

Integration of Quantitative and Qualitative Research

What Happened?Why?

API- 2014 31

Reverend Thomas Bayes, 1701 - 1761

Learning to use Bayesian Statistics to support quality improvement

API- 2014 32

Patient Focused Care: “Personalized Learning Systems”

“A Personalized Learning System is designed to make possible a more rigorous, collaborative, and individualized approach to care. Enable patients, families, clinicians, and researchers to work together to accelerate innovation, discovery, and the application of new knowledge…”

HBR Blog Network: A Personalized Learning System

for Improving Patient-Physician Collaboration,

Heather C. Kaplan, Jeremy Adler, Shehzad A. Saeed, Ian

Eslick, and Peter A. Margolis | 12:27 PM October 2, 2013

API- 2014 33

A growing appreciation of complex systems

API- 2014 34

The Need to Understand the Whole System…

When you are confronted by any complex social system, such as an urban center or a hamster, with things about it that you’re dissatisfied with and anxious to fix, you cannot just step in and set about fixing with much hope of helping.

This realization is one of the sore discouragements of our century… You cannot meddle with one part of a complex system from the outside without the almost certain risk of setting off disastrous events that you hadn’t counted on in other, remote parts.

If you want to fix something you are first obliged to understand…the whole system… Intervening is a way of causing trouble.

Lewis Thomas, 1974Biologist and EssayistFrom Business Dynamics by John Sterman, p.8

API- 2014 35

Advances in applicationof the Science of Improvement:Past, present, and future

Continue to add rigor to the methods of improvement.

Continue to develop the Science of Improvement on the foundation prepared by Deming.

Appreciation of a system

Understanding Variation

Theory of Knowledge Psychology

Values