2c - using data to improve - ihiapp.ihi.org/.../using_data_to_improve.pdf · lean management system...

Advanced Measurement for Improvement

Cambridge, MA • March 26-27, 2015

1

2C – Using Data to Improve

Advanced Measurement for Improvement Seminar

March 26-27, 2015

The Data Cycle

Measures identified

and defined

Data collection

process defined,

tested

A P

DS



2

Data Acquisition

Operational IT systems gather granular data on standard processes

� Clinical: Nursing, EHR, Labs, Pharmacy, etc.

� Administrative: Billing, scheduling, etc.

Supplemented by systems to gather clinical process data

� Institutional

� Ad-hoc

PDSA data is real-time, front-line, manual.

Interpretation and Application

Who needs to know what?

� What level of information

� How often? How soon?

Will the audience interpret the measures appropriately?

� How will you train them?

� How will you keep them consistent?

Will process owners know how to respond?

� How will you coach them?



3

Source: Virginia Mason Health System

Lean Management System

Ideal management system to support value-based production:

Leader standard work

Visual controls

Daily accountability and planning

Respect for people who do the work

Unity of purpose

Strategic intent, operational goals, and system views must be vertically aligned!

Mann, D. (2010). Creating a Lean Culture: Tools to sustain lean

conversions. Boca Raton, FL, CRC Press.



4

Systems Hierarchy

Macro-systems

e.g. trust, facility, region

Meso-systems

e.g. division, clinical dept, pathology, IT

Microsystems

e.g. unit, clinic, surgical team

Source: Virginia Mason Health System

“Catchball” process aligns levels



5

Reporting Improvement

Senior Leaders, Boards, Executive

Sponsors (Macro-system)

Percent of target sites engaged in key

improvement initiatives

Percent of target population exposed to interventions

Phase of intervention by site or project: Plan? Pilot?

Implementation? Spread?

Time-series family of key ‘current care’ and ‘population’

measures by site, with goals

Comparison to ‘best practice,’ national/regional datasets,

comparative benchmarks

Comparison to control sites

Source: Keith Mandel MD


Improvement Initiative Leaders, Department

Heads, etc. (Meso-system)

Time-series dashboard of all

‘current care’

and ‘population’ measures by site, with goals.

Key current care measures segmented by unit, patient

sub-population, risk groups. Measures matched to

domain of improvement work.

Current QI capability of site leaders and teams, other

‘foundational’ requirements (e.g. registry, EMR)

Degree of involvement/effort of QI teams

Data quality




6


Front-Line (Micro-system) Teams

Time-series dashboard of all

‘current care’

and ‘population’ measures by site, with goals.

Key current care measures segmented by unit, patient

sub-population, risk groups. Measures matched to

domain of improvement work.

PDSA measures for current process change testing.

Data quality


Exercise

For Your Own Project:

Identify the key data ‘customers’ and their relationship to (or role in) the project?

What is their degree of involvement in the project and familiarity with QI methods?

How can you leverage measurement to maximize their engagement in the work?

What information are they receiving now? Is it timely and accurate?

What are your ideas for improving data feedback?



7

Individuals or Groups Role in Project

Degree of

Involvement

(1=never – 5=daily)

Comprehension of

Methods and Goals Ideas for Engagement

SENIOR LEADERS, BOARDS, SPONSORS

IMPROVEMENT INITIATIVE LEADERS

FRONT LINE IMPROVEMENT TEAMS

Individuals or Groups Role in Project

Degree of

Involvement

(1=never – 5=daily)

Comprehension of

Methods and Goals Ideas for Engagement




Key Data

Customers

Currently Receiving

Information?

Time Lag,

Data Quality Ideas for Improvement



Percent of target population exposed to

interventions

Phase of intervention by site or project:

Plan? Pilot? Implementation? Spread?

Time-series family of key ‘current care’ and

‘population’ measures by site, with goals

Comparison to ‘best practice,’

national/regional datasets, comparative

benchmarksComparison to control sites

Time-series dashboard of all ‘current care’

and ‘population’ measures by site, with

goals.

Key current care measures segmented by

unit, patient sub-population, risk groups.

Measures matched to domain of

improvement work.

Current QI capability of site leaders and

teams, other ‘foundational’ requirements

(e.g. registry, EMR)


Data quality



goals.




improvement work.

PDSA measures for current process change

testing.

Data quality




Currently Receiving

Information?

Time Lag,

Data Quality Ideas for Improvement



Percent of target population exposed to

interventions

Phase of intervention by site or project:

Plan? Pilot? Implementation? Spread?

Time-series family of key ‘current care’ and

‘population’ measures by site, with goals

Comparison to ‘best practice,’

national/regional datasets, comparative

benchmarksComparison to control sites



goals.




improvement work.

Current QI capability of site leaders and

teams, other ‘foundational’ requirements

(e.g. registry, EMR)


Data quality



goals.




improvement work.

PDSA measures for current process change

testing.

Data quality




Are They Being

Served?



8

Monitoring the System of Care

• Alignment

• The measure ‘cascade’

• Strategic measure deployment

Dynamic & Static Views of a Process

0

10

20

30

40

50

60

70

80

90

100

3/1/2

008

3/8/2

008

3/15

/2008

3/22

/200

8

3/29

/200

8

4/5/

2008

4/12

/200

8

4/19/

2008

4/26/2

008

5/3/

2008

5/10/2

008

5/17

/200

8

5/24/2

008

5/31

/200

8

6/7/

2008

Control charts show

change over time

Histogram, radar charts,

etc. show cross-

sectional ‘snapshots’ at

a point in time 0

2

4

6

8

10

12

14

16

18

20

0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100



9

Caldwell, C. (1995). Mentoring Strategic Change in Health Care: An

Action Guide. Milwaukee, ASQC Quality Press.

Strategic Intent and Strategic Measures

Short Term – This year’s goals

� Cash flow & cost reduction

� Productivity, net revenue, receivable days

� Meet current clinical targets

� CHF readmits

Mid Term – Next year’s goals

� Increase market share

� Customer satisfaction, complaints

Longer Term – 3 year goals

� Increase organization agility

� # Improvement projects, improvement project cycle time

Caldwell, C. (1998). Results-driven management: Strategic quality deployment. The handbook for managing change in health care. C. Caldwell.

Milwaukee, ASQ Quality Press: 37-87.



10

Strategic Intent and Strategic Measures

Dimensions of system performance

Rate of innovation and improvement

Reduce non-value-added costs

Improve cash flow

Increase customer satisfaction

Progressively integrate the organization as a system (additional business units, standard practice, IT)

� Vertical

� Horizontal

Source: Caldwell, C. (1998)

West Paces Ferry Quality Dimensions c.1992

Productivity Sales Development Customer Loyalty




11

West Paces Ferry Level 1 Measures c.1992

Productivity Sales Development Customer Loyalty

Cost per member per

month1

Target doctor recruits Net revenue from new

products

Days to resolve a

complaint

Cash flow percent

prior year (growth)

Corporate contracts QI projects completed Health status – quality

of life

Cost of poor quality Public awareness of

brand

Employee satisfaction

– open communication

Patient brag

Income percent prior

year

Market share QI project percent

complete

Operating expense

percent prior year

Readmit percent2

1WPF was an integrated delivery system 2Quality target for corporate strategy


Kano – Customer Judgment as a Basis for Performance Appraisal

Kano, N. (1984). "Attractive Quality and Must-Be Quality." Journal of the Japanese Society for Quality Control 14(2): 39-48.

I

II

III

III. Delightful. Unexpected and

exciting

II. Normal. A satisfactory

experience

I. Expected. Below this level repels

customers



12

Radar Chart: Quality Dimensions

I

II

III

Productivity

Productivity

Development

Sales

Patient BragCost of poor

quality


Suboptimized Systems




13

Level 1 Radar Chart in Action


The Information Cascade

Macro-systems

e.g. system, trust, facility, region

Meso-systems

e.g. service line, division, clinical dept, pathology, IT

Microsystems

e.g. unit, clinic, surgical team



14

Levels of Measurement

1 - Strategic measures

• Derived from strategic dimensions

(e.g. Balanced Scorecard)

• Target current, mid, long term goals

• Align with strategic plan

2 - Division measures

• Structural units comprising key

organizational functions

• Most L3 are operational

‘management indicators’

3 - Business process indicators

• Measures of high-level process

effectiveness and efficiency

• Components may have different

owners

4 - Core mainstay and support process

indicators

• Single process owner

• This is where QI work is focused

(1)

(2)

(3)

(4)

(Levels)

Ma

na

ge

me

nt V

iew

Macro

Micro

Meso

Micro

Admin errors

per 100 scripts

Wrong patient

per 100 scripts

% errors

intercepted

Non-path orders

% cases

Allergy alerts

per 100 scripts

Medication

errors % dsch

Prescribing errors

per 100 scripts

Moving up:

• Cause-effect theory (e.g. driver diagram, clinical evidence)

• Observed correlation (e.g. regression models)

• Aggregation

Data flow to more macro levels

Management ‘line of sight’

‘Line of Sight’ Measures




15

# Calls to rapid

response team

Environment

Hand hygiene

compliance

‘Line of Sight’ Measures

Percent

inpatient

mortality

Compliance with

“bundles”

% Surgical

bundle

% Pressure

ulcer bundle

% CL bundle

% VAP bundle

Hospital

Acquired

Infection

rates

% Sepsis

bundle

L1 L2 L3 L4

AggregationDriver Model

Observed

correlation,

clinical

evidence

Aggregation Methods

• Individual Patient Data to Population

� Average, median, distribution of patients: Cost, Time, Scores,

etc.

� Percent conforming: Protocol-driven care

� Count of events: Falls, Mortality, ADEs, etc.

• Micro to Meso to Macro

� Numerators and denominators summed across units

� Overall averages, medians

� Average unit performance

• Aggregating Across Different Measures

� Staging systems

� Build composite measures or indices



16

Staging System

Griffin, F. A. and D. C. Classen (2008). "Detection of adverse events in surgical patients using the Trigger Tool

approach" Qual Saf Health Care 17(4): 253-258.

Discussion

Consider how the aim of your project fits into your organization’s strategic goals:

Do the key measures that track the success of your project fit into a measure cascade within the organization? What would that look like?

Do you have recommendations for your client regarding a strategy for operational measurement?



17

Dashboards

Examples

Why not ‘Red-Yellow-

Green’?

An ideal alternative

© 2009 Institute for Healthcare Improvement, R. Lloyd

Tables require perusal



18

“Dashboard” = Summary of Performance Measures

“Radar” chart provides a snapshot view of multiple key quality indicators for a single unit:“Where are we right now?”


© by R. Scoville

“Small multiples” view compares many units on a single dimension (& over time):“What changes do we see, and where?”



19

Bed Occupancy Dashboard

Picture of ER display system to control be utilization

Source: Rostow (2002)

Source: Provost, Murray & Britto (IHI Forum 2010)

‘Traffic light’ emphasizes goals…But more about this later….



20

Who Uses Hospital Dashboards?

Survey: “Who is given the scorecard you supplied to us, and how frequently?” N=139

Kroch et al. (2006)

Who Uses Hospital Dashboards?

“Shorter, more focused dashboards that are reviewed on a

frequent basis are associated with higher performance.

According to the results of this dashboard analysis,

hospitals that use dashboards with fewer measures are

more likely to be in the high-performance group, suggesting

that higher-performing hospitals have developed

dashboards that focus on areas they see as critical for

quality. Furthermore, performance data are more

actionable when such data are consistently reviewed by the

board on a relatively frequent basis.”

Kroch et al. (2006)



21

A Common Type of Dashboard


This ‘specifications’ view does not provide a predictive view of system dynamics

How Is Error Rate Doing?




22

How is Perfect Care Doing?


Alternative

A view where

Each measure is displayed on an appropriate control chart

All control charts are on same page to see the whole system

Advantages

More accurately assess meaning of system changes

Become aware of system interrelationships

Appreciate dynamic complexity

Base decisions for action on improvement signals

HOWEVER…

Requires the viewer to understand variation!




23

Control Chart Dashboard


SPN Dashboard Report Fall 2010



24

Small Multiples: One site, all measures

Source: Dentaquest Institute

Small Multiples

One measure, all

sites

Source: Dentaquest Institute

2c - using data to improve - ihiapp.ihi.org/.../using_data_to_improve.pdf · lean management system...

Documents