control charts: theory and use - vermont oxford … aqc/2019_jsq_slides...subgroup size subgroup...

Control Charts: Theory and Use

Heather

Disclosures

I have no conflicts of interest to disclose or resolve

Objectives

Examine the “anatomy,” structure and statistical basis of a control chart

Review the basic “types” of control charts.

Using examples, apply the rules for detecting special cause variation in control charts

Control Charts

The Shewhart chart (a.k.a. control chart) is a statistical tool used to distinguish between common cause and special cause variation

Chart Title

60

70

80

90

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24Run Order

Mea

sure Center Line

Upper Limit

Lower Limit

Provost, LP and Murray S. The Health Care Data Guide. 2011

0

5

10

15

20

25

30

35

1/1/

17

2/1/

17

3/1/

17

4/1/

17

5/1/

17

6/1/

17

7/1/

17

8/1/

07

9/1/

17

10/1

/17

11/1

/17

12/1

/17

Measure

A control chart is a run chart with some differences

Run chart: Center line is the median.

Control chart: Center line is often the mean.

Add control limits that reflect variability in data or the extent of common cause variation KEY

Mean

Upper Control Limit (UCL)

Lower Control Limit (LCL)

From Run Charts to Control ChartsVa

lue

Time

±2

SD

95.

4%±

3 S

D 9

9.7%

Relationship to Probability Theory

± 2 SD 95.4%

± 3 SD 99.7%

Constructing a Control Chart

Underlying data distribution dictates population parameters. Parameters dictate:

• Measure of central tendency (the “centerline”)• Measure of variability standard deviation values for

the upper and lower control limits.

Underlying distribution depends on type of data being observed (e.g., normal/Gaussian, Poisson, binomial, geometric)

Need to know what type of data you have to construct the proper type of control chart!

Continuous Data1. Numerical value for each unit in a group

Discrete (Integer) Data2. Classification: Presence or not of an attribute3. Count: How many attributes occur in sample

Type of data

Sample Size

Type of Chart

Math(software)

Constructing Control Charts

Types of Data & Control Charts

Healthcare Systems Engineering Institute

Common cause probability model Example

Disc

rete

Classification: Binomial

Parameter: p

Patient develops an SSI (Y/N)

Count: Poisson

Parameter: λ

Number of catheter-associated HAIs

Cont

inuo

us Normal

Parameters: m, s

Time to deliver thrombolytics

Type of data

Sample Size

Type of Chart

Math(software)

Constructing Control Charts

Single Observation

Multiple Observations• Equal Sample Size or Area of Opportunity• Unequal Sample Size or Area of Opportunity

Which Control Chart To Use

Type of Data

Discrete / Attribute(data is counted or classified)

Continuous / Variable(data is measured on a scale)

Count(events/errors are counted; numerator can be greater

than denominator)

Classification(each item is classified; numerator cannot be

greater than denominator)

Equal or fixed area of

opportunity

Unequal or variable area of opportunity

Equal or unequal

subgroup size

Subgroup size = 1(each subgroup is single observation)

Subgroup size > 1(each subgroup has multiple observations)

C chartCount of events

U chartEvents per unit

P chartPercent classified

X and MR chartsIndividual measures and

moving range

X-bar and S chartsAverage and standard

deviation

Gupta M and Kaplan HC, Clinics in Perinatology, 2017.

Quiz: Determine the Right Chart

Type of Data

Discrete / Attribute(data is counted or classified)

Continuous / Variable(data is measured on a scale)

Count(events/errors are counted; numerator can be greater

than denominator)

Classification(each item is classified; numerator cannot be

greater than denominator)

Equal or fixed area of

opportunity

Unequal or variable area of opportunity

Equal or unequal

subgroup size

Subgroup size = 1(each subgroup is single observation)

Subgroup size > 1(each subgroup has multiple observations)

C chartCount of events

U chartEvents per unit

P chartPercent classified

X and MR chartsIndividual measures and

moving range

X-bar and S chartsAverage and standard

deviation

A surgical service tracks a sample of 10 patients each week and records whether or not each patient received antibiotics on time.

Gupta M and Kaplan HC, Clinics in Perinatology, 2017.

Control Charts for Discrete Data (1)

Classification data

• P chart: Percent of observations with a given attribute• Measure of variability comes from binomial distribution

Centerline = p-bar = Average of the Statistic

UCL = CL + 3 σs

LCL = CL - 3 σs

100×=∑ ndp

( )in

pppUCL −+=

1003

( )in

pppLCL −−=

1003

σs from the binomial

distribution

Provost, LP and Murray S. The Health Care Data Guide. 2011 Slide courtesy of Terri Byczkowski, PhD, CCHMC

P-chart Calculations

Example: Late-Onset Sepsis

Performance Metric: Percent of infants discharged with Late-Onset Nosocomial Sepsis

Subgroup: variable number of infants discharged in a given month

MonthInfants with

Late InfectionPatients

Discharged4/1/2006 10 615/1/2006 13 816/1/2006 19 947/1/2006 20 788/1/2006 7 779/1/2006 18 77

10/1/2006 16 8411/1/2006 12 8312/1/2006 15 76

1/1/2007 17 902/1/2007 16 733/1/2007 16 1004/1/2007 13 755/1/2007 16 996/1/2007 12 887/1/2007 22 1058/1/2007 16 919/1/2007 19 93

Example: Late-Onset Sepsis

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

4/1/

065/

1/06

6/1/

067/

1/06

8/1/

069/

1/06

10/1

/06

11/1

/06

12/1

/06

1/1/

072/

1/07

3/1/

074/

1/07

5/1/

076/

1/07

7/1/

078/

1/07

9/1/

0710

/1/0

711

/1/0

712

/1/0

71/

1/08

2/1/

083/

1/08

4/1/

085/

1/08

6/1/

087/

1/08

8/1/

089/

1/08

10/1

/08

11/1

/08

12/1

/08

1/1/

092/

1/09

3/1/

094/

1/09

5/1/

096/

1/09

7/1/

098/

1/09

9/1/

0910

/1/0

911

/1/0

912

/1/0

9

Month

Y Axis: Proportion of Infants D/C with Late-Onset

Infection

Centerline: Average Proportion of Infants with Late-Onset Infection (over 45 months)

Control Limits

P-chart: % of VLBW infants with Late-Onset Infection

Quiz: Determine the Right Chart

Type of Data

Discrete / Attribute(data is counted or classified)

Continuous / Variable(data is measured on a scale)

Count(events/errors are counted; numerator can be greater

than denominator)

Classification(each item is classified; numerator cannot be

greater than denominator)

Equal or fixed area of

opportunity

Unequal or variable area of opportunity

Equal or unequal

subgroup size

Subgroup size = 1(each subgroup is single observation)

Subgroup size > 1(each subgroup has multiple observations)

C chartCount of events

U chartEvents per unit

P chartPercent classified

X and MR chartsIndividual measures and

moving range

X-bar and S chartsAverage and standard

deviation

The NICU is tracking the number of unplanned extubations each month as compared to total ventilator days.

Gupta M and Kaplan HC, Clinics in Perinatology, 2017.

Control Charts for Discrete Data (2)

Count Data• “C” Chart (as in count), plots the raw number of instances• “U” Chart (as in unit) plots the number of instances per

opportunities to observe

• Measure of variability comes from Poisson distribution

Example: Catheter-Associated InfectionsMetric: Catheter-Associated Infection Rate

Data obtained from infection control as reported to CDC. Each day, number of catheters is counted. This is used to obtain catheter days each month. Number of infections (catheter-associated) occurring each month is also reported.

Subgroup: Monthly

Unit Count: number of infections per opportunity (catheter day)

Month # InfectionsCatheter

Days10/1/2008 8 221211/1/2008 15 306412/1/2008 15 30071/1/2009 6 27832/1/2009 14 24993/1/2009 4 26924/1/2009 8 27845/1/2009 14 27726/1/2009 9 26907/1/2009 10 31458/1/2009 16 31719/1/2009 12 3209

10/1/2009 11 307611/1/2009 17 274912/1/2009 7 2759

Example: Catheter-Associated InfectionsY Axis: CA-Infection Rate

per 1000 line days

Centerline: Average CA-Infection rate (over 24 months)

Control Limits

U-Chart

Quiz: Determine the Right Chart

Type of Data

Discrete / Attribute(data is counted or classified)

Continuous / Variable(data is measured on a scale)

Count(events/errors are counted; numerator can be greater

than denominator)

Classification(each item is classified; numerator cannot be

greater than denominator)

Equal or fixed area of

opportunity

Unequal or variable area of opportunity

Equal or unequal

subgroup size

Subgroup size = 1(each subgroup is single observation)

Subgroup size > 1(each subgroup has multiple observations)

C chartCount of events

U chartEvents per unit

P chartPercent classified

X and MR chartsIndividual measures and

moving range

X-bar and S chartsAverage and standard

deviation

A call center samples 20 calls each day & records the average amount of time it takes for the scheduler to handle the call.

Gupta M and Kaplan HC, Clinics in Perinatology, 2017.

Control Charts for Continuous Data

Two charts Charts of Value or Sample Mean

• “X”Chart plots an individual value • “Xbar” Chart plots the sample average

Charts of Variation-S (average & standard deviation)• “MR” chart plots the moving range of the individual values• “S” chart plots the standard deviation of the sample

Measure of variability comes from normal (Gaussian) distribution

Example: C-section Decision to IncisionPerformance Metric: Time to incision following decision

for emergent c-section

What it means operationally: Minutes between decision to do c-section and time of incision

How it is observed: 10 charts sampled per week

Subgroup: The 10 charts sampled each week

Summary stats for the subgroups:

X-bar: The average decision to incision time for the 10 charts sampled each week

S: The standard deviation of decision to incision times for the 10 charts sampled each week.

What we want to see: process behavior over 30 weeks

WeekDecision to

Incision Time (minutes)

X-bar Standard Deviation

404536495035345039474952364950423850445044523649464136524351

1

2

3

42.5 6.4

46 5.6

45 6.1

Example from Benneyan, Int J Six Sigma and Competitive Advantage, 2008

C-section Incision: X-Bar & S Charts

S chart looks at the variation withinsubgroups. High variation within subgroups it makes it difficult to interpret variation between subgroups.

X-bar chart looks at the variation betweensubgroups.

Example from Benneyan, Int J Six Sigma and Competitive Advantage, 2008

Why two charts (Xbar & S)?

2 types of possible process changes (unnatural variation)

Mean or standard deviation

Either can change without the other

One chart to detect each type of change

Change in mean

Change in SD

How to Interpret a Control Chart

Goal to identify common or special cause variation and take appropriate action

Probability-based rules

Rules designed to balance Type I (alpha error, p<0.05) and Type II errors

Rules for Identifying Special Cause

Rules for Identifying Special Cause

Gupta M and Kaplan HC, Clinics in Perinatology, 2017.

Quiz: Interpretation

Points outside control limits?

Runs of 8 or more consecutive points on one side of the centerline?

Trends of 6 or more consecutive points increasing or decreasing?

Two of three consecutive points near the outer control limits?

Yes

Yes

No

Benneyan JC, et al. Qual Saf Health Care. 2003;12:458-464.

LCL

UCL

Yes

Quiz: Interpretation

This process appears to be in control, i.e. no special cause variation, only common cause variation.

Points outside control limits?

Runs of 8 or more consecutive points on one side of the centerline?

Trends of 6 or more consecutive points increasing or decreasing?

Two of three consecutive points near the outer control limits?

No

No

No

Benneyan JC, et al. Qual Saf Health Care. 2003;12:458-464.

No

Stable Process Process defined and predictable Range of variation (performance)

intrinsic to the process• Common Cause variation:

sampling error, noise; no signals

Changing results from a stable process requires a new process… current process designed to get what it gets

noise

Unstable Process Process not defined, unpredictable Range of variation not intrinsic–

influenced by external factors• Special cause: outside chance

variation – signals Changing results achieved by an unstable

process begins with removing the special causes to establish core process Learning from experience with unstable

processes is limited… all one can say is that no one knows what will happen next! signal

The Goal: Standardize then Improve

Standard process

Improved process

05

101520253035404550

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58

1 2 3Smal

ler i

s bet

ter

Unstableprocess

c/o J. Benneyan

Why Not Just Use a Run Chart?

0

3

6

9

12

15

18

21

24

27

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29

Num

ber A

ppt <

30

days

Month

0

3

6

9

12

15

18

21

24

27

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29

Num

ber A

ppt <

30

days

Month

Appointment AccessNumber pts appt < 30 days of request

Pure chaos!No standard process at all

Why Not Just Use a Run Chart?

Why Control Charts Over Run Charts?

Allow you to distinguish between common cause and special cause variation

More sensitive / more powerful in detecting changes

Estimate capability of a stable process more accurately predict performance

But… more difficult to generate

Take Home Points1. Control charts allow you to distinguish common cause and

special cause variation.

2. Key to control chart use is statistically-derived control limits to assess variation.

3. The type of data determines the type of control chart.

4. Probability-based rules should be used to identify special cause variation and interpret control charts.

5. Control charts are generally more powerful than run charts if you have enough data, but run charts can still be very useful.

References Benneyan, J.C., R.C. Lloyd, and P.E. Plsek, Statistical process control as a tool

for research and healthcare improvement. Qual Saf Health Care, 2003. 12(6): p. 458-64.

Benneyan, J.C., The design, selection, and performance of statistical control charts for healthcare process improvement. Int J Six Sigma and Competitive Advantage, 2008. 4(3):p.209-239.

Gupta, M, and H Kaplan, Using Statistical Process Control to Drive Improvement in Neonatal Care: A Practical Introduction to Control Charts. Clinics in Perinatology, 2017. 44:627-644.

Provost, L.P. and S.K. Murray, The Health Care Data Guide: Learning From Data for Improvement. 1st ed. 2011, San Francisco, CA: Jossey-Bass. 445 p.