informs rocky mtn presentation 03-17-11

1Rocky Mountain INFORMS, March 17, 2011

Quantitative Approaches to Improve Healthcare Access and Quality

Rocky Mountain INFORMS Chapter Meeting

A panel presentation, featuring the work of:Linda LaGanga, Ph.D.1,3

Steve Lawrence, Ph.D.2C.J. McKinney, Ph.D. Candidate1,4

Antonio Olmos, Ph.D.1Michele Samorani, Ph.D. Candidate2

1. Mental Health Center of Denver2. University of Colorado-Boulder3. University of Colorado-Denver

4. University of Northern Colorado


Healthcare Issues we address

To overbook or not? If we schedule them, will they come? What would Deming do to improve

healthcare? To achieve efficiency and effectiveness

of healthcare


Where is our work developed and documented? Experience and data from

the Mental Health Center of Denver Community mental health center serving over 14,000

people per year Surveys and interviews of other healthcare

providers/systems Presented at INFORMS annual conferences Other conferences:

Production & Operations Management Society Decision Sciences Institute Mayo Clinic Conference on OR/Systems Engineering in

Healthcare American Evaluation Association


Read more about it…

Decision Science Journal (May, 2007) Journal of Operations Management

(2010, in press) Conference presentations and proceedings at

http://www.outcomesmhcd.com/Pubs.htm Research posters on the wall

opposite this room


Appointment Scheduling and Overbooking Clinic Overbooking to Improve Patient Access

and Increase Provider Productivity LaGanga, L. R., & Lawrence, S. R. (2007).

Clinic overbooking to improve patient access and provider productivity.Decision Sciences, 38(2), 251 – 276.


Simple Overbooking Example


Model Assumptions Number of patients booked, K:

E(K) = SK = N S = Show rate, N = target n of patients K = N/S

Patients scheduled at even intervals throughout the day T = N/K = S Inter-appointment times compressed by the show rate

Patients arrive on time with probability S Patient service times deterministic

Added variability in final version


Overbooking: Best Case

No patients waitWaiting

No overtimeD9D7D5D3D1Service

X10A9X8A7X6A5X4A3X2A1Arrivals

Expected number of patients (5) arrive, evenly spacedBest Case

65.554.543.532.521.510.50Start Time

13121110987654321Time Slot

OvertimeRegular Time

10 appointment slots / session; 50% show rate

5 patients seen; no provider idle time; no patients wait; no clinic overtime


Overbooking: Bunched Early

W7W5W3W2Waiting

No overtimeD7D5D3D2D1Service

X10X9X8A7X6A5X4A3A2A1Arrivals

Expected number of patients (5) arrive, bunched earlyCase 1

65.554.543.532.521.510.50Start Time

13121110987654321Time Slot



5 patients seen; no provider idle time; 4 patients wait; no clinic overtime


Overbooking: Late Arrival

No patients waitWaiting

OT D10ID7D5D3D1Service

A10X9X8A7X6A5X4A3X2A1Arrivals

Expected number of patients (5) arrive, one late arrivalCase 2

65.554.543.532.521.510.50Start Time

13121110987654321Time Slot



5 patients seen; 10% provider idle time; no patients wait; 10% clinic overtime


Overbooking: Bunched Late

W9W8Waiting

OTD9D8D7IID3D1Service

X10A9A8A7X6X5X4A3X2A1Arrivals

Expected number of patients (5) arrive, bunched lateCase 3

65.554.543.532.521.510.50Start Time

13121110987654321Time Slot



5 patients seen; 20% provider idle time; 2 patients waiting; 20% clinic overtime


Overbooking: Extra Arrival

W9W7W5W3W2Waiting

OTD9D7D5D3D2D1Service

X10A9X8A7X6A5X4A3A2A1Arrivals

More patients arrive (6) than expected (5)Case 4

65.554.543.532.521.510.50Start Time

13121110987654321Time Slot



6 patients seen; no provider idle time; 5 patients waiting; 20% clinic overtime


Overbooking Utility Model


Overbooking Utility Model

Maximize clinic “utility” Trade-off Patient access (number of patients seen) Average patient waiting times Expected clinic overtime

Note that provider productivity is implicit in this model


Relative Benefits and Penalties

= Benefit of seeing additional patient = Penalty for patient waiting = Penalty for clinic overtime

The values of , , and don’t matter Just their ratios or relative importance


Utility Function

U SN

( )N OU U U A SN W O

Expected utility without overbooking

Expected utility with overbookingOU A W O

Expected net utility with overbooking


Utility Function Described

( )NU A SN W O

Utility Benefit ofPatients that

“Show”

Less Utility Benefitw/o Overbooking

Less PatientWaiting Penalty

Less ClinicOvertime Penalty

Net Utility Benefit from Overbooking (could be negative)


Simulation Experiments

Five clinic size levels N N = {10, 20, 30, 40, 50}

Ten show rates S S = {100%, 90%, … , 10%}

Full factorial experiment SN = 5 × 100 = 500 factor levels 10,000 replications per factor 500,000 observations


Regression Analysis

Results from simulation analyzed using regression analysis

Regression equations obtained Expected patient wait times Expected clinic overtime Expected provider productivity

All coefficients significant R2 = 98%+


Sensitivity to Service Uncertainty

-10

0

10

20

30

40

0.0 0.2 0.4 0.6 0.8 1.0

Service Time Variability

Aver

age

Net U

tility

N50R90N30R90N50R50N30R50N10R90N10R50N10R10N30R10N50R10

Average of net utility UN with overbooking as a function of service time variability cs , with and (=1, =0.5, τ =1.2)


Conclusions

Overbooking is one solution for appointment no-shows

Can significantly improve performance Patient access (more patients seen) Clinic utility

But with a cost Increased patient waiting & clinic overtime

Good for some clinics, not for others


Directions for Future Work

Scheduling policies Double booking Wave scheduling

Optimal overbooking policies Current overbooking policy is not “optimal” Dynamic programming

Nonlinear waiting & overtime functions Long waits much worse than short waits


Lean Options for Walk-In, Open Access, and Traditional

Appointment Scheduling in Outpatient Health Care Clinics

© 2008 – Linda LaGanga and Stephen Lawrence

Linda R. LaGanga, Ph.D.Director of Quality SystemsMental Health Center of DenverDenver, CO USA

Stephen R. Lawrence, Ph.D.Leeds School of BusinessUniversity of ColoradoBoulder, CO USA

Mayo Clinic Conference on Systems Engineering & Operations

Research in Health CareRochester, Minnesota – August 17, 2009

Additional information available at: http://Leeds.colorado.edu/ApptSched


Data Mining in Appointment Scheduling

Michele SamoraniPhD Candidate

Leeds School of Business, University of Colorado at Boulder


Finding Patterns with Data Mining


Young clients are more likely to keep appointments with no reminder call

DECISION TREE


If clients are under the age of 26.3 years old and have low average CRM (<.5), then they are more likely to keep their appointments

CLUSTERING


Using Data Mining to Schedule Appointments


Overbooking – Shortcomings

Suppose service time = 30 minutes

Little waiting time and no overtime

Some waiting time and a high overtime

If we could predict which patients show up and which don’t, we could obtain a more controllable schedule

1 0 1 10

9:00

9:20

9:40

10:00

10:20

10:40

11:00

11:20

11:40

1 1 1 0

12:00

0 1 1 10

9:00

9:20

9:40

10:00

10:20

10:40

11:00

11:20

11:400 1 1 1

12:00


The methodEvery time a visit request arrives:1)A classifier is used to predict if it shows or not (for each day)2)The visit request is scheduled by solving a stochastic program through column generation

Non‐controllable parameters•Service time•Revenue from seeing a patient•Clinic overtime cost•Waiting time cost

Controllable parameters•Number of slots K•Scheduling horizon h•Classification performance:

– Sensitivity (sn)– Specificity (sp)

How good we are at retrieving showing patients

How good we are at retrieving non‐showing patients


Productivity vs Punctuality Productivity: number of patients seen. It is increased by:

Punctuality: 1/(overtime + waiting time). It is increased by:


Real world case: MHCD

After playing for a few hours with the MHCD data set, I can achieve any of the following classification performances: sn = 0.9, sp = 0.5 sn = 0.7, sp = 0.7 sn = 0.6, sp = 0.8

Show rate Same day 1 day 2 days 3 days 4 days R

Low .74 .64 .65 .62 .61 .65

MHCD .87 .74 .75 .72 .71 .76

• Goal: Find the best policy for MHCD in terms of:– Overbooking– Open Access– Data Mining


Policy DM OB OA .

(min) (min) ∗ ∗

1 No No No 6.39 0.00 0.00 5.99 8 4

2 No No Yes 6.39 0.00 0.00 5.99 8 1

3 No Yes No 7.10 36.22 20.61 8.37 12 4

4 No Yes Yes 7.22 35.33 21.37 8.40 12 1

5

.6, .8 No No 6.82 0.00 0.00 6.44 8 5

.7, .7 No No 6.99 0.00 0.00 6.62 8 4

.9, .5 No No 7.36 0.00 0.00 7.00 8 5

6

.6, .8 No Yes 6.84 0.00 0.00 6.44 8 1

.7, .7 No Yes 6.83 0.00 0.00 6.43 8 1

.9, .5 No Yes 6.66 0.00 0.00 6.27 8 1

7

.6, .8 Yes No 7.24 21.11 14.96 7.78 12 3

.7, .7 Yes No 7.42 29.33 17.88 8.33 12 5

.9, .5 Yes No 7.58 40.78 23.56 9.03 12 2

8

.6, .8 Yes Yes 7.35 25.00 15.92 8.03 12 1

.7, .7 Yes Yes 7.44 28.44 18.51 8.28 12 1

.9, .5 Yes Yes 7.32 35.22 19.83 8.47 12 1

Data Mining

Overbooking

Open Access


Policy DM OB OA .

(min) (min) ∗ ∗

1 No No No 7.28 0.00 0.00 6.88 8 4

2 No No Yes 7.27 0.00 0.00 6.87 8 1

3 No Yes No 7.47 29.07 15.32 8.39 10 5

4 No Yes Yes 7.52 28.00 15.62 8.39 10 1

5

.6, .8 No No 7.49 0.00 0.00 7.11 8 5

.7, .7 No No 7.56 0.00 0.00 7.18 8 2

.9, .5 No No 7.85 0.00 0.00 7.47 8 2

6

.6, .8 No Yes 7.56 0.00 0.00 7.17 8 1

.7, .7 No Yes 7.59 0.00 0.00 7.19 8 1

.9, .5 No Yes 7.52 0.00 0.00 7.12 8 1

7

.6, .8 Yes No 7.60 20.73 13.26 8.14 10 2

.7, .7 Yes No 7.65 12.11 8.69 7.83 9 5

.9, .5 Yes No 7.86 15.22 9.81 8.18 9 2

8

.6, .8 Yes Yes 7.62 21.87 13.83 8.20 10 1

.7, .7 Yes Yes 7.64 24.87 14.53 8.36 10 1

.9, .5 Yes Yes 7.57 28.13 15.82 8.44 10 1


Conclusions Data mining can improve appointment scheduling in the

presence of no-shows If adopted in conjunction with overbooking, data mining can

either increase punctuality or productivity, depending on sensitivity and specificity

In case of low show rate, the advantage obtained by using overbooking is similar to the one obtained with data mining.

On the other hand, in case of high show rate, data mining is a superior technique

Interestingly, if we can achieve a decent classification performance, using open access is the worst choice

Thank you for your attention. Questions?


What about the scheduling horizon h? h does not have any significant impact by itself:

But its interaction with sn and sp is significant:


High sensitivity classifier

Classifier


Driving Clinical Quality Driving Clinical Quality Improvement through Mental Health Improvement through Mental Health Recovery Control ChartsRecovery Control Charts

CJ McKinney, MA*Antonio Olmos, PhDLinda Laganga, PhD

Mental Health Center of DenverDenver, CO, USA

* - Corresponding Author

INFORMS Annual Meeting 2009San Diego, CA

October, 11th, 2009


Literature Olmos-Gallo, P.A. DeRoche, K.K. (2010, August). Monitoring Outcomes

in Mental Health Recovery: The Effect on Programs and Policies.Advances in Mental Health (9)1, 8-16. http://amh.e-contentmanagement.com/archives/vol/9/issue/1/ contact P. Antonio Olmos for a copy of the publication

McKinney, C.J., Olmos-Gallo, P.A. McLean, C., LaGanga, L.R. (August 2010). Driving Clinical Quality Improvement through Mental Health Recovery Control Charts. Presented at the 3rd Annual Mayo ClinicConference on Systems Engineering & Operations Research in Health Care, Rochester, MN. Awarded First Place for Best Poster Presentation.

Clark, C.R., Olmos-Gallo, P.A. (2007). Performance Measurement: A signature approach to outcomes measurement improves recovery. National Council Magazine, 3, 26-28.

Glover, H. (2005). Recovery based service delivery: Are we ready to transform the words into a paradigm shift? Australian e-Journal for the Advancement of Mental Health, 4(3), www.auseinet.com/journal/vol4iss3/glovereditorial.pdf (accessed 15 May 2009)

Montgomery, D. C. (2005) Introduction to Statistical Quality Control, Fifth Edition. Hoboken, NJ: John Wiley and Sons, Inc.

Olmos-Gallo, P. A., DeRoche, K. K., McKinney, C. J., Starks, R., & Huff, S. (2009). The Recovery Markers Inventory: Validation of an instrument to measure factors associated with recovery from mental illness. Working paper


The Heart of Recovery Measurement


CheckCheck DoDo

PlanPlanActAct

Continuous Improvement


Quality Components in Mental Health Services

Performance

Reliability

Durability

Perceived Quality

Conformance to Standards

How well are MH services working? Are consumers improving in their recovery?

How often do we see improvements in recovery? How consistent are the outcomes across consumers?

How long does the consumer retain the recovery-supportive skills and tools taught through MH services?

How does the consumer perceive our ability to support MH recovery? Community?

Are we meeting program fidelity standards?

Quality Components Relationship to MH Services


Quality Control in Mental HealthQuality Control in Mental Health

Allocate and reallocate clinical resources more efficiently

Improve and maintain clinical program fidelity Reduce length of treatment, while sustaining same

level of recovery and recovery supportive factors Increase the number of consumers served, while

decreasing burden on case managers/therapists Identify most effective programs based upon

consumer needs


Mental Health Recovery Concept of Recovery has taken root

around the world Working Definition (MHCD): “A non-linear process of growth by which people move

from lower to higher levels of fulfillment in the areas of hope, safety, level of symptom interference, social networks, and activity.”

Federal Grant (SAMHSA) for Transformation to Recovery-Oriented Mental Health Systems

For information on the Recovery Transformation Summit, see http://www.gmhcn.org/files/RRecovery_Newsletter_Fall2010.pdf


Mental Health Recovery Outcomes

MHCD has developed 3 consumer specific recovery outcomes Consumer Recovery Measure – (Consumer Perspective)

Hope, Safety, Activity, Level of Symptom Management, Social Networks

Recovery Marker Inventory – (Clinician Perspective) Housing, Employment, Education, Active Growth, Participation, and Symptom Management

Recovery Needs Level – (Clinical Algorithm) Provides for one of 5 levels of treatment based upon clinical criteria

The examples in this presentation will utilize the Consumer Recovery Measure.


Relationship among Recovery Outcomes(1) Recovery Marker

Inventory (RMI)(Longitudinal data to support

clinical decision making)

(3) Consumer Recovery Measure (CRM)

(Consumer’s perception of their own recovery)

(2) Promoting Recovery in Organizations (PRO)

(Consumer’s perceptions of how well specific programs and staff are

promoting recovery)

To what degree is RECOVERY

happening for consumers at MHCD(Formative and summative

evaluation of recovery)

(4) Recovery Needs Level

(RNL)(Appropriate level of services)


Consumer Recovery Measure v3.0

The CRM V3.0 includes the 15 items listed below:2. Lately I feel like I’ve been making important contributions (active-growth)4. I have hope for the future (hope)5. I am reaching my goals (active growth)7. I have this feeling things are going to be just fine (hope)8. Recently my life has felt meaningful (hope)9. Recently, I have been motivated to try new things (active-growth)11. There are some people who cause me a lot of fear (safety)12. I get a lot of support during the hard times (social network)14. In most situations, I feel totally safe (safety)15. My life is often disrupted by my symptoms (symptom interference)16. Sometimes I’m afraid someone might hurt me (safety)17. I have people in my life I can really count on (social network)18. Life’s pressures lead me to lose control (symptom interference)19. I have friends or family I really like (social network)20. My symptoms interfere less and less with my life (symptom interference)21. When my symptoms occur, I am able to manage them without falling

apart (symptom interference)


Quality Control Issues in Recovery

Multiple sources of variability Measurement Consumer System

Changing environmental, treatment, and consumer specific factors affect outcome measurements.

Difficulty in detection of small changes due to large variability within and among consumers


Multilevel Modeling and Recovery

Multilevel modeling allows for the partitioning of variance among multiple levels of nesting, i.e. measures within consumers within therapists

Allows for regression based correction of expected outcomes for any unit at any level, i.e. conditional estimates based upon consumer characteristics in environment or treatment.

Can be used to simultaneously monitor multiple aspects of the system from measurements to clinical sites.

Based upon Mixed-Effects ANOVA design


Example of Multilevel modeling concepts

Consumer Level Effect

CRM Scores

=

Intake

+

Time in Tx =

=

Intercept

Intercept

Mood Disorder

Mood Disorder

+

+

=Intercept

ACT Tx+

=Intercept

ACT Tx+

=Intercept

ACT Tx+

=Intercept

ACT Tx+

Typical SLR Model System Level Effect

Higher Level Effects


Multilevel Regression Corrected Control Charts

CUSUM for Consumers (between consumer comparisons)

Allows for determination of a consumer’s progress as compared to peers in same treatment with environmental and demographic similarities


Example MRC-CUSUM Self Comparison


Example MRC-CUSUM Peer Comparison


Utilization of MRC-CUSUM

Improved allocation of resources – by allowing consumer comparison to peers

Identification of factors that may promote/inhibit recovery

Provide feedback regarding progress and relapse more quickly to clinicians


Multivariate Control Chart

Bivariate Control Chart for plotting of regression parameters (intercept and slopes)

Corrections may be made based upon environmental, treatment, and demographic characteristics


58

I II

IVIII


I II

IIIIV

BELOW AVG.

ABOVE AVG.

Decreasing

Increasing

Recovery InterceptRe

covery Slope

NOTE: ANY Outlier within a quadrant indicates it is farther away from the average than would be expected under typical circumstances.


Utilization of Bivariate Control Chart

Identify “outlying” consumers to help determine aspects of a program that promote self-perceived recovery, and those aspects that may be a deterrent to improvement in self-perceived recovery.

Allow for identification of consumers who may need further resources or different treatment.

Allows for overview of consumer progress, where comparisons over time may allow for evaluation of process changes and overall consumer effect.


Summary of Benefits

Allow for more efficient allocation of treatment and resources.

Identify program aspects that promote or deter improvement in self-perceived recovery.

Identify consumer in need of additional treatment or resources.

Allow for the identification of consumer and system factors that affect or interact with consumer outcomes and program effectiveness.

Being able to cater to differing needs of the wide variety of consumers served.

Identification of Episodes of Care


Moving forward in recovery models to drive quality improvement

Statistical Models

Information Technology

Knowledge Building& Dissemination:Learning CollaborativesStaff Involvement,Training


Future Directions to Drive Recovery System Improvement

Identify clinically significant patterns Expand to other recovery measures and aspects. Coordinate with data mining to identify

relationships between services and recovery outcomes

Automate quality control process Integrate fully into clinical quality review processes Develop accessible reporting and dashboard

systems for clinicians and managers


More information

If you would like to see more information concerning MHCD’s research and work with Recovery please visit:

http://www.outcomesmhcd.com/http://www.reachingrecovery.org/

Or contact [email protected]


Extra slides that were mentioned but not presented on 3/17/11 due to time limitations

From Mayo Clinic Conference on Operations Research & Systems Engineering in Healthcare

Lean Options for Walk-In, Open Access, and Traditional Appointment Scheduling in Outpatient Health Care Clinics (LaGanga & Lawrence, 2009)

Includes further development to appointment scheduling models toinclude metaheuristic optimization of overbooking levels

Comparison of traditional scheduling, open-access,and walk-in policies

Lean process improvement to reduce no-shows and expand intake capacity.

Condensed slide set. See http://www.outcomesmhcd.com/Pubs.htmfor complete, original presentation.

Driving Clinical Quality Improvement Through Mental Health Recovery Control Charts (McKinney, Olmos, McLean, LaGanga, 2010)

Poster presentation First Place Award









Additional information available at: http://www.outcomesmhcd.com/Pubs.htm


1. Background on Appointment Scheduling


Motivation Healthcare Capacity Funding restrictions Demand exceeds supply Serve more people with limited resources

Manufacturing Scheduling Resource utilization Maximize throughput

Healthcare Scheduling as the point of access

Maximize appointment yield


2. Lean Approaches

Rapid Improvement Capacity Expansion (RICE) Team January, 2008

Article in press, Journal of Operations Management (2010). Available at http://dx.doi.org/10.1016/j.jom.2010.12.005


Lean Approaches Reducing Waste Underutilization Overtime No-shows Patient Wait time

Customer Service Choice Service Quality Outcomes


Lean Process Improvement in Healthcare Documented success in hospitals

ThedaCare, Wisconsin Prairie Lakes, South Dakota Virginia Mason, Seattle University of Pittsburgh Medical Center Denver Health Medical Center

Influences Toyota Production System Ritz Carleton Disney

Hospitals to Outpatient Clinics run by hospitals Collaborating outpatient systems


Lean Process Improvement: One Year AfterRapid Improvement Capacity ExpansionRICE Results

Analysis of the1,726 intake appointments for the one year before and the full year after the lean project

27% increase in service capacity from 703 to 890 kept appointments) to intake new consumers

12% reduction in the no-show rate from 14% to 2% no-show

Capacity increase of 187 additional people who were able to access needed services, without increasing staff or other expenses for these services

93 fewer no-shows for intake appointments during the first full year of RICE improved operations.


Lean Process Improvement:RICE Project System Transformation

Appointments Scheduled and No-Show Rates

050

100150200250300350400450

Mon Tue Wed Thu Fri Mon Tue Wed Thu Fri

App

oint

men

ts

0%

5%

10%

15%

20%

AppointmentsNo-Show Rate

Year Before Lean Improvement

Year After Lean Improvement


How was this shift accomplished? Day of the week: shifted and added Tuesdays and Thursdays

Welcome call the day before Transportation and other information Time lag eliminated Orientation to Intake Assessment

Group intakes Overbooking Flexible capacity


Lean Scheduling Challenge Choice versus Certainty Variability versus Predictability Sources of Uncertainty / Variability No-shows Service duration Customer (patients’) Demand

Time is a significant factor Airline booking models?


3. Response to Overbooking


Sample Responses Campus reporter’s visit to student health

center “Not now and never will” Patient waits 15 – 20 minutes New administration, new interests

Morning News Radio “Overbooking leading to increased patient

satisfaction? That just doesn’t make any sense!” Public Radio Interviewer Benefits of increased access at lower cost


78

Other Responses Practitioners

Dentists General medicine Child advocacy

How should we overbook? Other options Lean Approaches Open Access (Advanced Access) Walk-ins


4. Enhanced Appointment Scheduling Model

0%

5%

10%

15%

20%

0 1 2 3 4 5 6 7 8 9 10 11 12

Number Waiting (k)

Prob

abili

ty


Objectives of Research Optimize patient flow in health-care clinics Traditionally scheduled (TS) clinic Some patients do not “show” for scheduled

appointments TS clinic wishes to increase scheduling flexibility Some capacity allocated to “open access” (OA)

appointments, OR Some capacity allocated to “walk-in” traffic

Balance needs of clinic, providers, and patients


Objectives of Research

Study impact of open access and walk-in traffic When is open access or walk-in traffic

beneficial? What mix of TS, OA, and WI traffic is

best? What are trade-offs of TS, OA, and WI

on clinic performance?


Relative Benefits and Penalties = Benefit of seeing additional client = Penalty for client waiting = Penalty for clinic overtime Numéraire of , , and doesn’t matter Ratios (relative importance) are important

Allow linear, quadratic, and mixed (linear + quadratic) costs


Linear & Quadratic Objectives

1, 1,1 1

ˆˆ 1ˆN k

jk N k N kj k k i k

U A k i kA

S

Linear Utility Function

Quadratic Utility Function 2 2

1, 1,1 1

ˆˆ 2 1 1ˆN k

jk N k N kj k k i k

U A k i kA

S

Benefit from patients served

Patient waiting penalties during normal clinic ops

Patient waiting penalties during clinic overtime

Clinic overtime penalties


Heuristic Solution Methodology 1. Gradient search

Increment/decrement appts scheduled in each slot Choose the single change with greatest utility Iterate until no further improvement found

2. Pairwise interchange Exchange appts scheduled in all slot pairs Choose the single swap with greatest utility Iterate until no further improvement found

3. Iterate (1) and (2) while utility improves4. Prior research: Optimality not guaranteed, but

almost always obtained


How does Open Access contribute to leaner scheduling?1. It provides a more reliable method of

overbooking.2. It eliminates the uncertainty of demand for

same-day appointments.3. It guarantees that patients will be seen when

they want.4. It reduces uncertainty caused by no-shows.5. It eliminates waste caused by unfilled

appointments.


5. Computational Results

0

1

2

3

4

5

6

7

8

9

10

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Net

Util

ity p

er P

rovi

der

Open Access (OA) Traffic (% of capacity)

Walk-ins

Open Access

-6.190

1

2

3

4

5

6

7

8

9

10

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Net

Util

ity p

er P

rovi

der


Walk-ins

Open Access

-6.19


Computational Trials 44 sample problems solved Session size N = 12 Appointment show rate = 70% Number of providers P = {1, 2, 4, 8} OA call-in rate = {0%, 10%, …100%} capacity With P = 4 and N = 12, then = 24 is 50% of capacity

Walk-in rate = {0%, 10%, …100%} of capacity With P = 4, then = 2 is 50% of capacity

Quadratic costs Parameters =1.0, =1.0, =1.5


Patients Seen

10

11

12

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Patie

nts

Seen

per

Pro

vide

r

OA or WI Traffic (% of capacity)

Walk-ins

Open Access

2 Providers (P=2)10

11

12

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Patie

nts

Seen

per

Pro

vide

r


Walk-ins

Open Access

2 Providers (P=2)

N=12, P=1, =0.7, =1.0, =1.0, =1.0, =1.5


Patient Waiting Time

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Expe

cted

Wai

ting

Tim

e / P

atie

nt


Walk-ins

Open Access

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Expe

cted

Wai

ting

Tim

e / P

atie

nt


Walk-ins

Open Access

N=12, P=1, =0.7, =1.0, =1.0, =1.0, =1.5


Clinic Overtime

0.0

0.5

1.0

1.5

2.0

2.5

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Exp

ecte

d P

rovi

der O

vert

ime

(dtim

e un

its)


Walk-ins

Open Access

0.0

0.5

1.0

1.5

2.0

2.5

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Exp

ecte

d P

rovi

der O

vert

ime

(dtim

e un

its)


Walk-ins

Open Access

N=12, P=1, =0.7, =1.0, =1.0, =1.0, =1.5


Provider Utilization

60%

65%

70%

75%

80%

85%

90%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Expe

cted

Pro

vide

r Util

izat

ion


Walk-Ins

Open Acess

60%

65%

70%

75%

80%

85%

90%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Expe

cted

Pro

vide

r Util

izat

ion


Walk-Ins

Open Acess

N=12, P=1, =0.7, =1.0, =1.0, =1.0, =1.5


Net Utility

N=12, P=1, =0.7, =1.0, =1.0, =1.0, =1.5

0

1

2

3

4

5

6

7

8

9

10

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Net

Util

ity p

er P

rovi

der


Walk-ins

Open Access

-6.190

1

2

3

4

5

6

7

8

9

10

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Net

Util

ity p

er P

rovi

der


Walk-ins

Open Access

-6.19


6. Insights and Recommendations


Managerial Implications TS appointments provide better clinic utility

than does WI traffic or OA call-ins Any WI or OA traffic causes some decline in utility

An all-WI or all-OA clinic performs worse than any clinic with some TS appointments Even a relatively small percentage of scheduled

appointments can significantly improve clinic utility Degree of improvement depends on number of

providers A mix of TS appointments with some OA or WI

traffic does not greatly reduce clinic performance (utility)


Insights from the Model Loss of utility with WI traffic is due to the long

right-tail of Poisson distribution Excessive patient waiting & clinic overtime

Loss of utility with OA traffic is due to uncertainty about number of OA call-ins

TS appts reduce patient waiting and clinic overtime Binomial distribution has truncated right tail

Multiple providers improves clinic utility Portfolio effect – variance reduction









Questions and comments? [email protected]([email protected]), [email protected]. Further information at http://www.outcomesmhcd.com/Pubs.htm


INTRODUCTION

Driving Clinical Quality Improvement Through Mental Health Recovery Control ChartsC.J. McKinney, Pablo A. Olmos, Cathie McLean, Linda R. LaGanga,

Division of Quality Systems, Mental Health Center of Denver, Denver, CO

Every community mental health center focuses on clinical quality. Benefits of effective service delivery support quality through:

• optimize resource allocation,• increase consistency in consumer outcomes, • increase service fidelity, • decrease administrative load on clinicians, and • increase access to consumer services.

This poster presents our development of a set of reliable and valid mental health recovery measures, which we combine for a multi‐perspective assessment of recovery progress, which anchors an objective clinical quality control system.

RECOVERY ASSESSMENTMHCD consistently collects, reviews, and analyzes data across all consumers on four different recovery‐oriented outcome measurement tools. The combined data from these assessments provide multi‐perspective viewpoints for a more comprehensive picture of the consumer’s recovery experience and what factors may be impacting their recovery. It also provides supporting information to ensure the consumer is placed at a level of care that appropriately reflects their needs.

Recovery Marker Inventory – Clinician AssessmentAssessments are recorded on seven factors associated with recovery: Employment, Learning/Education, Activity/Growth Orientation, Symptom Interference, Participation in Services, Housing, and Substance Use.

Documentation of this data provides the clinician with a longitudinal perspective – from both an overall standpoint, as well as more specific recovery dimensions. These observations can then be used to help guide clinical discussion with the consumer, and indicate treatment focus.

RECOVERY ASSESSMENT continuedRecovery Needs LevelThe Recovery Needs Level is a series of indicators that through an objective algorithm assigns the consumer to an appropriate clinical service level. The RNL is completed by the clinician every six months and as needed. The measure consist of 15 different dimensions such as the GAF, Residence, Case Management, Substance Abuse, and Service Engagement.

Promoting Recovery in OrganizationsThe PRO survey is completed by the consumer, and consists of 7 sections covering all major service positions at MHCD, i.e. front desk, nursing/medical, case management, and rehabilitation. This data is collected annually through a random sampling of consumers. The survey summaries are then utilized to determine how well the teams and system are promoting recovery ideals.

The Recovery Outcome Tools have enabled us to develop a quality review system to monitor individual consumer outcomes and recommend review in cases where the consumer may not be progressing as expected. We are able to do this in three ways:

1.The Consumer Recovery Profile provides a snapshot of a person’s current mental health recovery progress. It demonstrates through graphs and tables the current status of a consumer to aid in service planning.

QUALITY CONTROL CHARTS

Self‐Comparing Control Chart Peer‐Comparing Control Chart

Utilization Review Form

Consumer Recovery Measure – Consumer AssessmentWith the Consumer Recovery Measure, the consumer rates agreement or disagreement with statements regarding their current recovery experience. These responses gauge consumer perspective on five dimensions of recovery: Symptom Management, Sense of Safety, Sense of Growth, Sense of Hope, and Social Activity.

Graphic representation of this data is shared with the consumer to initiate clinical discussion about changes in these areas, what the consumer attributes the changes to, and possible relationships between categories. This promotes insight, and empowers the consumer to share their story in a new and different way.

2. The Recovery Change Chart automatically identifies consumers needing further review by flagging those with substantial change in their recovery outcomes. A flag occurs whenever a consumer deviates from their expected outcomes for an extended period of time or if the deviations are large.

3. The Utilization Review Process: When a consumer is “flagged” by the Change Chart they will be an automatic candidate for a utilization management review. This review is done by other clinicians reviewing a consumer’s medical record to determine if a gap in services has occurred and if other services should be considered. The recommendations from this review are forwarded to the program manager for further review and implementation.

CONCLUSION & FUTURE DIRECTIONS

For more information about research or mental health recovery at MHCD, please view conference

presentations on our website:www.outcomesmhcd.com

Consistent with continuous quality improvement, integration of these tools into the clinical workflow is a constantly evolving process. We feel the following are basic needs to meet, and opportunities for operational enhancement:

• Education of Clinical staff, Executive Management, Consumers, and other stakeholdersas to the value of outcomes data collection and analysis and integration into the clinical practice

• Technological ability to “communicate” with the Electronic Medical Record ‐ the Recovery Profile is connected to the Electronic Medical Record, so it can be easily accessed by clinicians by bringing the information to them, without having to log in or open other data storage sites

• Integration into the daily clinical work flow – clinicians can review outcomes information with consumers during individual sessions, so as to make the information more meaningful; it is employed as part of the Peer Review process; and can be used during six month case reviews

•Automation of Quality Review process – control charts “flag” concerning outcomes outliers and identify them for Utilization Management Review, so as to address and redirect treatment inefficiencies in a timely manner

• Exploration of “super performer” characteristics to identify benchmarks for teams/programs

• Consumer Recovery Portal – consumers will have access to their outcomes data for increased engagement in the recovery process

•Integrate physical and mental healthcare

••Maximize outcomes to improve human lives!Maximize outcomes to improve human lives!

Qualitative Identification of Service Outliers

QUALITY CONTROL CHARTS continuedPresented at the Mayo Clinic Conference on Systems Engineering & Operations Research in Health Care (August 2010), Rochester, MN. Awarded First Place for Best Poster Presentation.

informs rocky mtn presentation 03-17-11

Documents

rate patients

ns patients

overbooking clinic

clinic overtime9rocky

target n of patients

quality rocky mountain

room4rocky mountain

provider idle time