Stochastic models and data driven simulations for

healthcare operations

by

Ross Michael Anderson

B.S., Cornell University (2009)

Submitted to the Sloan School of Managementin partial fulfillment of the requirements for the degree of

Doctor of Philosophy

at the

MASSACHUSETTS INSTITUTE OF TECHNOLOGY

June 2014

© Massachusetts Institute of Technology 2014. All rights reserved.

Author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Sloan School of Management

May 15, 2014

Certified by. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Itai Ashlagi

Assistant Professor of Operations ManagementThesis Supervisor

Certified by. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .David Gamarnik

Professor of Operations ResearchThesis Supervisor

Accepted by . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Dimitris Bertsimas

Boeing Professor of Operations ResearchCo-director, Operations Research Center

2

Stochastic models and data driven simulations for healthcare

operations

by

Ross Michael Anderson

Submitted to the Sloan School of Managementon May 15, 2014, in partial fulfillment of the

requirements for the degree ofDoctor of Philosophy

Abstract

This thesis considers problems in two areas in the healthcare operations: KidneyPaired Donation (KPD) and scheduling medical residents in hospitals. In both areas,we explore the implications of policy change through high fidelity simulations. Wethen build stochastic models to provide strategic insight into how policy decisionsaffect the operations of these healthcare systems.

KPD programs enable patients with living but incompatible donors (referred to aspatient-donor pairs) to exchange kidneys with other such pairs in a centrally organizedclearing house. Exchanges involving two or more pairs are performed by arranging thepairs in a cycle, where the donor from each pair gives to the patient from the next pair.Alternatively, a so called altruistic donor can be used to initiate a chain of transplantsthrough many pairs, ending on a patient without a willing donor. In recent years,the use of chains has become pervasive in KPD, with chains now accounting for themajority of KPD transplants performed in the United States. A major focus of ourwork is to understand why long chains have become the dominant method of exchangein KPD, and how to best integrate their use into exchange programs. In particular,we are interested in policies that KPD programs use to determine which exchangesto perform, which we refer to as matching policies. First, we devise a new algorithmusing integer programming to maximize the number of transplants performed on afixed pool of patients, demonstrating that matching policies which must solve thisproblem are implementable. Second, we evaluate the long run implications of variousmatching policies, both through high fidelity simulations and analytic models. Mostimportantly, we find that: (1) using long chains results in more transplants andreduced waiting time, and (2) the policy of maximizing the number of transplantsperformed each day is as good as any batching policy. Our theoretical results arebased on introducing a novel model of a dynamically evolving random graph. Theanalysis of this model uses classical techniques from Erdos-Renyi random graph theoryas well as tools from queueing theory including Lyapunov functions and Little’s Law.

In the second half of this thesis, we consider the problem of how hospitals shoulddesign schedules for their medical residents. These schedules must have capacity

3

to treat all incoming patients, provide quality care, and comply with regulationsrestricting shift lengths. In 2011, the Accreditation Council for Graduate MedicalEducation (ACGME) instituted a new set of regulations on duty hours that restrictshift lengths for medical residents. We consider two operational questions for hospitalsin light of these new regulations: will there be sufficient staff to admit all incomingpatients, and how will the continuity of patient care be affected, particularly in a firstday of a patients hospital stay, when such continuity is critical? To address thesequestions, we built a discrete event simulation tool using historical data from a majoracademic hospital, and compared several policies relying on both long and short shifts.The simulation tool was used to inform staffing level decisions at the hospital, whichwas transitioning away from long shifts. Use of the tool led to the following strategicinsights. We found that schedules based on shorter more frequent shifts actually ledto a larger admitting capacity. At the same time, such schedules generally reduce thecontinuity of care by most metrics when the departments operate at normal loads.However, in departments which operate at the critical capacity regime, we foundthat even the continuity of care improved in some metrics for schedules based onshorter shifts, due to a reduction in the use of overtime doctors. We develop ananalytically tractable queueing model to capture these insights. The analysis of thismodel requires analyzing the steady-state behavior of the fluid limit of a queueingsystem, and proving a so called “interchange of limits” result.

Thesis Supervisor: Itai AshlagiTitle: Assistant Professor of Operations Management

Thesis Supervisor: David GamarnikTitle: Professor of Operations Research

4

Acknowledgments

I would like to thank my advisors, David Gamarnik and Itai Ashlagi. They have been

very generous with their time, and taught me to be a better problem solver, writer,

and presenter. Just as important, they have helped form my own perspective on what

problems are interesting and important. Finally, they have been great people to work

and socialize with for the last five years.

Thanks to Alvin Roth for serving on my committee and welcoming me into the

world of kidney exchange. He introduced me to important people in the field, and

has been a fun and interesting person to spend time with.

Chapter 4 is joint work with Yashodhan Kanoria. Yash has been everything one

could ask for in a co-author. He is a spectacular problem solver and also willing to

do the hard work when writing.

Thank you to everyone I worked with at Brigham and Women’s Hospital, espe-

cially Danielle Scheurer. Danielle guided David and myself through the complexity of

B&W to find a good problem, and was an advocate for our work within the hospital.

She also organized funding for our research.

I would like to thank all of our contacts at the APD, NKR, and UNOS. Without

their data, much of this research would not have been possible.

Thanks to Dimitris Bertsimas and Patrick Jaillet, the Co-Directors of the ORC.

Beyond serving department well, they both were willing to reach out personally to

me, both in discussing research and providing general advice. They were both ex-

traordinarily committed to the department and the welfare of all their students, and

I am grateful to have been a beneficiary.

I would like to thank Laura Rose and Andrew Carvalho, the administrators for

the ORC. I was not the most punctual when it came to turning my paperwork in on

time (this document included), but they made sure I never fell through the cracks.

John Tsitsiklis and Rob Freund were both excellent instructors, and I learned

quite a lot from them. John has an incredible ability to take something very complex,

determine what is important, and share it. I only took half of a class with Rob, but

5

it was excellent. I was also his TA for a summer and he was a pleasure to work for.

I would like thank all my friends from the ORC. In particular, I’m glad that

I have become so close with Adam, my roommate for three years, and other the

students from my year, Allison, Andre, Florin, Gonzalo, Maokai, and Vishal. I hope

we remain close in the years to come. Thanks to Theo and David, for being both

friends and mentors, to Phil, Michael, Mattheiu, Yehua, Nick, Will, Angie, Kris,

Fernanda, Nathan, John, Iain, Paul and all of the others for five years of good times.

The students of the ORC were the greatest part of MIT for me, both intellectually

and socially, and I consider myself lucky to have been a member.

I would like to thank Bobby Kleinberg, David Shmoys, David Williamson, and

Robert Bland from Cornell, for being great mentors and instructors. I would also like

to thank my roommates Ben, Brian, Bryant, Devin, Matt, Miles and Rob from my

time at Cornell. Had I not found my way into this group, I likely never would have

made it to MIT.

I would like to thank my family for all their support. One day, I will start paying

my cell phone bill and do my own taxes. Hopefully not too soon. I would also like to

say that I’m glad that my brother Greg has moved to Boston, and that I can be here

with him. You only get one family.

With some exceptions. I would like to thank the Relethfords for being a second

family to me for so many years. David is my oldest friend and played a large role in

making me who I am today. Our time at 23 Gorham (with Adam and Aileen) was

probably too much fun. For the second Thanksgivings, the long car rides through

upstate New York, and the many drafts of General Tso’s chicken at China 19, I thank

all of the Relethfords.

Last, I would like to thank Aileen. The last five years have been the happiest five

years of my life, and it has been because of you.

6

Contents

1 Introduction 15

1.1 Kidney Paired Donation . . . . . . . . . . . . . . . . . . . . . . . . . 16

1.1.1 Background & Related Literature . . . . . . . . . . . . . . . . 20

1.1.2 Algorithmic Results . . . . . . . . . . . . . . . . . . . . . . . . 22

1.1.3 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . 23

1.1.4 Dynamic Random Graph Results . . . . . . . . . . . . . . . . 25

1.1.5 Operational & Strategic Insights for KPD . . . . . . . . . . . 26

1.2 Scheduling Medical Residents in Hospitals . . . . . . . . . . . . . . . 32

1.2.1 Background & Related Literature . . . . . . . . . . . . . . . . 34

1.2.2 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . 35

1.2.3 Queueing Model Results . . . . . . . . . . . . . . . . . . . . . 39

1.2.4 Operational & Strategic Insights for Scheduling Medical Resi-

dents in Hospitals . . . . . . . . . . . . . . . . . . . . . . . . . 41

2 Scalable Algorithms for the Kidney Exchange Problem 43

2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

2.2 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

2.3 Algorithms for the KEP . . . . . . . . . . . . . . . . . . . . . . . . . 46

2.3.1 The Edge Formulation . . . . . . . . . . . . . . . . . . . . . . 46

2.3.2 The Cutset Formulation . . . . . . . . . . . . . . . . . . . . . 48

2.4 Algorithm Performance . . . . . . . . . . . . . . . . . . . . . . . . . . 50

2.4.1 Algorithm Performance on Clinical KPD Data . . . . . . . . . 50

2.4.2 Strength of Formulation . . . . . . . . . . . . . . . . . . . . . 53

7

2.5 Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

2.5.1 Long Maximum Cycle Length . . . . . . . . . . . . . . . . . . 56

2.5.2 Bounded Chain Lengths . . . . . . . . . . . . . . . . . . . . . 56

2.5.3 Two Stage Problems . . . . . . . . . . . . . . . . . . . . . . . 57

2.6 Proofs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

2.6.1 Proof of Cutset Separation . . . . . . . . . . . . . . . . . . . . 61

2.6.2 Proof of Strength of Formulation . . . . . . . . . . . . . . . . 61

3 Data Driven Simulations for Kidney Exchange 71

3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

3.2 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

3.3 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

3.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

4 Dynamic Random Graph Models for Kidney Exchange 83

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

4.2 Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

4.3 Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

4.4 Computational Experiments . . . . . . . . . . . . . . . . . . . . . . . 94

4.5 Two-way Cycle Removal . . . . . . . . . . . . . . . . . . . . . . . . . 95

4.6 Three-way Cycle Removal . . . . . . . . . . . . . . . . . . . . . . . . 97

4.7 Chain Removal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

4.8 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

4.9 Proofs of Preliminary Results . . . . . . . . . . . . . . . . . . . . . . 120

5 Data Driven Simulations for Scheduling Medical Residents in Hop-

sitals 123

5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

5.2 Materials and Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 124

5.2.1 Simulation Model of Patient Flow and Assignment to Doctors 124

5.2.2 Physician’s Assistants and Admitting . . . . . . . . . . . . . . 127

8

5.2.3 Resident Admitting Shifts, Schedules, and Policies . . . . . . . 128

5.2.4 Patient Flow for Reassigned Patients . . . . . . . . . . . . . . 136

5.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

5.3.1 Performance Metrics . . . . . . . . . . . . . . . . . . . . . . . 138

5.3.2 Assessing Policies at Historical Patient Arrival Rate . . . . . . 140

5.3.3 Performance Analysis under Increased Patient Volume . . . . 143

5.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

5.4.1 Key Insights . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

5.4.2 Assumptions and Limitations of the Model . . . . . . . . . . . 150

5.5 The Markov Chain Throughput Upper Bound . . . . . . . . . . . . . 152

5.6 Statistical Analysis of Patient Flows . . . . . . . . . . . . . . . . . . 158

5.6.1 Patient Arrival and Departure Data . . . . . . . . . . . . . . . 158

5.6.2 A Statistical Model for Patient Arrivals . . . . . . . . . . . . . 158

5.6.3 A Statistical Model for Patient Departures . . . . . . . . . . . 161

6 Queuing and Fluid Models for Scheduling Medical Residents in Hos-

pitals 165

6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165

6.2 Model, Assumptions and Main Results . . . . . . . . . . . . . . . . . 168

6.3 Numerical Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180

6.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181

6.5 Stability Conditions for Two Schedules. Proof of Theorem 6.1 . . . . 184

6.6 Uniform Bounds for Stationary Performance Measures . . . . . . . . 189

6.7 Fluid Model Approximations. Proof of Theorem 6.2 . . . . . . . . . . 198

6.8 Long Run Behavior of the Fluid Model . . . . . . . . . . . . . . . . . 201

6.9 Interchange of Limits. Proof of Theorem 6.3 . . . . . . . . . . . . . . 213

6.10 Convergence of Reassignments in the Fluid Limit . . . . . . . . . . . 217

6.11 Proof of Lemma 6.9 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222

6.12 Null Recurrence and Transience . . . . . . . . . . . . . . . . . . . . . 228

9

A Lyapunov Functions 233

A.1 Positive Recurrence . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233

A.2 Moment Bounds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234

A.3 Moment Bound with Unbounded Downward Jumps . . . . . . . . . . 237

A.4 Null Recurrence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241

B Random Graphs 243

B.1 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243

B.2 Proofs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244

10

List of Figures

1-1 Examples of cyclic kidney exchanges. . . . . . . . . . . . . . . . . . . 17

1-2 An example of a chain in kidney exchange. . . . . . . . . . . . . . . . 17

1-3 An example of an instance of the KEP. . . . . . . . . . . . . . . . . . 19

2-1 An example of a cutset constraint. . . . . . . . . . . . . . . . . . . . . 49

2-2 A pathological instance of the KEP. . . . . . . . . . . . . . . . . . . . 55

2-3 An instance of the KEP motivating the Cycle Formulation. . . . . . . 63

2-4 An instance of the KEP where the Cutset Formulation is strictly better

than the Subtour Formulation. . . . . . . . . . . . . . . . . . . . . . . 69

2-5 An instance of the KEP where the Subtour Formulation is strictly

better than the Cycle Formulation. . . . . . . . . . . . . . . . . . . . 70

3-1 Simulation results for batching. . . . . . . . . . . . . . . . . . . . . . 76

3-2 Hard-to-match pairs matched with batching. . . . . . . . . . . . . . . 76

3-3 Simulation results for fewer chains. . . . . . . . . . . . . . . . . . . . 78

3-4 Hard-to-match pairs matched with fewer chains. . . . . . . . . . . . . 79

3-5 Simulation results adjusting maximum cycle length. . . . . . . . . . . 80

3-6 Hard-to-match pairs matched with various maximum cycle lengths. . 80

4-1 An illustration of chain matching under the greedy policy. . . . . . . 90

4-2 An illustration of cycle matching under the greedy policy . . . . . . . 90

4-3 Average waiting by batch size for chain and cycle removal. . . . . . . 95

5-1 An example of a resident scheduling policy for GMS. . . . . . . . . . 129

5-2 Sensitivity of dropped patients to patient the arrival rate for Oncology. 144

11

5-3 Sensitivity of jeopardy reassignments and total reassignments to the

patient arrival rate for Oncology. . . . . . . . . . . . . . . . . . . . . 145

5-4 Sensitivity of dropped patients to patient the arrival rate for Cardiology.146

5-5 Sensitivity of jeopardy reassignments and total reassignments to the

patient arrival rate for Cardiology. . . . . . . . . . . . . . . . . . . . . 146

5-6 Sensitivity of dropped patients to patient the arrival rate for GMS. . 147

5-7 Sensitivity of jeopardy reassignment and total reassignments to the

patient arrival rate for GMS. . . . . . . . . . . . . . . . . . . . . . . . 147

5-8 Sensitivity of jeopardy reassignment and total reassignments to the

patient arrival rate for GMS (additional policies). . . . . . . . . . . . 148

5-9 Average arrivals by hour of day, day of week, and month of year, On-

cology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160

5-10 Average arrivals by hour of day, day of week, and month of year, Car-

diology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160

5-11 Average arrivals by hour of day, day of week, and month of year, GMS. 160

5-12 Distribution of patient length of stay by hour of day in patient arrival

time. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

6-1 The relationship between the various theorems in our “interchange of

limits” argument. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178

6-2 Steady state patients in system in the fluid limit for LS and DA. . . . 182

6-3 Steady state reassignments and jeopardy in the fluid limit for LS and

DA. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182

12

List of Tables

2.1 Performance of algorithms for the KEP on real instances. . . . . . . . 51

2.2 Performance of algorithms for the KEP on very large instances. . . . 51

5.1 The types of shifts used to to build schedules for residents. . . . . . . 130

5.2 Schedules considered for teams of residents. . . . . . . . . . . . . . . 132

5.3 Policies considered for GMS. . . . . . . . . . . . . . . . . . . . . . . . 135

5.4 Policies considered for Cardiology. . . . . . . . . . . . . . . . . . . . . 135

5.5 Policies considered for Oncology. . . . . . . . . . . . . . . . . . . . . . 135

5.6 Performance of the Oncology Department under three different policies

for scheduling residents. . . . . . . . . . . . . . . . . . . . . . . . . . 141

5.7 Performance of the Cardiology Department under five different policies

for scheduling residents. . . . . . . . . . . . . . . . . . . . . . . . . . 142

5.8 Performance of the GMS Department under six different policies for

scheduling residents. . . . . . . . . . . . . . . . . . . . . . . . . . . . 143

5.9 Patients that can be admitted per shift and capacity for teams of res-

idents under various schedules, GMS & Cardiology. . . . . . . . . . . 156

5.10 Patients that can be admitted per shift and capacity for teams of res-

idents under various schedules, Oncology. . . . . . . . . . . . . . . . . 156

5.11 Throughput upper bounds for teams of residents under various sched-

ules, GMS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

5.12 Throughput upper bounds for teams of residents under various sched-

ules, Cardiology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

13

5.13 Throughput upper bounds for teams of residents under various sched-

ules, Oncology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

5.14 Throughput upper bounds for PA teams by department. . . . . . . . 157

14

Chapter 1

Introduction

In this thesis, we look at a variety of problems arising in two areas of healthcare

operations: Kidney Paired Donation (KPD) and scheduling medical residents in hos-

pitals. These application areas will be discussed in detail in the subsequent sections.

While these application areas share little from a medical perspective, more abstractly,

both can be viewed as moderately large scale systems with complex stochastic dy-

namics and rich decision spaces, where simple and interpretable solutions are desired.

We take a dual approach to these problems, considering them from both an opera-

tional and a strategic perspective. From an operational perspective, we build high

fidelity simulation tools powered by historical data that can reproduce scenarios of

what would have happened under various policies. While this approach allows us

to quantitatively estimate the impact of a policy, it does little to provide systematic

explanations of the advantages of a policy, particularly when the simulations produce

seemingly counter-intuitive results. In order to better understand our problems from

a strategic perspective, we design and analyze families of stochastic models. These

models capture the key ideas driving the sometimes surprising simulation outcomes,

yet are simple enough to reasonably be studied analytically. Our models are based

on an innovative blend of models arising in queueing theory and the theory of ran-

dom graphs, and their analysis is interesting in its own right. As these models are

all moderately large scale stochastic systems and we are interested in the long run

average behavior, there are some common technical tools we use in their analysis. In

15

particular, we use the Lyapunov function technique to approximately measure steady

state performance. We then consider asymptotic limits of our large scale stochas-

tic systems, and obtain limit theorems that asymptotically characterize the long run

performance metrics.

We now discuss our two application areas, namely KPD and scheduling medical

residents in hospitals in depth.

1.1 Kidney Paired Donation

As of November 2013, there are more than 98,700 patients in the United States on

the cadaver waiting list for kidney transplantation [63]. Many of these patients have

a friend or family member willing to be a living kidney donor, but who is biologically

incompatible. Kidney Paired Donation (KPD) arose to allow these patients with

willing donors (hereby referred to as patient-donor pairs) to exchange kidneys, thus

increasing the number of living donor transplants and reducing the size of the cadaver

waiting list.

In KPD, incompatible patient-donor pairs can exchange kidneys in cycles with

other such pairs so that every patient receives a kidney from a compatible donor

(e.g. see Figure 1-1). Additionally, there are a small number of so-called altruistic

donors. These individuals are willing to donate their kidney to any patient without

asking for anything in return. In KPD, an altruistic donor can be used to initiate a

chain of transplants with incompatible pairs, ending with a patient on the deceased

donor waiting list that has no associated donor (e.g. see Figure 1-2). Often, chains

are planned in segments, where instead of immediately ending the chain on a patient

from the deceased donor waiting list, the donor from the final pair in a segment is

used to begin another segment after new patient-donor pairs arrive. Such donors are

called bridge donors. Once the segment is executed, the bridge donors and altruistic

donors are essentially identical for the purpose of planning future transplants, and

are collectively referred to as non-directed donors (NDDs).

When donors agree to participate in a kidney exchange, there are no legal con-

16

d1

p1

p2

d2

d1

p1

d2

p2

d3

p3

Figure 1-1: Left: A cyclic exchange involving two patient-donor pairs. Each pair isrepresented by a node, where the blue half of the node represents the donor and thered half represents the patient. Right: A cycle with three patient donor pairs.

d0

p1

d1

d2

p2

p3

d3

d4

p4p5

Figure 1-2: A chain exchange involving an altruistic donor, d0, four patient-donorpairs, and a patient with no donor p5. Each pair is represented by a node, where theblue half of the node represents the donor and the red half represents the patient.

tractual tools to enforce the exchange. For example, in the two way cycle from Figure

1-1, suppose that d1 donates to p2 today with the understanding that d2 will donate

to p1 tomorrow. Then tomorrow, d2 either (a) changes his mind about the transplant

or (b) becomes too sick to donate. Now p1 has no recourse, while p2 has just gotten

a kidney in exchange for nothing. Thus p1 has been irrevocably harmed, in that he

now has nothing left to trade and will be unable to participate in future exchanges.

To prevent irrevocable harm, KPD programs always perform all the transplants in a

cycle simultaneously. Consequently, KPD programs rarely organize cyclic exchanges

involving more than three patient donor pairs, as it is too difficult logistically to

organize more than three simultaneous transplants.

However, simultaneity is not always necessary to prevent irrevocable harm in

an exchange. In particular, we need only that for every patient donor pair in the

exchange, the patient receives a kidney at a time no later than the time the donor gives

his kidney. While for cycles, this condition implies simultaneity, for chains initiated

by an altruistic donor, it does not. In a chain with nonsimultaneous transplants, it

is still possible for a donor to renege on a proposed exchange after his associated

patient has received a kidney. While this outcome is very undesirable, as perhaps an

17

alternative chain could have resulted in more total transplants than the broken chain,

it does not leave any patient irrevocably harmed. The benefit of allowing chains to

be performed nonsimultaneously is two-fold: chains can be made longer without the

logistical complexities of simultaneous surgeries, and patients need not wait for every

member of the chain to arrive before they receive their transplant.

In all currently operating KPD programs, the decision of which transplants to

perform is decided centrally by the program. We refer to any operational restrictions,

e.g., a maximum cycle length or maximum chain segment length (if one is applicable),

as rules, and the mechanism for deciding which transplants to perform subject to the

rules as the matching policy. Many KPD programs use a batching policy as their

matching policy, parametrized by n, where patient donor pairs arrive for n days, and

then a set of exchanges are selected to maximize the number of transplants performed

that day. We refer to this parameter n as the match run time. In the special case

when n = 1, i.e. the KPD program optimizes for the current pool each day, we refer

to this matching policy as the greedy policy.

At a high level, we seek to address two problems in KPD. First, for a fixed pool of

patient-donor pairs and NDDs, we want to solve optimization problems over the set of

feasible transplants. In particular, we focus on the problem of finding the maximum

(possibly weighted) number of transplants, organized into cycles and chains, which

we refer to as the Kidney Exchange Problem (KEP). An example of a KEP instance

is shown in Figure 1-3. We consider several variations and extensions of this problem

as well.

Second, we want to explore how the rules and matching policy of a KPD program

affect key long run performance metrics, such as number of transplants performed

and time patients must wait to receive a transplant. We will focus our analysis on

comparing the batching policies (including the greedy policy), as these policies are

being used in practice. In particular, we assess the impact of changing the rules re-

stricting the maximum cycle and chain length and changing match run time. Perhaps

the more interesting question is deciding the match run time, as it is not obvious how

this quantity should be set to minimize the average patient waiting time. There is

18

n p1 p2 p3

p4 p5 p6 p7

Figure 1-3: An example of a KEP instance. The node labeled n is a non-directeddonor, and the remaining nodes p1 through p7 correspond to patient-donor pairs.Edges indicate possible transplants from the donor in the source node to the patientin the target node. In the optimal solution for this instance, indicated by the boldedges, we form the chain n, p1, p2, p3, p7, and the two-cycle with p5 and p6, leavingp4 unmatched.

an intuitive trade-off in setting the match run time described as follows. Optimizing

frequently reduces the time patients spend waiting between optimization runs. How-

ever, planning to transplant a patient donor pair in a two way cycle the day that they

arrive could prevent many transplants in a chain that can only be formed using this

pair with another pair that arrives the following day.

We approach this long run performance assessment from two perspectives. First,

for several exchange programs, we simulate the dynamics of a KPD pool under various

policies using their historical data. Note that these policies typically require solving

an optimization problem over the set of feasible transplants periodically, thus it is a

prerequisite that we can solve these optimization problems. Using this approach, we

found that, surprisingly, the greedy policy was essentially the best among all batching

policies in terms of average patient waiting time, and that this was true regardless of

any rules restricting the maximum cycle or chain length. This result suggests that

there is little to no loss of efficiency for existing KPD programs in using the greedy

policy.

This surprising observation motivated our second, analytical approach to long run

performance assessment. We designed a dynamic random graph model of a barter

exchange system with the goal of explaining this phenomena. In the model, a homo-

geneous stream of nodes (patient-donor pairs) arrive and directed edges (biologically

feasible transplants) are added randomly between the new node and existing nodes,

according to a single parameter p indicating the sparseness of the graph. A rule speci-

19

fies what type of exchanges are allowed (either two-cycles, two-cycles and three-cycles,

or the advancement of a single chain), and a policy dictates the strategy that is used

to remove nodes (select exchanges). For this model, under all of the aforementioned

rules, we show that asymptotically, as p → 0, the greedy policy achieves the mini-

mum possible average patient waiting time, up to constant factors. This result gives

us insight as to why greedy was optimal in our (more complex) simulation model of

a KPD pool.

In the subsequent sections, we first give some historical perspective on KPD. We

then explain our contributions: algorithms for solving the KEP and related optimiza-

tion problems, simulation results, and the analysis of our dynamic random graph

model. Finally, we give some interpretation of the implications of our results for

KPD.

1.1.1 Background & Related Literature

Integrating both cycles and chains in KPD was proposed in [67], where both the

chains and cycle have unlimited size. As organizing many surgeries simultaneously is

very logistically complex, the first implementations of KPD by New England Paired

Kidney Exchange and other clearinghouses used only two-way cyclic exchanges. After

a short period, clearinghouses have moved to allow three-way exchanges as well.

In [68], it was proposed to relax the requirement of simultaneity to the weaker

requirement that every patient-donor pair receive a kidney before they give a kidney.

As previously discussed, while this restriction still required all surgeries be performed

simultaneously for cycles, it did allow for non-simultaneous chains. Since the first

non-simultaneous chain was arranged [65], chain type exchanges have accounted for

a majority of the transplants in kidney exchange clearinghouses. Approximately 75

percent of the transplants in National Kidney Registry (NKR) and the Alliance for

Paired Donation (APD) are done through chains (these are two of the largest KPD

programs in the United States). Chains involving as many as 30 pairs have been

performed in practice, capturing significant public interest [70].

Variations of the KEP have been considered in the literature. As noted in [67],

20

when there is no maximum chain or cycle length, the problem can simply be solved

with a single linear program, using the integral network flow polyhedron. The special

case where chains are not permitted and only cycles of length two are used can be

exactly solved very efficiently, as it is equivalent to the maximum matching problem.

In [1], the special case of this problem where only cycles of length two and three are

used was shown to be NP -hard. See [16] for a stronger negative result in this special

case. However, integer programming techniques have been used by a variety of authors

to solve special cases of the KEP without chains or with chains of bounded length,

as first proposed in [69]. In [1], by improving the integer programming formulation of

[69] and devising good heuristics, the authors are able to solve KEP instances with

thousands of patient donor pairs, but without chains. Alternate IP formulations were

further explored for this special case in [20]. In [67], a heuristic to produce feasible

solutions when using chains and cycles of bounded length was suggested. However, no

optimization algorithm was given, opening a major algorithmic challenge to reliably

solve large scale instances of the general kidney exchange problem. The technique of

[1] was extended in [23] to solve very large instances with bounded cycles and bounded

chains. However, the algorithm became impractical when the maximum chain length

was larger than four or five, as the formulation required a (column generated) decision

variable for every chain with length at most the maximum chain length.

The techniques we use to solve the KEP, as described in this thesis, are similar to

techniques used to solve variations of the Traveling Salesman Problem (TSP), a well

known and difficult combinatorial optimization problem. The literature on this topic

is vast (see [46] and the references within). For several variations of the TSP that

are similar in spirit to the KEP, integer programming solvers have been developed

[13, 33, 34, 39] (this list is not intended to be comprehensive). In particular, our

formulation of the KEP most closely resembles a sparse, directed version of the Prize

Collecting TSP (PC-TSP). To the best of our knowledge, this particular variation

has not been studied previously.

21

1.1.2 Algorithmic Results

The primary algorithmic challenge we address is to find an algorithm to solve real

instances of the KEP without bounding the maximum chain length. Solving this

optimization problem is critical to the operations of KPD programs, which form long

chains in practice. Previously, these programs either only searched for short chains or

relied on heuristics that could lead to suboptimal solutions. Additionally, having fast

algorithms to solve the KEP is required to make the simulation of a KPD program

practical, which is the subject of the next section. Ultimately, we address this chal-

lenge by giving two new algorithms based on integer programming to solve the KEP.

One of these algorithms is motivated by an integer programming formulation of the

so-called Prize Collecting Traveling Salesman Problem (PC-TSP), another NP -hard

problem. The PC-TSP is a variant of the classical Traveling Salesman Problem (TSP),

one of the most widely studied NP -hard problems in combinatorial optimization.

We emphasize that in evaluating the success of our algorithm, we use real instances

drawn from historical data, as opposed to instances created by randomly generating

patient donor pairs or compatibility graphs. We do this because instances encountered

in practice are generated by the KPD pool dynamics. As a result, the statistics of

compatibility for the patient-donor pairs still waiting to be matched are complicated

(a patient-donor pair waiting to be matched is on average more difficult to match

than a patient-donor pair randomly selected from historical data, as the easy to

match patients are on average matched more quickly). Thus it is difficult to generate

representative instances randomly.

Running our algorithms on real data instances (typically with a few hundred

nodes) we find that:

For the vast majority of these instances, both algorithms can solve the problem

to optimality in a few seconds.

There are a few difficult instances where using our PC-TSP based algorithm

results in an improvement in run time by several orders of magnitude.

In addition to devising practical algorithms to solve these problems, we obtain several

22

theoretical results. In particular:

We prove that the integer programming formulation from our PC-TSP based

algorithm is the stronger of the two formulations, as measured by the value of

the linear programming relaxation.

We give a polynomial time algorithm to solve the separation problem for the

integer programming formulation from our PC-TSP based algorithm.

We devise an interesting pathological instance of the KEP that highlights the

worst case difficulty of the KEP and potential differences in performance of the

various integer programming formulations.

We also provide algorithms to address a variety of extensions to the KEP, including:

Solving the KEP with a large (but bounded) maximum cycle length and un-

bounded chains. (While in practice, cycles longer than three are rarely formed

due to logistical issues, there is some evidence that there may be benefits to

considering longer cycles [7].)

Solving the KEP with a large (but bounded) maximum chain length and short

cycles. (Thus our algorithms are also capable of solving the problem described

in [23].)

Solving a two stage version of the KEP, where a limited set of edges is selected

in stage one, then edges fail at random, and cycles and chains are built using

only the edges that were selected in phase one and did not fail. This signifi-

cantly generalizes the results of several other works [17, 24, 61], at the cost of a

computationally more expensive algorithm.

In solving all of these problems, a variety of heuristics were developed. The details

can be found in Chapter 2.

1.1.3 Simulation Results

We simulated to evolution of the KPD pool for NKR, a large exchange program, over

a multi-year horizon. As inputs to the simulation, we considered several factors, most

notably including:

The match run time,

23

The maximum size of an allowed cycle,

The number of altruistic donors (possibly none).

We focused on the following performance metrics: total number of transplants per-

formed and average waiting time to receive a transplant. Additionally, we looked at

these metrics for sub-populations of the exchange pool that are “difficult to match.”

We identified these sub-populations based on statistical properties of the historical

entrants to the NKR KPD program, most notably using Pair Match Power, as defined

in Chapter 3.

Qualitatively, our results are summarized below:

The greedy policy is as good as any batching policy. Under essentially all combi-

nations of inputs, patient waiting time is minimized or nearly minimized with

a match run time of one day (i.e. greedy policy). Additionally, there are no sig-

nificant increases in the total number of transplants until the match run time is

increased to at least three months, which gets only a few percent extra matches.

There are several practical reasons why a KPD would not want to wait three

months between much runs, as discussed in Section 1.1.5.

Disallowing chains results in substantially worse outcomes. While simulating

the NKR KPD pool, eliminating the use of chains reduced the total number of

transplants from 264 to 201, and increased average waiting time from 158 days

to 214 days.

At the current number of altruistic donors, rules changing the maximum cycle

length have essentially no effect on long run performance. Perhaps with fewer

altruistic donors in the system, the outcome would have been different, as in

[7].

There is little room to improve in total transplants relative to the greedy policy.

The number of transplants achieved by the greedy policy is within 15% of the

offline solution, an upper bound for any online solution.

Features notably absent in the simulations include:

Patient donor pair abandonment, where a pair permanently leaves the system

(typically due to illness). This can happen before a node is matched if it has

24

waited too long, including while the node is the bridge node in a chain.

Edge failures, where a transplant that has been planned must be canceled due

to a discovered incompatibility.

Time delays for transplants to be scheduled and performed, and for checking

compatibility (the cause of edge failures).

A discussion of how the inclusion of these features in our simulation model could

change our conclusions is briefly given in Section 1.1.5.

1.1.4 Dynamic Random Graph Results

In Section 1.1.3, we observed the surprising result that in essentially all settings, no

online policy could generate a lower average patient waiting time than the greedy

policy (using a match run time of one). This result was surprising because there

was a perceived tradeoff between match run time and average patient waiting time as

follows: Increasing the match run time would give us more information before deciding

which exchanges to make, potentially enabling smarter matches that reduced average

waiting time. However, increasing the match run time would force patients that

could be matched immediately to wait longer for an opportunities to be matched.

Additionally, we observed that without chains, substantially fewer transplants would

be made. In this section, we develop a model that analytically demonstrates why

these observations should occur. Our model is a new model of a dynamic random

graph that can be analyzed using a combination of tools from random graph theory

and queueing theory.

Our model is briefly summarized as follows. In each time period, a new node

v arrives (interpreted as a new incompatible pair), and for each other node in the

existing graph, a directed edge is added to and from v with probability p, i.i.d.

(corresponding to a potential transplant). After each arrival of a new node, there is

an option to remove nodes according to one of the three rules. The first two rules are

given below:

1. Remove any number of node disjoint directed two-cycles.

2. Remove any number of node disjoint directed two-cycles and three-cycles.

25

Under the third rule we delete chains. However, we need to maintain an extra part

of the state. In every time period, there is a single special node, the bridge donor.

The rule is:

3. Remove any directed simple (node disjoint) path starting from the bridge donor.

The final node in the path becomes the new bridge donor.

For each rule, we want to compare the greedy policy (where in every time period

we remove the maximum number of nodes) to arbitrary policies, or at least batching

policies (where we only attempt to remove nodes every n time periods, for some n

possibly depending on p). The most relevant performance metric in this model is long

run average node waiting time. It turns out that the total number of matches is not

actually a relevant metric in our infinite time horizon model, as we will show that all

nodes are eventually matched in finite time with probability one.

We discover that the steady state number of nodes in the system under the greedy

policy is Θ(1/p2) for two-cycles, Θ(1/p3/2) for two-cycles and three-cycles, and Θ(1/p)

for chains, as p → 0. Further, we show that every policy must have a steady state

number of nodes in system of at least Θ(1/p2) for two-cycles, and Θ(1/p) for chains,

thus we conclude that the greedy policy is optimal up to constant factors for the two-

cycle and chain rules. For the rule deleting two and three cycles, we show that every

monotone policy, which we define in Chapter 4 and includes all batching policies,

must have a steady state number of nodes in the system of at least Θ(1/p3/2), so we

conclude that the greedy policy is optimal up to constant factors among monotone

policies. Finally, we note that by applying “Little’s Law,” the steady state average

waiting time must equal to the steady average number of nodes in system. Thus our

conclusions apply to the steady state expected waiting times as well.

1.1.5 Operational & Strategic Insights for KPD

In this section, we discuss how our results relate to several practical aspects of KPD,

both at the operational and strategic levels.

At an operational level, our algorithmic results from Section 1.1.2 have two impor-

tant consequences for KPD. First, our algorithms enable KPD programs to efficiently

26

and optimally search for long chains in their daily operations. Our implementation

of our algorithm is currently used to by:

The APD, a large KPD program,

Several hospitals that run individual exchange programs, including Northwest-

ern Memorial Hospital (Chicago), Methodist Hospital (San Antonio), George-

town Medical Center (Washington DC), and Samsung Medical Center (Korea),

An “inter-exchange program” between the APD and NKR (the largest KPD

program in the US as measured by transplants arranged) that finds matches

between pairs registered in these two separate KPD programs.

Second, our algorithms give KPD programs a tool to evaluate the implications of

possible future policy changes. The improvement in solve time from hours or minutes

to seconds is of particular importance in this case, as at times it is desirable to

simulate many policies, each with hundreds of replications, creating a large burden of

computation. Our software has been used to run simulations to analyze a variety of

policy decisions for both NKR and the United Network for Organ Sharing (UNOS)

pilot exchange program.

At the strategic level, our results inform the discussions around several major

questions that have been debated in the KPD community for some time. First, our

work supports the proposal that long chains should be used in kidney exchange. When

long, non-simultaneous chains were first proposed, they were highly controversial [40].

The debate over the importance of long chains in KPD has largely already been re-

solved, in part due to the success of NKR and the APD (most of their transplants

were performed in non-simultaneous chains). Our results provide experimental evi-

dence and theoretical justification to support the observation that long chains have

been successful in practice. Specifically, in our (somewhat stylized) random graph

model, we found that using chains resulted in a patient waiting time which was an

order of magnitude smaller than relying solely on cycles. Likewise, in our simulation

experiments, we observed that organizing exchanges based on chains of unbounded

length resulted in more transplants and lower patient waiting time than using cycles

only.

27

Second, our results suggest that there is little loss of efficiency in using the greedy

policy, at least when compared to the batching policies. The optimality result for

the greedy policy in our theoretical model does not directly imply this, as in practice

compatibility is not homogeneous in KPD, and our result was asymptotic and only

up to constant factors. However, the generality of our result under all three rules for

making exchanges does suggest that the greedy policy is at least a near optimal way

to manage exchange pools. We note that in other dynamic models with heterogeneous

patient types [8], batching can improve average waiting time. However, our empirical

work suggests that the upside from batching is quite limited, particularly for existing

kidney exchange programs (the situation may change if exchange programs become

orders of magnitude larger).

Finally, our results tie into a larger question for kidney exchange, at least in the

United States. Namely, there has been considerable debate as to if there should be a

single national KPD program, both in the medical literature and the popular press

[71]. In particular, it has been suggested that competing exchanges are incentivized to

use a greedy policy so that they can match patients listed in multiple exchanges before

their competitors. In [71], a statement was made suggesting that this myopic behavior

wastefully uses easy to match pairs, “causing[ing] cherry-picking that undermines

optimization. . . It kind of creates this race to the bottom.” In particular, they are

suggesting that competition is incentivizing KPD programs to inefficiently organize

exchanges, under the assumption that the use of the greedy policy is inefficient.

Our result stating that the greedy policy is reasonably efficient suggests that the

loss of efficiency due to competition between exchange programs is not substantial.

Whether or not KPD should be nationalized in the United States is a very complicated

question involving many factors significantly outside the scope of this work. However,

our results suggest that from an efficiency perspective, competition may not be as

counterproductive as previously thought.

28

Discussion of Modeling Assumptions

As previously mentioned, there are some important practical features of kidney ex-

change that our simulations and theoretical models fail to account for, most notably

abandonment, edge failures, and time delays. We briefly discuss how the incorporation

of these factors into our models might affect our conclusions.

First, we consider abandonments, under the assumption that patients will aban-

don a constant rate, independent of how long they have already been waiting. In

such a model, the total number of abandonments would be proportionate to the av-

erage waiting time of a patient. Thus policies that have the least waiting time will

have the fewest abandonments. Without abandonments in our model, we found that

the average patient waiting time was made smallest by: (1) using long chains and

(2) using a greedy policy. Suppose that by the introducing abandonments into our

model, the result still held that using long chains and a greedy policy minimized

average patient waiting time. Under this assumption, by the italicized claim above,

using long chains and a greedy policy would minimize abandonments as well. Aban-

donments are clearly undesirable from the perspective of our performance metric of

total matches, as patients that abandon cannot be matched. Thus we conclude that

adding abandonments to our model would strengthen our arguments in favor of using

long chains and using a greedy policy.

A caveat in our argument is that we do not fully consider the tradeoffs involved

in using bridge donors vs. conducting exchanges simultaneously when considering

the cost of abandonment. It was suggested in [40], another simulation study, that

abandonment by bridge donors (who have little incentive not to abandon, as their

desired patient has already received a kidney) would result in long non-simultaneous

chains producing fewer total long run transplants than short simultaneous chains.

The tradeoffs involved in analyzing this problem are complex and somewhat beyond

the scope of this work. However, at NKR, the largest KPD program in the United

States, abandonment by bridge donors has been rare, (they had no broken chains in

2012 and only 3 broken chains in 2013, out of around 80 total chains), suggesting

29

that perhaps this is not a first order issue.

Finally, we briefly mention that our assumption of a constant abandonment rate

could likely be improved. The rate of abandonments caused by patients becoming too

sick to transplant would likely grow over time, as the longer a patient is on dialysis

waiting to receive a kidney, the more likely they are to develop further health compli-

cations. As our (heuristic) analysis above depended heavily on the abandonment rate

being constant, some of the conclusions on the effect of introducing abandonments

could change.

Next, we consider edge failures. We break the edge failures into two types that

are dealt with very differently from an operational perspective. The first is immedi-

ate failure, where a potential transplant is rejected immediately after it is proposed

because of a discovered biological incompatibility or a doctor deeming a donor unfit

for their patient. The second type is caused by abandonment, were after an exchange

is agreed upon and finalized, a donor or patient becomes unable to participate. In

practice, the first type of failure is common at many KPD programs, while the second

type of failure tends to be more rare. As the number of edge failures of the second

kind are rather small, we believe that their effect should be small and not change

our conclusions. Additionally, we mention that NKR has put considerable effort into

reducing failures of the first type, and has recently succeeded in making both failure

types rare.

At first glance, edge failures appear quite problematic for long chains, as a very

long chain is quite likely to have at least one failure, and every transplant after a point

of failure cannot be executed. However, the true cost of edge failures depends on the

recourse options after edges fail. Under the NKR model, if there is a single immediate

edge failure in a proposed exchange, all offers are withdrawn and a new set of offers is

made. Such a policy is only possible because the NKR match run time is only a day.

Under a greedy policy in combination with this offer withdrawal strategy, the effect of

immediate edge failures is essentially completely mitigated (and our conclusions will

remain unchanged). If instead, the match run time were for example three months,

then offer withdrawal recourse is infeasible, as the many iterations required to find

30

a proposed exchange with no immediate edge failures could take years. If under a

three month match run time, the recourse to immediate edge failure is to truncating

chains and cancel failed cycles, then forming very long chains (or long cycles) would

be of little value.

Finally, we mention that in Chapter 2, we develop a stochastic optimization

methodology for the KEP that enables optimizing the expected number of trans-

plants performed after edge failures occur and some recourse is taken. The technique

is very general and applicable to a wide range of recourse structures. For example,

when proposing exchanges, instead of requiring that the exchanges could all be feasi-

bly executed, one could propose a set of exchanges where some pairs receive or give

multiple offers. Then once the immediate failure information is realized, a maxi-

mum subset of these exchanges (which did not fail) could be executed in cycles and

chains. While understanding the long run implications of adding both edge failures

and complex recourse would require further study, it would appear that strategies

with complex recourse could benefit from using long chains.

Last, we consider time delays. In both our theoretical and simulation models, we

assumed that patients were transplanted as soon as they were matched. However, in

reality, there are many delays in the system that our models do not account for. As

many of our conclusions about greedy policies and using long chains were measured in

average patient waiting time, accounting for these delays could affect our conclusions.

Sources of delay not accounted for in our models include:

(i) After a set of transplants is proposed, a series of tests must be performed to

verify that the donors and patients are biologically compatible.

(ii) For simultaneous exchanges, a date must be found when all the doctors needed

are available.

(iii) For non-simultaneous chains, the transplants must be performed in the order

that they occur in the chain (so additional delays propagate down the chain).

In particular, note that a chain can be resumed from a bridge donor only when

all the previous transplants have been executed.

We now discuss how these delays could influence our conclusions. For (i), as the time

31

required to test for biological compatibility is short and continues to fall as KPD

programs improve their operations, we expect this to have a minimal impact on our

conclusions. Delay (ii) seems to strengthen our conclusions in favor of using long

non-simultaneous chains over cycles (particularly three way cycles), as with chains

transplants need not be scheduled simultaneously if doing so is inconvenient. Issue

(iii) of delays propagating down long chains and these delays not being accounted

for in our models will clearly lead us to overestimate the benefits of long chains. To

quantitatively estimate this effect would require further study and likely considerable

effort. However, we find it very unlikely that this effect would outweigh the reduction

in average waiting time gained by using chains.

1.2 Scheduling Medical Residents in Hospitals

Major hospitals face a difficult challenge of designing shift schedules for their resi-

dents that have capacity to treat all incoming patients, provide quality care, and are

compliant with regulations restricting shift lengths. Recently, there has been much

controversy surrounding the use of long shifts, and the resulting fatigue. In partic-

ular, fatigue during long resident shifts has been implicated as a cause of medical

errors [10, 37, 57], burnout, depression and other psychological problems [48, 72, 75],

and motor vehicle crashes [11]. In order to address these and related issues, the

ACGME instituted a new set of regulations on duty hours limiting the duration of

shift lengths to 16 hours for the first year residents [50]. The new regulations have

forced many academic medical centers to overhaul their shift schedules. Proponents

of the long shift are concerned with how this regulation will affect patient continuity

of care. They argue that reducing shift lengths will result in more patient handoffs

between caring practitioners, increasing the chance of miscommunication and acci-

dents [21, 64]. A particularly undesirable type of handoff is a reassignment, when a

patient is admitted temporarily by one doctor and then is transferred to a resident

for a permanent care. Reassignments are dangerous as they greatly increase the risk

of losing information that should be used in determining a course of treatment. Ad-

32

ditionally, long shift advocates additionally argue that reducing residents’ hours will

force hospitals to increase staffing levels to compensate for lost capacity to admit

patients [81, 82]. The impact of shift schedules on (a) the admitting capacity and (b)

the number of reassignments are two main questions we address.

First we approached these problems at an operational level. Working with Brigham

and Women’s (B&W) hospital, a major academic hospital in the Boston area, we built

a high fidelity discrete event simulator that given a resident schedule for the hospital,

can provide estimates of (a) capacity and (b) continuity of care using a variety of

performance metrics. Additionally, we developed a Markov chain based approxima-

tion for the capacity of a hospital’s resident schedule. Our approximation was far

more accurate than the simple bounds on capacity implied by making a basic rate

calculation using the number of residents available with either (i) the number of pa-

tients each resident can have in care simultaneously or (ii) the number of patients

each resident can admit per day. Using these tools, we were able to inform B&W

of the implications for each of the potential schedules they were considering when

transitioning to schedules satisfying the 16 hour shift length regulation. In contrast

to much of the existing literature on the effects of duty hour regulations which relies

on the perceptions of outcomes gathered in surveys (particularly when measuring the

effect on quality of care), our approach directly measures the relevant performance

metrics.

Our simulations led to a few surprising strategic conclusions. We found that

schedules based on shorter more frequent shifts that held total labor hours constant

actually led to a larger admitting capacity. At the same time, such schedules generally

reduce the continuity of care by most metrics when the departments operate at normal

loads. However, as a hospital department approached the critical capacity regime,

we found that the continuity of care improved in some metrics for schedules based on

shorter shifts, most notably in total reassignments, due to a reduction in the use of

overtime doctors.

To better understand these strategic insights, we developed a stylized queueing

model of patient flow in a hospital where analytic results on capacity and continuity

33

of care can be shown. In particular, we prove that a schedule with shorter more

frequent shifts has a greater admitting capacity than a schedule with long shifts.

Additionally, we show in an asymptotic scaling that the schedule based on shorter

more frequent shifts will provide a better continuity of care (measured by the number

of reassignments) when the patient load is high.

In the subsequent sections, we first give some background on medical resident

scheduling. We then discuss the results from our simulations and our analytic models.

Finally, we give some interpretation of our results for medical resident scheduling.

1.2.1 Background & Related Literature

To the best of our knowledge, this paper is the first study to quantitatively measure

the impact of duty hour restrictions (in particular, maximum shift lengths) on capac-

ity, i.e. issue (a). There have been some related studies [77, 81, 82], which suggested

that duty hour restrictions (reducing the total number of hours residents can work)

have caused hospitals to hire PAs and nurse practitioners to decrease the workload

of residents. Some previous studies have investigated the impact on continuity [82]

after New York state put a cap of 80 hours a week on resident work hours in 1989,

[48] when the ACGME created a similar national work hour restrictions in 2003, and

most recently [26] and the follow up [27] as well as [75] with the 2011 restrictions. All

of these studies have suggested that these regulations have not improved the quality

of care. However, these studies often aggregated resident and physician perceptions of

the change in the quality of care from before to after regulation, which is a subjective

metric [35, 82]. As noted in [35], while there have been many studies on the effect

of long shifts and imposing caps on duty hours in the last 30 years, very few studies

used randomized control design, and most used perceived outcomes instead of actual

outcomes, hence the contradictory results of these studies are inconclusive.

We now mention some other approaches considered in the field of operations re-

search for capacity management in hospitals. A very general survey of capacity man-

agement in healthcare is given in [43]. Simulation studies of capacity have been

done for medical resident schedules [25, 53] and various other hospital resources

34

[29, 51, 58, 66, 83]. However, as these simulations are incredibly sensitive to the

details of each hospital’s operations, the results do not generalize well to other hospi-

tals. There are some recent papers which propose a queueing model of patient flow in

a hospital, and then solve for important performance metrics, either analytically [85],

asymptotically [22, 85, 86], numerically [74], or with heuristic methods [44]. Although

not explicitly about healthcare, in [49, 59], fluid models for queues with time varying

arrival rates alternating between overloaded and underloaded periods are considered.

These models are more in the spirit of our work than the previously cited models

from a technical perspective, as transient behavior and “end of day effects” (see [45])

play a prominent role.

1.2.2 Simulation Results

We constructed a discrete event simulation tool in order to model a variety of resi-

dent shift schedules and their implications for the issues (a) and (b) above. In the

simulations, a sequence of patient arrival and departure times was given by a his-

torical dataset. The primary caregivers for these patients are the medical residents

and the Physician Assistants (PAs), who are organized into teams for the purposes of

admitting and caring for patients. Both individuals and teams are restricted in the

maximum number of patients they can admit per shift, and the maximum number

of patients they can have in care (where often the team limit is smaller than the

sum of the individual limits). When all available residents and PAs are at capacity,

patients are temporarily admitted by a doctor, then reassigned resident or PA for

long term care until discharge. There are two channels for reassignment, night floats

or an overtime doctor (referred to as a jeopardy admission). The latter is far less

desirable, as the overtime doctors typically leave soon after the night floats arrive, so

patients they admit are transferred from the overtime doctor to the night float and

then finally to their caring doctor (a resident or PA), giving additional opportunities

for miscommunication. Additionally, overtime doctors are expensive for the hospital.

We ran our tool against the historical data set of patients at B&W in three main

areas of services: General Medicine, Cardiology and Oncology. To determine the

35

impact on admitting capacity, we used the frequency of jeopardy as one performance

metric. To determine the impact on continuity, we collected several performance

metrics focusing on continuity of care in the critical period of the first 24 hours of a

patients time in the hospital, including:

the number of patients which had to be admitted by one care giver and then

permanently transferred to another, referred to as reassignments (e.g. a patient

is admitted by a jeopardy doctor or a night float and then is transferred to a

resident the following morning).

the frequency that the admitting doctor remained in the hospital for less than

hours after the patient arrived, and

the frequency that residents must admit another patient within two hours of

any admission. (Newly admitted patients take about two hours to “work up,”

a process requiring much of a resident’s attention. When a resident admits two

patients within a two hour window, they must work up two patients at the same

time, splitting their focus and increasing the chance of error.)

The performance metrics were computed for the existing schedules, which include

long (30 hours) shift lengths, as well as for alternative schedules with shorter more

frequent shifts, in compliance with the new ACGME regulations. In our comparisons,

we only consider schedules where the number of hours that residents will be eligible

to admit new patients is the same.

In our findings, the choice of resident schedule had a strong impact on most

of the performance metrics. In particular, we found that schedules with shorter

(about 10 hours) more frequent shifts are better capable of handling larger volumes

of patients. For example, in the General Medicine Service (GMS), a department at

B&W, jeopardy levels under the baseline patient load were 33 patients per year under

a schedule with short frequent shifts and 155 patients per year under a schedule with

long shifts. We also found that for all schedules, the number of jeopardy admissions

markedly increases with even modest increases in patient volume (more precisely, the

number of jeopardy admissions increases rapidly and non-linearly with an increase

in the volume of patients). Continuing our previous example using GMS, when the

36

patient load increased by 10%, we saw that the short shift schedule had an increase of

jeopardy patients to 256 patients per year, while the long shift schedule had a much

larger increase to 690 patients per year.

For the continuity of care in the first 24 hours of admission, our findings are more

subtle. Schedules based on short more frequent shifts contained gaps in resident

coverage, increasing total reassignments during off-peak hours. However, as these

schedules lead to an increased capacity (and thus reduced jeopardy instances), they

resulted in fewer reassignments during peak hours. Thus the effect of reducing shift

lengths on the total number of reassignments was somewhat data dependent and

was not uniform across departments. At the baseline level of patient load, we found

that in moving from a schedule based on long shifts to a schedule based on short

shifts, Cardiology would have an extra 1000 reassignments, Oncology would have

about 500 extra reassignments, and for GMS there would be essentially no change in

the number of reassignments (however, GMS used both intensive teaching units and

extra PAs relative to Oncology and Cardiology, skewing the results, see Chapter 3

for more details). At the same time, under an increased patient load, shorter shifts

caused fewer reassignments across all departments. Uninterrupted observation of a

patient for the first 6 hours since admission was challenging for all schedules; while

the percentage of patients receiving six hours of observation by their admitting doctor

was low for schedules with shorter shifts (15–20%, varying by department), it was still

only around 50% for schedules using long shifts.

Additionally, we performed some sensitivity analysis where we adjusted the rate

that patients arrived to the hospital, and measured how far it could be increased under

each resident schedule before the department would be over capacity (e.g. overtime

would be required regularly). We call maximum stable arrival rate the throughput

of the schedule. We then compared the throughput with two natural upper bounds,

computed by estimating the maximum rate each team of residents and PAs could

treat new patients, and summing this rate up over all teams in the department. The

first bound, the capacity upper bound on throughput, assumes that each resident is

constantly caring for their maximum number of patients in care, and from the average

37

length of stay, infers the average rate that the resident could admit new patients per

day. As this bound ignores the fact that residents are restricted in when they can

admit new patients by their schedule, we expect it to exceed the true throughput. The

second bound, the admitting upper bound on throughput, assumes that each resident

admits the maximum number of patients allowed on each shift, and averages over

the shift rotation schedule to obtain the average rate the resident could admit new

patients per day. As this second bound ignores that the resident may be unable to

admit because they have reached their capacity for total patients in care, we again

expect it to overestimate the throughput. We find that the minimum of these two

upper bounds is quite far from the true throughput. Additionally, we see that often,

for resident schedules with similar throughput upper bounds, the true throughput

is very different. This difference is most notable when comparing our long shift

and short shift schedules. This motivates our Markov chain throughput upper bound,

where capacity to admit patients and capacity to care for patients are accounted

for jointly. We refer to the bound as a Markov chain bound because to compute it

requires computing the steady-state solution of a small, finite state Markov chain,

which models the number of patients in care for a single team of residents over one

cycle of shift rotations. We compute this upper bound, and find that: (1) it is much

closer to the true throughput and (2) when used as a qualitative tool, it correctly

ranks different schedules by throughputs, unlike our previous bounds on throughput.

To summarize our results, we found that resident schedules have a dramatic impact

on the operational performance of hospitals. Specifically, schedules with shorter more

frequent shifts are more capable of handling large volumes of patients. Further, such

schedules will have comparable continuity of care in the first 24 hours of patient

admission to a traditional long shift schedule if the admitting medical staff are at

or near capacity, but may moderately decrease the continuity of care otherwise. We

developed a simulation tool to evaluate resident schedules, and simple model using a

finite state Markov chain to estimate a schedule’s throughput. While the study was

conducted with Brigham & Women Hospital data, we expect that similar findings

apply to other hospitals with large residency programs.

38

1.2.3 Queueing Model Results

In order to understand the results from Section 1.2.2, we developed an analytically

tractable queueing model of patient flow in a hospital. First we briefly describe our

model. Patients arrive according to a non-homogenous Poisson process with rate λ1

in one half of each day (say 10am till 10pm) and rate λ2 ≤ λ1 in the remaining

part of the day. We consider two stylized policies to schedule the residents, which

while significantly simplifying the actual polices, capture the salient features of these

policies. The formal description of the two policies is given in Section 6.2. Now we

just provide a high level description and discuss their main features.

The first policy we analyze in this paper is the Long Shifts (LS) policy. According

to this schedule two teams of residents with the same number of residents in each

team work for the duration of a day every other day, taking a day off after each day

on shift. Namely the first team works on days 2n + 1, n ≥ 0 and the second team

works on days 2n, n ≥ 1.

The second policy we analyze is called Daily Admitting (DA). According to the

DA policy, two teams of residents each with the same number of residents as for the

LS policy work every day during the high load (10am-10pm) half of the day, and a

are off-shift for the other half of the day. Both policies organize the residents into

two teams that are offset by a day in the rotating schedule, thus providing uniform

coverage. The main distinction between the LS and DA policies is that DA is based

on adopting shorter more frequent shifts.

The arriving patients are assigned to residents according to the following mech-

anism used both for the LS and DA schedules. Each resident has an upper bound

(capacity) on the number of patients he is allowed to have in care. Each arriving pa-

tient is assigned to a resident chosen uniformly at random from all residents on shift

who have not reached their capacity (in fact the analysis does not depend on how

patients are assigned to available residents for these policies). If all the residents are

at capacity at the arrival epoch, the patient joins the queue of unassigned patients

and is cared for temporarily by a doctor from a back-up supply of care providers.

39

We assume that we have an infinite supply of these providers, although using them

is undesirable (their use corresponds to a night float or jeopardy admission). These

patients are subsequently reassigned to a caring resident on the first come first serve

basis, as soon as one of them is available. Patients remain in the hospital for a random

exponentially distributed amount of time, beginning from when they are assigned to

a resident. We make this assumption because in practice, the newly arriving patients

without an assigned resident are stabilized but the treatment plan is not determined

until they are assigned to a resident.

We now summarize our results. First, we determine the throughput capacity of

each policy. Specifically, for a given number of residents, we compute the maximum

arrival rate at which patients can arrive before the queueing system becomes unstable,

i.e., the number of unassigned patients grows without bound. We show that DA has

a greater throughput capacity than LS, independent of the parameters of our model,

supporting our hypothesis from the previous section that using shorter, more frequent

shifts will increase capacity.

Next, we compare the number of reassignments under LS and DA (where, as in the

previous section, reassignments occur when a patient arrives and there is no resident

available to immediately treat them). In comparing policies, we are interested in the

expected number of reassignments per day in steady state. As direct steady state

analysis appears intractable, we instead resort to the method of fluid approximation

of the underlying queueing model. We analyze the long term behavior of the fluid

model and show that it converges to the unique steady state solution. The steady

state fluid solution carries important information about the long-term performance

of the underlying stochastic system. In particular, we prove an interchange of limits

result, that the steady state number of patients being treated and the number of pa-

tients waiting to be reassigned converges to the steady state fluid solution under the

appropriate rescaling. We obtain an implicit formula for the number of each type of

reassignment per day in the fluid limit that can be solved numerically (reassignments

in the first half of the day correspond to jeopardy reassignments, while reassignments

in the second half of the day correspond to night float reassignments). Under minor

40

technical assumptions, we also prove that the number of steady state reassignments

per day in the underlying stochastic model converges in the fluid rescaling to a nat-

ural function of the steady state fluid solution, thus justifying fluid approximation.

Computing the number of reassignments under each policy from the fluid steady state

solution, we find that the DA policy leads to fewer reassignments than the LS policy

when the load is high, and leads to more reassignments than the LS policy when

the load is low. Again, our model is consistent with our simulation observations. In

particular, using short, more frequent shifts will reduce reassignments if and only if

the system is heavily loaded.

1.2.4 Operational & Strategic Insights for Scheduling Medi-

cal Residents in Hospitals

In this section, we discuss the implications of our results to practical aspects of medical

resident scheduling, both at the operational and strategic levels.

At the operational level, we have two primary contributions. First, we produced

a high fidelity simulator of the operations at B&W hospital that can measure the

implications of using a resident shift schedule through a large number of metrics.

This tool was directly used by B&W to inform both resident scheduling decisions and

staffing level decisions in 2011. Additionally, we have developed the Markov chain

throughput upper bound that provides a way to quickly estimate the capacity of a

medical resident schedule. As the estimates generated by this tool do not depend on

any low level details of B&W’s operations, we would expect it to be of general use in

other hospitals.

At the strategic level, we have shown both through simulation and with analytic

models that if staffing levels and total hours available to admit patients are held

constant, but shorter more frequent shifts are used, then: (a) capacity will increase,

and (b) reassignments will decrease if and only if the system is heavily loaded. Since

most hospitals tend to operate at high load, our results lead to the conclusion that the

hospitals should consider implementing schedules with shorter more frequent shifts, as

41

it will increase the capacity to admit patients and reduce the number of reassignments.

In this sense the new regulation restricting further the length of shifts should not be

perceived as an impediment to efficient handling of patients at hospitals.

42

Chapter 2

Scalable Algorithms for the Kidney

Exchange Problem

2.1 Introduction

The Kidney Exchange Problem (KEP) is defined informally as follows: given an edge-

weighted directed graph and a maximum cycle length k, find a maximum weight

node-disjoint packing of cycles (of length at most k) and chains. KPD programs

solve the KEP to maximize the number of transplants performed. While the KEP

has been considered by other authors, existing algorithms have proven inadequate to

solve real world instances of the KEP while allowing for “long” chains. We consider

two algorithms based on integer programming formulations of the KEP. One of the

integer programming formulations considered is similar to a formulation for the Prize

Collecting Traveling Salesman Problem (PC-TSP). We give constraint generation

schemes to solve these IPs efficiently. We demonstrate the power of our algorithms by

running them on clinical data from several KPD programs. Finally, we give algorithms

to solve several extensions of the KEP that are of operational and/or theoretical

interest. These extensions include: (a) problems with a large maximum cycle length,

(b) problems with a large but bounded maximum chain length, and (c) two stage

stochastic optimization problems to accommodate for edge failure.

We now briefly explain the relationship between our problem and the PC-TSP.

43

Recall that in the TSP, one is given a list of cities and the cost of traveling between

pairs of cities, and the goal is to find a cycle visiting each city exactly once at the

minimum cost [12]. In the PC-TSP, again one must find a cycle visiting each city at

most once, but now one has the option of skipping some cities entirely and paying a

penalty (see [14], or more recently [42]). Qualitatively, the PC-TSP problem is similar

to the KEP in that one wants to find long paths in a graph (which the PC-TSP then

closes off as a cycle), without the need to visit every node. It differs in three respects:

(1) the solution for the KEP contains multiple disjoint large components (paths),

while in the PC-TSP solution contains only a single large component (a cycle), (2)

short cycles can be added to a solution for the KEP, (3) in the KEP, many edges are

missing from the graph (the TSP variant with missing edges is commonly referred

to as the sparse TSP). Despite these differences, we will see that our solution to the

KEP is similar to the solution for the PC-TSP.

Notation

We introduce some notation. Let G = (V,E) be a directed graph, and let w =

(w1, . . . , w|E|) be weights on the edges of G. For each v ∈ V , let δ−(v) be the edges

pointing to v and δ+(v) be the edges outgoing from v. Likewise, for a set of nodes

S ⊂ V , let δ−(S) be the set of edges into S and δ+(S) be the set of edges out of S.

For every S ⊂ V , let E(S) be the set of all edges with both endpoints in S. For a set

of edges D ⊂ E, let V (D) be the set vertices containing the endpoints of each edge in

D. Let C be the set of all simple cycles in G, where each cycle C ∈ C is represented

by a collection of edges, i.e. C ⊂ E. Let Ck be the subset of C consisting of cycles

which use k or fewer edges. For each v ∈ V , let Ck(v) be the cycles from Ck containing

an edge incident to v. Given a cycle C, let wC =∑

e∈C we be the total weight of the

cycle according to our weight vector w.

Suppose two formulations of an integer program are given. Without the loss of

generality, assume the underlying problem is of the maximization type. The linear

programming relaxation of an integer program is the value of the optimal solution

obtained when the integrality constraints are removed and the problem is solved as

44

a linear programming problem. Let Z1 and Z2 be the optimal solutions to the linear

programming relaxations for two different formulations.

Definition 2.1. If Z1 ≤ Z2 for every problem instance, then formulation one is

defined to be at least as strong as formulation two. We use the notation Z1 Z2.

If in addition there exists a problem instance such that Z1 < Z2, then we say that

formulation one is stronger than formulation two, and use the notation Z1 ≺ Z2.

Very often in practice, the stronger formulations greatly reduce the actual running

time of the integer programming problems [12].

Further, suppose that P1 and P2 are the polyhedrons for the linear programming

relaxations of our two formulations on the same set of variables. If P1 ⊂ P2 for every

problem instance, then trivially, Z1 Z2.

Organization

In Section 2.2, we define the KEP. In Section 2.3, we give two integer programming

based algorithms to solve the KEP. In Section 2.4 we analyze the performance of

our algorithms, both empirically using large scale instances of the KEP drawn from

clinical data, and theoretically using Definition 2.1. In Section 2.5, we show how to

solve the aforementioned extensions of the KEP, and discuss their operational and

theoretical relevance. Last, in Section 2.6, we give proofs of some of our the theoretical

results.

2.2 Problem Statement

An instance of the KEP is described as follows:

a list of non-directed donors (NDDs),

a list of patient-donor pairs (where the donor wants to donate to the paired

patient but is not compatible with this patient),

the compatibility information between all donors and patients

the “weight” or priority, of each potential transplant.

45

a bound k on the maximum cycle length

The goal is then to find a set of transplants, organized into cycles (of length at most

k) and chains initiated by NDDs that uses each donor and patient at most once and

maximizes the sum of the weights of all transplants performed. If all transplants have

weight one, then we are simply trying to find the arrangement which maximizes the

total number of transplants. In Section 2.5.2, we will show how this definition can be

supplemented to include an optional bound on the maximum chain length.

We now formalize this definition of the KEP in graph theoretic terms. We are given

a directed graph G = (V,E), a weight vector w = (w1, . . . , w|E|), and a nonnegative

integer parameter k. The set of nodes V is partitioned into sets N (the NDDs), and

P (the pairs of incompatible donors and patients). For u, v ∈ V , a directed edge from

u to v in E indicates that the donor in node u is compatible with the patient in node

v. As the nodes of N have no patient, they all must have in-degree zero (although

there can be nodes in P with in degree zero as well). The values we ∈ R for each edge

e ∈ E are weights for the edges, indicating the importance of this transplant, and our

goal is to find a maximum weight node disjoint cycle and chain packing, where the

cycles can use at most k nodes and the chains must originate from nodes in N . See

Figure 1-3 for an example.

2.3 Algorithms for the KEP

In this section, we present two algorithms based on integer programming formulations

for the KEP. Both of the IP formulations use an exponential number of constraints.

Thus special techniques are required to solve even moderate sized instances of these

integer programs, which will be described in subsequent sections.

2.3.1 The Edge Formulation

First, we give a straightforward integer programming formulation of the KEP, with

a binary variable for every edge, and constraints so that each node is used at most

once and no long cycles occur. The objective is to maximize a weighted number of

46

edges used. Formally, we use decision variables ye for e ∈ E and f iv (flow in) and f ov

(flow out) for v ∈ V , and solve:

max∑e∈E

weye (2.1)

s.t.∑

e∈δ−(v)

ye = f iv v ∈ V

∑e∈δ+(v)

ye = f ov v ∈ V

f ov ≤ f iv ≤ 1 v ∈ P, (2.2)

f ov ≤ 1 v ∈ N, (2.3)∑e∈C

ye ≤ |C| − 1 C ⊂ C \ Ck, (2.4)

ye ∈ 0, 1 e ∈ E.

Note that we introduce some auxiliary variables f iv and f ov for all v ∈ V to simplify

the formulation, although since they are defined by the equality constraints, they can

be eliminated. In words, (2.2) says that for the patient-donor pair nodes, the flow

out is at most the flow in, and the flow in is at most one, (2.3) says that for the NDD

nodes, the flow out is at most one, and (2.4), the “cycle inequalities,” say that for

any cycle |C| of length greater than k, the number of edges we can use is at most

|C| − 1, thus prohibiting long cycles when y is integral.

The number of constraints in the IP above is exponential in |E|, due to (2.4). As

a result, for large instances, we cannot simply enumerate all of these constraints and

give them directly to the IP solver. Instead, we use a simple recursive algorithm to

solve the problem. First, we relax all the constraints in (2.4) and solve the integer

program to optimality. Then we check if the proposed solution contains any cycles of

length greater than k. If so, we add the violated constraint from (2.4) and resolve.

We repeat this procedure until our solution contains no cycles longer than k. This

methodology is generally referred to using “lazy constraints.” The technique will

generally be successful if few constraints from (2.4) need to be generated on a typical

47

input. In the worst case, we might have to solve exponentially many IPs, and to

solve each IP may take an exponential amount of time (see Section 2.4.2 for a specific

example of pathological behavior). However, as we will show in Section 2.4.1, this

technique is often quite effective in practice. Finally, note that the efficiency of this

procedure relies on the fact that we can very quickly detect if any of the constraints

from (2.4) are violated for an integer solution, as we can (trivially) find the largest

cycle in a degree two graph in linear time.

2.3.2 The Cutset Formulation

The Cutset Formulation is closely related to a standard integer programming solution

for the TSP and PC-TSP. The name is derived from the Cutset Formulation of the

TSP, as in [12].

For each cycle C of length at most k, we introduce a new variable zC that indicates

if we are using the cycle C. We make the natural updates to (2.1) so the objective

value does not change when the same edges are used and to (2.2) so that edges cannot

be used both in a zC variable and a ye variable. Finally, we add (2.6) to prohibit

cycles longer than length k, similarly to (2.4). The formulation is:

max∑e∈E

weye +∑C∈Ck

wCzC

s.t.∑

e∈δ−(v)

ye = f iv v ∈ V

∑e∈δ+(v)

ye = f ov v ∈ V

f ov +∑

C∈Ck(v)

zC ≤ f iv +∑

C∈Ck(v)

zC ≤ 1 v ∈ P, (2.5)

f ov ≤ 1 v ∈ N,∑e∈δ−(S)

ye ≥ f iv S ⊂ P, v ∈ S (2.6)

ye ∈ 0, 1 e ∈ E,

zC ∈ 0, 1 C ∈ Ck.

48

S

v n

a

b

c

Figure 2-1: An example of a Cutset constraint from (2.6). The graph contains asingle NDD in green, labeled n. Observe that if node v is to be involved in any chain(i.e. f iv = 1), then we must use at least one of the edges a or b that go across the cutseparating S from the remaining nodes and NDD.

The constraint (2.6) is very similar to the cutset inequalities for the TSP [12] as

adapted to the PC-TSP by several authors (see [42] and the references within). Figure

2-1 provides a clarifying example explaining these constraints. Essentially, they work

as follows. Suppose that a chain is reaching some node v, and as a result, f iv equals

one. Now suppose that we cut the graph in two pieces such that half containing v

does not contain any of the NDD nodes from N . Since every chain begins at some

node in N (and thus does not begin in S), in order for our chain to reach v ∈ S, it

must use an edge that begins not in S and ends in S, i.e. and edge e ∈ δ+(S). Thus

our constraint requires that whenever there is flow into v, for every way that v can

be cut off from the NDDs N , there is at least this much flow over the cut.

Again, the integer programming formulation has exponentially many constraints

from (2.6), so we cannot enumerate them and give them all directly to the IP solver.

We could simply use the same recursive heuristic (“lazy constraints”) from the pre-

vious section to obtain a correct algorithm. Instead, our solution still relaxes the

constraints (2.6), but more aggressively attempts to find to violated constraints and

add them sooner, using a technique called “cutting planes” (or “user cuts”). The

method works as follows. The integer programming solver uses the classical branch

and bound algorithm to solve the cutset formulation, initially with all of the con-

straints (2.6) relaxed. However, at every node of the branch and bound tree, the

solver checks the fractional solution produced by the LP relaxation for constraints

49

from (2.6) that are violated, and adds them to the LP as they are encountered. In-

terrupting the solver to check for violated constraints is a standard feature of many

commercial IP solvers (e.g. CPLEX and Gurobi), and is commonly referred to adding

user cuts.

To apply this method, we needed an efficient algorithm that given a potentially

fractional solution, can either find a violated constraint from (2.6) or determine that

none exists. This problem is known in the field of optimization as the separation

problem.

Theorem 2.1. The separation problem for (2.6) can be solved by solving O(|P |)

network flow problems.

A proof of the result is given in Section 2.6. See [12] for more on the separation

problem and on the network flow problem. The solution to the separation problem

for the Cutset Formulation is very similar to the solutions to the separation problems

for the TSP and PC-TSP.

2.4 Algorithm Performance

In this section, we analyze the performance of our two algorithms, both theoretically

and empirically using clinical data.

2.4.1 Algorithm Performance on Clinical KPD Data

In this section, we compare the running times of our two algorithms using clinical

data from the NKR and APD KPD programs. In Table 2.1, we show the running time

of both algorithms on a series of “difficult” but realistic KEP instances encountered

in practice. All instances have a maximum cycle length of three. These instances are

realistic in that they were taken from the simulations described in Chapter 3, and

difficult in that at least one formulation either took a long time solve, or generated

a large number of constraints. Thus, these instances represent more of a worst case

than an average case situation for real data.

50

Instance Info Running time (s)

NDDs Patient-Donor Pairs Edges Edge Formulation Cutset Formulation

3 202 4706 0.18 0.25510 156 1109 4.425 1.0696 263 8939 16.186 11.0555 284 10126 28.063 16.036 324 13175 143.432 137.6666 328 13711 150.877 27.676 312 13045 1200∗ 1200∗

10 152 1125 10.388 0.2453 269 2642 13.896 0.056

10 257 2461 16.206 0.1137 255 2390 16.7 0.1086 215 6145 44.101 2.237

10 255 2550 103.112 0.1361 310 4463 177.582 0.151

11 257 2502 201.154 0.1546 261 8915 340.312 3.829

10 256 2411 347.791 0.1196 330 13399 522.619 6.507

10 256 2347 683.949 0.1217 291 3771 1200∗ 0.1638 275 3158 1200∗ 0.3064 289 3499 1200∗ 0.3763 199 2581 1200∗ 1.9437 198 4882 1200∗ 8.2552 389 8346 1200∗ 16.076

Table 2.1: Performance of the Edge and Cutset Formulations, for “difficult” realdata KEP instances. Timeouts (optimal solution not found) indicated by ∗. Forinstances above the midline, running time for the two algorithms was within an orderof magnitude, but for instances below the midline, the Cutset Formulation was atleast an order of magnitude faster.

Instance Info Edge Formulation Cutset Formulation

Instance NDDs Paired Nodes Edges Time (s) RAM (GB) Time (s) RAM (GB)

APD 47 931 190,820 1.79 1 104 25NKR 162 1179 346,608 3.074 1 314 37

Table 2.2: Performance of the Edge and Cutset Formulations on very large historicaldatasets. Performance is measured by running time (in seconds) and RAM consumed(in Gigabytes).

51

Note that to create snapshots that are representative of actual instances encoun-

tered by a KPD program, it is insufficient to simply take altruistic donors and donor

patient pairs at random from a historical data set, as the patients that are left over

after each match run statistically tend to be harder to match than a randomly selected

patient (the easy to match patients are more likely to be matched immediately).

Table 2.1 contains the running time of both algorithms on these difficult instances,

with a maximum attempted solve time of 20 minutes. In particular, in all instances

with the reported running time less than 20 minutes, the optimal solution was found.

The instances are separated into two broad groups. In the instances from the top half

of the table, the two algorithms took about the same amount of time (to within an

order of magnitude). In the instances on the bottom half of the table, the Cutset For-

mulation was much faster. Looking at the table, we make the following observations

in comparing the performance of the two algorithms:

Both algorithms are able to solve most instances to optimality quickly.

The Cutset Formulation solves all but one instance to optimality (in fact, this

instance was solved after several hours on an independent run).

The Cutset Formulation is usually faster, although for the easier of these difficult

instances, the difference is sometimes negligible.

On several inputs, the Cutset Formulation is orders of magnitude faster.

We stress again that these instances are the worst case instances, in that we only

showed results for problems where at least one algorithm had to generate a large

number of constraints. These worst case inputs are only a small fraction of all of the

simulated inputs, and generally speaking, both algorithms can solve most of these

instances to optimality very quickly.

To demonstrate that our algorithms can solve instances even larger than those

occurring in current KPD pools, we also ran our algorithms on the entire historical

data sets for the KPD programs NKR and APD. Each data set contains around 1000

patients (though arriving over the span of several years), making these instances much

larger than the instances described in Table 2.1. The running time for our algorithms

on these instances is shown in Table 2.2. We see that:

52

Both algorithms can solve both instances.

The Edge Formulation is much faster.

While the second point may seem surprising, we do have some explanation as to why

this is taking place. First and most importantly, these instances are substantially

different from the instances that KPDs encounter in practice, in that they do not

contain a disproportionately high fraction of hard to match patients. As a result,

there is a very large number of two and three cycles, making the number of variables in

the Cutset Formulation very large. Second, these instances are both “easy” instances,

in that very few of the constraints (2.4) and (2.6) must be added by the algorithm

to solve the integer programs, unlike the instances in Table 2.1. As suggested by

the results of the previous section, the advantages of using Cutset Formulation over

the Edge Formulation depend on the constraints (2.4) and (2.6) being binding in the

optimization problem.

For the purposes of comparing algorithms, it would be preferable to have more

realistic large scale instances beyond the two described above, but the current his-

torical data does not produce such large scale instances. In an attempt to produce

more realistic large scale instances from the historical data set, we experimented with

removing fractions of the altruistic donors at random. We found that these did not

significantly change in the performance of either algorithm.

2.4.2 Strength of Formulation

In this section, we give a theoretical result comparing our integer programming formu-

lations of the KEP using Definition 2.1. Let Zedg and Zcut be the values of the optimal

solutions to the linear programming relaxations of the Edge and Cutset Formulations,

respectively.

Theorem 2.2. The following relationship holds:

Zcut ≺ Zedg.

A proof of the result is given in Section 2.6. It is often the case that integer

53

programs formulations with stronger linear programming relaxations result in faster

running time [12]. This suggests that the Cutset Formulation should solve faster than

the Edge Formulation, as we saw was often the case in Section 2.4.1.

In Figure 2-2, we provide an example of a pathological instance of the KEP that the

Cutset Formulation can solve orders of magnitude faster than the Edge Formulation.

This instance has no altruistic donors, a maximum cycle length of three, and all edges

have weight one. Compared to the real instances we considered, this is a very small

instance, as it has only 30 nodes and 120 edges. The thirty nodes are arranged in

a single large directed cycle, with some additional edges. In particular, each node

has an edge pointing to the node two steps ahead and an edge pointing to the node

ten steps ahead (along the cycle). For example, node p2 has edges to p4 (two ahead)

and p12 (ten ahead). Also each node and has an incoming edge from the node nine

steps ahead (again along the cycle), e.g. node p2 has an incoming edge from p11.

As a result each node has four outgoing and four incoming edges, e.g. the incoming

edges to node p2 are from p1, p30, p11, and p22, while the outgoing edges are to p3,

p4, p12, and p23 (the incoming edge from p22 must be present under our definition

because p22 has an outbound edge to the node ten ahead, namely p2, and likewise

for the the outgoing edge to p23). This graph has two important properties: there

are no cycles of length three or less, but there are a very large number of cycles of

length thirty. As a result, there can be no cycles in any feasible solution, and the

Cutset Formulation will create no cycle variables (recall the maximum cycle length

was assumed to be three). Because there are no non-directed donors or short cycles,

the optimal solution is zero. It is easy to see that the linear programming relaxation

for the Cutset Formulation is also zero. By considering the cut where S = P and

v is arbitrary, as there are no edges going over this cut, the cutset inequality (2.6)

implies that the flow into v is zero. As v was arbitrary, we have that the flow in to

every node is zero, so the LP relaxation is zero. Thus for this instance, solving the

integer programming problem is no harder than solving a single linear programming

problem with the Cutset Formulation. Further, given that our proof that the LP

was zero only used |P | of the inequalities from (2.6), it is possible to solve the LP

54

p1 p2p3

p4

p5

p6

p7

p8

p9

p10

p11

p12

p13

p14p15p16

p17

p18

p19

p20

p21

p22

p23

p24

p25

p26

p27

p28

p29

p30

Figure 2-2: A pathological instance of the KEP that is very difficult for the EdgeFormulation but is solved trivially by the Cutset Formulation. The optimal solutionis zero.

very quickly depending on the strategy used to add violated constraints (in fact, our

solver adds all of these constraints in a single round of cut generation). As a result,

we are able to solve this instance almost instantly with the Cutset Formulation. In

contrast, for Edge Formulation, the linear programming relaxation has the optimal

value 29, as it can simply assign value 29/30 to every edge on the length-30 cycle.

Worse yet, after two hours of the running time, the best upper bound that the Edge

Formulation gives is still 30, as the high redundancy in the structure of the graph

results in many possible cycles of length 30. The constraints for each of these cycles

needs to be added to obtain an upper bound of 29. For similar reasons, branch and

bound is very ineffective.

55

2.5 Extensions

2.5.1 Long Maximum Cycle Length

In this section, we very briefly discuss how to solve the KEP with a maximum cycle

length significantly beyond three or four. The problem can be immediately solved

using our Edge Formulation for the KEP with no changes. In fact, for a fixed graph,

the Edge Formulation IP will likely become easier as the maximum cycle length

increases, as it simply corresponds to using fewer constraints.

However, the Edge Formulation was significantly worse than the Cutset Formula-

tion, both theoretically as defined by strength of formulation, and practically in terms

of running time on many instances. We would like to use the Cutset Formulation for

these large cycle problems. However, we would quickly run out of memory adding a

decision variable for every cycle in the graph, as the number of cycles grows rapidly

with the maximum cycle length. Even on a computer with extra memory, adding a

large number of variables that would mostly take the value zero would excessively

slow down the many linear programs solved internally when solving an IP. One pos-

sible solution would be to use the column generation scheme of [1] (see also [23])

to dynamically add variables to model in the middle of the optimization, much like

we add the Cutset constraints currently. However, using such a strategy is largely

incompatible with using the best commercial integer programming solvers and thus

would require significant effort to implement. The investigation of the viability of

this approach is a subject for future work.

2.5.2 Bounded Chain Lengths

We show how to adapt the Cutset Formulation to allow for a maximum chain length

`. With some additional work, this technique can also be used to adapt the Edge

Formulation, although we will not pursue this further. For each NDD n ∈ N and

each edge e ∈ E, we introduce auxiliary edge variables yne and likewise f i,nv and f o,nv

56

indicating flow that must begin at n. The formulation becomes:

max∑e∈E

weye +∑C∈Ck

wCzC

(y, z, f i, fo) ∈ Pcut∑n∈N

yne = ye e ∈ E (2.7)

∑e∈E

yne ≤ ` n ∈ N (2.8)

∑e∈δ−(v)

yne = f i,nv v ∈ V, n ∈ N (2.9)

∑e∈δ+(v)

yne = f o,nv v ∈ V, n ∈ N (2.10)

f o,nv ≤ f i,v ≤ 1 v ∈ V, n ∈ N (2.11)

ye ∈ 0, 1 e ∈ E,

zC ∈ 0, 1 C ∈ Ck

yne ∈ 0, 1 e ∈ E, n ∈ N.

The new constraints are briefly explained as follows. From (2.7), we have that each

edge used (ye) must be part of a chain beginning at some NDD n. From (2.8), we

obtain that each chain can use at most ` edges, thus giving the maximum chain

length. In (2.9) and (2.10), we just define auxiliary variables denoting if an edge used

in a chain starting at n comes into/out of v. Finally, in (2.11), we enforce that the

edges used in the chain starting at n are consecutive. The remaining constraints are

exactly the same as the PC-TSP constraints with no maximum chain length.

2.5.3 Two Stage Problems

Here we present a general framework for dealing with the possibility that after an

edge is selected, it might become ineligible for the matching, an event we refer to as

an “edge failure.” Edge failures occur commonly in practice for a variety of reasons,

e.g. a donor backs out, a patient dies, or a biological incompatibility is discovered.

57

We propose a two phase system for planning exchanges that anticipates edge fail-

ures occurring at random, and plans to maximize the number of transplants performed

once the failed edges have been identified and removed. In the first phase, a subset

of the edges in the graph are selected to be tested for edge failures. Operational

constraints restrict this set, where the basic idea is that it is not practical to check

all the edges. Some natural examples of phase one edge sets to test include:

Use at most m edges in phase one.

Each node has in degree at most mi and out degree at most mo.

The edges used in phase one must be a feasible solution to the KEP.

The only restriction on the rule used to select phase one edges is that there exists a

polyhedron P such that y ∈ P ∩ Z|E| iff y corresponds to a valid set of phase one

edges, i.e., the set of phase one edges must be describable as a MIP. After the phase

one selections are made, we learn which of the edges that we tested in phase one

failed, and in phase two, we solve the regular KEP using only edges that we checked

and did not fail in phase one. As we do not know which edges will fail before we make

our phase one decision, we use the objective of maximizing the expected weight of our

phase two KEP solution when picking our phase one solution. Next, we describe

the probabilistic framework we use for edge failures, and then the computational

technique used to compute our phase one solution.

We assume that there is a family of random variables Xe for e ∈ E, taking the

value one if the edge e can be used in the matching (if the edge does not fail) and

zero otherwise (if the edge fails). We make no assumptions about the independence

structure of the variables Xe. However, we do assume that we can jointly sample the

vector of Xe variables.

We now define a two stage stochastic integer optimization problem. We have

decision variables ye for e ∈ E which indicate the edges we wish to test in stage one.

In stage two, we observe our realization ω ∈ Ω of Xe(ω) for the edges where ye = 1

(the edges we tested), and then we form an optimal cycle and chain packing using

only edges that we tested in phase one and where Xe(ω) = 1. We select our phase one

edges y, integer and in P , to maximize the expected size of the phase two packing.

58

This problem can be solved using the method of sample average approximation,

as described and mathematically justified in [2, 54, 78]. Suppose that we sample the

vector of Xe jointly n times, and let xje for j = 1, . . . , n be the realization of Xe in the

jth sample. Let yje be one if we use edge e in realization j and zero otherwise, and

likewise let zjC be one if we use cycle C in the jth realization. Let P jcut be the Cutset

polyhedron the variables yje and zjC . Our formulation is then as follows:

maxn∑j=1

(∑e∈E

ceyje +

∑C∈Ck

cCzjC

)(2.12)

s.t. y ∈ P,

(yj, zj) ∈ P jcut,

yje ≤ ye e ∈ E, j = 1, . . . , n,

yje ≤ xje e ∈ E, j = 1, . . . , n,

zjC ≤ ye C ∈ Ck, e ∈ C, j = 1, . . . , n,

zjC ≤ xje C ∈ Ck, e ∈ C, j = 1, . . . , n,

ye ∈ 0, 1 e ∈ E,

yje ∈ 0, 1 e ∈ E, j = 1, . . . , n,

zjC ∈ 0, 1 C ∈ Ck, j = 1, . . . , n.

This model has a few very attractive features. First, it allows for a general proba-

bilistic model for edge failures, which in practice should be much more accurate than

simply i.i.d. edge failures. For example:

If an edge failed because the donor or receiver became ill or backed out, then

all edges involving that donor/receiver would be ruled out simultaneously.

If an edge failed because a receiver developed a new HLA antibody, then all

edges from donors with that HLA antigen incoming to this receiver would fail

simultaneously.

If an edge failed because a doctor or transplant center deemed a donor to be of

inadequate quality for the recipient (e.g. the donor was too old), then possibly

59

other edges pointing to the same doctor/transplant center would fail, but not

necessarily all of them, as a highly sensitized recipient may wish to accept a

kidney from this donor, while a standard recipient would not.

A salient feature of this model is that we have a great deal of flexibility in choosing

P (the set of edges we are allowed to pick in phase one). Our flexibility in choosing

P allows us to adapt to various operational constraints of actually running a kidney

exchange. Additionally, we can use P to try and influence “agents” (e.g. donors, re-

cipients, doctors, hospitals, and transplant centers) into taking actions that maximize

global welfare. For example, if we select more than one incoming edge to a node in

phase one, then the receiver, the doctor, the hospital, and the transplant center may

be incentivized to reject the worse of the two edges in order to try and get a higher

quality donor. One very simple fix is to restrict the edges tested in phase one so

that each node has an in-degree of at most one. Then as no one will receive multiple

offers, no one will be incentivized to turn down a kidney they otherwise would have

accepted.

Finally, note that it is at times desirable to add additional decision variables to

the phase one problem. For example, if we were to restrict our phase one solution to

be a feasible solution to the KEP, while we could take P = Pedg, it is computationally

more efficient to use the Cutset Formulation instead. One way of accomplishing this

is as follows, add a decision variables ye for e ∈ E and zC for each cycle C ∈ Ck, let

ye = ye +∑C∈Cke∈C

zC ,

and then take P to be the PC-TSP polyhedron applied to y and z, along with the

constraint above relating y to y and z. Further, note that the Cutset constraints for

the P jcut polyhedrons would automatically be implied by the Cutset constraints from

Pcut on (y, z) and thus could be eliminated.

60

2.6 Proofs

2.6.1 Proof of Cutset Separation

Proof of Theorem 2.1. Let (y, z) be the point for which we must determine if (2.6) is

satisfied. Following the well known procedure for Prize Collecting TSP, first we form

a directed weighted graph G = (V , E, w) where V = s ∪ V where s is an extra

node, and E = E ∪ (s, n) | n ∈ N, and weights we for e ∈ E are given by

we =

ye e ∈ E,

1 otherwise,

(the edges with we = 1 each go from the super source to a node in N).

Then for every v ∈ P where f iv > 0, we solve the max flow min cut problem with

source s and sink v. If we find a cut of weight less than f iv, then by taking S to be

the set of nodes on the sink side of the cut, we have found a violated constraint. As

we are optimizing over all cuts separating v from the super source and then checking

all v, we in fact check all the constraints from (2.6).

2.6.2 Proof of Strength of Formulation

Before proving the result, we introduce two auxiliary integer programming formula-

tions.

The Cycle Formulation

We propose an alternative formulation on the same set of variables as the Cutset

Formulation, called the Cycle Formulation. In the Cycle Formulation, all of the

variables and constraints are the same as the Cutset Formulation except that the

constraint (2.6) is replaced by

∑e∈C

ye +∑D∈CkD 6=C

|D ∩ C|zD ≤ |C| − 1 C ∈ C, (2.13)

61

The idea of the constraint is to prevent the edges of C from forming a cycle, unless

the variable zC is used (should the variable exist). Obviously, if ye = 1 for e ∈ C,

they would form a cycle, but the constraint prevents this. It would be more intuitive

to replace (2.13) with the simpler equation

∑e∈C

ye ≤ |C| − 1 C ∈ C. (2.14)

While this would produce a valid formulation of the KEP, we trivially have that this

would result in a formulation that is no better (in the sense of Definition 2.1), due to

polyhedron containment. In fact, replacing (2.13) by (2.14) results in a formulation

that is strictly worse. The example in Figure 2-3 shows an instance of the KEP

where if (2.13) is replaced by (2.14), then the LP relaxation of this modified Cycle

Formulation is worse than the LP relaxation of the Edge Formulation (without the

modification, the LP relaxations of the Edge and Cycle Formulations are the same

for this instance).

The Subtour Formulation

Our final formulation, the Subtour Formulation, is again on the same set of variables

as the Cycle and Cutset Formulations. The name is derived from the Subtour Elim-

ination Formulation of the TSP, as in [12]. In the Subtour Formulation, all of the

variables and constraints are the same as the Cutset (and Edge) Formulations except

that the constraint (2.6) is replaced by

∑e∈E(S)

ye +∑D∈CkV (D)⊆S

(|D| − 1)zD +∑D∈CkV (D)6⊆S

|D ∩ E(S)|zD ≤ |S| − 1 S ⊂ P. (2.15)

The idea of the constraint is that subset S of the nodes, the number of edges used to

make chains should be at most |S|−1. Again, in a slightly more intuitive formulation,

62

p1

p4

p2

p3

p5

3/4

3/4

3/4

1

1/4 1/4

p1

p4

p2

p3

p5

2/3

2/3

2/3

1

1/3 1/3

Figure 2-3: The figure above represents two fractional solutions to a single instance ofthe KEP. The purpose of this example is to demonstrates the necessity of using (2.13)instead of (2.14). In this instance, P = p1, . . . , p5, N = ∅, the edges are as indicatedin the figure above, and all edges have weight one. The numbers next to the edges in-dicate fractional solutions, namely ye for the Edge Formulation, and ye+

∑C∈Ck,e∈C zC

for the Cycle Formulation. Observe that the solution on the left has greater weightthan the solution on the right. The solution on the left is infeasible for the EdgeFormulation, as the constraint on the cycle (p1, p2), (p2, p3), (p3, p4), (p4, p1) is vi-olated. The solution on the right is optimal for the Edge Formulation. For theCycle Formulation, letting the cycle D = (3, 4), (4, 5), (5, 3), without the secondsum from the left hand side of (2.13), we could take zD = 1/4 and ye = 3/4 fore = (1, 2), (2, 3), (3, 4), (4, 1) and then fractional solution on the left would be feasi-ble. This would break the result that Zcyc Zedg. However, by including the variablezD in the constraint against the four cycle, we again have that the solution on theright is optimal for the Cycle Formulation.

63

we could replace (2.15) by

∑e∈E(S)

ye ≤ |S| − 1 S ⊂ P. (2.16)

However, this again produces a weaker formulation, as when replacing (2.13) by (2.14).

Additionally, observe that for any cycle C, letting S = V (C) be the set of vertices

incident to the edges of the cycle (so |C| = |S|), then as C ⊂ E(S),

∑e∈C

ye ≤∑

e∈E(S)

ye ≤ |S| − 1 = |C| − 1, (2.17)

i.e., (2.16) implies (2.14), so the (weakened) Subtour Formulation polytope is con-

tained in the (weakened) Cycle polytope and thus the (weakened) Subtour Formula-

tion must be at least as strong as the (weakened) Cycle Formulation. Ultimately, we

will show a similar result for (2.15) and (2.13).

Proof of Theorem 2.2

Throughout, we let Pedg, Pcyc, Psub, and Pcut be the polyhedrons for the linear pro-

gramming relaxations of the Edge, Cycle, Subtour and Cutset Formulations of the

KEP, respectively. Likewise, we let Zedg, Zcyc, Zsub, and Zcut be the values of the

optimal solutions to the linear programming relaxations of these formulations.

Proof of Theorem 2.2. We will instead show

Zcut ≺ Zsub ≺ Zcyc Zedg

which, as the relations ≺ and are transitive, implies the result.

First, we show that Pcut ⊆ Psub, which immediately implies that Zcut Zsub, as

the two formulations share the same objective function. It suffices to show that each

of the subtour elimination constraints from (2.15) are implied by the entire Cutset

Formulation. Fix S ⊂ P , and assume that y is feasible for the Cutset Formulation.

64

Fix some u ∈ S. First, we claim that

∑D∈CkV (D)⊆S

(|D| − 1)zD +∑D∈CkV (D)6⊆S

|D ∩ E(S)|zD

≤∑D∈Ck

V (D)∩S 6=∅

(|V (D) ∩ S| − 1)zD (2.18)

≤∑v∈Sv 6=u

∑D∈Ck(v)

zD. (2.19)

To justify (2.18), observe that for cycles D such that V (D) ⊆ S, we immediately have

|D| = |V (D)| = |V (D) ∩ S|, so for these zD terms, we |D| − 1 = |V (D) ∩ S| − 1. For

D such that V (D) 6⊆ S, we have two cases:

If V (D) ∩ S = ∅, then D ∩ E(S) = ∅ as well, so these terms can be dropped.

If D has ` vertices in S, where 0 < ` < |D|, then at most `− 1 of the edges of

D will have both endpoints in S.

Thus (2.18) has been shown. To justify (2.19), by a simple counting argument, we

have that:

If u 6∈ V (D), then the term zD will appear |V (D) ∩ S| times in (2.19),

If u ∈ V (D), then the term zD will appear |V (D) ∩ S| − 1 times in (2.19).

65

Thus (2.19) has been shown. Applying this inequality, we now have

∑e∈E(S)

ye +∑D∈CkV (D)⊆S

(|D| − 1)zD +∑D∈CkV (D)6⊆S

|D ∩ E(S)|zD

≤∑

e∈E(S)

ye +∑v∈Sv 6=u

∑D∈Ck(v)

zD

=∑v∈S

f iv −∑

e∈δ−(S)

ye +∑v∈Sv 6=u

∑D∈Ck(v)

zD (2.20)

= f iu −∑

e∈δ−(S)

ye +∑v∈Sv 6=u

f iv +∑

D∈Ck(v)

zD

≤∑v∈Sv 6=u

f iv +∑

D∈Ck(v)

zD

(2.21)

≤ |S| − 1, (2.22)

where (2.20) follows as for a set of nodes S, all edges incoming to a node in S have

there other endpoint either in S or outside of S, (2.21) follows from applying (2.6)

(multiplied by −1) for the set S and the vertex u, and (2.22) follows from applying

the upper bound from flow constraint (2.5) |S| − 1 times.

Next, we show that Psub ⊆ Pcyc and thus Zsub Zcyc. If suffices to show that for

any cycle C, (2.13) is directly implied by (2.15) taking S = V (C). To bound the first

term of the left hand side of (2.13), we have

∑e∈C

ye ≤∑

e∈E(S)

ye.

For the second term, we will partition D ∈ Ck, D 6= C into two sets, those where

V (D) ⊆ S and D 6= C, or those where V (D) 6⊆ S, i.e.,

∑D∈CkD 6=C

|D ∩ C|zD =∑D∈CkV (D)⊆SD 6=C

|D ∩ C|zD +∑D∈CkV (D) 6⊆S

|D ∩ C|zD.

66

For the first sum, we have |D ∩ C| ≤ |D| − 1, as D 6= C (and D 6⊂ C since both D

and C are simple cycles). Thus

∑D∈CkV (D)⊆SD 6=C

|D ∩ C|zD ≤∑D∈CkV (D)⊆SD 6=C

(|D| − 1)zD ≤∑D∈CkV (D)⊆S

(|D| − 1)zD

For the second sum, as C ⊂ E(S), we have |D ∩C| ≤ |D ∩E(S)| for all D ∈ Ck, and

thus

∑D∈CkV (D)6⊆S

|D ∩ C|zD ≤∑D∈CkV (D) 6⊆S

|D ∩ E(S)|zD.

Putting everything together, then applying (2.15) we have

∑e∈C

ye +∑D∈CkD 6=C

|D ∩ C|zD

≤∑

e∈E(S)

ye +∑D∈CkV (D)⊆S

(|D| − 1)zD +∑D∈CkV (D)6⊆S

|D ∩ E(S)|zD

≤ |S| − 1 = |C| − 1,

showing the claim.

To show that Zcyc Zedg, consider (y∗, z∗) ∈ Pcyc that is optimal for the Cycle

Formulation (the values of f iv and f ov are implied by y∗). If we let

xe = y∗e +∑

C∈Ck, e∈C

z∗C ,

then we claim that x ∈ Pedg (again with the values of the flow variables being deter-

mined by x). To show this, it suffices to verify (2.2), (2.3) and (2.4) hold for x. To

67

obtain (2.2), we have

∑e∈δ+(v)

xe =∑

e∈δ+(v)

(y∗e +

∑C∈Ck, e∈C

z∗C

)(2.23)

=∑

e∈δ+(v)

y∗e +∑

C∈Ck(v)

z∗C (2.24)

where in (2.23) we applied the definition of xe, and in (2.24), we used that Ck(v), the

set of cycles hitting v, is equal to the disjoint union over all e going out of v of the set

of cycles containing e (the union is disjoint as each cycle contains exactly one edge

out of v). Likewise, we have

∑e∈δ−(v)

xe =∑

e∈δ−(v)

y∗e +∑

C∈Ck(v)

z∗C .

Thus (2.5) from the Cycle Formulation implies (2.2) in the Edge Formulation. An

analogous argument immediately gives us (2.3) as well. Finally, to obtain (2.4), we

have for any cycle C with |C| > k,

∑e∈C

xe =∑e∈C

y∗e +∑D∈Cke∈D

z∗e

=∑e∈C

y∗e +∑D∈CkD 6=C

|D ∩ C|zD (2.25)

≤ |C| − 1, (2.26)

where in (2.25), we are counting, and using that |C| > k implies that there is no

D ∈ Ck such that D = C, and in (2.26) we are applying (2.13). Thus we conclude

68

p1

p2

p3

p4

p5

p6

p7

p8

Figure 2-4: Consider the family of problem instances on n ≥ 4 nodes where P =p1, . . . , pn, N = ∅, there are n edges forming a single cycle of length n, and we = 1for every edge. Above is the instance where n = 8. The optimal solution for the IPand the Cutset LP relaxation are both zero, but the Subtour LP relaxation has anoptimal solution n− 1 (each node has ye = (n− 1)/n).

that x is feasible. Using feasibility, we can obtain the result as follows:

Zedg ≥∑e∈E

cexe

=∑e∈E

ce

y∗e +∑C∈Cke∈C

z∗e

=∑e∈E

cey∗e +

∑C∈Ck

cCz∗e

= Zcyc.

In Figure 2-4, we give a family of problem instances where Zcut < Zsub. In Figure

2-5 we give an instance where Zsub < Zcyc.

69

p1

p2 p3

p4

p5p6

1

1/2

1

1/2

1

1/2

1/2

1/2

1/2

Figure 2-5: In the instance on six nodes above, where k = 3, N = ∅, P = p1, . . . , p6,and each edge has weight one, the IP optimum is zero. Taking ye to be the edgelabels in the figure above, we get a feasible solution to the LP relaxation of Zcyc = 6.However, the LP optimum for the Subtour Formulation is Zsub = 5. We can attainthis value by taking y(i,i+1) = 5/6 and y(6,1) = 5/6. To show that 5 is optimal, weapply the (2.15) taking S = P , to obtain that

∑e∈E(P ) ye ≤ 5, and then observe that∑

e∈E(P ) ye is equal to the objective function.

70

Chapter 3

Data Driven Simulations for

Kidney Exchange

3.1 Introduction

In this chapter, we consider how the types of exchanges performed (e.g. two-cycles,

three-cycles, and chains) and the strategy used to select exchanges (e.g. greedy,

batching with a match run time of n days) impact aggregate patient outcomes in

Kidney Paired Donation (KPD). In particular, we focus on the total transplants

performed and the average time patients wait to be transplanted. We quantify these

effects by simulating the dynamics of the National Kidney Registry (NKR) KPD

pool using historical clinical data. Most importantly, we investigate: (a) the value of

forming long chains with altruistic donors, and (b) the trade-off in average patient

waiting time when setting the match run time for the batching policies (see Section

1.1 for a discussion of this trade-off).

Organization

This chapter is organized as follows. In Section 3.2, we explain the methods used to

perform this simulation analysis. In Section 3.3 we give the results of our simulations.

In Section 3.4, we discuss the simulation outcomes and their implications.

71

3.2 Methods

Data

We simulate the NKR KPD pool over a two year time period from May 24, 2010 to

May 24, 2012. We initialize the pool by taking all donors and patients arriving be-

tween January 1, 2008 and May 24, 2010, and then removing the donors and patients

that NKR had actually matched before May 24th, 2010. This initial pool contains

63 patient-donor pairs, and an additional 410 pairs arrive over the course of the sim-

ulation. The dataset contains 75 altruistic donors and 244 patients on the waiting

list that have no associated donor. Compatibility between donors and patients is

determined primarily by blood type and HLA compatibility rules. Several additional

factors were also used including patient preferences (e.g. a patient is unwilling to

consider a donor over 60 years old) and previously attempted but failed cross match

tests from the NKR database.

Individual Metrics

In this section, we define several metrics designed to quantify the difficulty of matching

a patient, donor, or patient-donor pair in kidney exchange. These metrics were first

designed by NKR, and we use a slightly modified version.

These metrics are all defined relative to a population of patients and donors.

Throughout, we take this population to be all donors and patients in a patient-

donor pair from the historical NKR dataset. For each patient, the patient power is

the fraction of donors from the population that are biologically compatible with the

patient. Similarly, for each donor, the donor power is the fraction of patients from

the population that are biologically compatible. For each patient-donor pair, the pair

match power (PMP) is given by

(100 ∗ patient power of patient in pair) ∗ (100 ∗ donor power of donor in pair).

Finally, we mention that for patients that have more than one willing donor (the

72

NKR dataset contains several such patients), we need to adapt our definition. We

replace the “donor power” term with the following quantity: for a patient with mul-

tiple donors, the fraction of patients from the population that at least one donor is

compatible with.

Matching Rules and Policies

The matching rules specify the types of exchanges that can be made. Typically,

we will use the rule that chains of any length and cycles of length two and three

are allowed. The matching policy is the strategy that is used to determine which

exchanges to make (subject to the matching rules).

We will focus on a family of matching policies we refer to as the batching policies.

Each policy in the family is defined by a single parameter n, referred to as the match

run time. Exchanges are selected as follows. Patients and donors arrive for a period

of n days where no exchanges take place. Then, using all altruistic donors, patient-

donor pairs, and waiting list patients currently in the pool, an instance of the KEP

(see Chapter 2) is solved find the set of exchanges that maximizes the number of

transplants performed (all edges have weight one). The patients and donors that are

matched are removed from the pool, and the process is repeated every n days.

An important special case of the batching policies is the greedy policy, which every

day performs as many exchanges as possible (this is the batching policy with n = 1).

Simulation Model

The simulation requires three inputs:

1. A list of patient-donor pairs, altruistic donors, and waiting list patients, each

with the date they enter the system,

2. Matching rules to determine which exchanges are allowed,

3. A match run time n to select a batching policy.

The simulation maintains as its state the current date and the current KPD pool

of those waiting to be matched. The pool consists of patient-donor pairs, waiting

73

list patients, and non-directed donors (NDDs). The NDDs are the altruistic donors

combined with the bridge donors (see Section 1.1 for more on this distinction). The

pool is initialized using NKR’s historical data to represent the historical NKR pool

for May 24, 2010, and the current date is initialized to May 24th, 2010 as well. The

simulation then repeats the following steps until all the patients in the input have

arrived:

Advance the current time n days (e.g. one day for n = 1 or one week for n = 7).

Add the patient-donor pairs, altruistic donors, and waiting list patients that

have arrived during this n day period to the current pool.

Solve the KEP to determine a set of transplants to be performed.

Remove any patient-donor pair that was matched in a cycle from the pool.

For each chain ending on a waiting list patient, remove every patient-donor

pair, waiting list patient, and NDD in the chain from the pool. For each chain

ending on a patient-donor pair, remove every patient-donor pair and NDD in

the chain from the pool except the final patient-donor pair in the chain.

The donor from this pair is converted to an NDD (sometimes referred to as a

bridge donor) and remains in the pool.

System Performance Metrics

To evaluate simulation outcomes, we focus on the following long run performance

metrics:

Total Matches: The total number of transplants that were performed during

the simulation.

Average Waiting Time: The average over all patient-donor pairs of the following

quantity:

minmatch date, simulation end −maxarrival date, simulation start

where for pairs that were not matched by the end of the simulation, their “match

date” will be greater than “simulation end”.

74

Intuitively, the quantity average waiting time is capturing how much waiting occurred

in the simulation window. Importantly, the metric can be used to meaningfully

compare outcomes when the number of patient-donor pairs that were matched is

different, or even when the number of patient-donor pairs in the simulation is different.

We will also compute these metrics for sub-populations of “hard-to-match” pa-

tients. In particular, we consider patient-donor pairs that have a pair match power

of less than 20, although this distinction is somewhat arbitrary. For the NKR popu-

lation of patient-donor pairs, 182 of the 473 patient-donor pairs had a PMP less than

20, or 38%.

3.3 Simulation Results

Comparison of Batching Policies

In our first experiment we investigate the impact of the match run time on our

performance metrics of total transplants and average waiting time. We use the current

NKR rule allowing cycles of length up to three, and chains of unbounded length. The

results are summarized in Figure 3-1. We see that as the match run time increases

from one day to one year, the total number of patient-donor pairs matched increases

from 264 to 290. However, we also see that average patient waiting time increases

from 158 days to 216 days. Further, we observe for all match run times between a

day and a month, the performance is nearly identical. In Figure 3-2, we see that

disproportionately many of the additional matches gained by doing batching are for

hard-to-match pairs. In particular, when the batch size changes from one day to one

year, of the extra 26 transplants gained, 17 are for hard-to-match pairs (pairs with

PMP less than 20). The number of hard-to-match pairs matched increases by 33%,

while the number of easy-to-match pairs matched increases by only 4%.

75

1D 1W 2W 1M 3M 6M 1Y150

200

250

300

Match Run Time

Pat

ient-

Don

orP

airs

Mat

ched

1D 1W 2W 1M 3M 6M 1Y100

150

200

Match Run Time

Ave

rage

Wai

ting

Tim

e(d

ays)

Figure 3-1: Simulation results when using a batching policy and varying the matchrun time from one day to one year. Left: total patient-donor pairs matched. Right:average waiting time incurred by patient-donor pairs over the course of the simulation.

1D 1W 2W 1M 3M 6M 1Y0

20

40

60

Match Run Time

PM

P≤

20P

airs

Mat

ched

Figure 3-2: The number of hard-to-match patient-donor pairs (pairs with PMP lessthan 20) matched, as match run time changes from one day to one year.

76

Measuring the Value of Chains

In this experiment, we consider the implications of a reduced number of altruistic

donors. In particular, we run simulations using only 75%, 50%, 25%, and 0% of

the altruistic donors from the NKR data set, and see what the impact is on our

performance metrics. Throughout, we use the matching rule that cycles can be of

length two or three, and that chains can be of any length. However, when we eliminate

all of the altruistic donors, we are essentially using the rule that there can be no chains.

The only matching policy we use is the greedy policy.

The results are summarized in Figure 3-3. We see that as the number of altruistic

donors decreases, the total matches decreases substantially, and the average waiting

time increases. In particular, if we eliminate all of the chains, a total of 63 transplants

among paired nodes are lost. In addition, another 75 transplants to donors on the

waiting list will be lost, as now we cannot end our chains on the waiting list. Thus

each altruistic donor on average contributes nearly two transplants when donating

through KPD, while by donating directly to the deceased donor waiting list would

result in only a single transplant. This quantity should not be interpreted too literally,

as we cannot predict what would have happened if the simulation ran over a longer

time horizon (perhaps eventually, most of these pairs would have found an exchange

in a cycle).

Comparing our other metrics in the cases when we use either all or none of our

altruistic donors, we see that the use of chains gives additional positive patients

outcomes. Average waiting time for patient-donor pairs decreases by over 50 days

when chains are formed. In Figure 3-4, we see that disproportionately many of the

additional transplants gained from using the altruistic donors are hard-to-match pairs.

In particular, comparing using no altruistic donors and all of the altruistic donors, we

increase the number of hard-to-match pairs transplanted by about 100%, while the

total number of pairs transplanted increases by only 30%.

Finally, we note that the amount each altruist can contribute depends on the

number of patient-donor pairs available to enter an exchange with. As each additional

77

0 25 50 75 100150

200

250

Altruists Included (%)

Pat

ient-

Don

orP

airs

Mat

ched

0 25 50 75 100100

150

200

Altruists Included (%)

Ave

rage

Wai

ting

Tim

e(d

ays)

Figure 3-3: Simulation results when using a reduced number of altruistic donors.Left: total patient-donor pairs matched. Right: average waiting time incurred bypatient-donor pairs over the course of the simulation.

altruistic donor uses some number of these pairs when forming a chain, fewer remain

for subsequent altruists. Thus we would intuitively expect that additional altruistic

donors should have diminishing marginal value. While it appears that there may be

some evidence of this in Figure 3-3 and Figure 3-4, a more rigorous experimental

design would be needed to make any definitive conclusions. We leave this as a topic

for future research.

Adjusting the Maximum Cycle Length

In this experiment, we consider the implications of adjusting the maximum cycle

length. In particular, we run simulations using a maximum cycle length of zero, two,

three, and four. Throughout, we use the matching rule that chains can be of any

length, and our matching policy is the greedy policy.

The results are summarized in Figure 3-5. We see that changing the maximum

cycle length has very little effect on the total number of matches or average patient

waiting time. In Figure 3-6, we see that the number of hard-to-match pairs matched

is relatively unaffected by the maximum cycle length as well.

Surprisingly, performance is very slightly better when the maximum cycle length

is zero. While such a small improvement is likely to not be statistically significant

78

0 25 50 75 1000

20

40

Altruists Included (%)P

MP≤

20P

airs

Mat

ched

Figure 3-4: The number of hard-to-match patient-donor pairs (pairs with PMP lessthan 20) matched, as the number of altruistic donors is reduced.

in our experimental setup, a plausible explanation for this behavior is as follows.

The most likely two cycle to be formed is between two easy-to-match pairs. Thus

by eliminating two-cycles, we gain some easy-to-match pairs to use in chains. These

pairs, despite not forming two-cycles with hard-to-match pairs, still link to them.

Eventually, when a chain reaches these one of these easy-to-match pairs, we can

route the chain to a hard-to-match pair that would otherwise be unreachable, rather

than to the easy-to-match pair that would have formed a two-cycle with this pair.

3.4 Discussion

In this section, we summarize the key insights observed in our simulations of the NKR

KPD pool. Our findings are as follows:

The greedy policy produces nearly as many transplants as any batching policy,

and has the lowest average waiting time.

First, we note that this result is somewhat surprising, as a priori there is a trade-

off in setting the match run time with respect to the average patient waiting

time. For a complete discussion, see Section 1.1.

Second, the batching policies with a match run time between one day and

one month all had essentially the same performance. However, as discussed in

79

0 2 3 4150

200

250

Maximum Cycle Length

Pat

ient-

Don

orP

airs

Mat

ched

0 2 3 4100

120

140

160

Maximum Cycle Length

Ave

rage

Wai

ting

Tim

e(d

ays)

Figure 3-5: Simulation results when varying the maximum cycle length. Left: totalpatient-donor pairs matched. Right: average waiting time incurred by patient-donorpairs over the course of the simulation.

0 2 3 40

20

40

60

Maximum Cycle Length

PM

P≤

20P

airs

Mat

ched

Figure 3-6: The number of hard-to-match patient-donor pairs (pairs with PMP lessthan 20) matched, as the maximum cycle length is adjusted.

80

Chapter 1, the greedy policy is of particular interest as it is superior to batching

for a variety of practical reasons not captured by our metrics.

The use of chains results in significantly more transplants and reduced patient

waiting time, and hard-to-match patient-donor pairs are disproportionately the

beneficiaries of incorporating chain based exchanges.

According to our performance metrics, there is essentially no benefit in changing

the maximum cycle length, given the current number of altruistic donors.

We saw that regardless of whether the maximum cycle length was zero, two,

three or four, we did approximately the same number of transplants and patients

experienced the same average waiting time. If there were fewer altruistic donors,

the claim would no longer be true. For (an extreme) example, in the case where

there are no altruistic donors, obviously reducing the maximum cycle length to

zero would result in the loss of all transplants.

Despite cycles providing no benefit in our model, we would not recommend

eliminating their use completely, as cycles may have benefits not captured by

our model. Observe that when we eliminate cycles, since the total number of

patient transplants is the same as when using only chains, the chains must

on average become longer. As we discuss at some length in Section 1.1.5, our

model is missing features (as compared to an actual KPD exchange program)

that if incorporated, could potentially make transplanting a patient in a very

long chain worse than transplanting a patient in a short cycle (particularly a

two cycle).

There is very limited room to improve the total number of transplants beyond

the level attained by greedy policy.

Using a match run time of one year was not seriously considered as an imple-

mentable solution. If such a long match run time were used in practice, the

rate of patient abandonment (due to sickness) would more than cancel out the

increased number of transplants (see Section 1.1.5). Instead, it was supposed

to serve as a “upper bound” on the number of transplants that would be at-

tainable for an implementable strategy. We saw that the greedy policy was

81

able to produce about 90% of the total transplants produced using a match run

time of one year, suggesting that there is limited room for any policy to give an

improvement over greedy in total matches.

In fact, we can make this notion of an “upper bound” rigorous as follows.

Suppose that we take the match run time to be the entire two year time horizon.

Then essentially, we have an offline strategy where all of the arrivals are known

before any matching decisions are made. Thus we get a bound on the maximum

number of transplants achievable by any online matching strategy. It turns out

that using a two year match run time gives only 305 transplants, meaning that

the greedy policy is within 15% of optimal on total transplants.

82

Chapter 4

Dynamic Random Graph Models

for Kidney Exchange

4.1 Introduction

We consider the problem of efficient operation of a barter exchange platform for

indivisible goods. We introduce a dynamic model of barter exchange where in each

period one agent arrives with a single item she wants to exchange for a different item.

We study a homogeneous and stochastic environment: an agent is interested in the

item possessed by another agent with probability p, independently for all pairs of

agents. We consider three settings with respect to the types of allowed exchanges:

(a) Only two-way cycles, in which two agents swap their items, (b) Two or three-way

cycles, (c) (unbounded) chains initiated by altruistic donors who provide an item but

expect nothing in return. The goal of the platform is to minimize the average time

an agent waits to make an exchange.

In designing a strategy to minimize waiting time, there is a trade-off in determining

how quickly feasible swaps should be executed. For example, under a greedy policy

where swaps are made as soon as they are feasible, agents spend no time waiting

to make their exchange beyond the maximum arrival time of any member in the

exchange. However, under a batching policy where every n days, a set swaps is

selected to maximize the number of agents in an exchange, potentially more agents

83

can be matched. Notice that the greedy policy is essentially equivalent to the batching

policy for very small n. We observe the trade-off as n grows: we have the opportunity

to match more agents by taking n large, but each agent that is matched must wait

some additional time (beyond the maximum arrival time of the agents in their swap)

that grows with n for their swap to be executed.

Despite this perceived trade-off, we somewhat surprisingly find that in each of

these setting (a), (b) and (c) above, a policy that conducts exchanges in a greedy

fashion is near optimal, among a large class of policies that includes batching policies.

Further, we find that for small p, allowing three-cycles can greatly improve the waiting

time over the two-cycles only setting, and the presence of altruistic donors can lead

to a further large improvement in average waiting time. Specifically, we find that a

greedy policy achieves an average waiting time of Θ(1/p2) in setting a), Θ(1/p3/2)

in setting b), and Θ(1/p) in setting c). Thus, a platform can achieve the smallest

waiting times by using a greedy policy, and by facilitating three cycles and chains, if

possible.

Our findings are consistent with and provide explanation for empirical and com-

putational observations which compare batching policies in the context of kidney

exchange programs.

Organization

This chapter is organized as follows: We describe our model formally in Section 4.2

and state the main results of the chapter in Section 4.3. In Section 4.4, we describe

simulation results which support our theoretical findings, and suggest that greedy

beats any batching in an absolute, rather than approximate, sense, for each setting

we consider. In Section 4.5 we prove our main results for cycles of length two only,

in Section 4.6 we prove our results for two and three-cycles (technically the most

challenging), and in Section 4.7 we prove our results for chains. We conclude in

Section 4.8. Finally, Section 4.9 contains the proofs of some auxiliary results.

84

Notational Conventions

Throughout, R (R+) denotes the set of reals (nonnegative reals). We write that

f(p) = O(g(p)) where p ∈ (0, 1], if there exists C < ∞ such that |f(p)| ≤ Cg(p) for

all p ∈ (0, 1]. We write that f(p) = Ω(g(p)) where p ∈ (0, 1] if there exists C < ∞

such that f(p) ≥ Cg(p) for all p ∈ (0, 1]. Finally, f(p) = Θ(g(p)) if f(p) = O(g(p))

and f(p) = Ω(g(p))). We write that f(p) = o(g(p)) where p ∈ (0, 1], if for any C > 0,

there exists p0 > 0 such that we have |f(p)| ≤ Cg(p) for all p ≤ p0.

We let Bernoulli(p), Geometric(p), and Bin(n, p), denote a Bernoulli variable with

mean p, a geometric variable with mean 1/p, and a Binomial random variable which is

the sum of n independent identically distributed (iid) Bernoulli(p) random variables

(r.v.s). We write Xd= D when the random variable X is distributed according to the

distribution D. We let ER(n, p) be a directed Erdos Reyni random graph with n nodes

where every two nodes form a directed edge with probability p, independently for all

pairs. We let ER(n,M) be the closely related directed Erdos Reyni random graph

with n nodes and M directed edges, where the set of edges is selected uniformly at

random among all subsets of exactly M directed edges. Unfortunately this notation

for the two models makes them nearly indistinguishable. The reader will need to

infer from context which model we are referring to, as is common in the literature

on random graphs [52]. We let ER(nL, nR, p) denote a bipartite directed Erdos Reyni

random graph with two sides. This graph contains nL nodes on the left, nR nodes on

the right, and a directed edge between every pair of nodes containing one node from

each side is formed independently with probability p. Given a Markov chain Xt

defined on a state space X and given a function f : X → R, for x ∈ X , we use the

shorthand

Ex[f(Xt)]∆= E[f(Xt) | X0 = x].

85

4.2 Model

We first state our model for settings with only cyclical exchanges and no chains. Later

we augment it to accommodate altruistic/bridge donors and chains.

Consider the following model of a barter exchange where each agent arrives with

an item that she wants to exchange for another item. In our simple binary model,

each agent is (equally) interested in the items possessed by some of the other agents,

and not interested in the items possessed by the rest.

Compatibility graph representation. The state of the system at any time can

be represented by a directed graph where each agent is represented by a node, and a

directed edge (i, j) exists if agent j wants the item of agent i. Let G(t) = (V(t), E(t))

denote the directed graph of compatibilities observed before time t.

Dynamics. Initially the system starts in a state with any finite number of waiting

agents. We consider discrete times t = 0, 1, 2, . . . At each time, one new agent arrives1.

The new node representing this agent v has an incoming edge from each waiting agent

who wants the item of v, and an outgoing edge to each waiting agent whose item v

wants.

Stochastic compatibility model. The item of the new agent v is of interest to

each of the waiting agents independently with probability p, and independently, the

agent v is interested in the item of each waiting agent independently with probability

p. Mathematically, there is a directed edge (in each direction) with probability p

between the arriving node v and each other node that currently exists in the system,

independently for all nodes and directions.

Allocation and policies. An allocation in a compatibility graph is a set of disjoint

exchanges, namely a set of disjoint cycles and chains. We say that a node that is

part of an allocation is matched. When an allocation consisting of cycles is executed,

the compatibility graph is updated by eliminating the matched nodes and all their

incident edges. Immediately after the arrival of a new node, the platform can choose

1One can instead consider a stochastic model of arrivals, e.g., Poisson arrivals in continuous time.In our setting, such stochasticity would leave the behavior of the model essentially unchanged, andindeed, each of our main results extend easily to the case of Poisson arrivals at rate 1.

86

to perform one or more exchanges, based on its chosen policy. Here, a policy is a

mapping from the history of the system so far to an allocation. An exchange can

happen via a cycle, where a k-way cycle is a directed cycle in the graph involving k

nodes. It can also happen via a chain, which we define below.

Three types of settings (or technologies) are considered, differing by the exchanges

permitted in an allocation. In the first two settings, allocations can output only cycles

of length at most k, for k = 2, 3. These are termed Cycle Removal. In the third

setting, called Chain Removal, allocations consist of only a single chain originating

from a bridge node. We augment our model as follows for the chain removal setting.

Altruistic/bridge donors and chains. At the first time period, there is one

altruistic donor present in the system, possibly along with other regular agents, and

no further altruistic donors arrive to the system later. An altruistic donor is willing

to give an item without getting anything in return. We represent an altruistic/bridge

donor by a special bridge node.2 Bridge nodes can have only outgoing edges. For a

new arrival v, there is an edge from a bridge node to v with probability p, independent

of everything else. A chain is a directed path that begins with a bridge node. Once

a chain is executed by the platform, the last node in a chain becomes a bridge node

who can continue the chain in a later period. (All incoming edges to the last node in

the chain are eliminated.) Notice that only one bridge donor remains in the system

in the system at all times.

One natural policy that will play a key role in our results is the greedy policy. The

greedy policy attempts to match the maximum number of nodes upon each arrival.

Definition 4.1. The greedy policy for each of the settings is defined as follows:

Cycle Removal: At the beginning of each time period the compatibility graph

does not contain cycles with length at most k. Upon arrival of a new node,

if a cycle with length at most k can be formed with the newly arrived node,

it is removed, with a uniformly random cycle being chosen if multiple cycles

are formed. Clearly, at the beginning of the next time period the compatibility

2An example for a bridge node is a non-directed donor in kidney exchange programs.

87

graph again does not contain any cycles with length at most k. The procedure

is described on figure Figure 4-2.

Chain Removal: There is one bridge node in the system at the beginning of every

time period. This bridge node does not have any incoming or outgoing edges.

Upon the arrival of a new node at the beginning of a new time interval, the

greedy policy identifies an allocation that includes the longest chain originating

from the bridge node (breaking ties uniformly at random) and removes these

nodes from the system and the last node in the chain becomes a bridge node.

Note that such a chain has a positive length if and only if the bridge node has

a directed edge from it to the new node. Observe that the new bridge node has

in-degree and out-degree zero, so the process can repeat itself. This procedure

is described on figure Figure 4-1.

Under each of the settings, the system described above operated under the greedy

policy is a Markov chain with a countably infinite number of states, each state corre-

sponding to a compatibility graph, with a bridge node for the second setting, and no

bridge nodes for the first setting. Further, this Markov chain is irreducible since an

empty graph is reachable from any other state. This raises the question of whether

this Markov chain is positive recurrent. If the answer is positive one can further study

various performance measures.3 The performance measure we focus on in this paper

is the average (steady state) waiting time, which we define to be the average steady

state time interval between the arrival of a node and the time when this node is re-

moved4. We also consider policies other than the greedy policy, in general the class

of policies under which the system is stationary/periodic and ergodic in the t → ∞

limit. This includes5 the following class of policies that generalize Markov policies:

3The Markov chain turns out to be aperiodic for chain removal and also cycle removal, exceptfor cycle removal with k = 2 where it is periodic with period 2. In any case, average (steady state)waiting time, cf. (4.1), is a natural metric for any periodicity.

4One may instead consider a cost function that is not linear in waiting time, depending on theintended application. For this first work on dynamic barter exchange, we focus on the simple metricof average waiting time. We remark that our Theorems 4.3 (on chains) and 4.2 (on three-cycles)are scaling results that also hold for any cost function that is bounded above and below by linearfunctions of waiting time. Similarly, Theorem 4.1 (on two-cycles) leads to a Θ(1/p2) scaling resultfor any cost function of this type.

5More precisely, positive recurrent periodic Markov policies (that stabilize the system) lead to a

88

Definition 4.2. We call a policy a periodic Markov policy if it employs τ homogenous

first order Markov policies in round robin for some τ ∈ N.

In other words, a periodic Markov policy implements a heterogeneous first order

Markov chain, where the transition matrices repeat cyclically every τ rounds. Now

suppose the resulting Markov chain is irreducible and periodic with period τ ′. With-

out loss of generality, assume that τ is a multiple of τ ′ (if not, redefine τ as per

τ ← ττ ′). Now, clearly the subsequence of states starting with the state at time `

and then including states at time intervals of τ , i.e., times t = `, ` + τ, ` + 2τ, . . .

forms an irreducible aperiodic first order Markov chain. If this `-th ‘outer’ Markov

chain is positive recurrent, we conclude that it converges to its unique steady state,

leading to a periodic steady state for the original system with period τ . Define

W` ≡ Expected number of nodes in the system in the steady state

of the `-th outer Markov chain.

Thus, W` is the expected number of nodes in the system at times that are ` mod τ in

steady state. Then we define the average waiting time for a periodic Markov policy

as

W = (1/τ)τ−1∑`=0

W` . (4.1)

Note that this is the average number of nodes in the original system over a long

horizon in steady state. Recalling Little’s law, this is hence identical to the average

waiting time for agents who arrive to the system in steady state.

Remark 4.1. We state our results formally for this broad class of periodic Markov

policies, though our bounds extend also to other general policies that lead to a sta-

tionary/periodic and ergodic system in the t→∞ limit.

periodic and ergodic system. In any case we are not interested in policies that do not stabilize thesystem.

89

w1 w2 w3

h

w1 w2 w3

h a1a2

w1 w3

a1

Figure 4-1: An illustration of chain matching under greedy. Initially, h is the head ofthe chain (the bridge donor), and nodes w1, w2, and w3 are waiting to be matched,shown on the left. First, node a1 arrives, and his good is acceptable by both w1 andw3 but no one has a good acceptable by a1. As h’s good is not acceptable by a1, it isnot possible to move the chain. Then node a2 arrives. His good is acceptable by w2

and he is able to accept the good from h. The longest possible chain is shown in redin the center above. The chain is formed, h, a2, and w2 are removed, and w3 becomesthe new head of the chain (bridge donor). Edges incident to the matched nodes areremoved, as well as edges going in to w3. Note that in this case, the longest chainwas not unique; w1 could have been selected instead of w3.

n1 n2 n3 n4 n1 n2 n3 n4

n5 n6

n1 n3

n5

Figure 4-2: An illustration of cycle matching under the greedy policy, with a maximumcycle length of 3. Initially, nodes n1, n2, n3, and n4 are all waiting, as shown on theleft. Node n5 arrives, but no directed cycles can be formed. Then n6 arrives, formingthe three cycle n6 → n2 → n4 → n6. On the right, the three cycle is removed, alongwith the edges incident to any node in the three cycle. Note that when n6 arrives, asix cycle is also formed, but under our assumptions, the maximum length cycle thatcan be removed is a three cycle.

90

4.3 Main Results

We consider three different settings: a) two-way cycles only, b) two-way cycles and

three-way cycles, and c) unbounded chains initiated by altruistic donors. In each

setting we look for a policy that minimizes expected waiting time in steady state.

Two-way cycles only. Our first result considers only 2-way cycles:

Theorem 4.1. Under the Cycle Removal setting with k = 2, the greedy policy (cf.

Definition 4.1) achieves an average waiting time of ln 2/p2 +o(1/p2). This is optimal,

in the sense that for every periodic Markov policy, cf. Definition 4.2, the average

waiting time is at least ln 2/(− ln(1− p2)) = ln 2/p2 + o(1/p2).

The key fact leading to this theorem is that the prior probability of having a two-

cycle between a given pair of nodes is p2, so an agent needs Θ(1/p2) options in order

to find another agent with whom a mutual swap is desirable. This result is technically

the simplest to establish, but of equal interest in its implications. We prove Theorem

4.1 in Section 4.5.

Two-way cycles and three-way cycles. Our second result considers the case of

cycle removals with k = 3. Our lower bound in this case applies to a specific class of

policies which we now define.

Let G denote the global compatibility graph that includes all nodes that ever arrive

to the system, and directed edges representing compatibilities between them.

Definition 4.3. A deterministic policy (under either Chain Removal or Cycle Re-

moval) is said to be monotone if it satisfies the following property: Consider any pair

of nodes (i, j) and an arbitrary global compatibility graph G such that the edge (i, j)

is present. Let G be the graph obtained from G when edge (i, j) is removed. Let Ti

and Tj be the times of removal of nodes i and j respectively when the compatibility

graph is G and let Tij = min(Ti, Tj). Then the policy must act in an identical fashion

on G and G for all t < Tij, i.e., the same cycles/chains are removed at the same times

in each case, up to time Tij. This property must hold for every pair of nodes (i, j)

and every possible G containing the edge (i, j).

91

A randomized policy is said to be monotone if it randomizes between deterministic

monotone policies.

Remark 4.2. Consider the greedy policy for cycle removal defined above. It is

easy to see that we can suitably couple the execution of greedy on different global

compatibility graphs such that the resulting policy is monotone. The same applies to

a batching policy which matches periodically (after arrival of x nodes), by finding a

maximum packing of node disjoint cycles and removing them6.

Note that the class of monotone policies includes a variety of policies in addition

to simple batching policies. For instance, a policy that assigns weights to nodes and

finds an allocation with maximum weight (instead of simply maximizing the number

of nodes matched) is also monotone.

Theorem 4.2. Under the Cycle Removal setting with k = 3, the average waiting time

under the greedy policy (cf. Definition 4.1) is O(1/p3/2). Furthermore, there exists

a constant C < ∞ such that, for any monotone policy that is periodic Markov (see

Definitions 4.3 and 4.2), the average waiting time is at least 1/(Cp3/2).

Theorem 4.2 says that we can achieve a much smaller waiting time with k = 3,

i.e., two and three-cycle removal, than the removal of two-cycles only (for small p).

Further, for k = 3 greedy is again near optimal in the sense that no monotone policy

can beat greedy by more than a constant factor. Theorem 4.2 is proved in Section

4.6. The proof overcomes a multitude of technical challenges arising from the complex

distribution of the compatibility graph at a given time, and introduces several new

ideas.

We remark that we could not think of any good candidate policy in our homoge-

neous model of compatibility that violates monotonicity but should do well on average

waiting time. As such, we conjecture (but were unable to prove) that our lower bound

on average waiting time applies to arbitrary and not just monotone policies.

The following fact may provide some intuition for the Θ(1/p3/2) scaling of average

6Note that such a policy is periodic Markov with a period equal to the batch size.

92

waiting time7: In a static directed Erdos-Renyi graph with (small) edge probability

p, one needs the number of nodes n to grow as Ω(1/p3/2) in order to, with high

probability, cover a fixed fraction (e.g., 50%) of the nodes with node disjoint two and

three cycles8. Our rigorous analysis leading to Theorem 4.2 shows that this coarse

calculation in fact leads to the correct scaling for average number of nodes in the

dynamic system under the greedy policy, and that no monotone policy can do better.

Our result leaves open the case of larger cycles, i.e. k > 3, under the greedy,

arbitrary monotone and arbitrary general policies. Based on intuition similar to the

above, we conjecture that under the Cycle Removal setting with general k, the greedy

policy achieves the average waiting time of Θ(p−kk−1 ), and furthermore for every policy

the average waiting time is lower bounded by Ω(p−kk−1 ).

Unbounded chains initiated by altruistic donors. Our final result concerns the

performance under the Chain Removal setting.

Theorem 4.3. Under the Chain Removal setting, the greedy policy (cf. Definition

4.1) achieves an average waiting time of O(1/p). Further, there exists a constant

C <∞ such that even if we allow removal of cycles of arbitrary length in addition to

chains, for any periodic Markov policy, cf. Definition 4.2, the average waiting time

is at least 1/(Cp).

Thus, unbounded chains initiated by altruistic donors allow for a further large

reduction in waiting time relative to the case of two-way and three-way cycles, for

small p. In fact, as stated in the theorem, removal of cycles of arbitrary length (and

chains), with any policy, cannot lead to better scaling of waiting time than that

achieved with chains alone. In particular, greedy is near optimal among all periodic

Markov policies for chain removal.9

7Recall that the average number of nodes is the same as the average waiting time, using Little’slaw.

8The expected total number of three cycles is n3p3 and the expected number of node disjointthree cycles is of the same order for n3p3 . n. We need n3p3 ∼ n in order to cover a given fractionof nodes with node disjoint three cycles, leading to n & 1/p3/2. For n ∼ 1/p3/2, the number oftwo-cycles is n2p2 ∼ 1/p = o(n), i.e., very few nodes are part of two-cycles.

9One may ask what happens in the setting where chains, two-cycles and three-cycles are allallowed. We argue in Remark 4.3 that, for small p, this setting should be very similar to the settingwith chains only.

93

Theorem 4.3 involves a challenging technical proof presented in Section 4.7.

The intuition for the Θ(1/p) scaling of waiting time is as follows: Since an agent

finds the item of another agent acceptable with probability p, it is not hard to argue

that no policy can sustain an expected waiting time that is o(1/p); see our proof of the

lower bound in Theorem 4.3 for a formalization of this intuition. On the other hand,

under a greedy policy, the chain advances each time a new arrival can accept the item

of the bridge donor, which occurs typically at Θ(1/p) intervals. One might hope that

if there are many agents waiting, then typically, the next time there is an opportunity

to advance the chain, we will be able to identify a long chain that will eliminate more

agents than the number of agents that arrived since the last advancement. Our proof

shows that this is indeed the case.

4.4 Computational Experiments

We conducted simulation experiments which measure the average waiting times for

nodes under Chain Removal and Cycle Removal with k = 2 and k = 3. For each of

these matching technologies/settings, we simulated the performance of the batching

policy with the batch size of x nodes, and compute the results for various values of

x. For each scenario, we simulated a time horizon with 3500 arriving nodes, and

measured the average number of nodes in the system after the the arrival of the

1000th node. (The first 1000 arrivals serve the role of a “warm-up” period.) 50 trials

were conducted for each scenario simulated.

Figure 4-3(a) illustrates that when p = 0.1, the greedy policy, which corresponds

to the batching policy with the batch size x = 1 performs the best among all batch

sizes x. In addition, observe the significant difference between average waiting times

corresponding to the Chain Removal setting on the one hand and the Cycle Removal

setting with k = 2 on the other hand10. Figures 4-3(b),(c) and (d) provide similar

10We see that the difference between waiting times under chain removal and cycle removal withk = 3 is less pronounced. One reason for this could be that there are long intervals betweenconsecutive times when a chain can be advanced, leading to a poor constant factor for chain removal.These intervals can be shortened by using non-maximal chains, and this may significantly improvethe constant factor.

94

results for the cases p = 0.08, 0.06 and 0.04.

(a) p=0.1 (b) p=0.08

(c) p=0.06 (d) p=0.04

Figure 4-3: Average waiting time under the Chain Removal, Cycle Removal with k = 2, and Cycle

Removal with k = 3, with batching sizes x = 1, 2, 4, 8, 16, 32, 64

4.5 Two-way Cycle Removal

In this section we consider Cycle Removal with k = 2. The greedy policy correspond-

ing to the Cycle Removal setting when k = 2 is simple to characterize, since, as we

show below, the underlying process behaves as a simple random walk. We will observe

that the random walk has a negative drift when |V(t)| ≥ log(2)/p2, and obtain a tight

characterization of waiting time under greedy using a simple coupling argument. The

key idea for the lower bound is that regardless of the implemented policy, the rate at

which 2-cycles which will be eventually removed are formed must equal to the half of

the rate at which new nodes arrive, which is equal to unity. Further, the probability

that we do not form any cycles which will be eventually removed is lower bounded

by the probability that we do not form any cycles at all. This probability depends

only on the number of nodes in the system, the desired quantity.

95

Proof of Theorem 4.1. We first compute the expected steady state waiting time under

the greedy policy. Observe that for all t ≥ 0,

|V(t+ 1)| =

|V(t)|+ 1 with probability (1− p2)|V(t)|,

|V(t)| − 1 with probability 1− (1− p2)|V(t)|.

Let ε > 0 be arbitrary. If |V(t)| > (1 + ε) ln(2)/p2, then there exists a sufficiently

small p = p(ε) such that for all p > p(ε)

P(|V(t+ 1)| = |V(t)|+ 1) = (1− p2)|V(t)| ≤ 1

21+ε.

Let q = 1/21+ε < 1/2, and let Xt be a sequence of i.i.d. random variables with

distribution

Xt =

1 with probability q,

−1 with probability 1− q.

Let S0 = 0 and for t ≥ 1, St+1 = (St +Xt)+, so St is a Birth-Death process. Letting

r = q/(1− q) < 1, in steady state P(S∞ = i) = ri(1− r) for i = 0, 1, . . . , so

E[S∞] = r/(1− r) = q/(1− 2q) =1

21+ε − 2.

We can couple the random walk |V(t)| with St such that |V(t)| < (1 + ε) ln(2)/p2 +St

for all t. This yields

E[|V(∞)|] ≤ (1 + ε)ln(2)

p2+ E[S∞] ≤ (1 + ε)

ln(2)

p2+

1

21+ε − 2.

Thus for every ε > 0, we have

limp→0

E[|V(∞)|]− ln(2)/p2

1/p2≤ ε ln(2).

As ε was arbitrary, the result follows.

96

Now we establish the lower bound on |V(∞)|. Let v be a newly arriving node

at time t, and W be the nodes currently in system that are waiting to be matched.

Let I be the indicator that at the arrival time of v (just before cycles are potentially

deleted), no 2-cycles between v and any node in W exist. Let I be the indicator that

at the arrival time of v, no two cycles that will be eventually removed that are between

v and any node in W exist (in particular, I depends on the future). Thus I ≥ I a.s.

Let Vt be the number of vertices in the system before time t such that the cycle which

eventually removes them has not yet arrived. We let V∞ be the distribution of Vt

when the system begins in steady state. By stationarity

0 = E[Vt+1 − Vt] = EV∞ [2I − 1],

giving E[I] = 1/2. Intuitively, in steady state, the expected change in the number of

vertices not yet “matched” must be zero. Thus we obtain

1

2= E[I] ≥ E[I] = E[E[I | |V(∞)]] = E

[(1− p2)|V(∞)|] ≥ (1− p2)E[V(∞)],

by Jensen’s inequality. Taking logarithms on both sides and rearranging terms, we

get

E[|V(∞)|] ≥ log(1/2)

log(1− p2)=

log(2)

− log(1− p2).

4.6 Three-way Cycle Removal

In this section we prove Theorem 4.2. The proof is far more involved than for the

case k = 2, especially the upper bound, and relies on delicate combinatorial analysis

of 3-cycles random graph formed by nodes present in the system in steady state and

those arriving over a certain time interval. We consider a time interval of the order

Θ(1/p3/2) and assume that the system starts with at least order Θ(1/p3/2) nodes in

97

the underlying graph. We establish a negative drift in the system and then, as in the

case of Chain Removal mechanism, rely on the Lyapunov function technique in order

to establish the required upper bound.

For the lower bound, we introduce a novel approach that allows us to prove a

matching lower bound (up to constants) for monotone policies by contradiction. The

rough idea is as follows: if the steady state expected waiting time is small (in this

case smaller than 1/(Cp3/2) for appropriate C), then a typical new arrival sees a

small number of nodes currently in the system, and so typically does not form a two

or three-cycle with existing nodes or even the next few arrivals. Thus, the typical

arrival typically has a long waiting time, which contradicts our initial assumption of

a small expected waiting time.

Preliminaries

We first state a number of propositions and lemmas that will enable our proof of

Theorem 4.2. Proofs of these preliminaries are deferred to Section 4.9.

We begin by stating (without proof) the following version of the classical Chernoff

bound (see, e.g. Alon and Spencer 5).

Proposition 4.1 (Chernoff bound). Let Xi ∈ 0, 1 be independent with P(Xi =

1) = pi for 1 ≤ i ≤ n. Let µ =∑n

i=1 pi.

(i) For any δ ∈ [0, 1] we have

P(|X − µ| ≥ µδ) ≤ 2 exp−δ2µ/3 (4.2)

(ii) For any R > 6µ we have

P(X ≥ R) ≤ 2−R (4.3)

Next, we state a straightforward combinatorial bound: In a directed graph, a set

M of node disjoint three-cycles is said to be maximal if no three-cycle can be added

to M so that the set remains node disjoint.

98

Proposition 4.2. Given an arbitrary directed graph G, let N be the number of three-

cycles in a largest in cardinality set of node disjoint three-cycles in G. Then, any

maximal set of node disjoint three-cycles consists of at least N/3 three-cycles.

Finally, let Gt denote the global compatibility graph that includes all nodes that

ever arrive to the system up to time t, and directed edges representing compatibilities

between them. Denote by Wt the set of nodes out of 0, 1, . . . , t still present in the

system at time t. The following is a key property of monotone policies:

Lemma 4.1. Under any monotone policy, for every two nodes i, j arriving before

time t (namely i, j ≤ t) and every subset of nodes W ⊂ 0, 1, . . . , t containing nodes

i and j

P((i, j) ∈ Gt|Wt =W) ≤ p.

In words, pairs of nodes still present in the system at time t are no more likely to be

connected at time t than at the time they arrive.

The following corollary follows immediately by linearity of expectations.

Corollary 4.1. Let Wt = |Wt| and let Et be the number of edges between nodes in

Wt. Then, under a monotone policy, E[Et|Wt] ≤ Wt(Wt − 1)p.

Proposition 4.2 and Lemma 4.1 are proved in Section 4.9.

Proof of Theorem 4.2

Proof of Theorem 4.2: the performance of the greedy policy. Suppose at time zero we

observe W ≥ C3/p3/2 nodes in the system with an arbitrary set of edges between

them. Here C is a sufficiently large constant to be fixed later. Call this set of nodes

W . Consider the next T = 1/(Cp3/2) arrivals, and call this set of nodes A. Wlg,

label the times of these arrivals as 1, 2, . . . , T , and use the label t for the node that

arrives at time t. Let At ⊆ 1, 2, . . . , t − 1 be the subset of nodes in A that have

arrived but have not been removed before time t. Similarly define Wt to be the set

99

of nodes from W which are still in the system immediately before time t. Note that,

in particular, W1 =W .

Let N be the number of three cycles removed during the time period [0, T ], that

include two nodes from W . Let κ = 1/C2 and consider the event

E1 ≡ |AT+1| −N ≥ 2κ/p3/2 (4.4)

Introduce the event

E2 ≡ There exists a set of disjoint 2 and 3 cycles in A

with cardinality at least 3/(C3p3/2) . (4.5)

First suppose that the event E1 does not occur. Then

|AT+1| ≤2

C2p3/2+N ≤ 1

16Cp3/2+N , (4.6)

for C sufficiently large. Also event E2 implies that (again for C sufficiently large) at

most 9/(C3p3/2) ≤ 1/(16Cp3/2) nodes in A leave due to internal three-cycles or two

cycles. Since T = 1/(Cp32 ), then applying Eq. (4.6), at least 7/(8Cp3/2) − N other

nodes in A also leave before T + 1. These other nodes belong to cycles of one of the

following types:

(i) A three cycle containing another node from A and a node from W .

(ii) A two cycle with a node from W .

(iii) A three cycle containing two nodes from W . There are exactly N nodes of this

type.

Exactly N nodes in A are removed due to cycles of type (iii) above, so we infer that

at least 7/(8Cp3/2) − 2N nodes in A are removed due to cycles of type (i) or (ii)

above, meaning that at least (1/2)(7/(8Cp3/2)−2N) nodes inW are removed as part

of such cycles. Clearly, 2N nodes in W are removed as part of cycles of type (iii).

100

It follows that

|WT+1| ≤ |W| − 2N − 1

2

(7

8Cp3/2− 2N

)≤ |W| − 7

16Cp3/2−N . (4.7)

Combining Eqs. (4.6) and (4.7), we deduce that

|WT+1|+ |AT+1| ≤ |W| −3

8Cp3/2. (4.8)

We also have that the number of nodes in the system increases by at most T =

1/(Cp3/2). We will show that P(E1∪E2) ≤ ε = 1/9. Before establishing this claim we

show how this claim implies the result. We have

E[|WT+1|+ |AT+1| − |W|] ≤ εT − (1− ε) 3

8Cp3/2= − 2

9Cp3/2, (4.9)

i.e., the number of nodes decrease by at least 2/(9Cp3/2) in expectation. We now

apply Proposition A.4 to the embedded Markov chain observed at times which are

multiples of T . Namely, let Ti = i · T , and take Xi = G(Ti) = (V(Ti), E(Ti)) and

define V (Xi) = |V(Ti)|. If we let Di be the set of nodes that are deleted in some cycle

during the time interval [Ti, Ti+1), we obtain a decomposition

V (Xi+1) = |V(Ti+1)| = |V(Ti)|+ T − |Di| = V (Xi) + T − |Di|. (4.10)

Since T > 0 is deterministic it is trivially independent from G(Ti). Thus the assump-

tions on decomposing V from (A.8) are satisfied. The assumption that G | V (G) < n

is finite for every n is satisfied as there are only finitely many graphs with n nodes.

We take α = C3/p3/2 making B = G | |V(G)| ≤ C3/p3/2. We can take C1 = 1 as T

is deterministic. We can take C2 = 3, as trivially |Di| ≤ 3T since each newly arriving

node can be in at most one three cycle (thus making Dk = Dk in Proposition A.4).

Finally, we can take λ = 2/9, as by (4.9),

E[T − |Di|] ≤ −2

9Cp3/2= −2

9E[T ].

101

Thus by applying Proposition A.4, we obtain that

E[|V(T∞)|] ≤ max

α,

max1, C2 − 12C1

λE[Ak]

(2 +

2

λ

)= max

C3

p3/2,

4

2/9

1

Cp3/2

(2 +

2

2/9

)=

11C3

p3/2,

for C sufficiently large. Finally, since the embedded chain is observed over determin-

istic time intervals, the bound above applies to the steady-state bound. We conclude

E[|V(∞)|] ≤ 11C3

p3/2.

It remains to bound P(E1) and P(E2) to complete the proof. We do this below.

We claim that P(E2) ≤ ε/4. We first show that there is likely to be a maximal set of

node disjoint three-cycles in A of size less than 2/(3C3p3/2). This will imply, using

Proposition 4.2, that the maximum number of node disjoint three-cycles in A is at

most 2/(C3p3/2). Reveal the graph on A and simultaneously construct a maximal set

of node disjoint three-cycles as follows: Reveal node 1. Then reveal node 2. Then

reveal node 3 and whether it forms a three-cycle with the existing nodes. If it does

remove this three-cycle. Continuing, at any stage t if a three-cycle is formed, choose

uniformly at random such a three-cycle and remove it.

Since this process corresponds to a monotone policy (cf. Definition 4.3), then

using Corollary 4.1, the residual graph immediately before step t contains no more

than 2(t−1

2

)p edges in expectation, as the number of nodes is no more than t− 1. It

follows that the conditional probability of three-cycle formation at step t is no more

than E[Number of three-cycles formed] = 2(t−1

2

)p3. It follows that we can set up a

coupling such that the total number of three-cycles removed (this is a maximal set of

edge disjoint three-cycles resulting from our particular greedy policy) is no more than

Z =∑T

t=1Xt where Xt ∼ Bernoulli(2(t−1

2

)p3) are independent. Now E[Z] = 2

(T3

)p3 ≤

1/(3C3p3/2). Using Proposition 4.1 (i), we obtain that P(Z ≥ 2/(3C3p3/2)) < ε/8, for

large enough p, establishing the desired bound on the number of node disjoint three

102

cycles. We have shown that the probability of having more than 2/(C3p3/2) node

disjoint three cycles in A is less than ε/8.

Let Z ′ be the number of two cycles internal to A. Then Z ′ ∼ Bin((T2

), p2).

Hence, E[Z ′] ≤ 1/(C2p) and P(Z ′ ≥ 1/(C3p3/2)) ≤ ε/8 for sufficiently small p using

Proposition 4.1 (ii). It follows that the probability of having more than 1/(C3p3/2)

node disjoint two cycles in A is less than ε/8. Now P(E2) ≤ ε/4 follows by union

bound.

We now show P(E1) ≤ 3ε/4. To prove this, we find it convenient to define two

additional events. Denote by N (S1,S2) the (directed) neighborhood of the nodes in

S1 in the set of nodes S2, i.e., N (S1,S2) = j ∈ S2 : ∃i ∈ S1 s.t. (i, j) ∈ E. Abusing

notation, we use N (i,S) to denote the neighborhood of node i in S. Further, we find

it convenient to define Bt = N (t,At). Define

E3,t ≡ |At| ≥ κ/p3/2, and |Bt| < κ/(2p1/2), (4.11)

and E3 = ∪0≤t≤TE3,t. Define

E4,t ≡ |At| ≥ κ/p3/2, and |N (Bt,Wt)| < C3κ/(8p), (4.12)

and let E4 = ∪0≤t≤TE4,t. We make use of

E1 ⊆ (Ec4 ∩ E1) ∪ E4 ⊆ (Ec4 ∩ E1) ∪ E3 ∪ (E4 ∩ Ec3)

⇒ P(E1) ≤ P(Ec4 ∩ E1) + P(E3) + P(E4 ∩ Ec3)

Reveal the edges between t and At when node t arrives. The existence of each edge

is independent of the other edges and the current revealed graph. Thus we can bound

the probability of the event E3,t using Proposition 4.1 (i) by 2 exp(−1/(12C2p1/2)) for

large enough C. It follows that for sufficiently small p, we have

P(E3) ≤ 2T exp(−1/(12C2p1/2)) ≤ ε/4 . (4.13)

103

We now bound P(Ec4 ∩ E1). Let Nt be the number of three cycles removed before

time t of type (iii) (recall that type (iii) three cycles include two nodes from W).

Define Zt ≡ |At| −Nt. Define

E5,t ≡ Node t is part of a three-cycle of type (i) .

Note that

If E5,t then |At+1| = |At| − 1, Nt+1 = Nt if such a three cycle is removed and

|At+1| = |At|, Nt+1 = Nt + 1 if a three cycle of type (iii) is removed instead. In

either case, we have Zt+1 = Zt − 1.

With probability one we have |At+1| ≤ |At|+ 1 and Nt+1 ≥ Nt. It follows that

Zt+1 ≤ Zt + 1.

Now suppose Zt ≥ κ/p3/2 and Ec4,t. Clearly Zt ≥ κ/p3/2 ⇒ |At| ≥ κ/p3/2 and

hence Ec4,t ⇒ |N (Bt,Wt)| ≥ C3κ/(8p) = C/(8p). Revealing the edges between from

N (Bt,Wt) to t, we see that

P(E5,t|Zt ≥ κ/p3/2, Ec4,t) ≥ 1− (1− p)C/(4p) ≥ 3/4 , (4.14)

for large enough C and small enough p, independent of everything so far. So, infor-

mally, if Ec4,t then Zt is a bounded above by a random walk with a downward drift

whenever Zt ≥ κ/p3/2. We now formalize this.

Define the random walk (Zt)t≥1 as follows: Let Z1 = 0. Whenever Zt = 0, we

have Zt+1 = 1, else

Zt+1 =

Zt + 1 w.p. 1/4

Zt − 1 w.p. 3/4(4.15)

So (Zt)T+1t=1 is a downward biased random walk reflected upwards at 0.

Proposition 4.3. There exists C <∞ such that for any T ∈ N and ν > 0, we have

P(ZT+1 ≥ ν) ≤ CT exp(−ν/C).

The proof is omitted, as this is a standard result for random walks with a negative

104

drift. Using Proposition 4.3, we have that for sufficiently small p,

P(ZT+1 ≥ κ/(2p3/2)) ≤ ε/4 .

Let τ be the first time at which event E4,t occurs for t ≤ T , and let τ = T + 1 if

E4 does not occur. We now show that the following claim holds:

Claim 4.1. We can couple Zt and Zt such that for all t < τ , whenever Zt ≥ κ/p3/2

we have Zt+1 − Zt ≥ Zt+1 − Zt.

Proof of Claim. If E5,t occurs, then (see above) we know that Zt+1 = Zt − 1 and

Zt+1 − Zt ≥ −1 holds by definition of Z. Hence, it is sufficient to ensure that

Zt+1 = Zt+1 whenever Ec5,t occurs. But this is easy to satisfy since Eq. (4.14) implies

that

P(Ec5,t|Zt ≥ κ/p3/2, Ec4,t) ≤ 1/4 ,

whereas P(Zt+1 = Zt + 1) = 1/4. This completes the proof of the claim.

The following claim is an immediate consequence:

Claim 4.2. We have Zt ≤ Zt + dκ/p3/2e for all t ≤ τ .

Proof of Claim. The claim follows from Claim 4.1 and a simple induction argument.

It follows that

P(ZT+1 ≥ 2κ/p3/2, τ = T + 1) ≤ P(ZT+1 ≥ κ/p3/2, τ = T + 1)

≤ P(ZT+1 ≥ κ/p3/2)

≤ ε/4 .

Thus we obtain

P(Ec4 ∩ E1) ≤ ε/4 . (4.16)

105

Finally, we bound P(E4 ∩Ec3). For any S ⊆ A, let W∼S ⊆ W be the set of waiting

nodes that would have been removed before T + 1 if (hypothetically) the nodes in

S had no incident edges in either direction, but we left all other compatibilities

unchanged. Define the event E6 as follows: for all S ⊆ A such that |S| = κ/(2p1/2),

the bound

|N (S,W\W∼S)| ≥ C/(8p) (4.17)

holds.

Claim 4.3. The event E6 occurs whp.

Before proving the claim, we show that it implies P(E4 ∩ Ec3) ≤ ε/4. Suppose that

E6 and Ec3 occur. Consider any t such that |At| ≥ κ/p3/2. Since Ec3 , we have that

|Bt| ≥ κ/(2p1/2). Take any S ⊆ Bt such that |S| = κ/(2p1/2). Notice that for our

monotone greedy policy, cf. Remark 4.2, the set of waiting nodes that are removed

before time t must be a subset of W∼S , i.e., we have we have Wt ⊇ W\W∼S . Since

E6 occurs, it follows that |N (S,Wt)| ≥ C/(8p) ⇒ |N (Bt,Wt)| ≥ C/(8p). Thus we

have Ec4 . This argument just established that

E6 ∩ Ec3 ⊆ Ec4 ∩ Ec3

⇒ Ec6 ∩ Ec3 ⊇ E4 ∩ Ec3 .

It follows that P(E4 ∩ Ec3) ≤ P(Ec6 ∩ Ec3) ≤ P(Ec6) ≤ ε/4 using Claim 4.3, as required.

Proof of Claim 4.3. Consider any S ⊆ A such that |S| = κ/(2p1/2). Clearly, since

each node in A can eliminate at most 2 nodes in W , we have |W∼S | ≤ 2|A\S| ≤

2|A| = 2/(Cp3/2). It follows that |W\W∼S | ≥ C3/p3/2 − 2/(Cp3/2) ≥ C3/(2p3/2) for

large enough C. Now notice that by definition W∼S is a function only of the edges

between nodes inW∪(A\S), and is independent of the edges coming out of S. Thus,

for each node i ∈ W\W∼S independently, we have that each node in S has an edge to i

independently w.p. p. We deduce i ∈ N (S,W\W∼S) w.p. 1−(1−p)κ/(2p1/2) ≥ κp1/2/3

106

for small enough p, i.i.d. for each i ∈ W\W∼S . It follows from Proposition 4.1 (i)

that

|N (S,W\W∼S)| < C3

2p3/2· κp

1/2

3· 3

4=

κC3

8p=

C

8p

occurs w.p. at most 2 exp− (1/4)2 · C/(6p) · (1/3)

≤ exp(−C/(300p)) for small

enough p. Now, the number of candidate subsets S is(1/(Cp3/2)

κ/p1/2

)≤ (1/(Cp3/2))κ/p

1/2 ≤

exp(1/pε+1/2) for small enough p. It follows from union bound that |N (S,W\W∼S)| <C8p

for one (or more) of these subsets S with probability at most exp(−C/(300p)) ·

exp(1/pε+1/2) ≤ exp(−C/(400p))p→0−−→ 0. Thus, whp, |N (S,W\W∼S)| < C

8poccurs

for no candidate subset S, i.e., event E6 occurs whp.

Proof of Theorem 4.2: lower bound for monotone policies. Denote bym the expected

steady state number of nodes in the system, which by Little’s Law equals the expected

steady state waiting time. Suppose m ≤ 1/(Cp3/2), where C is any constant larger

than 36. Fix a node i, and reveal the number of nodes W in the system when i

arrives. Notice W ≤ 3m occurs with probability at least 1 − 1/3 = 2/3 in steady

state by Markov’s inequality. Assume that W ≤ 3m holds. Let W denote the nodes

waiting in the system when i arrives, and let A be the nodes that arrive in the next

3m time slots after node i arrives. Now, if node i leaves the system within 3m time

slots of arriving, then i must form a two or three cycle with nodes in A ∪W . The

probability of forming such a cycle is bounded above by

E[Number of two cycles between i and A ∪W|W ] (4.18)

+ E[Number of three cycles containing i and two nodes from A ∪W|W ] . (4.19)

Clearly,

E[Number of two cycles between i and A ∪W|W ] ≤ 6mp2 ≤ 1/C (4.20)

107

for p sufficiently small. To bound the other term we notice that

E[Number of three cycles containing i and two nodes from A ∪W|W ]

= p2 · E[Number of edges between nodes in A ∪W|W ] (4.21)

We use Corollary 4.1 to bound the expected number of edges between nodes in W at

the time when i arrives by W (W − 1)p and notice that other compatibilities (j1, j2)

for j1, j2 6⊆ W are present independently with probability p. Hence, we have

E[Number of edges between nodes in A ∪W|W ] ≤ |W∪A|(|W∪A|−1)p ≤ 6m(6m−1)p .

Using Eq. (4.21) we infer that

E[Number of three cycles containing i and two nodes from A ∪W|W ]

≤ 6m(6m− 1)p3 ≤ 36/C2 ≤ 1/C (4.22)

for C > 36. Using Eqs. (4.20) and (4.22) in (4.19), we deduce that the probability of

node i being removed within 3m slots is no more than 2/C.

Combining, the unconditional probability that node i stays in the system for

more than 3m slots is at least (2/3)(1−2/C) > 1/3 for large enough C. This violates

Markov inequality, implying that our assumption, m ≤ 1/(Cp3/2), was false. This

establishes the stated lower bound.

The following conjecture results if we assume that nt concentrates, and that typical

number of edges in a compatibility graph at time t with nt nodes is close to what it

would have been under an ER(nt, p) graph.

Conjecture 4.1. For cycle removal with k = 3, the expected waiting time in steady

state under a greedy policy scales as√

ln(3/2)/p3/2+o(1/p3/2), and no periodic Markov

policy (including non-monotone policies) can achieve an expected waiting time that

scales better than this.

Here the constant√

ln(3/2) results from requiring (under our assumptions) that

108

a newly arrived node forms a triangle with probability 1/3. (Thus the “drift” is zero,

as with probability 1/3, we form a triangle which gives a net loss of two nodes, and

with probability 2/3, we form no triangle and gain a node.)

Our simulation results, cf. Figure 4-3, are consistent with this conjecture: the

predicted expected waiting time for greedy from the leading term√

ln(3/2)/p3/2 is

W = 80 for p = 0.04, W = 43 for p = 0.06, W = 28 for p = 0.08 and W = 20.1

for p = 0.1. If proved, this conjecture would be refinement of Theorem 4.2. A proof

would require a significantly more refined analysis for both the upper bound and the

lower bound.

4.7 Chain Removal

In this section we prove Theorem 4.3. At any time there is one bridge donor in the

system. Under a greedy policy, the chain advances when the newly arrived agent can

accept the item of the bridge donor. The basic idea to show that greedy achieves

O(1/p) waiting time is to show that when there are more than C/p waiting nodes

just after we move the chain forward, then, on average, the next time the chain moves

forward, it will remove more nodes than were added in the interim. This “negative

drift” in number of nodes is crucial in establishing the bound (following which we

again use Proposition A.4 to infer a bound on the expected waiting time). The lower

bound proof is based on the idea that the waiting time for a node must be at least

the time for the node to get an in-degree of one. The lower bound is again proved by

contradiction.

Preliminaries

We will need a simple result on the tails of geometric random variables below.

Lemma 4.2. There exist p0 and κ0 such that for all p < p0 and all κ > κ0, if

109

X ∼ Geometric(p), then

E[XIX< 1

p√κ

or X>κp

]≤ 2

κp.

The proof is in Section 4.9. Additionally, we need Corollary B.2 from Appendix

B showing that there will be a long path in a bipartite directed Erdos-Renyi random

graph with high probability. We show this using a result of [3] (see [55] for a more

recent reference).

Proof of Theorem 4.3

We introduce the following notation. Let G(t) = (V(t), E(t), h(t)) be the directed

graph at time t describing the compatibility graph at time t. Here h(t) is a special

node not included in V(t) that is the head of the chain, which can only have out-going

edges. We denote by G(∞) = (V(∞), E(∞), h(∞)) the steady-state version of this

graph (which exists as we show below).

According to the greedy policy, whenever h(t) forms a directed edge to a newly

arriving node, a largest possible chain starting from h(t) is made. Thus before the

new node arrives, h(t + 1) will always have an in degree and out degree of zero (as

explained in Section 4.2), and we can only advance the chain when a newly arriving

node has an in edge from h(t). We refer to these periods between chain advancements

as intervals. Let τi for i = 1, 2, . . . , denote the length of the ith interval, so that

τi ∼ Geometric(p). Let T0 = 0 and Ti =∑i

j=1 τj for i = 1, 2, . . . , be the time at the

end of the ith interval. Additionally, let Ai be the set of nodes that arrived during

the ith interval [Ti−1, Ti], so that |Ai| = τi, and let Wi be the set of nodes that were

“waiting” at the start of the i-th interval, namely at time Ti−1. Thus, right before

the chain is advanced, every node in the graph is either in Wi, Ai or it is h(t) itself.

The intuition for the upper bound on waiting time for the greedy policy in The-

orem 4.3 is as follows. We will use the Lyapunov function argument (Proposition

A.4) to argue that for some C, if there are at least C/p nodes in the graph at the

start of an interval [Ti, Ti + τi+1], then the number of vertices deleted in an interval

110

is on average greater than the number of vertices that arrive in that interval, i.e.

we have a negative drift on the number of vertices. We lower bound the number of

nodes removed in the ith interval by the length of a longest path in the bipartite

graph formed by putting nodes in Ai (the newly arrived nodes) to the left part of the

graph, and putting nodes Wi (the nodes from the previous interval) to the right part

of the graph, and maintaining only edges between these two parts (thus in particular

preserving the bipartite structure). We bound the expected size of a longest path on

this subgraph using Corollary B.2. Observe, that the length of a longest path on our

bipartite graph is at most 2|Ai|. This will enable us to truncate the downward jumps

when applying Proposition A.4.

Proof of Theorem 4.3: performance of the greedy policy. We apply Proposition A.4,

taking as our Markov chain Xi = G(Ti), and our Lyapunov function V (·) to be

V (G(Ti))∆= |V(Ti)|. For a constant C > 0 to be specified later, we let α from Propo-

sition A.4 be α = C/p. Thus our finite set of exceptions is B = G = (V , E , h) : |V| ≤

C/p, the directed graphs with at most C/p nodes. Obviously our state space is

countable and B is finite. Let Pi be the path of nodes that are removed from the

graph in the ith interval. Thus

|V(Ti)| = |V(Ti−1)|+ |Ai| − |Pi|.

By taking Ai = |Ai| = τi and Di = |Pi|, we have that V (·) satisfies the form of

(A.8) and the independence assumptions on Ai and Di. As τi ∼ Geometric(p),

E[|Ai|2] ≤ 2/p2 = 2E[|Ai|]2, so we can take C1 = 2. We set C2 = 2, and so to apply

Proposition A.4, we must find λ > 0 such that for every graph G 6∈ B,

EG [|Ai| −min|Pi|, 2|Ai|] ≤ −λE[|Ai|], (4.23)

where EG[·] denote the expectation conditioned on the event Gi−1 = G. We create

an auxiliary bipartite graph G ′i = (Ai,Wi, E ′i), where Ai are the nodes on the left,

Wi are the nodes on the right, and E ′i is the subset of E(Ti) consisting of edges (u, v)

111

such that either u ∈ Ai, v ∈ Wi or vice versa (thus ensuring G ′i is bipartite). We let

v′i ∈ Ai be the node newly arrived at Ti that h(Ti − 1) connected to. Finally, we let

P ′i be the longest path in G ′i starting at v′i. Trivially, |P ′i| ≤ |Pi|. Observe that G ′i is a

ER(|Ai|, |Wi|, p) bipartite random graph. Thus we apply Corollary B.2 to show |P ′i|

is appropriately large with high probability. In particular, given arbitrary ε > 0 and

κ > κ0 > 1, where κ0 is to be specified later, find C and p0 according to Corollary B.2.

Then G(Ti−1) 6∈ B implies |Wi| = |V(Ti−1)| ≥ C/p. Then if p < p0 and a ∈[

1p√κ, κp

],

then by Corollary B.2,

P(|P ′i| < 2|Ai|(1− ε)

∣∣∣Ai| = a)≤ ε. (4.24)

We define the events Ei and Fi by

Ei =

Ai 6∈

[1

p√κ,κ

p

]Fi = |P ′i| < 2|Ai|(1− ε) .

We define Zi∆= 2|Ai|(1 − ε)IEci∩F ci . Thus Zi ≤ |P ′i| ≤ |Pi| from the definition of the

event Fi, and Zi ≤ 2|Ai| by construction. We now use this to get an upper bound

(4.23) as follows. First, we have:

EG [|Ai| −min|Pi|, 2|Ai|] ≤ EG [|Ai| − Zi]

= E [|Ai|IEi ] + E[|Ai| − Zi

∣∣∣Eci

]P(Ec

i ), (4.25)

where in (4.25), we used that Zi is zero on Ei. Now noting that for all a ∈[

1p√κ, κp

],

i.e. in the event Eci , we have

P(Zi = 0

∣∣∣ |Ai| = a)

= P(Fi

∣∣∣ |Ai| = a)≤ ε,

by (4.24), and therefore

P(Zi = 2(1− ε)|Ai|

∣∣∣ |Ai| = a)

= P(F ci

∣∣∣ |Ai| = a)≥ 1− ε.

112

as well. We now compute that

E[|Ai| − Zi

∣∣∣Eci

]=

∑a∈

[1

p√κ,κp

]E[|Ai| − Zi

∣∣∣ |Ai| = a]P(|Ai| = a

∣∣∣Eci

)

=∑

a∈[

1p√κ,κp

]E[|Ai| − Zi

∣∣∣Fi ∩ |Ai| = a]P(Fi

∣∣∣ |Ai| = a)P(|Ai| = a

∣∣∣Eci

)

+∑

a∈[

1p√κ,κp

]E[|Ai| − Zi

∣∣∣F ci ∩ |Ai| = a

]P(F ci

∣∣∣ |Ai| = a)P(|Ai| = a

∣∣∣Eci

)

≤∑

a∈[

1p√κ,κp

]E[|Ai|

∣∣∣Fi ∩ |Ai| = a]· ε · P

(|Ai| = a

∣∣∣Eci

)

+∑

a∈[

1p√κ,κp

]E[|Ai| − 2(1− ε)|Ai|

∣∣∣F ci ∩ |Ai| = a

]· (1− ε) · P

(|Ai| = a

∣∣∣Eci

)

≤ εE[|Ai|

∣∣∣Eci

]+ (1− ε)E

[(−1 + 2ε)|Ai|

∣∣∣Eci

]= (−1 + 4ε− 2ε2)E

[|Ai|

∣∣∣Eci

]≤ (−1 + 4ε)E

[|Ai|

∣∣∣Eci

].

Now combining this with (4.25), we have

E [|Ai|IEi ] + E[|Ai| − Zi

∣∣∣Eci

]P(Ec

i ) ≤ E [|Ai|IEi ] + (−1 + 4ε)E[|Ai|

∣∣∣Eci

]P(Ec

i )

= E [|Ai|IEi ] + (−1 + 4ε)E[|Ai|IEci

]P(Ec

i )

≤ 2

κp+ (−1 + 4ε)

(1

p− 2

κp

)(4.26)

≤ −1

p+

4ε

p+

4

κp,

= −1

p

(1− 4ε− 4

κ

),

where in (4.26) we used Lemma 4.2 twice. Now, we let δ∆= 4ε + 4/κ, and observe

that we can make δ arbitrarily and the inequality will still hold for sufficiently small

113

p by our choice of ε and κ0. As we have

EG [|Ai| −min|Pi|, 2|Ai|] ≤ −1

p(1− δ) = −E[|Ai|](1− δ),

we can apply Proposition A.4 with λ = (1− δ) to obtain that

E[|V(T∞)|] ≤ max

α,

max1, C2 − 12C1

λE[Ak]

(2 +

2

λ

)= max

C

p,

2

1− δ1

p

(2 +

2

1− δ

)

Finally, recall that we are working with the “embedded Markov chain” as we are

only observing the process at times Ti. We can relate the actual Markov chain to the

embedded Markov chain as follows:

E[|V(∞)|] = limt→∞

1

t

t∑s=0

|V(s)| (4.27)

= limn→∞

1

Tn

Tn∑s=0

|V(s)| (4.28)

= limn→∞

n

Tnlimn→∞

1

n

n∑i=1

Ti∑s=Ti−1

|V(s)|

= p limn→∞

1

n

n∑i=1

(|V(Ti−1)|(Ti − Ti−1) +

(Ti − Ti−1)(Ti − Ti−1 + 1)

2

)(4.29)

≤ p

(limn→∞

1

n

n∑i=1

|V(Ti−1)|(Ti − Ti−1) +1

n

n∑i=1

(Ti − Ti−1)2

). (4.30)

Here (4.27) follows from the positive recurrence of G(t). We have (4.28) as an → a

implies that for every subsequence ani , we have ani → a as well, and using that as

Tn → ∞ a.s. We obtain the left term in (4.29) by observing that Tn is the sum of

n independent Geometric(p) random variables and then applying the SLLN. For the

right term of (4.29), we simply use that |V(s+ 1)| = |V(s)|+ 1 for s ∈ [Ti−1, Ti − 1],

and then the identity∑n

i=1 i = n(n+ 1)/2.

We now considering each sum from (4.30) independently. For the first sum, observ-

114

ing that |V(Ti−1)|(Ti−Ti−1) is a function of our positive recurrent Markov chain G(Ti),

we have that there exists a random variable X∗ such that |V(Ti−1)|(Ti − Ti−1)⇒ X∗

and the average value of |V(Ti−1)|(Ti−Ti−1) converges to E[X∗] a.s. The convergence

in distribution |V(Ti−1)|(Ti−Ti−1)⇒ X∗ implies the existence of |V(Ti−1)|(Ti− Ti−1)

that converges to X∗ a.s. Putting these together, we have

limn→∞

1

n

n∑i=1

|V(Ti−1)|(Ti − Ti−1) = E[X∗] (4.31)

= E[

limi→∞|V(Ti−1)|(Ti − Ti−1)

]≤ lim inf

i→∞E [|V(Ti−1)|(Ti − Ti−1)] (4.32)

= lim infi→∞

E [|V(Ti−1)|]E[Ti − Ti−1] (4.33)

=1

pE[|V(T∞)|]. (4.34)

Here we have (4.31) by the ergodic theorem for Markov chains, (4.32) by Fatou’s

lemma, (4.33) by the independence of Ti−Ti−1 from V(Ti−1), and (4.34) by Theorem

2 from [80] (alternatively, (4.34) can be shown a little extra work using a simpler

result from [47]).

For the second sum, as Ti − Ti−1 = τi are i.i.d. Geometric(p), by the SLLN,

limn→∞

1

n

n∑i=1

(Ti − Ti−1)2 = E[τ 21 ] =

2− pp2≤ 2

p2

Thus

E[|V(∞)|] ≤ p

(1

pE[|V(T∞)|] +

2

p2

)= E[|V(T∞)|] +

2

p,

showing the result, as we have for the embedded process that E[|V(T∞)|] = Ω(1/p).

Finally, we mention that in moving from the “embedded Markov chain” back to

the original Markov chain we make use of the fact that τi is light tailed in the sense

that E[τ 2i ] = O((E[τi])

2), to obtain a bound of O(1/p) of the steady state expected

115

number of nodes in the system.

The proof for the lower bound in Theorem 4.3 is based on the following key idea:

the waiting time for a node must be at least the time for the node to get an in-degree

of one. Using this, if the steady state average waiting time is w = o(1/p), then by

Little’s Law when a typical vertex v arrives there are only o(1/p) vertices in system,

so v is likely not to have any in edges connecting with any of these existing nodes.

After w steps, the number of newly arrived nodes is w = o(1/p), so v is likely not to

connect to any of these nodes either. Then the idea is to show that v will be in the

system for greater than w steps with high enough probability (i.e. with probability

at least 1/3), contradicting that the expected waiting time v is w.

Proof of Theorem 4.3: lower bound. Let C = 24. We will show that the expected

steady state waiting time w is at least 1/(Cp) for all p, giving the result. Assume for

contradiction that there exists p such that w ≤ 1/(Cp). By Little’s law we have that

w = E[|V(∞)|] ≤ 1/(Cp) as well. Let i be a node entering at steady state, and let Wi

be the waiting time of node i. LetW be the set of nodes in the system when i arrives,

so W d= |V(∞)|, and define the event E1 = |W| ≤ 3w. By Markov’s inequality,

P(E1) ≥ 2/3. Note that i cannot leave the system until it has an in degree of at least

one. Let A be the first 3w arrivals after i, and let the event E2 be the event that

either a node from W or a node from A has an edge pointing to i. We have

P(E2) = P(Bin(|W|+ 3w, p) ≥ 1),

making

P(E2 | E1) ≤ P(Bin(6w, p) ≥ 1) ≤ P(Bin(6/(Cp), p) ≥ 1) ≤ 6

C=

1

4

using the definition of E1, then that w ≤ 1/(Cp), then Markov’s inequality, and

finally that C = 24. Thus

w = E[Wi] ≥ 3wP(Ec2) ≥ 3wP(Ec

2|E1)P(E1) ≥ 3w(1− 1/4)(2/3) = 3w/2 > w

116

providing the contradiction.

Remark 4.3. Consider, instead, a setting where two and three-cycles can be removed

in addition to chains. Theorem 4.3 tells us that under any policy, we still have a lower

bound of 1/(Cp) on the expected waiting time (in fact, this holds for arbitrarily long

cycles). It also tells us that under a greedy policy that executes only chains (ignoring

opportunities to conduct two and three-cycles), the expected waiting time is O(1/p).

Further, it is not hard to see that this policy misses two and three cycle opportunities

for O(p) fraction of nodes. As such, we conjecture that a greedy policy that executes

two and three-cycles in addition to chains will have almost identical performance to

greedy with chains only, and in particular, the expected waiting time will still be

O(1/p).

4.8 Conclusion

Overcoming the rare coincidence of wants is a major obstacle in organizing a suc-

cessful barter marketplace. In this chapter, we studied how the policy adopted by

the clearinghouse affect agents’ waiting times in a thin marketplace. We investigated

this question for a variety of settings determined by the feasible types of exchanges,

which are largely driven by the technology adopted by the marketplace. We also

studied how the feasible types of exchanges affect the waiting times. Our study of

such marketplaces is motivated in part by questions arising in the design of kidney

exchange programs.

We studied these questions in a dynamic model with a stylized homogenous

stochastic demand structure. The market is represented by a compatibility graph:

agents are represented by nodes, and each directed edge, which represents that the

source agent has an item that is acceptable to the target agent, exists a priori with

probability p. Exchanges take place in the form of cycles and chains, where chains are

initiated by an altruistic donor who is willing to give away his item without asking

anything in return. The key technical challenge we face is that in our dynamic setting,

the compatibility graph between agents present at a particular time has a complicated

117

distribution that depends on the feasible exchanges and the policy employed by the

clearinghouse.

We analyzed the long run average time agents spend waiting to exchange their

item, for small p, in a variety of settings depending on the feasible exchanges, chains,

2-way cycles, 2 and 3-way cycles, or chains. Our main finding is that regardless of

the setting, the greedy policy which attempts to match upon each arrival, is approxi-

mately optimal (minimizes average waiting time) among a large class of policies that

includes batching policies. Under the greedy policy, with cycles of length two, the

steady state average waiting time is of order Θ(1/p2), while allowing both length two

and three cycles leads to a steady state average waiting time of Θ(1/p3/2). Finally,

exchanges based on chains lead to a steady state average waiting time of Θ(1/p).

Thus, three-way cycles and chains lead to large improvement in waiting times rela-

tive to two-cycles only. Simulations in each setting support these findings, showing

that greedy beats batching for small p and also for moderate values of p.

We do not model competition between clearinghouses. In the presence of such

competition, where the same agents may participate in multiple clearinghouses, there

is an incentive for clearinghouses to complete exchanges at the earliest to avoid agents

completing an exchange in a different clearinghouse. One may worry that such an

incentive may lead clearinghouses to hurt social welfare when they adopt a greedy-like

policy. However, our work suggests that this is not the case, since greedy may be

near optimal also from the users’ perspective (we find that it approximately minimizes

expected waiting times).

Though we motivated our work primarily in the context of centralized market-

places, our results also have implications for decentralized marketplaces. One im-

plication is that while organizing longer cycles and chains may require a centralized

marketplace, our results imply that this may be an option worth considering (only

two-way exchanges are typically possible in a decentralized marketplace). Another

implication is more subtle: the fact that greedy is near optimal (for two-cycles only)

suggests that decentralized marketplaces typically cannot improve outcomes by hid-

ing possible matches; simply informing participants of available matches should be

118

nearly optimal.

Although we do not model important details of kidney exchange clearinghouses,

our findings are consistent with computational experiments that show that the greedy

policy is optimal among batching policies. In addition, our findings can serve as

foundations for the importance of using chains and cycles of size more than 2 when

the kidney exchange pool contains many “hard-to-match” patient-donor pairs, as

explained in Section 1.1.3.

Our work raises several further questions and we describe here a few of these.

Allowing for heterogeneous agents or goods may lead to different qualitative results

in some settings. For example, if Bob is a very difficult-to-please agent who is willing

to accept only Alice’s item but they are not both part of any single exchange, it may

be beneficial to make Alice wait for some time in the hope of finding an exchange

that can allow Bob to get Alice’s item (note that such a policy is not monotone).

In particular, when chains or cycles of more than two agents are permitted, some

waiting may improve efficiency in the presence of heterogeneity (some evidence for

this is given by Ashlagi et al. [8]).

In kidney exchange, the existence of easy-to-match and hard-to-match pairs (again

see Section 1.1.3) creates the following problem. Many hospitals are internally match-

ing their easy-to-match pairs, and enrolling their harder-to-match pairs to centralized

multi-hospital clearinghouses [6]. An important question is how much waiting times

of hard-to-match pairs will improve as the percentage of easy-to-match pairs grows.

Allowing for agents’ departures and outside options are other issues worth ex-

ploring. Designing “good” mechanisms that make it safe for agents to participate

in barter exchanges is an important direction (see a concurrent work by Akbarpour

et al. [4] who study how to make it safe for agents to report their “deadlines” in a

homogenous sparse marketplace that conducts pairwise exchanges).

119

4.9 Proofs of Preliminary Results

Proof of Proposition 4.2. We prove the result by contradiction. Assume that a maxi-

mal set of node disjoint three-cycles W contained fewer than N/3 three-cycles. Then

there must be a three-cycle X from a largest set of node disjoint three-cycles such

that for every three-cycle Y ∈ W , X and Y have no nodes in common. This yields

a contradiction, as we could then add X to W to make a larger set of node disjoint

three-cycles, thus making W not maximal.

Proof of Lemma 4.1. We assume that the removal policy is deterministic. The proof

for the case of randomized policies follows immediately. Fix any two nodes i, j which

arrive before time t (namely i, j ≤ t). Given any directed graph G on nodes 0, 1, . . . , t

(that is nodes arriving up to time t) such that the edge (i, j) belongs to G, denote by

G the same graph G with edge (i, j) deleted. Let W be any subset of nodes 0, 1, . . . , t

containing i and j. Recall that we denote by Gt the directed graph generated by

nodes 0, 1, . . . , t and by Wt the set of nodes observed at time t. Note that, since the

policy is deterministic, graph Gt uniquely determines the set of nodes Wt.

We have

P(Wt =W) =∑G

P(G) +∑G

P(G),

where the first sum is over graphs G containing edge (i, j) such that the set of nodes

observed at time t isW when Gt = G, and the second sum is over graphs G containing

edge (i, j), such that when Gt = G, the set of nodes observed at time t is W . Note,

however that by our monotonicity assumption, if Gt = G implies Wt = W , then

Gt = G also implies Wt =W . Thus

P(Wt =W) ≥∑G

(P(G) + P(G)),

where the sum is over graphs G containing edge (i, j) such that Gt = G implies

Wt = W . At the same time note that P(G) = P(G)(1 − p)/p since it corresponds to

120

the same graph except edge (i, j) deleted. We obtain

P(Wt =W) ≥∑G

P(G)(1 + (1− p)/p) =∑G

P(G)/p.

We recognize the right-hand side as P(Wt =W|(i, j) ∈ Gt). Now we obtain

P((i, j) ∈ Gt|Wt =W) = P(Wt =W|(i, j) ∈ Gt)P((i, j) ∈ Gt)/P(Wt =W)

≤ P((i, j) ∈ Gt)

≤ p,

and the claim is established.

Proof of Lemma 4.2. By the memoryless property of the geometric distribution, for

all t > 0,

E[X | X > t] = t+ E[X] = t+1

p. (4.35)

Thus for all sufficiently large κ we have

E[XIX>κ

p

]= (1− p)

κp

(κ

p+

1

p

)(4.36)

≤ e−κ1 + κ

p(4.37)

≤ 1

2κp, (4.38)

where (4.36) follows from (4.35), (4.37) follows as (1 − p)1/p ≤ e−1 for all p (take

logarithms), and finally (4.38) holds provided κ ≥ κ0 for appropriately large κ0.

For the remaining term, we have that for all sufficiently large κ and sufficiently

121

small p,

E[XIX≤ 1√

κp

]= E[X]− E

[XIX> 1√

κp

]=

1

p− (1− p)

1√κp

(1√κp

+1

p

)(4.39)

≤ 1

p−(

1− pe

) 1√κ(

1√κp

+1

p

)(4.40)

≤ 1

p− (1− p)

1√κ

(1− 1√

κ

)(1√κp

+1

p

)(4.41)

=1

p− (1− p)

1√κ

1

p+ (1− p)

1√κ

1

κp

≤ 2√κ

+1

κp(4.42)

≤ 3

2κp(4.43)

where (4.39) follows from (4.35). To obtain (4.40), by Taylor’s theorem, (1− p)1/p =

e−1(1−p/2)+o(p) as p→ 0, thus for sufficiently small p, we have (1−p)1/p ≥ e−1(1−p).

In (4.41) we use that e−x ≥ 1 − x, in (4.42), we use that for all x sufficiently small,

1 ≥ (1 − x)n ≥ 1 − 2xn, and (4.43) follows by taking κ sufficiently large. Thus the

result is shown.

122

Chapter 5

Data Driven Simulations for

Scheduling Medical Residents in

Hopsitals

5.1 Introduction

In 2011, the ACGME instituted a new set of regulations on duty hours that restrict

shift lengths for medical residents. We consider two operational questions for hospitals

in light of these new regulations: will there be sufficient staff to admit all incoming

patients, and how will the continuity of patient care be affected, particularly in a first

day of a patients hospital stay? To address these questions, we built a discrete event

simulation tool using historical data from a major academic hospital, and compared

several policies using both long and short shifts. Using our simulation tool, we will

find that schedules based on shorter more frequent shifts actually lead to a larger

admitting capacity. At the same time, such schedules generally reduce the continuity

of care by most metrics when the departments operate at normal loads. However,

in departments which operate at the critical capacity regime, we found that the

continuity of care improved in some metrics for schedules based on shorter shifts, due

to a reduction in the use of overtime doctors. In contrast to much of the existing

123

literature on the effects of duty hour regulations, our approach directly measures the

relevant performance metrics, rather than relying on the perceptions of outcomes

gathered in surveys.

In our simulations, we find that the relationship between how residents are sched-

uled and the capacity of the hospital to admit patients is quite complex. In particular,

the capacity is not well estimated by simply considering the number of residents in

conjunction with any one of: (a) the number of hours residents are available to admit

patients, (b) the number of patients each resident is allowed to admit per shift, (c) the

number of patients each resident is allowed to have in care simultaneously. Instead,

these constraints interact in highly non-trivial ways. We observe that the capacity of

two schedules can differ greatly even when the same number of residents are used and

long run averages are constant for (a), (b), and (c). We develop the Markov chain

throughput upper bound as a simple model that can quickly compute the interaction

between these constraints to estimate the capacity of a schedule.

Organization

The chapter is organized as follows. In Section 5.2, we explain how our simulation

model works. In Section 5.3, we give the simulation results. In Section 5.4 we discuss

our results and the conclusions we draw for scheduling medical residents in hospitals.

In Section 5.5, we give the details of how our Markovian chain throughput upper bound

is computed. Finally, in Section 5.6, we give some additional details on the statistical

models used to scale up or down the historical data set.

5.2 Materials and Methods

5.2.1 Simulation Model of Patient Flow and Assignment to

Doctors

We now describe a model for evaluating a hospital departments ability to admit and

treat patients under a particular staffing schedule. As input, the model takes the

124

historical data of every patient treated by the department over some period of time

(e.g. a year), including the time they arrive and the time they leave the hospital.

Another input to the model is a list of teams of admitters. Each team is either a

resident team or a physicians assistant (PA) team. Patients are admitted to specific

members of teams. Each team follows a schedule specifying when they are eligible to

admit patients. Schedules give typically a four to nine day rotation, and specify on

which days of the rotation and at what times the team on shift i.e. eligible to admit

patients. Further, each team is subject to some additional constraints:

(C1) maximum number of patients in care allowed for a team,

(C2) maximum number of patients admitted per shift allowed for a team,

(C3) maximum number of patients in care allowed for an individual resident or

PA,

(C4) maximum number of patients admitted per shift allowed for an individual

resident or PA.

Most of the resident constraints are dictated by ACGME work hour restrictions,

while most of the PA constraints are based on hospital policy as a practical measure.

For example, if a team of two admitters had a constraint of type (C3) saying each

admitter could only have 10 patients in care and a constraint of type (C1) saying the

team could only have 18 patients in care, then it would be possible for one admitter

to have 10 patients and the other 8, but having 11 and 7 or 10 and 9 would not be

allowed. The model further specifies a selection rule that determines which of the

eligible admitters on shift will be assigned to an arriving patient. Throughout, we

use the following rule: when a patient arrives, we look at all teams on shift with an

eligible admitter, and select one such team uniformly at random. We then select a

team member uniformly at random from the eligible admitters in this team. We do

this as hospital data suggested that there was not a systematic procedure determining

which patients were assigned to which admitters.

Given these inputs, our simulation model works as follows. We define the set of

event times to be the times of patient arrivals, patient departures, and the beginning

of admitter shifts. For each admitter, we maintain a list of current patients in care.

125

We also maintain a list of each patient that has arrived but has not yet been assigned

to an admitter. We then iterate through the event times chronologically and update

the lists. There are three possibilities, based on the type of the event.

When the next event is a patient arrival, we find the admitting teams that

are on shift. From these teams, we find the members that satisfy constraints

C1-4. Assuming there is such an admitter, we use the selection rule to choose

who receives this patient. However, if there is no available admitter, i.e. every

admitter that is currently on shift violates at least one of C1-4, then that patient

is added to the list of patients that have arrived but not yet been assigned to

an admitter. These patients are admitted temporarily by either the night float

service or the jeopardy service (to be transferred to a resident team or PA team

later).

When the next event is a patient departure, if the patient has been assigned

to an admitter, we remove that patient from list of patients in care by the

admitter. Otherwise, the patient must be in the list of patients not yet assigned

to an admitter, and we remove them from this list.

When the next event is the beginning of a new shift, we take the patients in the

list of patients not yet assigned to an admitter, and attempt to assign them to

an admitter on shift satisfying constraints C1-4 using the selection rule. The

patients that have been waiting longest are assigned first.

If at the beginning of a new shift, there is insufficient capacity to admit all

patients waiting for a reassignment, the remaining patients are “dropped” from

the simulation. In reality, the hospital is forced to declare a state of emergency

census and the capacity of each team is increased. Under a properly functioning

schedule, this happens very infrequently. Thus the number of dropped patients

in a simulation is a good indicator of the system operating at an unsustainable

load. Note that the load on the admitting staff is to some degree shielded from

extreme demands by the number of beds in the hospital. No matter how fast

patients are actually arriving, from the perspective of the physicians there can

126

only be as many inpatients as there are beds.1

Note that while in principle, an admitter could take a new patient from the list of

unassigned patients after another patient departed, we do not allow for this in our

model. In fact, in practice at B&W, once a resident or team of residents has reached

capacity on one of the constraints C1-4, the resident does not admit any new patients

for the remainder of the shift, even if patients depart and the resident is eligible again.

This occurs as the admission nurses assign the patients to admitters manually, and

are only notified when an admitter becomes ineligible. As there can only be patients

unassigned to an admitter when all the admitting teams are capped, we know that the

team which just experienced a departure is capped as well, so our simulator actually

operates as B&W does.

This specifies the dynamics of our model. We use it to collect statistics on the

performance metrics of interest, e.g. the fraction of patients who do not receive an

admitter immediately upon arrival, or the percentage of time that a particular team

is capped.

5.2.2 Physician’s Assistants and Admitting

The physician’s assistant (PA) teams for B&W each provide 2 PAs in the hospital 24

hours a day, 7 days a week (each team consists of around 6 individuals, but there are

always 2 present). These teams can have up to 15 patients in care simultaneously,

and have no other formal restrictions. B&W employed one such team for GMS in the

period 2007-2011. For 2011-2012, they added a second team for GMS. The Cardiology

department only had a team for 2011-2012, and the Oncology department only had

a team for 2010-2012. B&W felt that admitting a patient to a PA service provided

the same quality of care as admitting to a resident.

1Additionally, in most academic medical centers there are boarders, which are patients that donot have an inpatient bed, but are admitted to an inpatient team and stay in the emergency room.

127

5.2.3 Resident Admitting Shifts, Schedules, and Policies

Here we describe the scheduling system used to determine when residents are on shift,

and the total number of residents and PAs required for each department. As far as

the residents are concerned, the system consists of three layers, shifts, schedules, and

policies. Residents are put in teams typically of size 2-3, and teams are organized

into groups, each with 2-4 teams. Shifts specify when a resident is on shift or off

shift for a 24 hour period from 7am-7am. A schedule applies to a group of teams of

residents. It takes a sequence of shifts (usually 4 or 6) and assigns each resident to

rotate through these shifts. It also specifies the offset between residents on the same

team and the offset between teams on the rotation. Finally, a policy assigns schedules

to every group of team of residents, and further specifies an offset between groups

in a department. Generally, offsets are chosen so that the residents provide coverage

uniformly throughout the rotation. A policy also specifies the number of PA teams a

department will be assigned.

As an example, we now describe in detail the policy by GMS for the 2010-11

academic year. The shifts used were the Long Shift (L) that ran from 7am to noon

the following day, the Short Shift (S) that ran from 7am to 2pm, and the Off Shift

(O) where residents were not admitting new patients. At the schedule level, we have

two groups of teams. For the first group, there were four teams with two residents

each, called GMS A-D. They all followed the four day rotating schedule Long Shift,

Off, Short Shift, Off (LOSO). Within a team, there was no offset between the two

residents, but between teams, there was a one day stagger, ensuring that every day

there was a team on Long Shift. The second group of teams formed the intensive

teaching unit (ITU). It had two teams, ITU 1 and ITU 2, and each team had three

residents. The residents followed the 6 day rotating schedule Long Shift, Off, Off,

Short Shift, Short Shift, Off (LOOSSO). Within a team, each resident was offset by a

day on the rotating schedule, and between teams, there was a three day offset on the

rotating schedule, again insuring that every day there was one resident on Long Shift.

The policy for GMS was simply to use the GMS A-D group and the ITU 1-2 group.

128

1 2 3 4 5 6 7 8 9 10 11 12

A1

A2

B1

B2

C1

C2

D1

D2

IA1

IA2

IA3

IB1

IB2

IB3

InternTeamGroup

LOSO

LOOSSO

A

B

C

D

IA

IB

Figure 5-1: The GMS policy for the 2010-11 academic year. The GMS A-D teamshave a 4 day schedule and the ITU 1-2 teams have a 6 day rotating schedule, so thesystem has an overall 12 day rotating schedule.

There was no need to specify any offset at the policy level as both groups of residents

have schedules which are symmetric. Figure 5-1 provides a diagram explaining all

the offsets between residents for the policy. Additionally, the policy allocated a single

team of 2 PAs to GMS (giving a PA capacity of 15 patients in care). There are several

additional details describing shift structure. The number of hours a resident must

spend at the hospital is longer than the shift, as it takes the resident approximately

two hours on average to treat a newly admitted patient. For some shifts, if the shift

falls on a weekend, the shift is canceled. Some shifts have different lengths in the

first and second semesters, to give more experienced residents more hours. Under

some policies where there are no residents admitting patients between 5am and 7am,

patients arriving at this time wait until 7am and then are admitted by the doctor

starting a new shift. This mechanism is called “Early Admit.” There is a maximum

number of patients a resident can admit while on this shift. This is set as realistically

129

Name Admitting Hours In hospital hours Weekends Early Admit Max Admits (Onc)

L 7am-5am (22 hours) 7am-Noon (29 hours) Yes No 5+2 (5+0)S 7am-2pm (7 hours) 7am-2pm+ (7+ hours) No No 2+1 (1+0)M 2pm-10pm (8 hours) 2pm-Midnight (10 hours) Sometimes No 3+1 (3+0)M’ 10am-6pm (8 hours) 10am-8pm (10 hours) Sometimes No 3+1 (3+0)D 7am-6pm (11 hours) 7am-8pm (13 hours) Yes Yes 4+1 N/AD’ 7am-7pm (12 hours) 7am-9pm (14 hours) Yes Yes 4+1 N/AN 6pm-5am (11 hours) 6pm-7am (13 hours) Yes No 4+1 N/AO None Variable Variable No 0 0

Table 5.1: The above shifts were used to construct schedules for residents.

each new admission takes two to three hours of work, and residents are not supposed

to take patients that force them to stay beyond the end of their shift. The maximum

number of admissions per shift is typically of the form “5+2,” meaning that the

resident can do five new admissions, but may do a total of seven admissions including

taking reassignment patients (see Section 5.2.4). If there are no reassignments, then

the resident can only admit 5 patients, but if the resident must admit more than 2

reassignments, then the resident can still only admit a maximum of seven patients for

the day. Also this maximum is dependent on the department. Specifically, doctors

from Oncology can admit fewer patients per shift as their admissions take more time.

All of these details are summarized for each type of shift in Table 5.1.

As previously described, a schedule assigns a rotating sequence of shifts to a group

of resident teams, and gives an offset within each team and between teams in the

rotating schedule. The schedules considered in this project are summarized in Table

5.2. Here average hours/week indicates the number of hours spend admitting. This

translates the 4 and 6 day schedules into 7 day averages accounting for weekends.

The offset information is omitted, but under each schedule, the coverage provided by

the residents is uniform throughout the rotation. We now give a brief discussion of

the motivation for each schedule.

1. LOSO– This was the default schedule that was in place for all three B&W

departments prior to the new regulations. By using long shifts, the schedule

provides long periods of patient observation following admission and reduces

the total number of daily shift changes, providing continuous ongoing care for

patients.

130

2. LOOSSO– This schedule was only used by the Intensive Teaching Unit teams

for GMS. As these teams are spending additional time on educational aspects of

patient care, they have a reduced capacity of 18 patients in care for 3 residents

(recall that LOSO has a capacity of 20 patients in care for 2 residents).

3. MMMO– This schedule was designed as an alternative to LOSO that with the

goal of increasing capacity and reducing jeopardy. When the project began,

due to a recent restructuring, several B&W departments found themselves in

a perpetual state of emergency census (when teams must operate above their

capacity) and jeopardy. Interestingly, we observed that by having many short

frequent shifts instead of occasional long shifts, the fundamental capacity of the

system is increased, an issue further explored in Section 5.5.

4. M’M’M’O– This schedule is nearly the same as MMMO, except the resident

shifts are shifted four hours earlier. This schedule was proposed as an alternative

to MMMO with the goals of improving resident quality of life (by allowing them

to get home from work at a reasonable hour) and increasing the overlap between

the hours when a resident was admitting and the hours when a resident was in

the hospital attending to other responsibilities such as treating and discharging

existing patients (these activities tend to take place in the morning).

5. D’OSO– This schedule essentially just takes the old LOSO schedule and makes

it regulations compliant by significantly reducing the length of the long shift.

Like M’M’M’O, this schedule has shifts ending during peak hours, which can

prevent residents from leaving on time. To try and ease the residents’ depar-

tures, the flex admitter is introduced for this schedule, as described in Section

5.2.4. Also notice that the team in care capacity of D’OSO is 15, while for

LOSO it was 20.

6. DOOONO– This schedule was designed to provide both day and night admit-

ting coverage by residents without using long shifts. The schedule also reduces

the number of patients in care per resident (a team cap of 20 for 3 residents, as

opposed to a team cap of 20 for 2 residents under LOSO). Additionally, under

DOOONO residents only can admit 2 days out of every 6 and ten patients every

131

Name Teams Residents/Team Team in Care Cap

LOSO 4 2 20LOOSSO 2 3 18MMMO 4 2 20M’M’M’O 4 2 20D’OSO 4 2 15DOOONO 2 3 20DOSOOONOO 3 3 20

Table 5.2: The following schedules were used to manage resident teams. Included areboth actual policies implemented by B&W as well as alternatives considered as partof this research.

six days, whereas under LOSO residents can admit 2 out of every 4 days and

10 patients every four days.

7. DOSOOONOO– This 9 day rotating schedule is essentially a variant of

DOOONO proposed for the cardiology department that provides uniform cover-

age using 9 total residents, as opposed to the 6 residents required for DOOONO.

Finally, we describe the policies. Recall that a policy was simply a list of schedules

assigned to groups, with an offset between each group saying where to start in the

rotating schedule. Additionally, a policy must specify the capacity of the PA teams.

For each policy, we define the throughput as follows. Suppose that there were

infinitely many patients waiting to be treated initially, but each patient only begins

recovering when they are assigned to a resident or PA. The throughput of the policy

is average rate that patients leave for the entire department (in patients per day).

The throughput of a policy is a good indicator of how a policy will perform under

an increasing patient load. Intuitively, as the average number of patients arriving

per day increases towards the throughput, the likelihood of jeopardy and dropped

patients increases.

The throughput is difficult to determine exactly, but there are two natural upper

bounds that we can easily observe. First, suppose that the residents could ignore

constraints (C2) and (C4), which restrict the number of patients they can admit per

day. Then as we are assuming that there are infinitely many patients waiting, the

residents would always have the maximum number of patients in care as allowed by

132

(C1) and (C3). For example, if an resident always has ten patients in care simul-

taneously, and the average patient stays for four days, then on average 10/4 = 2.5

patients will depart per day that were under the care of that resident. In reality,

the resident cannot maintain ten patients in care at all times because of constraints

(C2) and (C4) restricting the number of admissions per day, so we are overestimat-

ing the throughput. We call this estimate the capacity upper bound on throughput.

For our second upper bound, we instead assume that the residents can ignore con-

straints (C1) and (C3) restricting the number of patients they have in care, but must

obey the constraints (C2) and (C4) restricting the number of patients they can ad-

mit per day. For example, under LOSO, on an “L” day, the resident admits seven

patients, on an “S” day the resident admits 3 patients, and otherwise the resident

does not admit any patients. Thus the resident admits ten patients every four days,

or 10/4 = 2.5 patients per day. However, there is a small error in this computation,

as the “S” shift only occurs on week days, so really on average the resident admits

(7 + 3 · 5/7)/4 = 2.25 patients per day. Again in reality, the resident cannot always

admit all these patients, as sometimes the resident will reach ten patients in care and

be forced to stop admitting. Thus we have obtained a second upper bound on the

throughput, which we refer to as the admitting upper bound on throughput.

As the constraints (C1)-(C4) only affect individual residents and teams of resi-

dents, we can approximate the throughput of a policy by approximating the through-

put of each team of residents and the PAs, and then summing the results up. The

throughput of the PAs is equal to the capacity upper bound on throughput, as PAs

only have a constraint on the number of patients in care, not on the number of pa-

tients admitted per day. For example, for GMS, if the PAs have the capacity to treat

15 patients simultaneously and patients stay for an average of four days, then the

PAs add 15/4 = 3.75 to the policy throughput. To give a very simple example of

how to compute the throughput of a policy, suppose that GMS put both group 1

and group 2 on the schedule LOSO, that GMS had a PA service that could treat 15

patients simultaneously, and that the average length of stay was four days. Then the

PA service would have a throughput of exactly 3.75 patients per day. There would

133

be 8 teams each with two residents on LOSO, totaling 16 residents. Each resident

has a capacity upper bound of 2.5 patients per day, giving the residents a total ca-

pacity upper bound of 16 · 2.5 = 40 patients per day, and making the policy capacity

upper bound 43.75 patients per day. Similarly, each resident has an admitting upper

bound of 2.25 patients per day, making the admitting upper bound for all residents

16 · 2.25 = 36 patients per day, and thus making the admitting upper bound for the

policy equal to 39.75. As a result, we expect the throughput of the proposed policy

to be less than 39.75.

Using these two upper bounds, we can conclude that the true throughput must be

less than the minimum of these two upper bounds. One might suspect that the true

throughput would be equal to the minimum of these two upper bounds, or at least

that for two policies for which the minimum of these upper bounds is the same, that

the throughput should be the same. In particular, when we apply our upper bounds

for the policies Initial and Daily Admitting, we get the same bound on the throughput,

so we suspect that the policies have the same throughput. However, it turns out that

this is not the case. In Section 5.5, we derive a much more accurate approximation of

the throughput where (C1)-(C4) are all used, which we refer to as the Markov chain

throughput upper bound. When all four constraints are combined, the constraints

(C2) and (C4) on the number of patients that can be admitted per day cause the

residents to not always have 10 patients in care simultaneously, and constraints (C1)

and (C3) on the number of patients that a resident can have in care simultaneously

cause the resident to be unable to admit the maximum number of patients per shift,

thus making both upper bounds too large. While the upper bounds to provide a good

first order test to compare schedules, we will see in later sections that the Markov

chain throughput upper bound correctly predicts that policies using MMMO should

experience fewer dropped patients than policies using LOSO.

Select policies are given in Table 5.3 for GMS, Table 5.4 for Cardiology, and Table

5.5 for Oncology, along with the capacity upper bound on throughput, the admitting

upper bound on throughput, and the Markov chain throughput upper bound. A

detailed breakdown of throughput for each schedule is given in Section 5.5, along with

134

Throughput Upper Bound(Patients/Day)

Name Group 1 Group 2 PA Capacity Capacity Admitting Markov

Initial LOSO LOOSSO 15 30.4 33.2 25.9Daily Admitting MMMO LOOSSO 15 30.4 36.7 27.7Preserve Day D’OSO D’OSO 30 33.7 35.3 28.7Corkscrew DOOONO DOOONO 30 24.7 26.7 19.8Hybrid-15 DOOONO D’OSO 15 25.9 27.7 20.9Hybrid-30 DOOONO D’OSO 30 29.2 31.0 24.2

Table 5.3: A sample of policies considered for GMS.

Throughput Upper Bounds(Patients/Day)

Name Group 1 PA Capacity Capacity Admitting Markov

Initial LOSO 0 17.0 18.3 13.7Daily Admitting MMMO 0 17.0 21.7 15.3Triploid-0 DOSOOONOO 0 12.8 12.4 10.8Triploid-15 DOSOOONOO 15 16.0 15.6 14.0Triploid-30 DOSOOONOO 30 19.2 18.8 17.2

Table 5.4: A sample of policies considered for Cardiology.

some additional explanation for the calculations. In particular, schedules creating

teams where the (C1) constraint (team patients in care capacity) is not dominated

by the (C3) constraint (individual patients in care capacity), e.g. LOOSSO, require

some additional approximation.

Notice that policies using the schedules D’OSO, DOOONO and DOSOOONOO

have additional PA capacity. This is necessary to offset the reduction in the number

of patients residents can have in care and the number they can admit under these

schedules, i.e. the reduced throughput.

Throughput Upper Bound(Patients/Day)

Name Group 1 PA Capacity Capacity Admitting Markov

Initial LOSO 15 15.0 13.8 12.2Daily Admitting MMMO 15 15.0 18.7 13.9Daily Admitting Shifted M’M’M’O 15 15.0 18.7 13.9

Table 5.5: A sample of policies considered for Oncology.

135

5.2.4 Patient Flow for Reassigned Patients

Here we describe what happens to the patients that arrive when all admitting teams

are at capacity and are forced to wait. Such patients are referred to as reassignments,

as they are admitted by one doctor and then permanently transferred to another.

At B&W, three different groups provide this temporary care: the night floats, the

flex admitters, and the jeopardy service. Further, the flex admitters have two modes

of admission, flex forward and flex backward. We now briefly discuss medical and

financial implications of admitting patients through each of these mechanisms.

The night floats are residents and doctors that arrive in the evening and leave the

following morning. The exact hours when the night floats arrive and depart tends to

vary quite widely depending on the policy. The night floats primary responsibility

is cross coverage (providing care for patients that have already been admitted, but

whose doctors have left the hospital for the night). Night floats typically are covering

for many patients simultaneously, and as a result a large fraction of their time is

spent simply keeping existing patients stable. However, when all PAs and residents

on shift have reached capacity and can no longer take new patients, the night floats

watch over newly arriving patients temporarily. The following morning when there is

a shift rotation, these patients are handed off to a resident team or PA service. These

patients are referred to as night float reassignments. In our model, there is no limit

to the number of patients the night floats can admit.

Admissions by night floats are considered quite undesirable. When a patient is

admitted, the admitter spends typically around two hours with the patient while

determining a course of treatment. This can be a very substantial fraction of the

total time and attention they will receive from their doctor. Thus when a patient

is admitted by one doctor and then permanently transferred to another, information

can be lost. Additionally, there are underlying issues with the night float system that

compound these problems. For all patients treated by night floats, both reassignments

and patients in cross coverage, it has been historically difficult to enforce protocols

for the night floats to document their observations and communicate them with the

136

resident receiving the patient the following morning. In the case of reassignments,

these lapses in communication are particularly dangerous as many critical decisions

regarding the course of treatment are made soon after the patient arrives, making

this a particularly bad time for a mistake. Thus it is desirable to have schedules with

low night float admissions rates.

Next we consider the jeopardy service. If all the admitters on shift have reached

capacity before the night floats arrive, a state of jeopardy is declared. Doctors are

called in for overtime to temporarily admit and treat new patients until the night

floats arrive. These patients are handed off to the night floats, and then handed off

again to a resident or PA the following morning.

Admissions by jeopardy are significantly worse than an admission directly to the

night floats, both from a financial and medical perspective. As jeopardy relies on

doctors being paid overtime, jeopardy admissions are much more expensive than

other types of admissions. Additionally, patients are now handed off twice before

they reach their caring doctor, doubling the opportunity for a communication failure.

It was the expectation of B&W that jeopardy should occur infrequently.

Finally, the flex admitters were a newly created service for the 2011-12 year. They

admit new patients from 4-8pm daily, but only when resident and PA service capacity

is exhausted. The flex admitter is used only when the schedule D’OSO is used, namely,

by GMS under the policies Preserved Day and Hybrid. For the purposes of our model,

the flex admitter is not constrained in the number of patients they can temporarily

hold.

The flex admitter has two modes of admission, flex forward and flex backward.

The modes differentiate which doctor will ultimately treat the patient. When the

flex admitter takes the patient, if there is a resident whose shift ended within the last

two hours that is not yet at capacity (and on a team that is not yet at capacity),

the patient will be handed back to this doctor the following day, resulting in a flex

backward admission. When there are no such residents available, the patient will be

handed off to the night floats at 8pm and then handed off again to a resident or PA

the following morning, resulting in a flex forward.

137

The flex forward is somewhat analogous to a jeopardy admission, but it is slightly

better in that no overtime pay is necessary. However, the real motivation of this

mechanism is to reduce jeopardy admissions in exchange for flex backward admis-

sions. Flex backward admissions are desirable as the resident can observe the patient

with the flex admitter for a portion of the admitting process and have a face to face

discussion with the flex admitter about the patient before the resident leaves for the

day. Thus the resident will know more about this patient than a resident beginning

shift the following day, significantly reducing information lost due to miscommunica-

tion.

5.3 Results

We now discuss the result of simulating the policies described above in our model.

First, we list the performance metrics that we compute in our simulation studies,

as well as the motivation and goals behind the selecting the schedules and policies

that we simulated. Then, we look at simulations of these policies and compare their

performance.

5.3.1 Performance Metrics

We now briefly summarize how we measure the quality of each policy

1. Reassignments– As described in Section 5.2.4, a reassignment occurs when all

residents and PAs are at capacity when a patient arrives, so that patient must

be temporarily admitted by another doctor and then transferred to a resident

or PA the following day. The four types of reassignments are listed below from

least to most desirable:

(a) Jeopardy reassignment– These patients are admitted by doctors working

overtime before the night floats arrive. They are handed off to the night

floats, and then handed off again to a resident or PA the following morning.

Using jeopardy requires the hospital to pay overtime wages.

138

(b) Flex forward reassignment– These patients are admitted by the flex ad-

mitter. These patients are also handed off to night floats and then to a

resident or PA the following morning.

(c) Night float reassignment– These patients are temporarily admitted by the

night floats and then reassigned to a resident or PA the following morning.

(d) Flex backward reassignment– These patients are admitted by the flex ad-

mitter with the support of a resident in the final two hours of their shift.

The patient is watched temporarily by the night floats, and then returned

to the same resident the following morning.

2. Jeopardy Days– This is simply the number of days in the year that a jeopardy

reassignment occurred. As usually only a single jeopardy doctor is required to

deal with all jeopardy admissions for a day, the cost of jeopardy is driven by

the number of days of occurrence, not the number of patients.

3. Dropped Patients– These are patients that should have been reassigned, but at

the time of reassignment to a resident or PA, the capacity was exhausted (as

described in Section 5.2.1). Such patients are a sign that the policy is chronically

over capacity.

4. Observation < 6 hours– This is the fraction of patients admitted by the residents

that were viewed continuously for less than 6 hours following admission by the

resident before the residents shift ended. Here we are only accounting time

on shift in the observation hours. Note that while residents spend typically

80 hours a week in the hospital, only 40 of these are hours on shift. Thus

depending on how the remaining time in the hospital is allocated, this statistic

could significantly underestimate the actual frequency of at least six hours of

observation. For example, under LOSO, on an “L” day, residents would remain

in the hospital until noon the following day, so effectively there would always

be less than 6 hours of observation.

5. Shift runs late– Here, we indicate the frequency that a resident must admit a

patient in the final two hours of a shift (among all days with any shift other

than “off”). This figure is important as admitting a patient requires two to three

139

hours of work, so the resident will be forced to work past the end of their shift.

This can ultimately lead to residents violating restrictions on hours worked per

week, so it is important to anticipate with what frequency such admissions will

occur. Historically, both and B&W and at other hospitals, there have been

problems getting residents out of the hospital when their shifts end during peak

admitting hours.

6. Interarrival < 2 hours– Here we measure the frequency that a residents succes-

sive admission times are separated by less than two hours. We use two hours

as it takes about two hours to admit a new patient. Thus we are measuring

how often residents are forced to deal with two or more patients simultaneously.

From a quality of care perspective, it is desirable that this number be low.

5.3.2 Assessing Policies at Historical Patient Arrival Rate

First, we compare the performance of the policies for the 2009-10 historical patient

arrival data for each of our three departments, Oncology, Cardiology, and GMS.

In Table 5.6, we give the performance of these policies in the Oncology depart-

ment. In all of our simulations, the Oncology department has four teams each of

two residents, and a PA service with a capacity of 15 patients in care that admits

patients 24 hours a day, 7 days a week. We consider three policies from Table 5.5,

Initial, Daily Admitting, and Daily Admitting Shifted. We see that under all three

policies, we do not drop patients, implying that they provide sufficient admitting ca-

pacity to treat incoming patients. We see that Initial has some jeopardy, but both

Daily Admitting and Daily Admitting Shifted have none. However, we see that Ini-

tial uses significantly fewer night float admissions than Daily Admitting and Daily

Admitting Shifted. We see that Initial is more likely to provide 6 hours of continuous

observation than either Daily Admitting or Daily Admitting Shifted. As previously

remarked in Section 5.3.1, the policy Initial actually will observe nearly all patients

for at least 6 hours, as this figure only accounts for time spent on shift, not time in

the hospital. Although the hours in hospital not on shift have not been specified for

Daily Admitting or Daily Admitting Shifted, if these policies were implemented these

140

Schedule Initial Daily Admitting Daily Admitting Shifted

Night Float Admissions 242 676 1298Jeopardy Days 20 0 0Jeopardy Admissions 43 0 0Dropped Patients 0 0 0Observation < 6 Hours (%) 47.0 83.7 95.4Shift Runs Late (%) 15.9 13.0 18.5Interarrival < 2 Hours (%) 38.0 19.3 12.8

Table 5.6: Performance of the Oncology Department under three different policies forscheduling residents. This simulation is based on 2009-10 historical data. There were3246 patients of the course of the year in the simulation.

hours would likely be before the shift began, so that the residents could be present

in the morning for rounds and to discharge patients.

Finally, we see that under Initial it is much more likely that the patient interarrival

time is less than two hours than under either Daily Admitting or Daily Admitting

Shifted. This is occurring as Daily Admitting and Daily Admitting Shifted have more

residents on hand admitting simultaneously at peak hours when patients are arriving

more frequently.

In comparing Daily Admitting and Daily Admitting Shifted directly, we see that

Daily Admitting Shifted puts significantly more load on the night floats. This occurs

as the peak arrival process is between 2pm and 10pm, so half of the peak arrivals

are missed by Daily Admitting Shifted and are forced onto the night floats. We also

see that Daily Admitting Shifted has a higher fraction of patients with less than 6

hours of observation before the end of the admitters shift. This occurs as we only

meet the 6 hour threshold when a patient arrives in the first two hours of the shift,

and there are far fewer arrivals between 10am and noon than between 2pm and 4pm.

We see an increase in the likelihood that a shift will run late under M’M’M’O, as this

schedule has residents end their shifts at the peak arrival time of the day. However,

this policy will provide residents with a higher quality of life, as they are released

from the hospital at an earlier hour of the day.

Next, in Table 5.7, we look at the performance of the policies from Table 5.4,

Initial, Daily Admitting, and Triploid. Under the first two policies, a group of 8

141

Schedule Initial Daily Admitting Triploid-0 Triploid-15 Triploid-30

Night Float Admissions 445 1313 473 125 5Jeopardy Days 17 0 210 63 1Jeopardy Admissions 24 0 760 144 3Dropped Patients 0 0 156 10 0Observation < 6 Hours (%) 44.8 84.8 70.5 61.3 60.4Shift Runs Late (%) 16.6 14.9 25.7 29.7 31.7Interarrival < 2 Hours (%) 40.9 18.9 41.4 37.5 32.3

Table 5.7: Performance of the Cardiology Department under five different policies forscheduling residents. This simulation is based on 2009-10 historical data. There were3257 patients of the course of the year in the simulation.

residents and no PAs are used. Under the Triploid policy, nine residents are used,

but individually residents are constrained to admit patients more slowly and care for

fewer total patients. To offset this, we consider 3 levels of PA staffing, no PAs (as

under the first two policies), referred to as Triploid-0, one PA team (giving a capacity

of 15 patients), Triploid-15 and two PA teams (giving a capacity of 30 patients),

referred to as Triploid-30.

The comparison between Initial and Daily Admitting for Cardiology is very similar

to the relation of these policies for Oncology. We see that Initial puts less load on

the night floats than Daily Admitting, but that Daily Admitting puts less (in fact

no) load on jeopardy than Initial. Again, Daily Admitting is less likely to have 6

continuous hours of observation than Initial, but is also less likely to have to admit

twice in a two hour window than Initial.

Looking at the three levels of PA staffing for the Triploid policy, it is immediate

from the dropped patient and jeopardy statistics that Triploid-0, i.e. not increasing

PA staffing, is infeasible. Under Triploid-15, we still have the occasional dropped

patient and quite a bit of jeopardy, suggesting that we are at the edge of the capacity

of the system. However, in Triploid-30, the schedule actually performs quite well.

Between the PAs and the residents, there is sufficient capacity to admit essentially all

arriving patients and the night floats and jeopardy services are rarely needed. The

policy performance is comparable to Initial and Daily Admitting in other metrics as

well, although it is benefiting from having a greater staffing level.

142

Schedule Initial Daily Admitting Hybrid-15 Hybrid-30 Preserved Day Corkscrew

Night Float Reassignments 1450 1523 1585 812 1395 803Jeopardy Days 67 7 72 12 3 149Jeopardy Reassignments 242 16 252 25 7 775Flex Forward Reassignments 0 0 527 79 17 0Flex Backward Reassignments 0 0 267 311 304 0Dropped Patients 10 0 478 38 28 316Observation < 6 Hours (%) 56.6 78.4 66.1 59.5 73.0 61.2Shift Runs Late (%) 21.9 24.5 20.3 26.1 32.3 19.5Interarrival < 2 Hours (%) 40.5 29.5 38.8 37.8 24.8 41.6

Table 5.8: Performance of GMS under two different policies for scheduling residents.This simulation is based on 2009-10 historical data. There were 7218 patients of thecourse of the year in the simulation.

Finally, in Table 5.8 we compare several the performance of the policies from

Table 5.3 for the GMS department. In these policies, GMS used two groups of teams

of residents, and has a PA patient in care capacity of either 15 under Initial, Daily

Admitting, and Hybrid-15, and 30 under Hybrid-30, Preserved Day and Corkscrew.

Under Initial, we see that GMS is at the brink of instability, with some dropped

patients and many days of jeopardy. Comparing Initial with Daily Admitting, we see

that Daily Admitting has about 100 more night float admissions, but over 200 fewer

jeopardy admissions, giving an overall improvement on reassignments. This is due

to the hybridization effect of MMMO reducing jeopardy admissions and LOOSSO

providing night coverage.

Looking at the other schedules, we see that Preserved Day has less capacity than

Initial and that Corkscrew has less capacity than Preserved Day, to the point where

the dropped patients necessitate increasing the PA capacity to 30. Preserved Day

provides inadequate night float coverage, and corkscrew lacks capacity, but hybridized

(in Hybrid 30) together they produce a schedule that takes the better properties from

both components. We see that even with the hybridization, we cannot reduce the PA

level back to 15 without dropping patients.

5.3.3 Performance Analysis under Increased Patient Volume

In this section, we consider performance of the policies under an increased and de-

creased patient volume. We look only at dropped patients, jeopardy reassignments,

and total reassignments. Here, the total number of reassignments is the sum of the

143

2915-10%

3246 3586+10%

3865+20%

4292+30%

0

100

200

300

Patients per Year

Dro

pp

edP

atie

nts

Sensitivity of Dropped Patients toPatient Arrival Rate (Oncology)

InitialDaily Admitting

Daily Admitting Shifted

Figure 5-2: Sensitivity of dropped patients to patient arrival rate. Simulations basedon 2009-10 historical data for Oncology.

jeopardy reassignments, night float reassignments, and when appropriate the flex

forward reassignments (we do not count the flex backward reassignments as from a

quality of care perspective, these reassignments arent particularly harmful, see Section

5.2.4).

First, we consider the Oncology department under an increasing patient load, as

shown in Figure 5-2 and Figure 5-3. We observe first that the Initial policy begins

to break down (i.e. experience dropped patients) with only a 10% increase in patient

volume, and completely fails with a 20% increase, while Daily Admitting and Daily

Admitting Shifted experience no dropped patients until a 30% increase in patient

volume. Next, we observe that when the load is increased by 20%, Daily Admitting

causes fewer total reassignments than Initial, despite the fact that Daily Admitting

is reassigning most patients arriving off peak hours.

For Cardiology, in Figure 5-4 and Figure 5-5, we observe that Initial and Daily

Admitting both have almost no dropped patients, even when the patient arrival rate

is increased by 20% (results for Triploid policies are in the appendix). We also see

144

2915-10%

3246 3586+10%

3865+20%

4292+30%

0

500

1,000

1,500

2,000

Patients per Year

Rea

ssig

nm

ents

Sensitivity of Total Reassignments toPatient Arrival Rate (Oncology)

InitialDaily Admitting

Daily Admitting Shifted

2915-10%

3246 3586+10%

3865+20%

4292+30%

0

200

400

600

800

Patients per YearR

eass

ignm

ents

Sensitivity of Jeopardy Reassignments toPatient Arrival Rate (Oncology)

InitialDaily Admitting

Daily Admitting Shifted

Figure 5-3: Performance of Oncology Department with various policies under chang-ing patient volume. This simulation is based on 2009-10 historical data.

that unlike Oncology, Daily Admitting is always causing more reassignments than

the other policies. This is both due the fact that Cardiology is not supported by any

PAs under Daily Admitting (as it was with Oncology), and because the system is not

critically loaded to the point of Initial failing.

Finally, in Figure 5-6, Figure 5-7 and Figure 5-8 we see that for the GMS depart-

ment, all policies are right at the edge of their capacity, giving very poor performance

with only a 10% increase in the number of patients in the system. One should keep

in mind that Initial, Daily Admitting, and Hybrid 15 are using fewer PAs than the

other policies (thus we present there results in separate figures).

5.4 Discussion

5.4.1 Key Insights

Here we summarize general insights revealed through our simulation analysis on the

influence of the resident policy and the arrival rate of patients on the number dropped

145

2926-10%

3257 3558+10%

3914+20%

4245+30%

0

20

40

Patients per Year

Dro

pp

edP

atie

nts

Sensitivity of Dropped Patients toPatient Arrival Rate (Cardiology)

InitialDaily Admitting

Figure 5-4: Sensitivity of dropped patients to patient arrival rate. Simulations basedon 2009-10 historical data for Cardiology.

2926-10%

3257 3558+10%

3914+20%

4245+30%

500

1,000

1,500

Patients per Year

Rea

ssig

nm

ents

Sensitivity of Total Reassignments toPatient Arrival Rate (Cardiology)

InitialDaily Admitting

2926-10%

3257 3558+10%

3914+20%

4245+30%

0

200

400

600

Patients per Year

Rea

ssig

nm

ents

Sensitivity of Jeopardy Reassignments toPatient Arrival Rate (Cardiology)

InitialDaily Admitting

Figure 5-5: Performance of Cardiology Department with various policies under chang-ing patient volume. This simulation is based on 2009-10 historical data.

146

6508-10%

7218 7901+10%

8633+20%

9429+30%

0

500

1,000

1,500

Patients per Year

Dro

pp

edP

atie

nts

Sensitivity of Dropped Patients to Patient Arrival Rate (GMS)

InitialDaily Admitting

Figure 5-6: Sensitivity of dropped patients to patient arrival rate. Simulations basedon 2009-10 historical data for GMS.

6508-10%

7218 7901+10%

8633+20%

9429+30%

2,000

4,000

6,000

Patients per Year

Rea

ssig

nm

ents

Sensitivity of Total Reassignments toPatient Arrival Rate (GMS)

InitialDaily Admitting

6508-10%

7218 7901+10%

8633+20%

9429+30%

0

1,000

2,000

3,000

Patients per Year

Rea

ssig

nm

ents

Sensitivity of Jeopardy Reassignments toPatient Arrival Rate (GMS)

InitialDaily Admitting

Figure 5-7: Performance of GMS with various policies under changing patient volume.This simulation is based on 2009-10 historical data.

147

6508-10%

7218 7901+10%

8633+20%

9429+30%

0

2,000

4,000

6,000

Patients per Year

Rea

ssig

nm

ents

Sensitivity of Total Reassignments toPatient Arrival Rate (GMS)

Preserved DayHybrid-15Hybrid-30Corkscrew

6508-10%

7218 7901+10%

8633+20%

9429+30%

0

1,000

2,000

3,000

Patients per YearR

eass

ignm

ents

Sensitivity of Jeopardy Reassignments toPatient Arrival Rate (GMS)

Preserved DayHybrid-15Hybrid-30Corkscrew

Figure 5-8: Performance of GMS with various policies under changing patient volume.This simulation is based on 2009-10 historical data.

patients and reassignments. In comparing policies, it is important to remember that

there are two distinct groups of policies playing under different rules: the first group

comparing Initial and Daily Admitting policies, and the second group comparing

Preserved Day, Hybrid, Corkscrew, and Triploid policies. For the first group, the

constraints on the total number of patients in care are less restrictive than for the

second group. However, the first group also generally was assigned lower PA staffing

levels. As many variables are being changed at once, it is difficult to draw conclusions

from comparisons made between these two groups of policies. For all three depart-

ments, the Initial policy and the Daily Admitting policy actually hold these variables

relatively constant, and thus provide the best tools for comparison.

We now give three key insights:

1. Policies with more frequent shorter shifts have a greater capacity than policies

with long infrequent shifts and thus are superior under heavy patient loads.

We can directly observe this behavior in comparing Initial and Daily Admit-

ting for all three departments. We see that as the patient load increases, more

patients are dropped under the Initial policy than the Daily Admitting policy.

148

As an additional indicator, we see that the Initial policy always results in more

instances of jeopardy than Daily Admitting. Note that beyond comparing long

shifts versus shorter more frequent shifts, the two policies use equivalent re-

sources. Each resident can admit up to ten patients every four days and have

up to ten patients in care simultaneously. Additionally, the requirements for

number of residents, PAs, and admitting hours per four day rotation are all

about the same. The difference in capacity arises because admitting more fre-

quently allows residents to better maintain close to 10 patients in care at all

times. A mathematical justification of this phenomenon is further explored in

Section 5.5 and again in Chapter 6.

2. As the patient load increases, the number of reassignments and dropped patients

increases rapidly.

This can be observed for all policies, although the effect is most noticeable for

Oncology under the Initial policy, and under all policies in GMS. This kind of

non-linear performance degradation as the system approaches capacity is typical

in capacitated systems.

3. Schedules in which the admitting hours of doctors are aligned with the arrival

rate of patients result in significantly fewer reassignments.

This can most clearly be seen in comparing the policies Daily Admitting and

Daily Admitting Shifted for the Oncology department. Daily admitting focuses

all admitting capacity on the window 2pm-10pm, while Daily Admitting Shifted

focuses all admitting capacity on the window 10am-6pm. Under Daily Admit-

ting, we have 676 total reassignments, while under Daily Admitting shifted,

we have 1298 total reassignments. Additionally, observing that the number of

dropped patients under these policies is almost the same, we see that capacity

and alignment are two independent issues.

We also see that for GMS, the Preserved Day Schedule results in many more

reassignments than Hybrid-30, as Preserved Day fails to cover the latter half of

peak admitting hours as well.

149

5.4.2 Assumptions and Limitations of the Model

We now identify some of the assumptions and limitations of our study. First, we

look at the alignment between the measured outcomes optimized for by our choice of

shift schedules, and actual hospital goals. Next, we briefly discuss possible sources

of modeling error. Finally, we consider several studies contradicting the basic as-

sumptions motivating shorter shifts, and discuss some plausible interpretations of

seemingly contradictory evidence.

The goals of medical residency programs include (a) delivering high quality pa-

tient care, (b) providing residents with a good educational experience, (c) providing

residents with a safe work environment and healthy work life balance, and (d) pro-

viding services economically. Our study does not consider the effect of shift schedules

on (b), although a recent study found that residents perceived that the post reg-

ulation shift schedules left less time for education [27]. For (d), schedules with a

greater capacity correspond to requiring fewer total interns to treat a fixed number

of patients, suggesting that shorter more frequent shifts are financially more viable.

Furthermore, jeopardy admissions are costlier than night float admissions. Therefore,

since schedules with shorter more frequent shifts reduce jeopardy admissions while

increasing night float admissions, such schedules are more cost effective. However,

as we do not explicitly account for the increased expense of additional night floats

required, a more careful analysis is warranted. Additionally, we are ignoring indirect

changes in expenses, e.g. if quality of care is increased, some expensive accidents

may be avoided. In [62], a more comprehensive estimate of the cost for restricting

duty hours was estimated, but at a less granular scale (i.e. costs for the national res-

idency program, not a single hospital). Regarding (a) and (c), in light of the studies

referenced in the introduction, one might assume that a schedule relying on shorter

shifts that allows residents to sleep more regularly and have more sleep hours in total

should improve the quality of care (due to a reduction in errors caused by fatigue),

reduce occupational hazards for residents, and improve resident quality of life. This

opinion was held by some residents prior to implementing the 2011 regulations [26],

150

although there was a strong concern that a loss in continuity of care would cause

more accidents than those prevented by reducing fatigue. Our study addresses (a)

by quantifying the change in continuity of care when using schedules with shorter

more frequent shifts, focusing on the first 24 hours of a patient’s stay in the hospital

(e.g. as measured by reassignments and frequency of six hours of observation after

admission), but makes no further attempt to quantify the benefits for (a) and (c) in

using shorter shifts due to reduced fatigue.

Next, we briefly discuss potential sources of modeling error. In particular, we

note that our simulation model is an extreme simplification of how hospitals actually

operate. As a result, our conclusions may only be valid under certain assumptions

implicit in our model. We quickly give two such examples. First, we assume that the

only bottleneck in admitting patients is the availability of doctors. However, in an

actual hospital setting, there are many potential bottlenecks to admitting patients,

including beds, nurses, and specialized equipment. We made this assumption because

the practitioners in B&W suggested that this was correct up to first order in B&W

in 2009-11. However, if the patient load increased 20% as in some of our sensitivity

analysis, most likely another resource would become a bottleneck. Second, we con-

sidered the distribution of arrival times and departure times exogenous. However,

adjusting schedules would likely have some effect on both. A small but non-negligible

fraction of patient arrivals are scheduled by doctors at their own will, so changing

their hours would likely change their incentives when scheduling patients. Addition-

ally, patients may adapt to avoid waiting times if under some schedules, arriving a

certain time of day often resulted in long waits. Departure times are determined by

when doctors choose to discharge patients, so again, altering their schedules could

have unintended consequences. Thus, we suggest using some caution when applying

these results, particularly against expecting certain quantitative outcomes.

Several recent studies of the aftermath of the 2011 regulation have challenged

some of the assumptions we adopted on a number of grounds. In [27, 76], the authors

find (by issuing surveys to residents) that after the change in regulation, residents are

on average not getting much more total sleep, some residents are violating new duty

151

hour restrictions, more medical accidents take place, and the rates of depression and

burnout are not lower. There are many possible explanations as to why the authors

may have reached these somewhat counter-intuitive conclusions. For example, if

the duty hour restrictions were not enforced, we would not expect any improvement

in resident quality of life. Instead, if duty hours were enforced but hospitals were

not properly prepared to adapt operationally, residents might experience additional

stress on the job and have more errors due to general disorganization in the new

system (particularly in the handoff protocols). Finally, as these studies were based

on surveying residents and physicians perceptions, they are subject to bias in the

perceptions of the participants over something which is an increasingly political issue.

These difficulties, whether perceived or actual, may improve with time as hospitals

adapt their operations to improve handoffs, and more senior residents, who may feel

victimized by the changes in regulations (which increased their work load to reduce the

workload on first year residents) graduate from the residency program, as suggested

in [35] after the 2003 regulations. Additionally, one should note that there are a few

new studies challenging the notion that fatigue results in worse medical outcomes

[30, 84]. However, these studies were performed as retrospective analyses, with many

potential confounding factors that could lead to erroneous conclusions.

5.5 The Markov Chain Throughput Upper Bound

In this section, we give a probabilistic analysis of the long run average rate that

a resident or team of residents can treat patients, as used to compute the Markov

chain throughput upper bound in Section 5.2.3. Using this tool, we give mathematical

justification for why a schedule that has short, frequent, and evenly spaced shifts

such as MMMO should have a greater capacity than a schedule with fewer, longer,

unevenly spaced shifts such as LOSO. First, we consider a simple model where a

single resident that only has the capacity to treat one patient at a time must treat an

infinite incoming stream of patients. We consider two schedules for the resident and

compute the long run average number of patients treated under each schedule. In this

152

special case, we prove that short evenly spaced shifts have a greater capacity than

long shifts. Then, we generalize the model and provide an algorithm to compute the

maximum rate that the residents and PAs of an entire department can treat patients.

In the basic model, we have an infinite number of time periods t = 1, 2, . . ., each

representing a single day. The resident has a four day rotating schedule that specifies

if the resident is on shift or off shift. We consider two schedules for the resident.

The first, which we refer to as Consecutive (C), is On, On, Off, Off, and the second,

which we refer to as Spread (S) is On, Off, On, Off. At the start of each day, if the

resident is on shift and does not currently have a patient in care, the resident takes

this new patient. Then each day, regardless of whether or not the resident is on shift,

the patient in care leaves with probability p (a parameter of the model, 0 < p < 1),

and stays otherwise. We now determine the long run average number of patients that

our resident will treat under schedules (S) and (C). Under policy (S), for each day

t = 1, 2, . . ., we define DS(t) be one if a patient departed in time period t and zero

otherwise. Likewise, under policy (C), for each time period t = 1, 2, . . . , we define

DC(t) to be one if a patient departed in time period t and zero otherwise. Using this

notation, the long run average number of patients treated by our resident under each

policy, denoted DS(∞) and DC(∞), is given by

DC(∞) = limT→∞

1

T

T∑t=1

DC(t)

DS(∞) = limT→∞

1

T

T∑t=1

DS(t).

By the law of large numbers, we can compute the long run average behavior by instead

directly computing the average behavior in a four day cycle, namely,

DC(∞) = E

[1

4

4∑t=1

DC(t)

]=

1

4

4∑t=1

E[DC(t)] =1

4

(p+ p+ (1− p)p+ (1− p)2p

),

DS(∞) = E

[1

4

4∑t=1

DS(t)

]=

1

4

4∑t=1

E[DS(t)] =1

4(p+ (1− p)p+ p+ (1− p)p) .

153

We briefly justify this calculation. Any day the resident is on shift, the probability of

discharging a patient will be p, independent of the past. Likewise, if a resident was

on shift the previous day, and is off shift today, then the probability of discharging a

patient can be computed as P(no discharge yesterday)P(discharge today) = (1− p)p,

independent of events occurring greater than one day ago. By induction, it is simple

to show that if a resident last was on shift n days ago, then the probability of discharge

today is given by (1−p)np, and is independent of events occurring earlier than n days

previous. Observe that the terms E[DC(1)] and E[DC(2)] are equal to the probability

of discharging a patient while on shift, and the terms E[DC(3)] and E[DC(4)] are equal

to the probability of discharging one day after being on shift and two days after being

on shift, respectively. This justifies the calculation for DC(∞). The computation of

DS(∞) is justified similarly.

We observe that for every possible value of p (the probability patients depart each

day),

DS(∞)−DC(∞) =1

4(1− p)p− 1

4(1− p)2p =

1

4(p2 − p3) > 0,

i.e. DS(∞) > DC(∞).

From this calculation, we can conclude the resident will in the long run treat more

patients under (S) than under (C).

We now extend the model so that residents can treat multiple patients simulta-

neously, but must obey a capacity on the maximum number of patients in care and

the maximum number of patients admitted per day (according to a schedule). Again,

we have an infinite number of time periods t = 1, 2, . . ., each corresponding to a day.

For now, we will restrict ourselves to the schedules from Section 5.2.3 where the (C1)

constraint (team patients in care capacity) is dominated by (C3) constraint (individ-

ual patients in care capacity), i.e. LOSO and MMMO. We will let θ denote such a

schedule. Let cθ denote the maximum number of patients in care for a resident. Let

aθ(t) be the maximum number patients that can be admitted on day t under policy θ.

For example, under LOSO, aθ(1) = 7, aθ(2) = 0, aθ(3) = 3, aθ(4) = 0, aθ(5) = 7, . . ..

For t = 0, 1, . . . ,, let Xθ(t) be the number of patients in care in time period t. Finally,

154

let Dθ(t) be the number of patients that depart in period t. We will assume that

Dθ(t) has the distribution Bin(Xθ(t − 1), p), i.e. each patient in care departs with

probability p, independently of the other patients, as in the previous section. We

assume that an initial condition Xθ(0) is given, and then the dynamics of our model

are given by

Xθ(t) = minXθ(t− 1) + aθ(t)−Dθ(t), c

for t = 1, 2, . . .. When aθ(t) is on an n day rotating schedule, then Xθ(t) is a Markov

chain with period n on the state space 0, 1, . . . , c. It is easy to see that the process

Xθ(nt), t = 0, 1, 2, . . . , will be a time homogeneous Markov chain on the same state

space with a single recurrent class. Thus we can solve for the long run behavior

of Xθ(nt) by computing the stationary distribution, i.e. solving a system of c + 1

linear equations. Once the stationary distribution is known, it is straight forward to

compute the expected number of departures per n day rotation (and thus per day).

To improve the accuracy of our results, we make the following enhancement to the

model. For shifts that do not occur on weekends, we make aθ(t) random, taking the

value zero with probability 2/7, and the maximum number of admissions otherwise.

While it would be more accurate to make the day of the week as well as the day in

the n day cycle both part of the state, it would increase the number of states from

ncθ to potentially 7ncθ with little practical gain.

Finally, we discuss the case when constraint (C1), resident team capacity, is not

dominated by constraint (C3), individual resident capacity. When this happens, on

our particular problem instances the (C1) constraint is much more restrictive than the

(C3) constraint. Therefore it is a reasonable approximation to simply ignore the (C3)

constraint and view the entire team as a single resident with the combined admitting

schedule of all residents on the team. This technique works very well for D’OSO

where the team capacity is 15 patients but the individual capacity is ten patients,

and there are two residents on a team with no offset in their schedules (i.e. they

both have “D”’ on the same days). However, for other schedules where the residents

are staggered, e.g. LOOSSO, things are a little more complicated. We would to say

155

Schedule Residents cθ aθ(1) aθ(2) aθ(3) aθ(4) aθ(5) aθ(6) aθ(7) aθ(8) aθ(9)

LOSO 1 10 7 0 3∗ 0MMMO 1 10 4 4∗ 4 0LOOSSO 3 20 10 7 7 3∗ 3∗ 3∗

D’OSO 2 15 10 0 6∗ 0DOOONO 3 20 10 5 5 0 5 5DOSOOONOO 3 20 5 5 8 3 3 0 5 5 5

Table 5.9: The values of cθ and aθ(t) used for each schedule for GMS and Cardiology.Values of aθ(t) marked with ∗ take the value zero with probability 2/7, to approximatethat some shifts are skipped on weekends.

Schedule Residents cθ aθ(1) aθ(2) aθ(3) aθ(4)

LOSO 1 10 5 0 1∗ 0MMMO 1 10 3 3∗ 3 0M’M’M’O 1 10 3 3∗ 3 0

Table 5.10: The values of cθ and aθ(t) used for each schedule for Oncology. Valuesof aθ(t) marked with ∗ take the value zero with probability 2/7, to approximate thatsome shifts are skipped on weekends.

that the capacity is [7, 0, 0, 3, 3, 0] + [0, 7, 0, 0, 3, 3] + [3, 0, 7, 0, 0, 3] = [10, 7, 7, 3, 6, 6].

However, on the first day, when it falls on weekend, we only lose part, not all of

the capacity. While this could be correctly accounted for, instead, for simplicity,

when this situation occurs, we just assume that the admitting capacity equals ten,

regardless of whether it is a weekday or weekend.

We use this technique to make the Markov chain throughput upper bound for res-

idents and teams of residents. We omit the details of the calculations but summarize

the inputs to the calculation in Table 5.9 for GMS and Cardiology and Table 5.10

for Oncology, and the results of the calculation in Table 5.11 for GMS, in Table 5.12

for Cardiology, and in Table 5.13 for Oncology. Note that the “Residents” column

indicates the number of individuals that computation is for. Using these quantities,

it is straightforward to compute the Markov chain throughput upper bound for each

policy, as you only need to multiply the estimate for a resident (or team of resident)

by the number of residents (teams of residents), and then add in the PA capacity, as

given in Table 5.14 (multiplied by the capacity of the PA service divided by 15).

Another improvement to the model would be to replace the infinite stream of

patients waiting to be treated by a random number of patients arriving each day.

156

Throughput Upper Bound(Patients/Day)

Schedule Residents Capacity Admitting Markov

LOSO 1 2.25 2.29 1.78MMMO 1 2.25 2.71 2.00LOOSSO 3 4.50 5.79 4.17D’OSO 2 3.37 3.57 2.75DOOONO 3 4.50 5.00 3.26

Table 5.11: The long run average number of departures per day under each schedulefor GMS.

Throughput Upper Bound(Patients/Day)

Schedule Residents Capacity Admitting Markov

LOSO 1 2.13 2.29 1.71MMMO 1 2.13 2.71 1.91DOSOOONOO 3 4.26 4.14 3.61

Table 5.12: The long run average number of departures per day under each schedulefor Cardiology.

Throughput Upper Bound(Patients/Day)

Schedule Residents Capacity Admitting Markov

LOSO 1 1.58 1.43 1.23MMMO 1 1.58 2.04 1.44

Table 5.13: The long run average number of departures per day under each schedulefor Oncology.

Department Average Patient Stay Throughput of Capacity 15 PA Team(days) (patients/day)

GMS 4.44 3.37Cardiology 4.70 3.19Oncology 6.32 2.37

Table 5.14: Average patient length of stay by department and corresponding PAthroughput.

157

Then, as in reality, the resident would occasionally be idle. Analyzing such a model is

significantly more complicated than the above analysis, and is the subject of Chapter

6. However, the a key finding that chapter is essentially as follows: Suppose that each

day the average arrival rate of patients is λ. Then the number of patients waiting to

be treated will increase to infinity as t→∞ if and only if λ exceeds the throughput

of the policy.

5.6 Statistical Analysis of Patient Flows

5.6.1 Patient Arrival and Departure Data

The patient data we use is from the B&W Oncology, Cardiology and General Medicine

departments. Recall that a resident or PA is assigned to a patient, and in turn

each patient is assigned to a bed. At B&W, these events occur simultaneously. For

about 75% of patients, we have this exact time. The remaining 25% of patients

were transfers from other hospitals. For these patients, we use as a proxy the time

the patient reached the bed, as for transfer patients these times are relatively close.

For all patients, we know the time the patient was discharged. Each of the three

departments has specialist teams that instead of taking any new arriving patient,

only take new patients with a particular ailment, and take all such patients. For

example, General Medicine has two PA teams, “REN-PA” for patients with renal

failure and “PUL-PA” for patients with chronic pulmonary disease. As such patients

are treated separately in an independent system, we excluded them from our model.

5.6.2 A Statistical Model for Patient Arrivals

In order to simulate a schedule with a patient loads above the historical patient load,

we need to create additional patients to augment the historical data. For each of the

three departments we considered, we used the same procedure described below. We

assume that the patient arrival process is a non-stationary Poisson process, where for

each hour of the day, the rate is constant. We let λhn denote the arrival rate for hour

158

h for day n of the simulation. Let d(n) be the day of the week for simulation day n

and let m(n) be the month of the year for simulation day n. Further, we assume that

for each hour of the day h, each day of the week d, and each month of the year m,

there exists constants λh, λd and λm such that

λhn = λh + λd(n) + λm(n).

We produce estimates of our unknowns λh of λh for each hour h, λd of λd for each day

d, and λm of λm for each month m by solving the regularized least squares problem

below. Let ahn denote the actual number of arrivals we observed in hour h of day n.

We choose a small constant ε > 0 to be our regularizer, and then solve

minλh,λd,λm,λhn

365∑n=1

24∑h=1

(ahn − λhn)2 + ε

(24∑h=1

λh +7∑d=1

λd +12∑m=1

λm

)subject to λh + λd(n) + λm(n) = λhn h = 1, . . . , 24, n = 1, . . . , 365.

The regularization is necessary only for technical reasons and the solution was rela-

tively insensitive to the choice of ε. The optimization problem can be solved efficiently

by most commercial packages for scientific computing (e.g. SciPy).

To assure the reader that the patient arrival rate does depend on all three of these

variables and allow the reader the visualize these relationships, we plot the average

number of patients per hour, the average number of patients per day of week, and

the number of patients per month of the year using the 2009-10 data for Oncology

(Figure 5-9), Cardiology (Figure 5-10) and GMS (Figure 5-11).

Given these estimators, to increase the arrival rate by 5%, we would generate a

non stationary Poisson process with a fixed rate in each hour of the day h and each

day of the simulation n of .05 · λhn. We would then add these artificial arrival times

to the historical data.

We use a simpler technique to reduce the arrival rate by 5%. We simply select

5% of the patients at random from the historical data, and remove them from the

simulation.

159

12am 6am12pm

6pm12am

0

0.5

1

1.5

Hour of Day

Arr

ival

sP

erH

our Oncology Hourly

S M T W R F S S

68

101214

Day of Week

Arr

ival

sP

erD

ay

Oncology Daily

Jan Apr Jul Oct Jan

280300320340360

Month of Year

Arr

ival

sP

erM

onth

Oncology Monthly

Figure 5-9: For the Oncology Department in 2009-10, the average number of arrivalsby hour of day, day of week, and month of year.

12am 6am12pm

6pm12am

0

0.5

1

1.5

Hour of Day

Arr

ival

sP

erH

our Cardiology Hourly

S M T WR F S S5

10

15

Day of Week

Arr

ival

sP

erD

ay

Cardiology Daily

Jan Apr Jul Oct Jan

380

400

420

440

Month of Year

Arr

ival

sP

erM

onth

Cardiology Monthly

Figure 5-10: For the Cardiology Department in 2009-10, the average number of ar-rivals by hour of day, day of week, and month of year.

12am 6am12pm

6pm12am

0.5

1

Hour of Day

Arr

ival

sP

erH

our GMS Hourly

S M T WR F S S14

16

18

20

Day of Week

Arr

ival

sP

erD

ay

GMS Daily

Jan Apr Jul Oct Jan

500

550

600

Month of Year

Arr

ival

sP

erM

onth

GMS Monthly

Figure 5-11: For the GMS in 2009-10, the average number of arrivals by hour of day,day of week, and month of year.

160

5.6.3 A Statistical Model for Patient Departures

For the synthesized patient data from the previous section, we need a statistical model

to determine their length of stay in the hospital. Here it is problematic to assume

any parametric form. In Figure 5-12, we see a sinusoidal pattern with a period of one

day in the length of stay. Looking closely, splitting our patients by the hour of the

day that they arrived, we see that while the length of stay is dependent on the arrival

time, the time of day of departure is not. Specifically, regardless of when a patient

arrived, they will typically leave around noon. However, the number of days a patient

stays in the hospital appears to depend on the time of day arrived. For example, a

patient arriving late at night is unlikely to depart the following day at noon.

To systematically test the relationship between the arrival hour of day, day of

week, and month of year with the patient length of stay, we used a regression model

similar to the model used in the previous section. We found that the hour of day

and day of week were strongly correlated to the patient length of stay, but that the

month of year was not particularly relevant.

To actually generate the random length of stay each synthesized patients, instead

of fitting our data to some parametric distribution and then sampling, as we did in

the previous section, we instead draw from the empirical distribution in our data.

Specifically, if we had a synthesized arrival in hour h and day of week d, we looked at

all patients that arrived in the same hour of the day on the same day of the week, and

picked one of their length of stays uniformly at random. Because we had hundreds

of data points for every hour day pair and were generating relatively few points, this

was a reasonable approach. Had we not ruled out the month of year dependence, we

would have had insufficient data points for this method to give reliable results.

Finally, we describe the statistical test used to determine that the month of the

year was not an important factor in determining the length of stay for patients, but

that the hour of the day and the day of the week were important. We assumed that

when a patient arrived on day n during hour h, they would have an average length

of stay of `hn hours Using the same notation as in the previous section, we assumed

161

that there were constants `h, `d(n) and `m(n) such that

`hn = `h + `d(n) + `m(n).

Then for each patient i = 1, . . . , k, where k is the number of patients treated in the

department for a year, let ai be the actual length of stay for that patient, hi be the

hour of day of the patient arrival, ni be the day of the year of the patient arrival,

mi be the month of year of patient arrival, and di be the day of week of the patient

arrival. We then computed our estimates by solving

minˆh,ˆd,ˆm,ˆhn

k∑i=1

(ahini − ˆhini)

2 + ε

(24∑h=1

ˆh +

7∑d=1

ˆd +

12∑m=1

ˆm

)subject to ˆ

hi + ˆd(ni) + ˆ

m(ni) = ˆhini i = 1, . . . , k,

Running the test, we observed that the influence of the month term in determining

ˆhn was an order of magnitude smaller than the day of week term or hour of day term

and could safely be ignored. The hour of day term was more important than the day

of week term. There was some discrepancy in the day of week terms was between

weekdays and weekends.

162

24 48 72

0

5

Hours since arrival

Pat

ient

dep

artu

res

Monday 7pm arrivals

0 24 48 72

0

2

4

6

Hours since arrival

Pat

ient

dep

artu

res

Friday 7pm arrivals

0 24 48 72

0

1

2

3

Hours since arrival

Pat

ient

dep

artu

res

Monday 10am arrivals

24 48 72

0

2

4

6

Hours since arrival

Pat

ient

dep

artu

res

Monday 11pm arrivals

Figure 5-12: The distribution for patients’ length of stay is heavily influenced by thetime of day the patient arrives, as patients all tend to leave the hospital at midday.Patient length of stays exceeding 100 hours not shown above.

163

164

Chapter 6

Queuing and Fluid Models for

Scheduling Medical Residents in

Hospitals

6.1 Introduction

Major hospitals face a difficult challenge of designing shift schedules for their resi-

dents that satisfy demand, provide quality care, and are compliant with regulations

restricting shift lengths. Motivated by empirical work conducted by the authors at

the Brigham and Women’s (B&W) Hospital in Boston, we analyze the impact of shift

lengths on two key performance metrics. The first metric is admitting capacity—the

largest patient arrival rate sustainable by a given shift schedule. The second metric

is the number of reassigned patients—the number of patients admitted temporarily

by one doctor and then permanently transferred to a resident.

We build a queueing model to compare two shift scheduling policies that are

representative of the alternatives encountered in hospitals: one where residents work

long shifts on alternating days, called Long Shifts (LS), and another where residents

admit patients daily in short shifts, called Daily Admitting (DA). We determine the

admitting capacity for our queueing model under each policy. Then we construct a

165

fluid model—a large scale approximation of the underlying queueing model. We show

that for each policy, the fluid model has a unique steady state solution. Finally, we

establish an interchange of limits between the stochastic and fluid models in steady

state. We use these results to compare the key performance metrics under the two

policies.

Our analysis shows that the DA policy has a greater capacity to admit patients

than the LS policy for all parameter choices. Furthermore, we numerically establish

the existence of a threshold value, such that the number of reassigned patients is

smaller for the DA policy than for the LS policy if and only if the arrival rate of

patients is greater than the threshold value. Since most hospitals operate at near

critical loads, our two findings lead to the conclusion that schedules which rely on

shorter more frequent shifts than those found in practice would increase admitting

capacity and reduce the number of reassigned patients.

Organization

The remainder of this chapter is organized as follows. The queueing model and its

fluid limit are described Section 6.2 and the main results are stated there. In Section

6.3 we numerically solve for the steady state behavior of the fluid model and discuss

the performance implications for our queuing model. Then we give some concluding

remarks in Section 6.4. The proofs of the main results are in the following sections.

In Section 6.5, we exactly characterize the stability of our queueing model under each

policy using a simple linear Lyapunov function type argument. In Section 6.6, we

use quadratic Lyapunov functions to bound the expected steady state queue length.

In Section 6.7, we prove the existence of the fluid limits, applying the results in

[60]. Then in Section 6.8, we show that the fluid limit has a consistent periodic long

run behavior under each policy, where the solution in each period is characterized

by a simple system of differential equations. In Section 6.9, we prove that the long

run solution to the fluid model approximates the steady state queue lengths of the

underlying queueing model. Justifying this requires an argument for an “interchange

of limits.” As in [38], we use our moment bound from Section 6.6 to show tightness

166

of the rescaled stationary distributions, and then we follow the technique of [31] and

similarly [79] to prove the interchange of limits. In Section 6.10, we use the result

of Section 6.9 to show that the long run number of daily reassignments converges

in the fluid rescaling converges to a natural function of the fluid limit. Finally, we

have two rather technical sections: Section 6.11, where we show several elementary

properties of the solution to a differential equation, and Section 6.12, where we use

another Lyapunov function argument to distinguish between the null recurrent and

transient cases in our queueing model.

Summary of Notation

We conclude with a summary of the mathematical notation used in the chapter.

Throughout, R (R+) denotes the set of (nonnegative) reals, and likewise, Z (Z+)

denotes the set of (nonnegative) integers. For a vector x ∈ Rn, ‖x‖p = (∑n

i=1 xpi )

1/p

is the `p-norm. The `1 ball of radius r is denoted Br(x) = y ∈ R2 | ‖x − y‖1 <

r. For x, y ∈ R, x ∧ y = minx, y and (x)+ = maxx, 0. We define f(t−) as

limτt f(τ) when the limit exists. We let Exp(µ), Pois(λ), and Bin(n, p), denote an

exponential random variable with mean 1/µ, a Poisson random variable with mean

λ, and a Binomial random variable with mean np and variance np(1−p), respectively

(these moments characterize the distributions). If the sequence of random vectors Xn,

n = 1, 2, . . . , converges weakly (in distribution) to X as n → ∞, we say Xn ⇒ X.

For a stochastic process X(t) in either discrete or continuous time, Ex[X(t)] denotes

E[X(t) | X(0) = x]. A sequence of continuous time vector valued stochastic processes

Xn(t) on a common probability space Ω converges almost surely (a.s.) and uniformly

on compact sets (u.o.c.) to a deterministic function x(t) if for every t > 0 and almost

every ω ∈ Ω,

limn→∞

sup0≤s≤t

‖Xn(s, ω)− x(s)‖1 = 0,

where ‖ · ‖1 is the 1-norm for vectors. See [19] for more details. As in [28] sections

11.2-11.3, for functions f : Rn → R, we let ‖f‖L = supx,y∈Rn |f(x)− f(y)|/‖x− y‖1

167

denote Lipschitz semi-norm (when f is a Lipschitz function, the value of this norm is

the smallest Lipschitz constant that f satisfies). We let ‖f‖BL = ‖f‖L + ‖f‖∞. This

quantity is a true norm.

6.2 Model, Assumptions and Main Results

We begin by introducing our model of residents admitting and treating a flow of in-

coming patients. The patients are assumed to arrive according to a non-homogeneous

Poisson process. For each k ∈ Z+, the process has rate λ1 over the time intervals,

[k, k+½) and rate λ2 < λ1 over the time intervals [k+½, k+1). Let let λ = (λ1 +λ2)/2

denote the average arrival rate and

λ(t) =

λ1 t ∈ [k, k + ½),

λ2 t ∈ [k + ½, k + 1).

The intervals [k, k + 1) represent, for example, 24 hour cycles, where [k, k + ½) is the

portion of the day, say from 10am to 10pm, in which the vast majority of patients

arrive (see Figure 5-9). The residents are combined into two teams, A and B, identical

in size, which are eligible to admit patients (are on shift) according a schedule to be

described below. Each team has capacity c > 0 bounding the maximum number

of patients the team can have in care. Each arriving patient is assigned to one of

the residents on a team, chosen uniformly at random, provided that at least one

of the teams on shift has not reached its capacity c. Note, that this is equivalent

to saying that each resident has in care capacity c/N , where N is the number of

residents on each the team. If each on shift teams has reached its capacity, the

patient joins a single queue and is cared for by one of the back-up doctors until one of

the residents is available, at which point the waiting terminates, using the First-In-

First-Out assignment policy. The availability occurs either when one of the assigned

patients leaves the hospital freeing the capacity of one of the teams, or when one of

the teams with load less than c begins a shift.

168

At any time, each team is in one of two states, on shift or off shift, as specified

by a policy. Patients remain assigned to a team until they leave the hospital. The

durations of hospital stays are assumed to be i.i.d. and exponentially distributed

with rate µ. That the random length of treatment time each patient requires begins

accumulating at the moment of assignment to a team, and continues to accumulate

when team is off shift. How this corresponds to actual practices is explained in the

introduction.

We consider two scheduling policies controlling when each team is on shift, Long

Shifts (LS) motivated by LOSO, and Daily Admitting (DA), motivated by MMMO

(see the introduction for descriptions of LOSO and MMMO). LS is a two day rotating

schedule. Team A is on shift on odd days, i.e. [2k + 1, 2k + 2) for all k ∈ Z+, and

off shift otherwise. Similarly, team B is on shift for even days, i.e. [2k, 2k + 1) for

all k ∈ Z+, and off otherwise. In DA, both teams A and B are on shift every day for

the first half of each day, i.e. [k, k + ½) for all k ∈ Z+, and off otherwise, effectively

creating a single team with double the capacity.

To state our results, it will be convenient to introduce the following quantities

describing the dynamics of our model. For t ≥ s ≥ 0, let A(s, t) denote the number

of patients that arrive in the time interval [s, t] according to our non-homogeneous

Poisson process. For each policy θ ∈ LS,DA, let Qθ(t) denote the number of

patients in the queue not yet assigned to a team plus the number of patients assigned

to the teams that are on shift at time t. For each θ ∈ LS,DA, let Rθ(t) denote

the total number of patients currently assigned to teams which are off shift at time t.

Further, for each θ ∈ LS,DA we introduce the random vector Sθ(t) = (Qθ(t), Rθ(t)),

which we take to be the state of our system. The processes Qθ(t), Rθ(t), and Sθ(t)

are assumed to be right-continuous with left limits. For every s < t, we let Donθ (s, t)

denote the total number of patients which departed in the time interval [s, t] from

the teams which were on shift in this period. We define Doffθ (s, t) analogously.

169

Under the policy LS, for each k ∈ Z+ and each t ∈ [0, 1), QLS and RLS satisfy

QLS(k + t) = QLS(k) + A(k, k + t)−DonLS(k, k + t), (6.1)

RLS(k + t) = RLS(k)−DoffLS(k, k + t). (6.2)

On [k, k + 1), we see that in distribution QLS(t) behaves exactly as the total number

of customers in system for an M(t)/M/c queue with arrival rate λ(t), service rate

µ, and initial value QLS(k). Similarly, on [k, k + 1), RLS(t) behaves as an M/M/c

system with no arrivals, service rate µ, and initial value RLS(k). At each time k ∈ Z+

a transition occurs: the off shift team switches to on shift and vice versa, and patients

that are waiting can get assigned to the team rotating on shift. In terms of QLS and

RLS, this can be described as follows:

QLS(k) = (QLS(k−)− c)+ +RLS(k−),

RLS(k) = QLS(k−) ∧ c.

We define operator Γ : R2+ × R+ → R2

+ by

Γ(q, r;κ)∆= (q − κ)+ + r, q ∧ κ), (6.3)

and note that we can equivalently write

SLS(k) = Γ(SLS(k−); c). (6.4)

The equations (6.1), (6.2) and (6.4) along with a distribution of the initial states

SLS(0) completely determine the distribution of SLS(t) for all t ∈ R+. It is immediate

that on integer times, the process SLS(k) is a two dimensional Markov chain on the

countable state space Z+ × 0, 1, . . . , c. Moreover, without loss of generality we can

restrict the state space to the set,

SLS∆= 0, . . . , c2 ∪ (q, c) | q ∈ Z+, (6.5)

170

as this set is the image of Γ(·; c) when we take the domain to be Z+ × 0, 1, . . . , c.

Thus SLS(k) ∈ SLS for all k ≥ 1 with probability one.

We now describe similar relations for the policy DA. For every k ∈ Z+ and every

t ∈ [0, ½),

QDA(k + t) = QDA(k) + A(k, k + t)−DonDA(k, k + t),

RDA(k + t) = 0,(6.6)

where on [k, k + ½), the process QDA(t) has the same distribution as an M/M/2c

queue with an arrival rate of λ1, a service rate of µ and an initial value of QDA(k).

At time k + ½, both teams move off shift, resulting in the transition

QDA(k + ½) = (QDA((k + ½)−)− 2c)+,

RDA(k + ½) = QDA((k + ½)−) ∧ 2c.

Equivalently, in terms of Γ,

SDA(k + ½) = Γ(SDA((k + ½)−); 2c). (6.7)

On [k + ½, k + 1),

QDA(k + ½ + t) = QDA(k + ½) + A(k + ½, k + ½ + t),

RDA(k + ½ + t) = RDA(k + ½)−DoffDA(k + ½, k + ½ + t).

(6.8)

Now on [k + ½, k + 1), QDA(k + ½ + t) changes according to a Poisson process with

arrival rate λ2, and R(t) is an M/M/2c queue with no arrivals and service rate µ. At

integer times, we have a second shift change, this time leading to

QDA(k + 1) = QDA((k + 1)−) +RDA((k + 1)−),

RDA(k + 1) = 0.

171

Equivalently, in terms of Γ,

SDA(k + 1) = Γ(SDA((k + 1)−); 0). (6.9)

Equations (6.6), (6.7), (6.8) and (6.9) along with the distribution over the initial

states SDA(0) determine the distribution of SDA(t) for all t ∈ R+. Again on integer

times, the process SDA(k) is a Markov chain on the countable state space. However,

now the state space is the one-dimensional set

SDA∆= Z+ × 0, (6.10)

as RDA(k) = 0 for all k ∈ Z+.

We are now ready to discuss our main results. We define the stochastic process

Sθ(t) to be stable if the embedded discrete time process Sθ(k) for k ∈ Z+ is positive

recurrent, and unstable otherwise. Normally we define stability for this type of prob-

lem as positive Harris recurrence of the process Sθ(t), t ∈ R+. However, it is easy

to see that in our case these definitions are equivalent, and further that the former

definition is much easier to work with.

Observe that under both policies, Rθ(k) is bounded hence Qθ(k) is the only po-

tential source of instability. Thus when Sθ(t) is unstable, with probability one, the

number of patients waiting to be assigned to a resident team will grow without bound.

Note that in reality, when a hospital has a large number of patients waiting, it reroutes

incoming patients to other hospitals to reduce congestion, so instability would actu-

ally correspond to the hospital frequently being forced to turn patients away—clearly

a very undesirable situation.

We now discuss conditions under which Sθ(t) is stable. Before formally stating

our results, we provide some intuition. Let Lc(t) and L2c(t) be the number of patients

in system for an M(t)/M/c queue and M(t)/M/2c queue both driven by the arrival

process A(0, t), respectively. For LS, it is not difficult to see that we can couple SLS(t)

172

with Lc(t) and L2c(t) such that surely, for every t,

L2c(t) ≤ QLS(t) +RLS(t) ≤ Lc(t).

The inequalities hold as the process SLS(t) has capacity between c and 2c at all times

t. Similarly, we can couple SDA(t) and L2c(t) such that

L2c(t) ≤ QDA(t) +RDA(t).

Recall from basic queueing theory that the process Lc(t) is positive recurrent iff λ < cµ

and L2c(t) is positive recurrent iff λ < 2cµ. In light of our coupling, we thus expect

the maximum throughput (the largest λ = (λ1 + λ2)/2 such that Sθ(t) is stable) of

SLS(t) to lie between cµ and 2cµ, and likewise we expect the maximum throughput

of SDA(t) to be at most 2cµ. This suggests that we need to determine to what extent

each policy can utilize the 2c total capacity available to treat patients, or conversely

how much forced idling is caused under each policy by a team’s inability to admit

new patients when off shift. To this end, we let

ρLS∆=

λ

c(1− e−µ) + cµ, (6.11)

ρDA∆=

λ

2c(1− e−µ/2) + cµ. (6.12)

We intend to show that Sθ(t) is positive recurrent iff ρθ < 1 for each θ. These values

of ρθ imply that

λ∗LS∆=

λ

ρLS

= c(1− e−µ) + cµ, λ∗DA∆=

λ

ρDA

= 2c(1− e−µ/2) + cµ,

give the maximum throughput of SLS(t) and SDA(t), respectively. Thus our main

stability result is as follows.

Theorem 6.1. For each θ ∈ LS,DA, the process Sθ(t) is positive recurrent when

ρθ < 1, null recurrent when ρθ = 1, and transient when ρθ > 1. Namely, the process

173

Sθ(t) is stable iff ρθ < 1. Furthermore, ρDA < ρLS. In particular, the Daily Admitting

policy has a greater maximum throughput.

The intuition behind the result is that the queue will be stable as long as condi-

tional on the queue being large, the expected number of arrivals per day is less than

the expected number of departures per day. Independent of the initial queue length,

we expect λ arrivals per day. For the sake of argument, assume the initial queue

length were infinite, so the resident teams are only idle when they are off shift and

have completed caring for their initial patients.

Under the policy LS, in a single day the team on shift has c patients in care at all

times each recovering at rate µ, producing Pois(cµ) departures. Thus the expected

number of departures from the team on shift is cµ. For the team off shift, as we

assumed there were infinitely many patients initially in care, we begin with all c

capacity utilized. Again patients depart at rate µ, but now when they leave they are

not replaced. The probability a patient will depart is P(Exp(µ) ≤ 1) = 1 − e−µ. As

whether or not each patient departs is independent, we have Bin(c, 1−e−µ) departures,

so the expected number of departures from the team off shift is c(1− e−µ). Thus the

expected change for the number of patients in the system is given by

−γLS∆= λ− cµ− c(1− e−µ).

Recalling that we expect the system to be stable when γLS > 0, we see from (6.11)

that this is equivalent to ρLS < 1. Performing a similar computation for DA, we see

that in a single day there are Pois(cµ) on shift departures and Bin(2c, 1 − e−µ/2) off

shift departures, giving an expected change in the number of patients in system of

−γDA∆= λ− cµ− 2c(1− e−µ/2).

Thus the queue should be stable if γDA > 0, or equivalently from (6.12), when ρDA < 1.

174

To show ρDA < ρLS, it suffices to show that 2− 2e−µ/2 > 1− e−µ, which follows since

1− 2e−µ/2 + e−µ = (1− e−µ/2)2 > 0. (6.13)

As cµ + 2c(1 − e−µ/2) < 2cµ, we see that DA still results in fewer expected

departures than an M/M/2c queue. However, if we are willing to consider schedules

with more shift changes per day, we can achieve an expected number of departures

arbitrarily close to our “upper bound” of 2cµ by generalizing the policy DA. Given

k > 0 integer and even, consider the schedule where both teams are on shift for

[i/k, (i+ 1)/k) for all i even (the case of i = 2 is simply the policy DA). This divides

the day into k equally sized pieces, where for k/2 such pieces both teams are on

shift, and for the remaining k/2 periods both teams are off shift. We see immediately

that independent of k, each team still spends half of each day on shift. In this half

day on shift, our two teams’ 2c capacity will again have Pois(cµ) departures. Now

in each of our off shift periods, the probability of a patient leaving is P(Exp(µ) ≤

1/k) = 1 − e−µ/k, so we have Bin(2c, 1 − e−µ/k) off shift departures in each of our

k/2 off shifts, or Bin(kc, 1 − e−µ/k) off shift departures per day. Thus the expected

off shift departures per day is kc(1− e−µ/k). Letting k →∞, we see through Taylor

expansion that our off shift departures tend to cµ, giving 2cµ total departures as with

the M/M/2c queue. While in practice, we cannot have arbitrarily short shifts, we do

see a general trend that shorter shifts increase capacity.

The stability property however is not the only relevant performance measure. An

important quantity to look at is the number of patient reassignments (i.e. the number

of arriving patients forced to wait due to the non-availability of residents, as discussed

in the introduction). For each policy θ ∈ LS,DA, we can easily verify that Sθ(k)

is irreducible and aperiodic on Sθ. Thus under the condition ρθ < 1, there exists

a unique steady state distribution for Sθ(t), and we denote this random vector by

Sθ(∞). Analyzing Sθ(∞) directly appears to be intractable. Instead, we resort to

the method of fluid approximation, which we now define.

Given the parameters of our queueing model λ1, λ2, µ and c, we consider a sequence

175

of approximate models n = 1, 2, . . ., where we change the parameters so that in the

nth model, λn1 = λ1n, λn2 = λ2n, µn = µ, and cn = cn. In words, the rate of

patient recovery is fixed, but the patient arrival rates and patient capacity scale

up linearly. For each policy θ ∈ LS,DA, we let Qnθ (t), Rn

θ (t) and Snθ (t) be the

corresponding processes. We let Snθ be the set Sθ as defined in (6.5) and (6.10) for

the processes Snθ (t). The process associated with fluid rescaling is defined as Snθ (t)/n.

We immediately note by (6.11) and (6.12) that ρθ does not change with n, so the

stability criteria for each Snθ (t) is the same. Thus for ρθ < 1 the sequence Snθ (∞)/n

is well defined. Our next main result is that as n → ∞, the sequences Snθ (t)/n and

Snθ (∞)/n converge meaningfully to some deterministic process sθ(t) = (qθ(t), rθ(t)),

and its unique fixed point limk→∞ sθ(k), respectively. We now provide details.

For LS, we define the process sLS(t) = (qLS(t), rLS(t)) on R+ × [0, c] inductively

on intervals [k, k + 1). For each interval, consider the system of ordinary differential

equations (ODEs)

qLS(t) = λ(t)− µ(qLS(t) ∧ c), (6.14)

rLS(t) = −µrLS(t). (6.15)

At integer times k ≥ 1, the process jumps as did SLS(k). Specifically, we let

sLS(k) = Γ(sLS(k−); c). (6.16)

In analogy with SLS, we will show that at integer times k ≥ 1 this process is actually

restricted to

TLS∆= [0, c]2 ∪ R+ × c. (6.17)

We now give a similar construction for sDA(t) = (qDA(t), rDA(t)) on R+×[0, 2c]. Again

for each interval [k, k + ½), we let rDA(t) = 0 and define qDA(t) by

qDA(t) = λ1 − µ(qDA(t) ∧ 2c). (6.18)

176

At times k + ½, k ∈ Z+, we let

sDA(k + ½) = Γ(sDA((k + ½)−); 2c).

For each interval [k+ ½, k+ 1), qDA(t) and rDA(t) are defined by the following ODEs:

qDA(t) = λ2,

rDA(t) = −µrDA(t).

Again at integer times k ≥ 1, we define

sLS(k) = Γ(sDA(k−); 0).

We let

TDA∆= R+ × 0. (6.19)

We will show that this is the set of possible values sDA(k) can take for integer k ≥ 1.

Proposition 6.1. For every θ ∈ LS,DA, and every sθ(0) ∈ Tθ, sθ(t) exists and is

uniquely defined for all t ∈ R+. Further, for all integer k ≥ 1, sθ(k) ∈ Tθ.

The result is shown in Section 6.8. We now formally relate Sθ(t) to sθ(t).

Theorem 6.2. For each θ ∈ LS,DA, if Snθ (0)/n→ sθ(0) a.s., then

limn→∞

Snθ (t)

n= sθ(t),

a.s. and u.o.c.

While this theorem allows us to approximate Sθ(t) by the simpler process sθ(t),

we have not established any relationship between Sθ(∞) and sθ(k) as k → ∞. We

do this next, but first we need some definitions.

Suppose we are given a discrete time dynamical system on a state space X ⊂ Rn

177

Snθ (k) Snθ (∞) W i,nθ (∞)

sθ(k) sθ(∞) wiθ(∞)

Theorem 6.1

k →∞

Theorem 6.2 limn→∞

Snθ (k)

n

Theorem 6.3

k →∞

Theorem 6.3 limn→∞

Snθ (∞)

nCorollary 6.1 lim

n→∞

W i,nθ (∞)

n

Figure 6-1: A diagram explaining how each of our theorems relate our stochasticprocess and the fluid limit, for finite times, at steady state, and then finally for thesteady state number of reassignments.

defined by f : X → X , i.e. xk+1 = f(xk) for all k. A point x∗ is defined to be attractive

if for all x0 ∈ X ,

limn→∞

xn = x∗.

Note that there can be at most one attractive point. We now state our next result

relating sθ(∞) and Sθ(∞).

Theorem 6.3. For each policy θ ∈ LS,DA, the sequence sθ(k) has a unique attrac-

tive point sθ(∞) ∈ Tθ iff ρθ < 1. Moreover, when ρθ < 1, the following convergence

in probability takes place:

limn→∞

Snθ (∞)

n= sθ(∞).

Notice that condition for the existence of an attractive point for sθ(t) is exactly

the same as the stability condition for Sθ(t). In the second claim of Theorem 6.3,

we are essentially justifying an interchange of limits, as informally we are “equating”

limn→∞ limk→∞ Snθ (k)/n with limk→∞ limn→∞ Snθ (k)/n, as shown in the left half of

Figure 6-1.

We now use this result to approximate the steady state number of reassignments

178

in the queueing model. For each θ ∈ LS,DA and each k ∈ Z+, let W 1θ (k) (resp.

W 2θ (k)) be the number of arriving patients during [k, k+ ½) (resp. [k+ ½, k+ 1)) that

are forced to wait a nonzero amount of time before assignment to a resident, i.e. the

number of reassignments. Similarly, when ρθ < 1, we define W 1θ (∞) (resp. W 2

θ (∞))

to be the steady state number patients forced to wait during [0, ½) (resp. [½, 1)). Next

we define variables for the fluid approximations of these quantities. Let bθ(t) be

bLS(t) = IqLS(t)≥c, (6.20)

bDA(t) =

IqDA(t)≥2c t ∈ [k, k + ½),

1 t ∈ [k + ½, k + 1),

(6.21)

i.e. bθ(t) is the indicator that the on shift teams are saturated. We define

w1θ(k) =

∫ k+½

k

bθ(t)λ1dt, (6.22)

w2θ(k) =

∫ k+1

k+½

bθ(t)λ2dt. (6.23)

When ρθ < 1, we let w1θ(∞) = w1

θ(0) and w2θ(∞) = w2

θ(0) assuming the fluid system

begins in steady state, i.e. sθ(0) = sθ(∞). We next argue that w1θ(∞) and w2

θ(∞)

asymptotically describe the steady state number reassignments. Let W 1,nθ (k) and

W 2,nθ (k) be the number of reassignments for Snθ (t) from our fluid approximation.

Then

Corollary 6.1. For policy LS, assuming λ1, λ2 6= cµ, and for policy DA, assuming

λ1 6= 2cµ, the following convergence in probability takes place:

limn→∞

W 1,nθ (∞)

n= w1

θ(∞),

limn→∞

W 2,nθ (∞)

n= w2

θ(∞).

The case when λj = cµ for either j = 1, 2 presents some annoying technical

difficulties. As realistically we will never have exact equality, we do not pursue this

179

issue further. This sequence of results justifies approximating W 1θ (∞) and W 2

θ (∞)

by w1θ(∞), w2

θ(∞), respectively. The result of Corollary 6.1 are summarized in the

right half of Figure 6-1.

6.3 Numerical Results

In this section, we numerically solve for the steady state solution of the fluid model of

each policy. We then compare the cost of the reassignments in a single day starting

at steady state under each policy as we vary the average arrival rate. We relate our

numerical observations to our empirical observations from Figure 5-3.

Throughout this section, we use the following parameters in our model: µ = 1/2,

c = 40, λ1 = 9λ/5, and λ2 = λ/5. Our choice of µ and c imply that λ∗LS ≈ 35.7388

and λ∗DA ≈ 37.6959. The value of c and the ratio of λ1 to λ2 were chosen to be

representative of a department from a large hospital such as B&W. The value of µ

must be chosen more carefully. In light of Remark 6.1, we set µ to control the relative

sizes of the average length of stay and length of time between shifts. At B&W under

the policy LOSO, there is a long shift every four days and the average patient length

of stay is four days. Thus in our model we set the average length of stay (1/µ) to be

two days as the policy LS has a long shift every two days.

In Figure 6-2, we fix λ at 34 and observe the steady state behavior of our two

policies in the fluid limit over the course of a day. Notice that λ < λ∗LS < λ∗DA, so

under both policies the fluid model is stable, but heavily loaded. We see that for both

policies, under these particular parameters, the number of patients being treated by

the teams on shift plus the number of patients waiting, qθ(t), increases over the first

half of day. For LS, the capacity of 40 for the teams on shift (as indicated by the

dotted black line) is exceeded, and resulting in some reassignments. For DA however,

as both teams are working during the first half of the day, we stay below the capacity

of 80 patients and have no reassignments. In the second half of the day, under LS

we see that the backlog of patients subsides and we return below 40 patients by the

end of the day. For DA, as both teams are off shift during the second half of the day,

180

we see a jump at time 1/2 between qDA and rDA and then small backlog of arrivals

accumulate in the second half of the day.

In Figure 6-3, we show the number of reassignments for each policy in [0, ½) and

[½, 1) as we vary λ. The dotted vertical lines indicate λ∗LS and λ∗DA, the largest patient

arrival rates such that LS and DA are stable. We see that our observation from Figure

6-2, that LS had many reassignments in [0, ½) while DA had no reassignments in this

period, is typical when the system is heavily loaded (for λ near λ∗LS). We also see

in Figure 6-3 that when λ is low, both polices cause no reassignments in the first

half of the day, and only DA causes reassignments in the second half the day. As

λ increases towards λ∗LS, we see LS begin to reassign nearly all patients, while DA

continues to only reassign patients arriving in the second half of the day. Finally, for

very large λ, we eventually see DA reassigning some patients during the first half of

the day. While for these particular parameter settings, we only see DA reassignments

in the first half of the day for λ so large that LS is unstable, this does not hold for all

parameter settings. Interestingly, we see that under DA for λ near λ∗DA, the number

of reassignments does not approach λ, while it does for LS. This is occurring as under

these parameters, we have more patients leaving than arriving in the second half of

the day, creating some spare capacity during the start of the first half of the following

day.

Comparing Figure 6-3 with our empirical observations from Figure 5-3 we see

that the relationship between LS and DA is qualitatively similar to the relationship

between the B&W policies LOSO (labeled Initial in the figure) and MMMO (labeled

Daily Admitting in the figure). Most importantly, we have preserved the property

that shorter more frequent shifts are the better policy when the patient load is heavy.

6.4 Conclusion

We have developed a queueing model to determine the effect of long shifts in med-

ical resident schedules on the hospital’s capacity to admit patients and the quality

of care delivered. Our model was motivated by the empirical work from Chapter 5

181

0 0.2 0.4 0.6 0.8 1

25

30

35

40

45

Time of day

Pat

ients

qLS(t)rLS(t)

capacity

0 0.2 0.4 0.6 0.8 1

0

20

40

60

80

Time of dayP

atie

nts

qDA(t)rDA(t)

capacity

Figure 6-2: Steady state number of patients in system in the fluid limit under eachpolicy. Parameter values: λ = 34, λ1 = 9λ/5, λ2 = λ/5, µ = 1/2, c = 40.

30 32 34 36 38

0

10

20

30

40

λ

Pat

ients

Rea

ssig

ned

w1LS(∞)

w2LS(∞)

w1DA(∞)

w2DA(∞)

30 32 34 36 38

0

10

20

30

40

λ

Pat

ients

Rea

ssig

ned

LS TotalDA Total

Figure 6-3: Steady state number of patient reassignments in the fluid limit for differentλ under each policy. Parameter values: λ1 = 9λ/5, λ2 = λ/5, µ = 1/2, c = 40.

182

on scheduling medical residents for B&W hospital. In this paper, we compared the

stylized schedules Long Shifts (LS), where residents worked 24 hour shifts on alter-

nating days, and Daily Admitting (DA), where residents worked every day but only

during peak arrival hours. We used Lyapunov function techniques to characterize the

stability of our queueing model under each policy. We found that DA has a greater

capacity to admit patients than LS for all parameter choices. To analyze the long-run

performance of our queueing model, we first considered the associated fluid model,

which is a deterministic system with periodic dynamics. We showed that under each

policy, when the queueing model is stable, the fluid model had a unique periodic

steady state solution. We showed that our queueing model under the fluid rescaling

converges to the fluid model on finite time intervals. Then we used an interchange of

limits argument to show that the steady state queue lengths under the fluid rescaling

converge to the unique steady state solution of the fluid model. We use these results

to approximate the steady state number of reassignments in our queueing model by

the steady state behavior of the fluid model. Numerically solving for the steady state

of the fluid model under various parameter choices, we found evidence suggesting

the existence of a threshold value on the arrival rate such that DA causes fewer re-

assignments than LS iff the arrival rate exceeds the threshold value. These results

substantiate the main empirical findings in Chapter 5. The issue of resident schedules

is currently quite pertinent, as new regulations restrict residents to a maximum shift

length of 16 hours [50]. Our work contributes to understanding the implication of the

new regulation. As hospitals tend to operate in heavily loaded regimes, we find that

schedules relying on shorter more frequent shifts could increase capacity and reduce

reassignments.

183

6.5 Stability Conditions for Two Schedules. Proof

of Theorem 6.1

In this section, we prove Theorem 6.1, characterizing the stability of Sθ(t). We show

these results using the method of Lyapunov functions, using Proposition A.1 and

Proposition A.2. The statement of these results and theirs proofs can be found in

Appendix A, Section A.1.

Let the Lyapunov function V : Sθ → R+ be defined by V (q, r) = q + r for the

Markov chain Sθ(k). We first analyze the drift under the policy LS, namely

E[V (SLS(1)) − V (SLS(0))]. Let A∆= A(0, 1), Don

LS∆= Don

LS(0, 1), and DoffLS

∆= Doff

LS(0, 1)

denote the number of arrivals and departures in a single day under LS.

Lemma 6.1. We have

−γLS = limq→∞

E(q,c)[V (SLS(1))− V (SLS(0))] = inf(q,r)∈SLS

E(q,r)[V (SLS(1))− V (SLS(0))].

(6.24)

Additionally, there exists a constant CLS > 0 depending only on µ such that

sup(q,r)∈SLS

E(q,r)[(V (SLS(1))− V (SLS(0)))2] ≤ λ2 + λ+ CLS(c2 + c). (6.25)

Proof. First, observe that the value of the Lyapunov function does change at time 1:

V (SLS(1)) = V (Γ(SLS(1−); c)) = V (SLS(1−)), (6.26)

as applying Γ does not change the number of patients in system. Thus for ` = 1, 2,

E(q,r)

[(V (SLS(1))− V (SLS(0)))`

]= E(q,r)

[(V (SLS(1−))− V (SLS(0)))`

]= E(q,r)

[(A−Don

LS −DoffLS)`

]. (6.27)

Let DonLS

d= Pois(cµ) and Doff

LSd= Bin(c, 1 − e−µ) such that Don

LS, DoffLS, and A are inde-

pendent. As DonLS(0, t) for 0 ≤ t < 1 has the distribution of the departure process for

184

an M(t)/M/c queue, we can couple DonLS with Don

LS such that regardless of SLS(0),

DonLS ≤ Don

LS. (6.28)

For the off shift departures, as the patient length of stay is exponential, each of the

r = R(0) ≤ c patients in care will depart in the interval [0, 1) with probability 1−e−µ

independently of other patients. Thus DoffLS

d= Bin(r, 1 − e−µ), so trivially it can be

coupled with DoffLS such that

DoffLS ≤ Doff

LS, (6.29)

with equality when r = c. From (6.28) and (6.29) we obtain that for any initial

condition S(0) = (q, r) ∈ SLS

A−DonLS −Doff

LS ≥ A− DonLS − Doff

LS.

Taking expectations and then the infimum over all (q, r) ∈ SLS, we obtain that

inf(q,r)∈SLS

E(q,r)[V (SLS(1))− V (S(0))] ≥ λ− cµ− c(1− e−µ) = −γLS.

Observe that for any realization where Q(0) = q and DonLS ≤ Don

LS ≤ q − c, we also

have QLS(t) ≥ c for all t ∈ [0, 1−), and thus DonLS = Don

LS. As a result,

E(q,c)[DonLS] ≥ E(q,c)

[Don

LSIDonLS≤q−c

]= E

[Don

LSIDonLS≤q−c

]. (6.30)

Since almost surely

limq→∞

DonLSIDon

LS≤q−c= Don

LS,

by monotonicity of expectation and then the Monotone Convergence Theorem, we

185

obtain that

lim infq→∞

E(q,c)[DonLS] ≥ lim

q→∞E[Don

LSIDonLS≤q−c

] = E[DonLS] = cµ.

Combining this inequality with (6.28), we obtain limq→∞ E(q,c)[DonLS] = cµ and thus

limq→∞

E(q,c)[V (SLS(1))− V (SLS(0))] = λ− c(1− e−µ)− cµ = −γLS.

Lastly, to show (6.25), using independence, (6.28), and (6.29),

sup(q,r)∈SLS

E(q,r)[(A−DonLS −Doff

LS)2]

≤ E[A2] + E[(DonLS)2] + E[(Doff

LS)2] + 2E[DonLS]E[Doff

LS]

= λ2 + λ+ c2µ2 + cµ+ c(1− e−µ)e−µ + (c(1− e−µ))2 + 2c2µ(1− e−µ).

We now give an analogous result to the previous lemma for DA. As the proof is

nearly the same, some details have been omitted. As before, let DonDA

∆= Don

DA(0, ½)

and DoffDA

∆= Doff

DA(½, 1), give the number of departures under policy DA in a single day

(note that there is no one on shift during [½, 1) and no one off shift during [0, ½) under

DA).

Lemma 6.2. We have

−γDA = limq→∞

E(q,0)[V (SDA(1))− V (SDA(0))] = inf(q,r)∈SDA

E(q,r)[V (SDA(1))− V (SDA(0))].

(6.31)

Additionally, there exists a constant CDA > 0 depending only on µ such that

sup(q,r)∈SDA

E(q,r)[(V (SDA(1))− V (SDA(0)))2] ≤ λ2 + λ+ CDA(c2 + c). (6.32)

Proof. Again, for any initial state SDA(0) = (q, 0), V (SDA(1)) = V (SDA(1−)) and

186

V (SDA(½)) = V (SDA(½−)), as applying Γ at times ½ and 1 does not change the number

of patients in system. Thus for ` = 1, 2,

E(q,0)[(V (SDA(1))− V (SDA(0)))`] = E(q,0)

[(A−Don

DA −DoffDA)`

]. (6.33)

Let DonDA

d= Pois(cµ) and Doff

DAd= Bin(2c, 1 − e−µ/2) such that Don

DA, DoffDA, and A are

independent. As DonDA(0, t) for 0 ≤ t < ½ has the distribution of the departure process

for an M(t)/M/2c queue, we can couple DonDA(0, ½−) with Don

DA such that regardless

of SDA(0),

DonDA ≤ Don

DA. (6.34)

For the off shift departures, as the patient length of stay is exponential and thus mem-

oryless, each of the RDA(½) ≤ 2c patients in care will depart in the interval [½, 1) with

probability 1 − e−µ/2 independently of other patients. Thus DoffDA

d= Bin(RDA(½), 1 −

e−µ/2), so it can be coupled with DoffDA such that

DoffDA ≤ Doff

DA, (6.35)

with equality when RDA(½) = 2c. From (6.34) and (6.35) we obtain that for any

(q, 0) ∈ SDA,

A−DonDA −Doff

DA ≥ A− DonDA − Doff

DA,

and thus by taking expectations

inf(q,r)∈SDA

E(q,r)[V (SDA(1))− V (SDA(0))] ≥ λ− cµ− 2c(1− e−µ/2) = −γDA.

As before, to complete showing the three term equality in (6.31), it suffices to show

limq→∞ E(q,0)[V (SDA(1))−V (SDA(0))] = −γDA. Given QDA(0) = q, for any realization

such that DonDA ≤ Don

DA ≤ q − 2c, we have QDA(t) ≥ 2c for all t ∈ [0, ½−), and thus

DonDA = Don

DA. Further, QDA(½−) ≥ 2c ensures RDA(½) = 2c and thus DoffDA = Doff

DA. As

187

a result,

E[DonDA] ≥ E

[Don

DAIDonDA≤q−2c

]= E

[Don

DAIDonDA≤q−2c

],

E[DoffDA] ≥ E

[Doff

DAIDonDA≤q−2c

]= E

[Doff

DAIDonDA≤q−2c

].

We can apply the Monotone Convergence Theorem as before but now on both

DonDAIDon≤q−2c and Doff

DAIDon≤q−2c to obtain the desired limit. The rest of the proof

is exactly as in the previous lemma.

Proof of Theorem 6.1. Suppose ρθ < 1, i.e. γθ > 0. From (6.24) of Lemma 6.1, we

obtain that

−γLS = limq→∞

E(q,c)[V (SLS(1))− V (SLS(0))].

Similarly, from (6.31) of Lemma 6.2, we

−γDA = limq→∞

E(q,0)[V (SDA(1))− V (SDA(0))].

Recall by (6.5) that for every (q, r) ∈ SLS, when q ≥ c, we must have r = c, and by

(6.10) for every (q, r) ∈ SDA we have r = 0. Thus the sets

Bθ =

(q, r) ∈ Sθ

∣∣∣∣ E(q,r)[V (Sθ(1))− V (Sθ(0))] > −γθ/2,

are finite. Observe that for both LS and DA, (A.1) is satisfied by (6.25) and (6.32),

respectively. Applying Proposition A.1, taking B = Bθ and γ = γθ/2, we conclude

that Sθ(k) is positive recurrent.

Now suppose instead that ρθ ≥ 1, i.e. γθ ≤ 0. In the setting of Proposition A.2,

for both θ we take Bθ = (0, 0) and observe that (A.2) is trivially satisfied by taking

y = (q, r) for any nonzero (q, r) ∈ Sθ. The condition in (A.3) is satisfied by (6.25) for

188

LS and (6.32) for DA. Finally, (A.4) is satisfied as by (6.24) and (6.31), we have

inf(q,r)∈Sθ

E(q,r)[V (Sθ(1))− V (Sθ(0))] = −γθ ≥ 0.

Thus from Proposition A.2 we conclude that Sθ(k) is either null recurrent or tran-

sient.

It remains to show that Sθ(k) is null recurrent when ρθ = 1 and transient when

ρθ > 1. To do so, we use another Lyapunov function argument, Theorem 3.2 from

[56], (see also [32] section 3.6). However, as the statement of this theorem is rather

technical, we defer this part of the proof to Section 6.12.

Finally, that ρDA < ρLS follows from (6.13).

Remark 6.1. Note that as µ → 0, (1 − e−µ/2)2 → 0, so by (6.13) we see that

ρLS− ρDA → 0 as well. Intuitively, if patients take many days to recover, the amount

of forced idle time due to not being able to admit patients while off shift will be

negligible. Conversely, when µ is large,

ρLS ≈λ

cµ+ c, ρDA ≈

λ

cµ+ 2c.

In this regime, nearly all patients recover in each off shift. When c is also large, we

see DA has a larger stability region, due to the off shifts for each team being shorter.

While this regime isn’t particularly relevant in the hospital setting, where µ ≈ 1/4,

it exposes another interesting and relevant parameter, namely the length of the time

between shifts relative to the recovery rate µ.

6.6 Uniform Bounds for Stationary Performance

Measures

In this section, we consider the sequence of systems under the fluid rescaling Snθ (t)/n

with the assumption that ρθ < 1, and give bounds on the expected stationary queue

lengths that are independent of n. We will again use the Lyapunov function technique,

189

namely Proposition A.3. The statement of this result and a proof can be found in

Appendix A, Section A.2.

First, we need a property of sample paths of the M(t)/M/m queue. For every

initial queue length q ∈ Z+, we create a separate queue length process with the same

arrival and service rates on a common probability space Ω. Let f : Z+×R+×Ω→ Z+

map an initial queue length q, a time t, and a realization ω ∈ Ω to the number of

patients in the M(t)/M/m system length at time t. The queues are coupled such that

they share a single common Poisson process determining arrival times, and a single

independent common Poisson process determining potential departure times (which

only result in departures when there are patients in care). The relationship between

the arrival process, the potential departure process, and the actual departures is the

same as the relationship between A(0, t), Don(0, t) and Don(0, t) from Section 6.5.

Lemma 6.3. For every realization ω ∈ Ω, every time t ∈ R+, and all initial queue

lengths q, r ∈ Z+ such that q ≥ r, f(·, t, ω) satisfies

0 ≤ f(q, t, ω)− f(r, t, ω) ≤ q − r.

Namely, f(·, t, ω) is monotone increasing and 1-Lipschitz continuous with respect to

the `1 norm in the initial queue length.

Proof. Fix ω, and consider q, r ∈ Z+, q > r. Let τ0 = 0 and for i = 1, 2, . . . , let τi

be the time of the ith event from our processes driving arrivals and departures. It

suffices to prove that

0 ≤ |f(q, τi, ω)− f(r, τi, ω)| ≤ q − r, (6.36)

for all τi, as the queue length can only change at the times of these events. Trivially

the claim holds at τ0. Suppose the claim holds until τi. At time τi+1:

1. Suppose the event was an arrival. Then f(q, τi+1, ω) = 1 + f(q, τi, ω) and

190

f(r, τi+1, ω) = 1 + f(r, τi, ω), so

f(q, τi+1, ω)− f(r, τi+1, ω) = f(q, τi, ω)− f(r, τi, ω),

thus (6.36) holds.

2. Suppose the event was a potential departure. By our inductive hypothesis, we

must be in one of the two cases below:

(a) f(q, τi, ω) = f(r, τi, ω). Then by our coupling, the system under initial

condition q and under initial condition r must both either have an actual

departure or have no departure at τi+1. As the change in queue lengths

will be the same, by the same reasoning as when we have an arrival, we

continue to satisfy (6.36).

(b) f(q, τi, ω) > f(r, τi, ω). Now either both systems have a departure, or only

the system with initial queue length q has a departure (as it has more

active servers), which with the inductive hypothesis implies

f(q, τi+1, ω)− f(r, τi+1, ω) ≤ f(q, τi, ω)− f(r, τi, ω) ≤ q − r.

When both systems experience an actual departure, f(q, τi+1, ω)−f(r, τi+1, ω) =

f(q, τi, ω)−f(r, τi, ω) ≥ 0 where the inequality is by the inductive hypoth-

esis. When only the system under initial condition q has a departure, we

still have

f(q, τi+1, ω)− f(r, τi+1, ω) = f(q, τi, ω)− 1− f(r, τi, ω) ≥ 0,

where the inequality holds by initial assumption for this case. Thus (6.36)

holds.

Next we establish a few properties of Γ, defined in (6.3).

191

Lemma 6.4. For all κ ≥ 0 and all (q, r), (q′, r′) ∈ R2+, the operator Γ(·;κ) satisfies

nΓ

((q, r)

n;κ

)= Γ(q, r;nκ), (6.37)

‖Γ(q, r;κ)− Γ(q′, r′;κ)‖1 ≤ ‖(q, r)− (q′, r′)‖1. (6.38)

Namely, with respect to the `1 norm, Γ(·;κ) is 1-Lipschitz continuous. Further, each

component of Γ(q, r;κ) increases monotonically in both q and r.

Proof. The first property, follows immediately by definition of Γ, as

nΓ

((q, r)

n;κ

)= n

(q

n∧ κ,

( qn− κ)+

− r

n

)=(q ∧ nκ, (q − nκ)+ − r

)= Γ(q, r;nκ).

To show the second part, we find that

‖Γ(q, r;κ)− Γ(q′, r′;κ)‖1 = |(q − κ)+ + r − (q′ − κ)+ − r′|+ |(q ∧ κ)− (q′ ∧ κ)|

≤ |(q − κ)+ − (q′ − κ)+|+ |r − r′|+ |(q ∧ κ)− (q′ ∧ κ)|.

Without loss of generality, suppose q ≥ q′. Now by considering the exhaustive cases

q′ ≥ κ, q > κ > q′, and κ ≥ q, the Lipschitz continuity follows trivially. The

monotonicity property is an immediate consequence of (6.3).

Corollary 6.2. For θ ∈ LS,DA, fix any s, s ∈ Sθ. Assume that the processes Sθ(t)

has Sθ(0) = s and let Sθ(t) be a version of Sθ(t) with instead Sθ(0) = s. There is a

coupling between these processes such that for all t ≥ 0,

‖Sθ(t)− Sθ(t)‖1 ≤ ‖s− s‖1.

Moreover, s ≥ s componentwise implies Sθ(t) ≥ Sθ(t) componentwise for all t.

Proof. It suffices to show the claim for all t ∈ (0, 1], as then the result follows by

induction. For θ = LS, for all t ∈ (0, 1), QLS(t) and QLS(t) are M(t)/M/c queues

with the same arrival and service rates. Thus by Lemma 6.3, we obtain that QLS(t) is

monotone increasing and 1-Lipschitz in q. Likewise, RLS(t) and RLS(t) are M(t)/M/c

192

queues with arrival rate zero and the same service rate, so again by Lemma 6.3, we

obtain that RLS(t) is monotone increasing and 1-Lipschitz in r. As QLS(t) does not

depend on r and RLS(t) does not depend on q, the claim holds for t ∈ (0, 1). For

t = 1,

∥∥∥SLS(1)− SLS(1)∥∥∥

1=∥∥∥Γ(SLS(1−); c)− Γ(SLS(1−); c)

∥∥∥1

≤∥∥∥SLS(1−)− SLS(1−)

∥∥∥1

(6.39)

≤ ‖s− s‖1 , (6.40)

where (6.39) follows from Lemma 6.4 and (6.40) follows from our analysis of the case

t ∈ (0, 1). Monotonicity follows as each component of Sθ(1) is the composition of

monotone increasing functions and thus monotone increasing in every input, again

by Lemma 6.4 and our analysis of the case t ∈ (0, 1). For θ = DA, the proof is very

similar.

We also need a simple uniform bound on a sequence of Poisson random variables.

Lemma 6.5. If Xnd= Pois(γn), then for all k > 2γ(e− 1) and all n = 1, 2, . . . ,

P(Xn ≥ kn) ≤ e−kn/2.

Proof. We have

P(Xn ≥ kn) ≤ exp(−kn)E[exp(Xn)] = exp(−kn) exp(γn(e− 1)) ≤ exp(−kn/2).

We now analyze our system in the fluid scaling. As before let An∆= An(0, 1),

Don,nLS

∆= Don,n

LS (0, 1−), Don,nDA

∆= Don,n

DA (0, ½−), Doff,nLS

∆= Doff,n

LS (0, 1−), and Doff,nDA

∆=

Don,nDA (½, 1−). Using Lemma 6.5, we show that when ρθ < 1, the convergence as q

goes to infinity in Lemma 6.1 is uniform in n.

193

Lemma 6.6. We have

limk→∞

supn>0

E(nk,nc)

[V (SnLS(1))− V (SnLS(0))

n

]= −γLS.

Proof. From (6.27), we have that for all k and n,

E(nk,nc)

[V (SnLS(1))− V (SnLS(0))

n

]= E(nk,nc)

[An −Don,n

LS −Doff,n

n

]= λ− c(1− e−µ)− E(nk,nc)

[Don,n

LS

n

].

where the second equality follows as And= Pois(nλ) and Doff,n

LSd= Bin(nc, 1− e−µ) (see

discussion (6.29)). As in (6.28) we have Don,nLS

d= Pois(ncµ) coupled with Don,n

LS such

that Don,nLS ≥ Don,n

LS . Thus E[Don,nLS ] ≤ ncµ for all n and all k, so it suffices to show

that

lim infk→∞

supn>0

E(nk,nc)

[Don,n

LS

n

]≥ cµ.

As in (6.30), we find that

lim infk→∞

supn>0

1

nE(nk,nc)[D

on,nLS ] ≥ lim

k→∞supn>0

1

nE[Don,n

LS IDon,nLS ≤(k−c)n].

194

Now

limk→∞

supn>0

∣∣∣∣ 1nE[Don,nLS IDon,n

LS ≤(k−c)n]− cµ∣∣∣∣

= limk→∞

supn>0

∣∣∣∣ 1nE[Don,nLS IDon,n

LS ≤(k−c)n − Don,nLS ]

∣∣∣∣= lim

k→∞supn>0

1

nE[Don,n

LS IDon,nLS ≥(k−c)n]

≤ limk→∞

supn>0

1

n

√E[(Don,n

LS )2]E[IDon,nLS ≥(k−c)n]

≤ limk→∞

supn>0

1

n

√((ncµ)2 + ncµ) exp(−kn/2) exp(cn) (6.41)

= limk→∞

√((cµ)2 + cµ) exp(−k/2) exp(c) (6.42)

= 0.

Here (6.41) follows from Lemma 6.5, and (6.42) follows as when k, is large, the

supremum is attained by taking n = 1.

Lemma 6.7. We have

limk→∞

supn>0

E(nk,0)

[V (SnDA(1))− V (SnDA(0))

n

]= −γDA.

The proof is very similar to previous lemma and omitted. We can give the uniform

moment bounds for Snθ (∞).

Lemma 6.8. For each policy θ ∈ LS,DA, when ρθ < 1, there exists a constant Mθ

depending on λ, c and µ such that for every n > 0, E [V (Snθ (∞))] ≤Mθn.

Proof. As ρθ < 1, we have γθ > 0. By Lemma 6.6, there exists kLS such that for all

k > kLS and all n,

1

nE(nk,nc)[V (SnLS(1))− V (SnLS(0))] ≤ −γLS

2.

195

For any q > q,

0 ≤ E(q,r)[V (SnLS(1))]− E(q,r)[V (SnLS(1))] ≤ q − q,

by the monotonicity and 1-Lipschitz of Corollary 6.2. Thus

E(q,r)[V (SnLS(1))− V (SnLS(0))]− E(q,r)[V (SnLS(1))− V (SnLS(0))]

= E(q,r)[V (SnLS(1))]− E(q,r)[V (SnLS(1))] + q − q

≤ 0.

As a result, we obtain that for every n, for all q > kLSn,

1

nE(q,nc)[V (SnLS(1))− V (SnLS(0))] ≤ −γLS

2. (6.43)

We define kDA analogously using Lemma 6.7 and γDA. Let

bθ∆= max

kθ,

1

γθ

(λ2 + λ+ Cθ(c

2 + c))

,

where Cθ is as defined by (6.25) for LS and (6.32) for DA. Thus by Lemma 6.1 and

Lemma 6.2, for all s ∈ Snθ ,

Es[(V (Snθ (1))− V (Snθ (0)))2] ≤ (λn)2 + λn + Cθ((cn)2 + cn)

= n2λ2 + nλ+ Cθ(n2c2 + nc)

≤ bθγθn2. (6.44)

196

We let

Bnθ

∆= (q, r) ∈ SLS | q + r ≤ 2nbθ, (6.45)

U(q, r)∆= (q + r)2,

αnθ∆= 2nbθ,

βnθ∆= n2bθ(4λ+ γθ),

γnθ∆= nγθ/2.

and apply Proposition A.3 for each n, using U as our Lyapunov function and f(q, r) =

q + r. We observe that (A.6) holds trivially. For (q, r) ∈ Snθ we have

E(q,r)[U(Snθ (1))− U(Snθ (0))] = E(q,r)[(q + r + An −Don,nθ −Doff,n

θ )2]− (q + r)2

= 2(q + r)E(q,r)[An −Don,n

θ −Doff,nθ ]

+ E(q,r)

[(An −Don,n

θ −Doff,nθ )2

]= 2(q + r)E(q,r)[V (Snθ (1))− V (Snθ (0))]

+ E(q,r)[(V (Snθ (1))− V (Snθ (0)))2]

≤ 2(q + r)E(q,r)[V (Snθ (1))− V (Snθ (0))] + n2γθbθ, (6.46)

where (6.46) is a consequence of (6.44). Now, for (q, r) ∈ Bnθ , using (6.45) with (6.46),

we see that

2(q + r)E(q,r)[V (Snθ (1))− V (Snθ (0))] + n2γθbθ ≤ 4nbθE(q,r)[An(0, 1−)] + n2γθbθ

≤ n2bθ(4λ+ γθ)

= βnθ ,

197

showing that (A.7) holds. Finally, for s ∈ Snθ \Bnθ ,

E(q,r)[U(Snθ (1))− U(Snθ (0))] ≤ 2(q + r)E(q,r)[V (Snθ (1))− V (Snθ (0))] + n2γθbθ (6.47)

≤ −γθn(q + r) + n2γθbθ (6.48)

≤ −γθ2n(q + r)− γθ

2n(2nbθ) + n2γθbθ (6.49)

= −γθn2f(q, r),

where (6.47) follows from (6.46), (6.48) follows as s ∈ Snθ \Bnθ so we can apply (6.43),

and (6.49) follows as s ∈ Snθ \ Bnθ ensures q + r > 2nbθ. This shows that (A.5) is

satisfied for each n. Thus for every n we can apply Proposition A.3 to obtain that

E[f(Sn(∞))] ≤ αn +βn

γn= n (2bθ (4λ/γθ + 1)) ,

showing the result.

6.7 Fluid Model Approximations. Proof of Theo-

rem 6.2

In this section, we establish Theorem 6.2. First, we introduce the following additional

notation to be used throughout the section. Let uiθ be the time of the ith shift change

under policy θ, i.e. for i = 0, 1, 2, . . . , uiLS∆= i and uiDA

∆= i/2. Let ciθ be the number

of residents on shift during [ui, ui+1), i.e. ciLS∆= c for i = 0, 1, 2, . . . , ciDA

∆= 2c for even

i, and ciDA∆= 0 for odd i.

To prove the result, we will invoke a theorem from [60] that shows the convergence

of multidimensional Markovian queueing processes to its fluid limit under the so called

“uniform acceleration.” Consider a Markov process X(t) on state space Zm+ with

transition rates that depend both on the current state and the time. The process

X(t) is driven by a finite set of independent rate one exogenous Poison process Ei(t),

i = 1, . . . , k. The events from these processes trigger a “jump” vi ∈ Zm in X(t). For

each process i, there is a rate function αi(x, t) : Rm+ ×R+ → R+ that depends both on

198

the state x and the time t. Assume that for each i and t ∈ R+, αi(·, t) is γi-Lipschitz

in x where γi does not depend on t. We define X(t) by

X(t)∆= X(0) +

k∑i=1

viEi

(∫ t

0

αi(X(τ), τ)dτ

),

In Theorem 9.2 from [60], it is shown that this procedure uniquely defines the process

X(t). Next, we consider a deterministic process x(t) on Rm+ defined by

x(t)∆= x(0) +

k∑i=1

vi

∫ t

0

αi(x(τ), τ)dτ.

Again the existence and uniqueness of such an x(t) is shown in Theorem 11.4 from [60].

To approximate X(t) by x(t), we consider a sequence of processes Xn(t), n = 1, 2, . . . ,

defined by

Xn(t)∆= Xn(0) +

k∑i=1

viEi

(n

∫ t

0

αi

(Xn(τ)

n, τ

)dτ

),

i.e. X1(t) is the original process. A special case of their result is as follows.

Proposition 6.2 ([60], Theorem 2.2). If Xn(0)/n→ x(0) a.s., then

limn→∞

Xn(t)

n= x(t),

a.s. and u.o.c.

We now return to our model. On the intervals [uiθ, ui+1θ ) between shift changes,

our processes Sθ(t) and sθ(t) are of the form of X(t) and x(t) from the theorem.

Specifically, we can take v1 = (1, 0) and α1((q, r), t) = λ(uiθ + t) so that the arrival

199

process A(uiθ, uiθ + t) = E1(t) for t ∈ [uiθ, u

i+1θ ]. Similarly, we take v2 = (−1, 0) and

α2((q, r), t)∆= (ciθ ∧ q)µ =

(c ∧ q)µ θ = LS,

(2c ∧ q)µ θ = DA, i even,

0 θ = DA, i odd,

then Don(uiθ, t) = E2(t) for t ∈ [uiθ, ui+1θ ]. Finally, we take v3 = (0,−1) and

α3((q, r), t)∆= rµ so that Doff(uiθ, t) = E3(t) for t ∈ [uiθ, u

i+1θ ]. We satisfy the Lips-

chitz condition on αi(·, t) as α1 does not depend on the state and both α2 and α3 are

µ-Lipschitz in (q, r) independent of t.

Thus the proposition immediately yields that if Snθ (uiθ)/n → sθ(uiθ) a.s., then

Snθ (t)/n → sθ(t) u.o.c. From this point, the primary difficulty in proving Theorem

6.2 is showing that Sθ(t) jumping at each shift change does not ruin the convergence.

We can now prove the main result of the section.

Proof of Theorem 6.2. For each policy θ ∈ LS,DA, we will show by induction on

i that Snθ (t)/n → sθ(t) a.s. and uniformly on [0, uiθ]. The case of i = 0 holds by the

assumption of the theorem.

Suppose the claim holds for i. We notice that

sup0≤τ≤ui+1

θ

∥∥∥∥Snθ (τ)

n− sθ(τ)

∥∥∥∥1

=

max

sup

0≤τ≤uiθ

∥∥∥∥Snθ (τ)

n− sθ(τ)

∥∥∥∥1

, supuiθ≤τ<u

i+1θ

∥∥∥∥Snθ (τ)

n− sθ(τ)

∥∥∥∥1

,

∥∥∥∥Snθ (ui+1θ )

n− sθ(u

i+1θ )

∥∥∥∥1

.

By the definitions of uiθ and ciθ, it follows immediately that when applying Γ for the

shift change at time ui+1, we use Γ(·; ciθ). Recalling that Snθ and sθ are a.s. RCLL,

200

and applying (6.37) and then (6.38) from Lemma 6.4,∥∥∥∥Snθ (ui+1θ )

n− sθ(u

i+1θ )

∥∥∥∥1

=

∥∥∥∥Γ(Snθ ((ui+1θ )−);nciθ)

n− Γ(sθ((u

i+1θ )−); ciθ)

∥∥∥∥1

=

∥∥∥∥Γ

(Snθ ((ui+1

θ )−)

n; ciθ

)− Γ

(sθ((u

i+1θ )−); ciθ

)∥∥∥∥1

≤∥∥∥∥Snθ ((ui+1

θ )−)

n− sθ((u

i+1θ )−)

∥∥∥∥1

.

For τ ∈ [uiθ, ui+1θ ], we let Snθ (t) and sθ(t) be the continuous extension of Snθ (t) and

sθ(t), respectively, from [uiθ, ui+1θ ) to [uiθ, u

i+1θ ], i.e.,

Snθ (τ)∆=

Snθ (τ) τ < ui+1θ ,

Snθ ((ui+1θ )−) τ = ui+1

θ ,

sθ(τ)∆=

sθ(τ) τ < ui+1θ ,

sθ((ui+1θ )−) τ = ui+1

θ .

Thus we obtain that

sup0≤τ≤ui+1

θ

∥∥∥∥Snθ (τ)

n− sθ(τ)

∥∥∥∥1

= max

sup

0≤τ≤uiθ

∥∥∥∥Snθ (τ)

n− sθ(τ)

∥∥∥∥1

, supuiθ≤τ≤u

i+1θ

∥∥∥∥ Snθ (τ)

n− sθ(τ)

∥∥∥∥1

.

By induction, the first term in the above maximum goes to zero a.s. and in particular

Snθ (uiθ)/n→ sθ(uiθ) a.s. By Proposition 6.2, the second term converges to zero a.s. as

well, as previously discussed.

Remark 6.2. The result from [60] is actually stronger than what we suggested. It

implies that a.s. and u.o.c., as n → ∞, ‖Snθ (t)/n − sθ(t)‖1 ≤ O(log n). This can be

generalized to our case inductively in the same manner, but we do not pursue this

further.

6.8 Long Run Behavior of the Fluid Model

In this section, for each policy θ ∈ LS,DA, we show that the fluid limit at integer

times sθ(k), which is a deterministic discrete time dynamical system, has a simple

long run behavior. To show this, we need to recall definitions from Section 6.2. For

201

a set X ⊂ Rn, X 6= ∅, a function f : X → X , and an initial condition x0 ∈ X , let

xk, k ∈ Z+ be defined by f(xk) = xk+1. Recall from Section 6.2 that a point x∗ ∈ X

is attractive if for every x0 ∈ X , limk→∞ xk = x∗. As previously mentioned, such an

x∗ must be unique. Further, when f is continuous on X , it immediately follows that

f(x∗) = x∗, i.e. x∗ is a fixed point of f . First, we give a known (e.g. [18] page 183)

criterion for identifying attractive points.

Proposition 6.3. Suppose X is nonempty and compact, f : X → X is continuous on

X , and for every x,y ∈ X ,

‖f(x)− f(y)‖p < ‖x− y‖p,

for some p ≥ 1. Then there exists a unique attractive point x∗ ∈ X .

We now give a sufficient condition to ensure that in finite time xk will reach a

bounded set, such as the compact set in the previous theorem.

Proposition 6.4. If there is a function V : X → R+, γ > 0 and B ⊂ X such that

for all x ∈ X \B,

V (f(x))− V (x) ≤ −γ,

then for all x0 ∈ X \B, there exists m ≤ dV (x0)/γe such that xm ∈ B.

Proof. Let n = dV (x0)/γe + 1 and assume for contradiction that x0, . . . ,xn are all

not in B. Then

V (xn) = V (x0) +n∑k=1

V (xk)− V (xk−1) ≤ V (x0)− nγ < 0,

contradicting the non-negativity of V .

Finally, we give a criteria for instability.

202

Proposition 6.5. Suppose V : X → R+ is a continuous function such that supV (x) |

x ∈ X =∞, and for all x ∈ X ,

V (f(x))− V (x) ≥ 0.

Then an attractive point does not exist.

Proof. Assume for contradiction there were an attractive point x∗. Let x0 ∈ X be

such that V (x0) > V (x∗). Such a point exists as have assumed that the supremum

of V is infinite. However, as x∗ is attractive and V is continuous,

V (x∗) = limn→∞

V (xn) = V (x0) + limn→∞

n∑k=1

V (xk)− V (xk−1) ≥ V (x0),

contradicting V (x0) > V (x∗).

We now consider the differential equation that controls the evolution of the state

for the fluid model.

Lemma 6.9. Given parameters (γ,m, µ, x(0)) ∈ R4+, the differential equation

x(t) = γ − (x(t) ∧m)µ, (6.50)

has a unique solution. The solution x(t) is monotone in t and satisfies x(t) ≥ 0 for

all t ≥ 0. Further, if g : R+ → R+ is defined by g(x(0)) = x(½), then g is strictly

increasing and 1-Lipschitz. Finally, if we let

x∆=

m γ ≥ mµ,

m− (γ −mµ)/2 γ < mµ,

then x(0) ≥ x implies that:

(a) x(t) ≥ m for t ∈ [0, ½),

(b) x(½) = x(0) + (γ −mµ)/2,

(c) For all y > x(0), g(y)− g(x(0)) = y − x(0).

203

On the other hand, if x(0) < x, then each of (a), (b) and (c) above are violated. In

particular,

(a’) There exists s ∈ [0, ½) such that x(s) < m,

(b’) x(½) > x(0) + (γ −mµ)/2,

(c’) For all y > x(0), g(y)− g(x(0)) < y − x(0).

The proof is rather lengthy but no difficult, so it is deferred to Section 6.11. We

now use this to analyze the fluid limits of LS and DA. Let g1LS(q) and g2

LS(q) be the

function g from Lemma 6.9 when the parameters (λ1, c, µ, q) and (λ2, c, µ, q) are used,

respectively. Let fLS : TLS → TLS be given by

fLS(q, r)∆= Γ(g2

LS(g1LS(q)), re−µ; c). (6.51)

Similarly, let gDA(q) be the function from Lemma 6.9 using parameters (λ1, 2c, µ, q),

and let

h1DA(q, r)

∆= Γ(gDA(q), 0; 2c), (6.52)

h2DA(q, r)

∆= Γ

(q +

λ2

2, re−µ/2; 0

), (6.53)

fDA(s)∆= h2(h1(s)). (6.54)

We now prove Proposition 6.1.

Proof of Proposition 6.1. To prove existence and uniqueness of sθ(t), it suffices to

show that for every k ∈ Z+, sθ(t) exists and is uniquely defined for all 0 ≤ t ≤ k.

For k = 0, we are given sθ(0) in the statement of the proposition. Suppose the claim

holds for k. We now consider cases on θ.

LS – By induction sLS(k) is uniquely defined. As qLS(t) solves the differential equation

on [k, k + ½) as used to define g1LS(q) with initial condition qLS(k), and likewise

solves the differential equation on [k + ½, 1) used to define g2LS(q) with initial

condition qLS(k+½), we obtain by Lemma 6.9 that qLS(t) is uniquely determined

on [k, k+ 1). As rLS(t) = −µrLS(t) and by induction rLS(k) is uniquely defined,

204

we obtain that for t ∈ [k, k+ 1), r(t) = r(k)e−µ(t−k), uniquely defining sLS(t) on

that interval as well. Thus we immediately obtain that for every sLS(k) ∈ TLS,

sLS(k + 1) = fLS(sLS(k)),

showing the hypothesis.

DA – The argument is similar. Briefly, for all sDA(k) ∈ TDA,

sDA(k + ½) = h1(sDA(k)),

sDA(k + 1) = h2(sDA(k + ½)),

sDA(k + 1) = fDA(sDA(k)),

and at intermediate times in t ∈ (k, k + ½) and t ∈ (k + ½, k + 1), sDA(t) is the

unique solution to a linear ODE either with constant coefficients or of the type

from Lemma 6.9.

Finally, we must show that sθ(k) ∈ Tθ, k ≥ 1. For LS, we must show that rLS(k) < c

implies qLS(k) ≤ c. By definition

sLS(k) = Γ(sLS(k−); c) =

(qLS(k−)− c+ rLS(k−), c) qLS(k−) ≥ c,

(rLS(k−), qLS(k−)) qLS(k−) < c.

Suppose rLS(k) < c. Note that rLS(k) < c only in the second case. As qLS(k) =

rLS(k−) ≤ rLS(k − 1) ≤ c, we obtain that qLS(k) ≤ c, showing the claim. For DA,

must simply show that rDA(k) = 0, which holds as

rDA(k) = Γ(sDA(k−); 0) = (q(k−) + r(k−), 0).

Next we give some simple structural properties of the functions giLS and fθ that

will be needed in the analysis of the long run behavior of the fluid limits.

205

Lemma 6.10. There exists a unique qLS such that when qLS(0) ≥ qLS,

(a) qLS(t) ≥ c for t ∈ [0, 1),

(b) qLS(1−) = qLS(0) + λ− cµ,

(c) For q > qLS(0), g2LS(g1

LS(q))− g2LS(g1

LS(qLS(0))) = q − qLS(0).

and when qLS(0) < qLS, (a), (b) and (c) are violated. In particular,

(a’) There exists s ∈ [0, 1) such that qLS(s) < c,

(b’) qLS(1−) > qLS(0) + λ− cµ,

(c’) For q > qLS(0), g2LS(g1

LS(q))− g2LS(g1

LS(qLS(0))) < q − qLS(0).

Proof. For i = 1, 2, let qiLS be x from Lemma 6.9 when used to create giLS. It is easy

to see that properties (a), (b) and (c) will hold when for all i = 1, 2, properties (a),

(b) and (c) from the application Lemma 6.9 to create giLS hold, i.e. we have both

qLS(0) ≥ q1LS and qLS(½) ≥ q2

LS.

Similarly, it is easy to see that properties (a’), (b’) and (c’) will hold if there exists

i ∈ 1, 2 such that that properties (a’), (b’) and (c’) from the application of Lemma

6.9 to create giLS hold, i.e. if either qLS(0) < q1LS or qLS(½) < q2

LS.

As qLS(½) = g1LS(qLS(0)) and by Lemma 6.4 g1

LS is strictly increasing, there is a

threshold q∗ such that qLS(½) ≥ q2LS iff qLS(0) ≥ q∗. Thus by taking qLS = maxq∗, q1

LS,

we will have both qLS(0) ≥ q1LS and qLS(½) ≥ q2

LS iff qLS(0) ≥ qLS. This gives the

result.

Lemma 6.11. For θ ∈ LS,DA, fθ is 1-Lipschitz with respect to the `1 norm and

both outputs of the function fθ are monotonically increasing in both inputs.

Proof. For LS, the functions g1LS and g2

LS are monotonically increasing and 1-Lipschitz

by Lemma 6.9. Similarly re−µ as a function of r is increasing and 1-Lipschitz, and

by Lemma 6.4, Γ(·; c) is 1-Lipschitz and each component is monotonically increasing

in every input. Thus fLS(q, r) is a composition of monotone increasing 1-Lipschitz

functions and thus monotone increasing and 1-Lipschitz.

The argument is similar for DA. The functions gDA, q + λ2/2 as a function of q,

re−µ/2 as a function of r, Γ(·; 2c) and Γ(·; 0) are all 1-Lipschitz and monotonically

increasing in every input (by Lemma 6.9 for gDA and by Lemma 6.4 for Γ(·, 2c)

206

and Γ(·, 0)). Therefore fDA(q, r) is a composition of 1-Lipschitz monotone increasing

functions and thus 1-Lipschitz and monotone increasing.

We can now analyze the long run behavior of the fluid limits. First we will

show that when ρθ < 1, fθ restricted to some Tθ ⊂ Tθ has an attractive fixed point

using contractive mapping (Proposition 6.3). Then we will use a Lyapunov function

argument to show that the attractive point over Tθ is in fact attractive over all of Tθ(Proposition 6.4). Finally, we will use another Lyapunov function argument to show

that fθ has no attractive points when ρθ ≥ 1 (Proposition 6.5).

Proposition 6.6. The process sθ(k) has a unique attractive fixed point iff ρθ < 1.

Proof. Assume ρLS < 1. Recall qLS from Lemma 6.10, and let qDA be x from Lemma

6.9 as applied to create gDA. Let

Tθ∆= Tθ \ (q, r) | q > qθ. (6.55)

We now check the assumptions of Proposition 6.3 are satisfied by fθ restricted to

Tθ. We can immediately verify by definition that fθ is the composition of continuous

functions and thus continuous (recall that g1LS, g2

LS, and gDA are continuous by Lemma

6.9 and Γ(·, κ) is continuous for all κ by Lemma 6.4). Noting that qLS ≥ c by part

(a) of Lemma 6.10, we see that TLS = [0, c]× [0, c]∪ (q, c) | 0 ≤ q ≤ qLS and thus it

is a nonempty compact set. Likewise TDA = (q, 0) | 0 ≤ q ≤ qDA where by Lemma

6.9 we see that qDA ≥ 2c, thus TDA is a nonempty compact set as well. We still need

to check that fθ : Tθ → Tθ and that fθ is contractive on Tθ.

To show fLS : TLS → TLS, we have that for any (q, r) ∈ TLS,

fLS(q, r) ≤ fLS(qLS, c) (6.56)

= Γ(qLS + λ− cµ, ce−µ; c

)(6.57)

=(qLS + λ− cµ− c+ ce−µ, c

)(6.58)

≤ (qLS, c), (6.59)

207

where the inequalities are componentwise. Here (6.56) holds by Lemma 6.11. We

obtain (6.57) by Lemma 6.10 part (b). Then (6.58) holds as we have qLS + λ− cµ =

qLS(1−) ≥ c by part (a) of Lemma 6.10. Finally (6.59) holds as ρLS < 1 implies

λ− cµ− c+ ce−µ = −γLS < 0. Thus we obtain that fLS : TLS → TLS.

To show fDA : TDA → TDA, we first compute that

h1(qDA, 0) = Γ

(qDA +

λ1

2− cµ, 0; 2c

)(6.60)

=

(qDA +

λ1

2− cµ− 2c, 2c

). (6.61)

Here (6.60) follows from part (b) of Lemma 6.9 on gDA, and (6.61) follows from part

(a) of the lemma. Thus for all (q, 0) ∈ TDA,

fDA(q, 0) ≤ fDA(qDA, 0) (6.62)

= h2

(q +

λ1

2− cµ− 2c, 2c

)= Γ

(q +

λ1

2− cµ− 2c+

λ2

2, 2ce−µ/2; 0

)=(q + λ− cµ− 2c+ 2ce−µ/2, 0

)≤ (qDA, 0). (6.63)

where (6.62) holds by the monotonicity of fDA and (6.63) holds as ρDA < 1. Again

the inequalities are componentwise. Thus we obtain that fDA : TDA → TDA.

We show that fLS is contractive on TLS with respect to the ‖ · ‖1 norm. Consider

(q, r), (q′, r′) ∈ TLS, such that (q, r) 6= (q′, r′). If q 6= q′, then by part (c’) of Lemma

6.10

|g2LS(g1

LS(q))− g2LS(g1

LS(q′))| < |q − q′|. (6.64)

Similarly, when r 6= r′, then

|re−µ − r′e−µ| < |r − r′|. (6.65)

208

Thus we obtain that

‖fLS(q, r)− fLS(q′, r′)‖1 = ‖Γ(g2LS(g1

LS(q)), re−µ; c)− Γ(g2LS(g1

LS(q′)), r′e−µ; c)‖1

≤ |g2LS(g1

LS(q))− g2LS(g1

LS(q′))|+ |re−µ − r′e−µ| (6.66)

< |q − q′|+ |r − r′| (6.67)

= ‖(q, r)− (q′, r′)‖1.

Here (6.66) holds by Lemma 6.4 and (6.67) follows from (6.64) if q 6= q′ and from

(6.65) if r 6= r′.

Showing that fDA is contractive on TDA with respect to the ‖ · ‖1 norm is very

similar to the LS case. Briefly, we observe that when q < qDA, that h1DA is strictly

contractive by (c’) of Lemma 6.9. It is easy to see that h2DA is non-expansive for all

(q, r) ∈ R2+. Thus fDA on TDA is the composition of a contractive function and a

non-expansive function and hence contractive.

Thus the assumptions of Proposition 6.3 are satisfied by fθ on Tθ when ρθ < 1.

This implies that once sθ(k) enters Tθ it will converge to a unique fixed point. For

the case ρθ < 1, it remains to show that for sθ(0) 6∈ Tθ, we reach Tθ in finite time.

To this end, we apply Proposition 6.4 using the Lyapunov function V (q, r)∆= q+r,

the set of exceptions as Tθ, and γ to be γθ (we have γθ > 0 as ρθ < 1). We now show

that the drift condition is satisfied. For LS, (q, r) 6∈ TLS implies q ≥ qLS ≥ c, thus we

must have r = c. Thus,

V (fLS(q, c))− V (q, c) = V (Γ(g2LS(g1

LS(q)), ce−µ; c))− (q + c)

= g2LS(g1

LS(q)) + ce−µ − (q + c) (6.68)

= q + λ− cµ+ ce−µ − (q + c) (6.69)

= −γLS.

Here (6.68) follows from the same argument justifying (6.26), and (6.69) follows from

209

part (b) of Lemma 6.10. For DA, (q, 0) 6∈ TDA implies q ≥ qDA. We obtain

h1(q, 0) =

(q +

λ1

2− cµ− 2c, 2c

),

just as we justified (6.60) and (6.61). Thus for such q,

V (fDA(q, 0))− V (q, 0) = V

(h2

(q +

λ1

2− cµ− 2c, 2c

))− q

= V (Γ(q + λ− cµ− 2c, 2ce−µ/2; 0))− q

= −γDA.

Thus the assumptions of Proposition 6.4 are satisfied, establishing the claim in the

case when ρθ < 1.

Finally, we show that sθ(k) has no attractive point when ρθ ≥ 1 by applying

Proposition 6.5. We again take V (q, r) = q + r, and immediately verify that it is

continuous and unbounded on Tθ. For LS, we compute that for any sLS(0),

V (sLS(1))− V (sLS(0)) = qLS(1−)− qLS(0) + rLS(1−)− rLS(0) (6.70)

=

∫ 1

0

λ(t)− µ(qLS(t) ∧ c)dt− rDA(0)(1− e−µ) (6.71)

≥ λ− cµ− c(1− e−µ) (6.72)

= −γLS

≥ 0, (6.73)

where (6.70) follows similarly to (6.26), (6.71) follows from the definition of qLS(t),

(6.72) follows as rLS(0) ≤ c and qLS(t) ∧ c ≤ c, and finally (6.73) follows as ρLS ≤ 1.

210

For DA, we compute that for any sDA(0) that

V (sDA(1))− V (sDA(0)) = qDA(1−)− qDA(0) + rDA(1−) (6.74)

=λ2

2+ qDA(½)− qDA(0) + rDA(½)e−µ/2

=λ2

2+ qDA(½) + rDA(½)− qDA(0)− rDA(½)(1− e−µ/2)

≥ λ2

2+ qDA(½) + rDA(½)− qDA(0)− 2c(1− e−µ/2) (6.75)

=λ2

2+ qDA

(½−)− qDA(0)− 2c(1− e−µ/2) (6.76)

=λ2

2+

∫ ½

0

λ1 − µ(qDA(t) ∧ 2c)dt− 2c(1− e−µ/2) (6.77)

≥ λ−∫ ½

0

2cµ dt− 2c(1− e−µ/2)

= −γDA

≥ 0, (6.78)

where (6.74) follows similarly to (6.26), (6.75) follows as rDA(½) ≤ 2c, (6.76) follows

as by Γ, qDA(½−)+rDA(½−1) = qDA(½), (6.77) follows from the definition of qDA(t), and

finally (6.78) holds as ρDA ≥ 1. Thus we see by Proposition 6.5 that ρθ ≥ 1 implies

that sθ(k) has no attractive point, completing the proof of Proposition 6.6.

Finally, we give a result providing a uniform bound on the distance moved towards

the fixed point in each iteration of the fluid model. This result will be useful in the

proof of Theorem 6.3. Let Vθ : Tθ → R+ be given by

Vθ(s)∆= ‖s− sθ(∞)‖1. (6.79)

Corollary 6.3. For each θ ∈ LS,DA, when ρθ < 1, for every z > 0, there exists

γ > 0 such that

infs∈Tθ\Bz(sθ(∞))

Vθ(fθ(s))− Vθ(s) ≤ −γ.

Proof. Recall from in the proof of Proposition 6.6 that for each θ, we defined sets Tθ

211

such that that fθ is contractive on Tθ and for all s ∈ Tθ \ Tθ,

V (fθ(s))− V (s) = −γθ < 0.

Suppose s = (q, r) 6∈ Tθ. Trivially s ≥ sθ(∞) componentwise as sθ(∞) ∈ Tθ. By

Lemma 6.11, as s ≥ sθ(∞) componentwise, we have fθ(s) ≥ fθ(sθ(∞)) = sθ(∞)

componentwise as well. Thus for s 6∈ Tθ, letting (q′, r′) = fθ(s),

Vθ(fθ(s))− Vθ(s) = ‖fθ(s)− sθ(∞)‖1 − ‖s− sθ(∞)‖1

= q′ − qθ(∞) + r′ − rθ(∞)− (q − qθ(∞) + r − rθ(∞))

= V (fθ(s))− V (s)

≤ −γθ.

For all s ∈ Tθ, s 6= sθ(∞), as fθ is contractive on Tθ, we have

Vθ(fθ(s)) = ‖fθ(s)− sθ(∞)‖1

= ‖fθ(s)− fθ(sθ(∞))‖1

< ‖s− sθ(∞)‖1

= Vθ(s).

Thus for s ∈ Tθ, Vθ(fθ(s)) − Vθ(s) ≤ 0, holding with equality only when s = sθ(∞).

Fix z from the statement of the Lemma. As Vθ(fθ(s)) − Vθ(s) is a composition of

continuous functions and thus continuous and as Tθ is compact (as shown in the

proof of Proposition 6.6), we have that given our z there exists ε > 0 such that

infs∈Tθ\Bz(sθ(∞)

Vθ(fθ(s))− Vθ(s) ≤ −ε.

Thus by taking γ = minε, γθ, we obtain the result.

212

6.9 Interchange of Limits. Proof of Theorem 6.3

In this section, we prove Theorem 6.3, showing that the rescaled steady state distri-

butions Snθ (∞)/n converge in probability to the fixed point sθ(∞) of the fluid limit

at integer times.

Recall that a set of random vectors Xn is defined to be tight if for every ε there

exists k such that for every n, P(‖Xn‖1 > k) ≤ ε. As a direct consequence of Lemma

6.8 and Markov’s inequality, we obtain:

Corollary 6.4. For each policy θ ∈ LS,DA, when ρθ < 1, the set of random vectors

Snθ (∞)/n is tight.

By Prokhorov’s theorem, this implies that Xn is relatively compact. That is,

for every subsequence Xni there exists a random vector X and a subsubsequence Xnij

such that Xnij⇒ X (see [15]). Thus for every subsequence ni there is a subsubse-

quence nij and a random vector Sθ such that as j →∞

Snijθ (∞)

nij⇒ Sθ.

Thus to show Theorem 6.3, it is sufficient to show that for every sequence ni, the

resulting Sθ equals sθ(∞) with probability one, as convergence in distribution to a

constant implies convergence in probability.

Proof of Theorem 6.3. First, we claim that for θ ∈ LS,DA,

fθ(Sθ)d= Sθ, (6.80)

where fLS and fDA are defined by (6.51) and (6.54), respectively. By Proposition 11.3.2

from [28], we can equivalently check that the Levy-Prokhorov distance between these

variables is zero, i.e. that for all g : Tθ → R such that ‖g‖BL ≤ 1, we have

E[g(fθ(Sθ))− g(Sθ)] = 0.

213

Here, we use the three term estimate as devised by [31], Chapter 4, Theorem 9.10, in

a similar continuous time interchange of limits argument. See [79] for similar but less

terse argument. Assume for every n that Snθ (0)d= Snθ (∞). Now for every n, we have

∣∣E[g(fθ(Sθ))− g(Sθ)]∣∣ ≤ ∣∣∣∣E [g(fθ(Sθ))− g

(fθ

(Snθ (0)

n

))]∣∣∣∣+

∣∣∣∣E [g(fθ

(Snθ (0)

n

))− g

(Snθ (1)

n

)]∣∣∣∣+

∣∣∣∣E [g(Snθ (1)

n

)− g

(Sθ)]∣∣∣∣ .

As fθ and g are continuous and g is bounded, g fθ is a bounded continuous func-

tion. For the first term, Snθ (0)/nd= Snθ (∞)/n ⇒ Sθ, so we can apply the Con-

tinuous Mapping Theorem and then the Bounded Convergence Theorem to ob-

tain that E[g (fθ (Snθ (0)/n))] → E[g(fθ(Sθ))] as n → ∞ along nij . By stationarity

Snθ (1)/nd= Snθ (0)/n

d= Snθ (∞)/n, implying that the third term converges to zero along

nij by a similar argument.

Finally we bound the second term. Let hnθ : Tθ → R and hθ : Tθ → R be given by

hnθ (s)∆= Ebnsc

[g

(Snθ (1)

n

)],

hθ(s)∆= g(fθ(s)),

so that∣∣∣∣E [g(fθ

(Snθ (0)

n

))− g

(Snθ (1)

n

)]∣∣∣∣ =

∣∣∣∣E [hθ (Snθ (0)

n

)− hnθ

(Snθ (0)

n

)]∣∣∣∣ . (6.81)

We need some properties of hnθ and h to make an estimate. First, we claim that for

all s, hnθ (s) → hθ(s) as n → ∞. As a consequence of Theorem 6.2, we have that for

every s ∈ Tθ, if Snθ (0) = bnsc so that Snθ (0)/n → s a.s., then Snθ (1)/n → fθ(s) a.s.

as well. By the continuity of g, it follows from that g(Snθ (1)/n) → g(fθ(s)) a.s. as

well. Noting that g(Snθ (1)/n) is bounded, we can apply the Bounded Convergence

214

Theorem to obtain that for s ∈ Tθ,

limn→∞

hnθ (s) = hθ(s). (6.82)

We can now bound (6.81) with a coupling argument. By the Skorokhod Representa-

tion Theorem, let Ω be a common probability space for Snθ (0) and Sθ such that for

ω ∈ Ω, Snθ (0, ω)→ Sθ(ω) a.s. Now we have∣∣∣∣hθ (Snθ (0, ω)

n

)− hnθ

(Snθ (0, ω)

n

)∣∣∣∣ ≤ ∣∣∣∣hθ (Snθ (0, ω)

n

)− hθ

(Sθ(ω)

)∣∣∣∣+∣∣hθ (Sθ(ω)

)− hnθ

(Sθ(ω)

)∣∣+

∣∣∣∣hnθ (Sθ(ω))− hnθ

(Snθ (0, ω)

n

)∣∣∣∣ .We claim each of these terms converges to zero a.s. The first term converges to zero as

hθ is a continuous function and Snθ (0, ω)/n→ Sθ(ω) a.s. The second term converges

to zero by (6.82). Let Snθ (t) be another version of the process Snθ (t) with the initial

condition Snθ (0) = bnSθc that is coupled to Snθ (t) as in Corollary 6.2. Then∣∣∣∣hnθ (Sθ(ω))− hnθ

(Snθ (0, ω)

n

)∣∣∣∣=

∣∣∣∣∣E[g

(Snθ (1)

n

)− g

(Snθ (1)

n

) ∣∣∣∣∣ Snθ (0) = bnSθ(ω)c, Snθ (0) = Snθ (0, ω)

]∣∣∣∣∣≤ ‖g‖BL

nE

[∥∥∥Snθ (1)− Snθ (1)∥∥∥

1

∣∣∣∣∣ Snθ (0) = bnSθ(ω)c, Snθ (0) = Snθ (0, ω)

]

≤ ‖g‖BL

n

∥∥bnSθ(ω)c − Snθ (0, ω)∥∥

1(6.83)

≤ ‖g‖BL

(∥∥∥∥Sθ(ω)− Snθ (0, ω)

n

∥∥∥∥1

+1

n

),

showing that the third term converges to zero a.s. as well. Here (6.83) follows from

Corollary 6.2. Finally, as hnθ and hθ are bounded by one, the Bounded Convergence

Theorem implies that the right hand side of (6.81) converges to zero. Thus we have

shown (6.80).

Now we show that Sθ = sθ(∞) a.s. We assume the conclusion is false to show

215

a contradiction. By assumption, there exists some sθ 6= sθ(∞) and ε > 0 such that

sθ(∞) 6∈ Bε(sθ) and

P(Sθ ∈ Bε(sθ)) > 0.

Let N be such that

P(‖Sθ‖1 > N) < P(Sθ ∈ Bε(sθ)). (6.84)

Let z = ‖s − sθ(∞)‖1 − ε and let Z = Bz(sθ(∞)) be the largest ball around sθ(∞)

disjoint from Bε(sθ). We now use Vθ from (6.79), and let

−d ∆= inf

s6∈ZVθ(fθ(s))− Vθ(s).

Note that d > 0 by Corollary 6.3. Let n ∈ Z+ be such that

nd > sup‖s‖1<N

Vθ(s),

(the supremum is bounded as s ∈ Tθ|‖s‖1 < N is compact and Vθ is continuous).

Let f(m)θ be the function fθ composed with itself m times. We now claim that for all

s ∈ Tθ,

f (n)(s) 6∈ Z implies that ‖s‖1 > N. (6.85)

We show the contrapositive using Proposition 6.4. We take our bounded set of ex-

ceptions as Z, use the Lyapunov function Vθ, and drift −d. The drift condition is

satisfied as d > 0. Thus ‖s‖1 < N implies that there exists m with 0 ≤ m ≤ n such

that f(m)θ (s) ∈ Z. Notice that Corollary 6.3 implies that fθ(Z) ⊂ Z. Thus ‖s‖1 < N

in fact implies that f(n)θ (s) ∈ Z, showing (6.85).

216

Thus we have the inequalities

P(‖Sθ‖1 > N) < P(Sθ ∈ Bε(sθ)) (6.86)

= P(f(n)θ (Sθ) ∈ Bε(sθ)) (6.87)

≤ P(f(n)θ (Sθ) 6∈ Z) (6.88)

≤ P(‖Sθ‖1 > N), (6.89)

where (6.86) follows from (6.84), (6.87) follows from (6.80), (6.88) follows as Z and

Bε(sθ) are disjoint, and (6.89) follows from (6.85). Thus we have obtained a contra-

diction, which shows that Sθ equals sθ(∞) with probability one. This completes the

proof.

6.10 Convergence of Reassignments in the Fluid

Limit

Before proving Corollary 6.1, we need a simple monotonicity result for the fluid limit

qθ(t).

Lemma 6.12. Under the assumptions of Corollary 6.1, for θ ∈ LS,DA, qθ(t) is

monotone on [0, ½) and [½, 1). Further, for any t∗ ∈ [0, 1] such that qLS(t∗) = c, (resp.

t∗ ∈ [0, ½) such that qDA(t∗) = 2c), qθ(t) is strictly monotone in a neighborhood of

t∗. Consequently, there exists ε0 such that for every ε < ε0 there exists δ such that

|qθ(t)− qθ(t∗)| < δ implies |t− t∗| < ε.

Proof. The monotonicity on [0, ½) and [½, 1) follows as on each such interval, qθ(t) is

the solution to the differential equation of the type from Lemma 6.9. For LS, for

any t∗ such that qLS(t∗) = c, we have qLS(t∗) = λ(t∗)− cµ. As we have assumed that

λ1, λ2 6= cµ, we have that qLS(t∗) 6= 0. Noting that qLS(t) is continuously differentiable,

it follows that qLS(t) is strictly monotone. A similar argument applies for DA.

Finally, we show the convergence of the number reassignments (the number of

217

patients forced to wait per day), completing the commutative diagram in Figure 6-1.

Proof of Corollary 6.1. We first show that W 1,nLS (∞)/n → w1

LS(∞) in probability.

Noting that the limit is a constant, it sufficient to show convergence in distribution.

By Proposition 11.3.3 from [28], W 1,nLS (∞)/n ⇒ w1

LS(∞) iff for all g : R+ → R such

that ‖g‖BL ≤ 1, we have

limn→∞

E[g

(W 1,n

LS (∞)

n

)]= g(w1

LS(∞)).

For each n we take each SnLS(0)d= SnLS(∞). Using Theorem 6.3 and the Skorokhod

Representation Theorem, we put the SnLS(0) on a common probability space such that

SnLS(0)/n → sLS(∞) a.s. We use this process to generate the W 1,nLS (∞) and w1

LS(∞)

all on a common probability space.

It follows from Lemma 6.12 that there can be at most one time t∗ ∈ [0, ½) such that

qLS(t∗) = c, and that qLS(t) must be strictly monotone at t∗. Let ε0 be from the lemma

and fix some ε ∈ (0, ε0). Let δ be from Lemma 6.12 such that |qLS(t) − qLS(t∗)| < δ

implies |t− t∗| < ε. Recall from Theorem 6.2 that when SnLS(0)/n→ sLS(0) a.s., then

sup0≤t≤½ ‖SnLS(t)/n− sLS(t)‖1 → 0 a.s. As |QnLS(t)/n− qLS(t)| ≤ ‖SnLS(t)/n− sLS(t)‖1,

we also have sup0≤t≤½ |QnLS(t)/n−qLS(t)| → 0 a.s. As a result, we also have convergence

in probability. In particular, for our δ,

limn→∞

P

(sup

0≤t≤½

∣∣∣∣Qn(t)

n− q(t)

∣∣∣∣ > δ

)= 0.

Let Enδ be the event

Enδ

∆=

sup

0≤t≤½

∣∣∣∣Qn(t)

n− q(t)

∣∣∣∣ > δ

2

,

i.e. limn→∞ P(Enδ ) = 0 for all δ. Let En

δ denote the complement of this event. We

218

have that∣∣∣∣E [g(W 1,nLS (∞)

n

)]− g(w1

LS(∞))

∣∣∣∣≤

∣∣∣∣∣E[g

(W 1,n

LS (∞)

n

)− g(w1

LS(∞))

∣∣∣∣∣ Enδ

]∣∣∣∣∣P(Enδ )

+

∣∣∣∣∣E[g

(W 1,n

LS (∞)

n

)− g(w1

LS(∞))

∣∣∣∣∣ Enδ

]∣∣∣∣∣P(Enδ )

≤ 2‖g‖BLP(Enδ ) + ‖g‖BLE

[∣∣∣∣W 1,nLS (∞)

n− w1

LS(∞)

∣∣∣∣∣∣∣∣∣ En

δ

], (6.90)

where in (6.90), we are using both that g is bounded by ‖g‖BL and has Lipschitz

constant at most ‖g‖BL. Letting n → ∞, we see the first term go to zero. For the

second term, we consider two cases:

1. Suppose that inft∈[0,½) |qLS(t) − c| = γ > 0. We can assume without loss of

generality that δ < γ, as we can always take δ smaller without interfering in

the convergence of our first term, and doing so will only increase the proposed

infimum. As δ < γ, we ensure that conditional on Enδ , for every time t ∈ [0, ½)

that ∣∣∣∣QnLS(t)

n− c∣∣∣∣ ≥

∣∣∣∣∣|qLS(t)− c| −∣∣∣∣Qn

LS(t)

n− qLS(t)

∣∣∣∣∣∣∣∣∣ (6.91)

= |qLS(t)− c| −∣∣∣∣Qn

LS(t)

n− qLS(t)

∣∣∣∣ (6.92)

≥ γ/2,

where (6.91) is the reverse triangle inequality and (6.92) follows by our choice

of δ. We have two further cases:

(a) If qLS(t) < c and thus QLS(t)/n < c for all t, then both W 1,nLS (∞) and

w1LS(∞) will be zero, so we will have that (6.90) converges to zero as

n→∞.

(b) Similarly, if qLS(t) > c and thus QLS(t)/n > c for all t, then W 1,nLS (∞) =

219

An(0, ½) and w1LS(∞) = λ1/2. Thus

E

[∣∣∣∣W 1,nLS (∞)

n− w1

LS(∞)

∣∣∣∣∣∣∣∣∣ En

δ

]=

1

P(Enδ

)E [∣∣∣∣W 1,nLS (∞)

n− w1

LS(∞)

∣∣∣∣ IEnδ ]=

1

P(Enδ

)E [∣∣∣∣An(0, ½)

n− λ1

2

∣∣∣∣ IEnδ ]≤ 1

P(Enδ

)E [∣∣∣∣An(0, ½)

n− λ1

2

∣∣∣∣] .The above converges to zero almost surely since E[An(0, ½)/n] → λ1/2 as

n→∞. Thus (6.90) converges to zero as n→∞.

2. Suppose instead that qLS(t) crosses c. Suppose λ1 > cµ, so by Lemma 6.12

qLS(t) is monotonically increasing. Let t∗ be the time such that qLS(t∗) = c.

Then

w1LS(∞) =

∫ ½

t∗λ1dt = (½− t∗)λ1.

We claim that conditional on Enδ ,

|t− t∗| ≥ ε implies that |QnLS(t)/n− c| ≥ δ/2. (6.93)

We will show the contrapositive. We have that when |QnLS(t)/n− c| < δ/2, then

|qLS(t)− c| ≤∣∣∣∣qLS(t)− Qn

LS(t)

n

∣∣∣∣+

∣∣∣∣QnLS(t)

n− c∣∣∣∣ < δ.

Now by Lemma 6.12, |qLS(t)− c| < δ implies |t− t∗| < ε, showing the claim.

Next, we claim that then conditional on Enδ , for all t > t∗ + ε, Qn

LS(t)/n > c.

220

Assume not for contradiction. Then

0 ≤ c− QnLS(t)

n− δ

2(6.94)

≤ c− qLS(t) +

∣∣∣∣QnLS(t)

n− qLS(t)

∣∣∣∣− δ

2

≤ c− qLS(t) (6.95)

< 0, (6.96)

giving a contradiction. Here (6.94) holds by (6.93) in conjunction withQnLS(t)/n <

c, (6.95) holds as we are assuming Enδ , and finally (6.96) holds as λ1 > cµ and

Lemma 6.12 implies that qLS(t) is increasing in t, qLS(t∗) = c, and t > t∗.

By an analogous argument, we can show that for all t < t∗ − ε, QnLS(t)/n < c.

As the number of reassignments W 1,nLS (∞) is the number of arrivals such that

QnLS(t) ≥ cn at the time t of arrival, we thus have that all arrivals An(t∗+ ε, ½),

will be reassignments, none of the arrivals An(0, t∗ − ε) will be reassignments,

and the remaining arrivals are to be determined. This implies

An(t∗ + ε, ½)IEnδ ≤ W 1,nLS (∞)IEnδ ≤ An(t∗ − ε, ½)IEnδ .

Thus

E

[∣∣∣∣W 1,nLS (∞)

n− w1

LS(∞)

∣∣∣∣∣∣∣∣∣ En

δ

]≤ E

[ ∣∣∣∣A1,nLS (t∗ + ε)

n− w1

LS(∞)

∣∣∣∣+

∣∣∣∣A1,nLS (t∗ − ε)

n− w1

LS(∞)

∣∣∣∣∣∣∣∣∣ En

δ

]

≤ E

[ ∣∣∣∣A1,nLS (t∗ + ε)

n− w1

LS(∞)

∣∣∣∣+

∣∣∣∣A1,nLS (t∗ − ε)

n− w1

LS(∞)

∣∣∣∣]/

P(Enδ )

≤ 2ελ1/P(Enδ ).

Letting n → ∞, the above converges to 2ελ1. As ε was arbitrary, the above

221

term and thus (6.90) must converge to zero as n→∞.

It is not hard to see that if instead λ1 < cµ, then as qLS(t) will be decreasing,

we will obtain a similar bound of the form

An(0, t∗ − ε)IEnδ ≤ W 1,nLS (∞)IEnδ ≤ An(0, t∗ + ε)IEnδ ,

After taking expectations, we could again show the convergence of (6.90) to

zero. Thus we conclude that W 1,nLS (∞)/n⇒ w1

LS(∞).

Showing that W 2,nLS (∞) ⇒ w2

LS(∞), W 1,nDA(∞)/n ⇒ w1

DA(∞), and W 2,nDA(∞)/n ⇒

w2DA(∞) is very similar and the details are omitted.

6.11 Proof of Lemma 6.9

Here we give a series of lemmas about the differential equation

x(t) = γ − µ(x(t) ∧m),

with x(0) ∈ R+, γ, µ,m > 0, that will ultimately allow us to prove Lemma 6.9. For

convenience, we let g(x, t) equal x(t) when x(0) = x (making the function g(x) defined

in Lemma 6.9 equal to g(x, ½)).

Lemma 6.13. The differential equation given by x(t) with initial condition x(0) ∈ R+

has a unique solution x(t) with x(t) ≥ 0 for all t ∈ [0, ½]. Further, for every x ∈ R+,

g(x, t) is either strictly increasing in t for all t, strictly decreasing in t for all t, or

equal to g(x, 0) for all t.

Proof. The differential equations

y(t) = γ −mµ,

z(t) = γ − z(t)µ,

with an initial condition y(s) ∈ R+ and z(s) ∈ R+ both have unique solutions for all

222

t > 0 given by

y(t) = y(s) + (t− s)(γ −mµ),

z(t) =γ

µ+ exp(−µ(t− s))

(z(s)− γ

µ

),

respectively. We claim that these two differential equations in combination determine

the path of x(t). Given our formula for x(t), we observe that x(t) ≥ m and x(t) = y(t)

implies x(t) = y(t). Suppose that for some s we have x(s) ≥ m and let y(s) = x(s).

Then for all t ≥ s such that y(t) ≥ m, we will have x(t) = y(t). Analogously, we

observe that x(t) < m and x(t) = z(t) implies that x(t) = z(t). Suppose that for

some s we have x(s) < m, and let z(s) = x(s). Then for all t ≥ s such that z(t) ≤ m,

we will have x(t) = z(t). Thus we can show that x(t) has a unique solution on [0, ½)

for all initial conditions by showing that there is a clean exchange at the boundary

t | x(t) = m. In particular, it suffices to show that we cross the boundary at most

one time, which follows from the monotonicity claim in the second part of the Lemma.

First however, we need to analyze the long run behavior of z(t). We can immedi-

ately see from z(t) that if z(0) = γ/µ, then z(t) = 0 so z(t) = γ/µ for all t. Similarly,

when z(t) < γ/µ, z will be strictly increasing at t, and when z(t) > γ/µ, z will be

strictly decreasing at t. Further, from the solution for z(t), we see that if at any time

s, z(s) < γ/µ then for all times t > s, we will still have z(t) < γ/µ. Namely, z will

approach γ/µ but never reach it. Likewise, when z(s) > γ/µ, we will have z(t) > γ/µ

for all t > s.

We can now finish the Lemma by considering three cases:

1. Suppose γ > mµ, or equivalently γ/µ > m. If x(0) < m, then as the attractive

point of z(t) is greater than m, we will have x(t) strictly increasing until either

time ½ or s such that z(s) = m, if s < ½. There is nothing left the prove in

the first case, so we consider the second. Once x(t) ≥ m, as γ ≥ mµ, we have

x(t) = y(t) = γ − mµ > 0, so x(t) will increase strictly and never again fall

before m. Thus in all cases, x(t) is strictly increasing for all t.

2. Suppose γ < mµ. Then if x(t) ≥ m, we will have x(t) = y(t) = γ −mµ < 0, so

223

x(t) will be strictly decreasing. Again there are two possibilities, either there

is a time s < ½ such that x(s) = m, or x(s) will not reach m before time ½.

As x(t) is uniquely defined in the second case, we need only consider the first

case further. Once we reach m, the dynamics of x(t) will be that of z(t). Recall

that z(t) will strictly decrease towards the fixed point γ/µ for all t > s when

z(s) < γ/µ. Thus for all x > γ/µ, g(x, t) is strictly decreasing in t. When

x(0) = γ/µ, then by our previous analysis of z(t) we have that g(x(0), t) = γ/µ

for all t. Finally, when x(0) < γ/µ, we know that x(t) will be strictly increasing

for all t towards γ/µ. Thus in all cases on x(0) the criteria of the Lemma are

met.

3. Suppose γ = mµ. Then for all x ≥ m, x(t) = γ −mµ = 0, so g(x, t) = g(x, 0).

For all x < m, by our analysis of z(t), we know that x(t) will be strictly

increasing towards γ/µ = m but never reach it.

Thus we can conclude that x(t) has a unique solution for all t ∈ [0, ½).

Lemma 6.14. When x > y, we have g(x, t) > g(y, t) for all t.

Proof. We will make a coupling argument. By Lemma 6.13, we know g(x, t) is either

strictly increasing in t, strictly decreasing in t, or constant. Assume for contradiction

that there is a time s such that g(y, s) ≥ g(x, s). We now consider cases:

1. Suppose that g(x, t) is strictly increasing in t. As g(y, t) is continuous in t, and

at time s, g(y, s) ≥ g(x, s) > g(x, 0), by the Intermediate Value Theorem there

must be some time r with 0 < r ≤ s such that g(y, r) = g(x, 0). But as x(t) is

not a function of t, only x(t), we thus obtain that g(y, s) = g(x, s−r) < g(x, s),

giving a contradiction.

2. Suppose that g(x, t) is constant. As g(y, t) is continuous in t, and at time s,

we have g(y, s) ≥ g(x, s) = g(x, 0), by the Intermediate Value Theorem there

is a time r ≤ s such that g(y, r) = g(x, 0). But then g(y, t) is constant at r,

contradicting that g(y, t) must either be strictly increasing, strictly decreasing,

or constant for all t.

3. Suppose that g(x, t) is strictly decreasing and g(y, t) is either strictly decreasing

224

or constant. Then by taking g(x, t)∆= −g(y, t) and g(y, t)

∆= −g(x, t), we can

apply cases one and two to g(x, t) to show the claim.

4. Finally, suppose that g(x, t) is strictly decreasing and g(y, t) is strictly increas-

ing. Under our assumption that g(y, s) ≥ g(x, s), again by the Intermediate

Value Theorem, there must be some time 0 < r ≤ s such that g(x, r) = g(y, r).

But as x(t) depends only x(t), not t, we would then have that for all t ≥ r,

g(x, t) = g(y, t). This creates a contradiction, as we have assumed that g(x, t)

is strictly decreasing in t and g(y, t) is strictly increasing in t.

Lemma 6.15. For x ≥ x as defined in Lemma 6.9, g(x, t) ≥ m for 0 ≤ t ≤ ½ and

g(x, ½) = x+ (γ− cµ)/2. For x < x, there exist 0 ≤ s < t ≤ ½ such that for τ ∈ (s, t),

g(x, τ) < m, and g(x, ½) < x+ (γ − cµ)/2.

Proof. We compute for 0 ≤ t ≤ ½ that

g(x, t) = x+

∫ t

0

γ − µ(m ∧ g(x, τ))dτ

≥ x+

∫ t

0

γ −mµdτ

= x+ t(γ −mµ). (6.97)

We first consider g(x, t) in cases:

1. Suppose that γ ≥ mµ. Then x = m, and as γ −mµ ≥ 0, we obtain from (6.97)

that g(x, t) ≥ m for all t ≥ 0.

2. Suppose that γ < mµ. Then x = m− (γ −mµ)/2, so by (6.97) for all t ≥ 0,

g(x, t) ≥ m+ (γ −mµ)

(t− 1

2

)≥ m.

We thus conclude that g(x, t) ≥ m for all 0 ≤ t ≤ ½. As a result, we can now make

225

the exact computation that for all x ≥ x,

g(x, ½) = x+

∫ ½

0

γ − µ(m ∧ g(x, t))dt

= x+

∫ ½

0

γ − µmdt (6.98)

= x+1

2(γ −mµ).

where (6.98) holds as g(x, t) ≥ g(x, t) ≥ m by Lemma 6.14 and then the above

analysis, making m ∧ g(x, t) = m for all t.

We now consider x < x, and find s and t such that for all τ ∈ (s, t), g(x, τ) < m,

as in the statement of the lemma. We consider cases:

1. Suppose γ > mµ and thus x = m. Then g(x, 0) = x < x = m so obviously we

can take s = 0 and t small to show the claim.

2. Suppose instead that γ < mµ and thus x = m− (γ−mµ)/2. For x < m, again

the claim obviously holds as then g(x, t) < m for all t ≤ ½. For x such that

m ≤ x < x, observe that as x = m− (γ −mµ)/2,

0 ≤ t∗∆=

x−mmµ− γ

<x−mmµ− γ

=1

2,

and thus

g(x, t∗) = x+ (γ −mµ)x−mmµ− γ

= m.

As x(t) = γ−µ(m∧x(t)) ≤ γ−mµ < 0 by our assumptions, we can take s = t∗

and t = ½.

Finally, using s and t from the statement of the lemma, we show that g(x(0), ½) >

226

x(0) + (γ − cµ)/2 when x(0) < x. We compute that

g(x, ½) = x+

∫ ½

0

γ − µ(g(x, τ) ∧m)dτ

≥ x+

∫ s

0

γ − µmdτ +

∫ t

s

γ − µg(x, τ)dτ +

∫ ½

t

γ − µmdτ

= x+ (γ −mµ)(½− (t− s)) +

∫ t

s

γ − µg(x, τ)dτ

> x+ (γ −mµ)/2.

The final inequality is strict as g(x, τ) < m for all τ ∈ (s, t).

Lemma 6.16. For all x > y ≥ x, where x is defined in Lemma 6.9,

g(x, ½)− g(y, ½) = x− y,

and for all x > y, y < x,

0 < g(x, ½)− g(y, ½) < x− y.

Proof. For x > y ≥ x, by Lemma 6.15, we have that

g(x, ½)− g(y, ½) = x+ (γ −mµ)/2− y − (γ −mµ)/2 = x− y.

For x > y, y < x, let s and t be from Lemma 6.15 such that for all τ ∈ (s, t),

g(y, τ) < m. Then

g(x, ½)− g(y, ½) = x− y + µ

∫ ½

0

(g(y, τ) ∧m)− (g(x, τ) ∧m)dτ.

As x > y, by Lemma 6.14 we have g(x, τ) > g(y, τ) for all τ , and thus g(x, τ) ∧m ≥

227

g(y, τ) ∧m for all τ , making the integrand nonpositive. Thus

g(x, ½)− g(y, ½) = x− y + µ

∫ t

s

(g(y, τ) ∧m)− (g(x, τ) ∧m)dτ

= x− y + µ

∫ t

s

g(y, τ)− (g(x, τ) ∧m)dτ

≤ x− y + µ

∫ t

s

g(y, τ)− g(x, τ)dτ

< x− y.

That g(x, ½)− g(y, ½) > 0 follows immediately from Lemma 6.14.

Proof of Lemma 6.9. The Lemma follows immediately from Lemma 6.13, Lemma

6.14, Lemma 6.15, and Lemma 6.16.

6.12 Null Recurrence and Transience

We distinguish between the null recurrent and transient cases using Proposition A.5

from Appendix A, Section A.4. As before, we will use the Lyapunov function V (q, r)∆=

q + r.

Lemma 6.17. For each θ ∈ LS,DA, the following limit exists, is finite, and is

non-zero:

Fθ∆= lim

q→∞(q,r)∈Sθ

E(q,r)

[(V (Sθ(1))− V (Sθ(0)))2] . (6.99)

Further,

sups∈Sθ

Es

[(V (Sθ(1))− V (Sθ(0)))4] <∞. (6.100)

Finally, when ρθ = 1, as q →∞, we have for r such that (q, r) ∈ Sθ that

E(q,r)[V (Sθ(1))− V (Sθ(0))] = O(exp(−q/2)). (6.101)

228

Proof. First consider θ = LS. Recall from (6.27) that for any ` ≥ 0,

E(q,c)

[(V (SLS(1))− V (SLS(0)))`

]= E(q,c)

[(A+Don

LS −DoffLS

)`].

Further, recall that we coupled DonLS with Don

LS that had distribution Pois(cµ) such

that DonLS < Don

LS, and DoffLS with Doff

LS that had distribution Bin(c, 1 − e−µ) such that

DoffLS ≤ Doff

LS. Using these couplings, we can show (6.100) just as we showed (6.25).

Recall that under our coupling, we have DoffLS = Doff

LS for initial (q, r) such that

r = c. Likewise, we have that DonLS was equal to Don

LS under an initial condition (q, r)

for those realizations where DonLS < q − c. This led to (6.30), namely that for q > c,

E(q,c) [DonLS] ≥ E

[Don

LSIDonLS<q−c

].

We now can show (6.101). Assuming ρθ = 1 and thus γθ = 0, we have

E(q,c)[V (SLS(1))− V (SLS(0))] = E[A−Doff

LS − DonLS

]+ E(q,c)

[Don

LS −DonLS

]≤ −γθ + E[Don

LSIDonLS≥q−c

]

≤

√E[(Don

LS

)2]E[IDon

LS≥q−c

]≤√

((cµ)2 + cµ)E[exp

(Don

LS − q + c)]

≤ exp(−q/2)√

((cµ)2 + cµ) exp(c) exp(cµ(e− 1))

= O (exp(−q/2)) ,

as q →∞. Here we use that for any random variable X, IX≥t ≤ exp(X − t).

It remains to show (6.99). We will show that

FLS = E[(A− DonLS − Doff

LS)2].

229

For any q > 0, observe that

E(q,c)

[(V (SLS(1))− V (SLS(0)))2] = E(q,c)

[(V (SLS(1))− V (SLS(0)))2 IDon

LS>q−c

]+ E(q,c)

[(V (SLS(1))− V (SLS(0)))2 IDon

LS≤q−c

].

For the second term, we have that

limq→∞

E(q,c)

[(V (SLS(1))− V (SLS(0)))2 IDon

LS≤q−c

]= lim

q→∞E[(A− Don

LS − DoffLS

)2

IDonLS≤q−c

](6.102)

= E[(A− Don

LS − DoffLS

)2]

(6.103)

= FLS,

where (6.102) follows by our coupling and (6.103) follows from the Monotone Con-

vergence Theorem. For the first term, we have

limq→∞

E(q,c)

[(V (SLS(1))− V (SLS(0)))2 IDon

LS>q−c

]≤ lim

q→∞

√E(q,c)

[(V (SLS(1))− V (SLS(0)))4]E(q,c)

[IDon

LS>q−c

]≤ lim

q→∞

√P(Don

LS − c > q)√

sups∈SLS

Es

[(V (SLS(1))− V (SLS(0)))4]

= 0,

where in the final equality we use (6.100). This shows (6.99) and thus the Lemma in

the case of LS. The case of DA is similar.

We now complete the proof of Theorem 6.1 by showing that for each θ ∈ LS,DA,

Sθ(k) is null recurrent when ρθ = 1 and transient when ρθ > 1.

Proof of Theorem 6.1. We have already established in section Appendix 6.5 that

when ρθ ≥ 1, Sθ(k) is either null recurrent or transient, and when ρθ < 1, Sθ(k)

is positive recurrent. Thus it suffices to show that Sθ(k) is recurrent when ρθ = 1

230

and transient otherwise. We proceed using Proposition A.5. We must check that the

assumptions of the proposition are satisfied by Sθ(k) for θ ∈ LS,DA. By (6.100)

from Lemma 6.17, (A.23) is satisfied. It is obvious that (A.22) is satisfied. Finally,

to check (A.21), we verify the sufficient condition (A.26). Recalling Corollary 6.2, it

is immediate that for all z > 0,

infs∈Sθ

P(V (Sθ(1)) ≥ z | Sθ(0) = s) = P(V (Sθ(1)) ≥ z | Sθ(0) = (0, 0)) > 0.

Now suppose ρθ = 1. We will show that (A.24) holds. For a constant bθ > 0,

we will take Bθ of the form (q, r) ∈ Sθ | q < bθ. By (6.101) from Lemma 6.17, as

q →∞, we have

E(q,r) [V (Sθ(1))− V (Sθ(0))] = O(exp(−q/2)),

and by (6.99) from Lemma 6.17, as q →∞,

E(q,r)

[(V (Sθ(1))− V (Sθ(0)))2]

2V (q, r)= Θ

(1

q

).

Thus by taking bθ sufficiently large and using that exp(−q/r) = o(1/q), we have

(A.24) for all s ∈ Sθ \Bθ, showing that Sθ(k) is null recurrent.

Alternatively, suppose that ρθ > 1. We now must show that (A.25) holds. We use

bθ to define Bθ in the same way. By Lemma 6.1 and Lemma 6.2 we have

limq→∞

(q,r)∈Sθ

E(q,r) [V (Sθ(1))− V (Sθ(0))] = −γθ > 0,

and again by (6.99) we have

E(q,r) [V (Sθ(1))− V (Sθ(0))]

2V (Sθ(0))= Θ

(1

q

).

Thus by taking bθ sufficiently large and ε = 1, we have (A.25), showing Sθ(k) is

transient, completing the proof.

231

232

Appendix A

Lyapunov Functions

Throughout, suppose Xk is a discrete time irreducible Markov chain on a countable

state space X . We now give a series of results characterizing the recurrence of Xk

and, for a function V : X → R+, the moments of V (Xk). The tools used to obtained

these results are commonly referred to as Lyapunov-Foster functions, or test functions.

These results are used extensively in Chapters 4 and 6.

A.1 Positive Recurrence

In this section, we give sufficient conditions to prove whether or not Xk is positive

recurrent. Distinguishing between the null recurrent and transient cases is addressed

in Section A.4 with similar techniques.

First we give a condition for the positive recurrence of Xk due to [36], see [9]

for a modern reference. We use Ex to denote the expectation operator conditional on

X0 = x.

Proposition A.1 (Foster et al. 36). If there exists a function V : X → R, γ > 0,

and a finite set B ⊂ X such that for all x ∈ B,

Ex[V (X1)− V (X0)] <∞, (A.1)

233

and for all x ∈ X \B,

Ex[V (X1)− V (X0)] ≤ −γ,

then Xk is positive recurrent.

A function V satisfying these properties is usually called a Lyapunov function.

Lyapunov functions can also be used to prove Xk is not positive recurrent when

the drift is nonnegative. The following is a special case of Proposition 5.4 from [9].

Proposition A.2. Suppose there exists a Lyapunov function V : X → R, a finite set

B ⊂ X and a state y ∈ X \B satisfying

supx∈B

V (x) < V (y), (A.2)

supx∈X

Ex[(V (X1)− V (X0))2

]<∞, (A.3)

infx∈X\B

Ex[V (X1)− V (X0)] ≥ 0. (A.4)

Then Xk is either null recurrent or transient.

A.2 Moment Bounds

Now, suppose that Xk is positive recurrent, and let X∞ denote the unique steady

state distribution. We now give a bound on the first moment of f(X∞) for any

function f . The result below is new, but similar to existing results from Gamarnik

and Zeevi [38], Glynn and Zeevi [41].

Proposition A.3. Suppose that Xt is positive recurrent and that there exist α, β, γ >

0, a set B ⊂ X and functions U : X → R+ and f : X → R+ such that for x ∈ X \B,

Ex[U(X1)− U(X0)] ≤ −γf(x), (A.5)

234

and for x ∈ B,

f(x) ≤ α, (A.6)

Ex[U(X1)− U(X0)] ≤ β. (A.7)

Then

E[f(X∞)] ≤ α +β

γ.

Proof. For every z > 0 let Uz : X → R+ and fz : X → R+ be given by

Uz(x)∆= minU(x), z, fz(x)

∆= f(x)IU(x)<z.

Trivially for all sufficiently large z, we have fz(x) = f(x) and Uz(x) = U(x) for all

x ∈ B, so (A.6) and (A.7) are satisfied by fz and Uz for all large z. We claim that

(A.5) is satisfied as well. Suppose that x is such that U(x) ≥ z. Then

Ex[Uz(X1)− Uz(X0)] = Ex[Uz(X1)]− z ≤ 0 = fz(x) = −γfz(x).

Alternatively, when x is such that U(x) < z, using that Uz(x) ≤ U(x) for all x,

Ex[Uz(X1)− Uz(X0)] = Ex[Uz(X1)− U(X0)]

≤ Ex[U(X1)− U(X0)]

≤ −γf(x)

= −γfz(x).

Thus for all z and all x ∈ X \ B, Ex[Uz(X1) − Uz(X0)] ≤ −γfz(x) in analogy with

(A.5). As for all z, Uz is bounded by construction, E[Uz(X∞)] is finite. By stationarity

235

and the finiteness of E[Uz(X∞)],

0 = E [EX∞ [Uz(X1)− Uz(X0)]]

=∑x∈B

P(X∞ = x)Ex[Uz(X1)− Uz(X0)] +∑x 6∈B

P(X∞ = x)Ex[Uz(X1)− Uz(X0)]

≤ βP(X∞ ∈ B)− γ∑x 6∈B

P(X∞ = x)fz(x)

≤ β − γE[fz(X∞)IX∞ 6∈B],

or rearranging terms,

E[fz(X∞)IX∞ 6∈B] ≤β

γ.

Thus

E[fz(X∞)] = E[fz(X∞)IX∞∈B] + E[fz(X∞)IX∞ 6∈B]

≤ αP(X∞ ∈ B) +β

γ

≤ α +β

γ.

Now by Fatou’s lemma, as fz(X∞)→ f(X∞) almost surely as z →∞,

E[f(X∞)] = E[

limz→∞

fz(X∞)]≤ lim inf

z→∞E[fz(X∞)] ≤ α +

β

γ,

giving the result.

Remark A.1. Observe that we need not assume that B is bounded.

236

A.3 Moment Bound with Unbounded Downward

Jumps

When applying Proposition A.3 in queuing problems, it is common to take f(·) as the

number of customers in system and U(·) as the square of the number of customers

in system. However, when analyzing systems where jumps downward are not easily

bounded, e.g., a queue with abandonment, it can be difficult to show that (A.5)

holds by naıvely evaluating the equation using this f(·) and U(·). In this section, we

specialize our analysis to a family of Markov chains with a certain queuing structure,

and then give a steady state moment bound that applies in the presence of large

downward jumps.

Given an irreducible aperiodic Markov chain Xk on a countable statespace X ,

suppose there exists a nonnegative function V : X → R+ which admits the following

decomposition

V (Xk) = V (Xk−1) + Ak −Dk, (A.8)

where the Ak ≥ 0 is an i.i.d. sequence such that Ak is independent from state Xk,

while the Dk ≥ 0 may depend on Xk and Ak, but not on k directly. Specifically,

Dk is a function of Xk and Ak. Ak and Dk are interpreted as the number of arrivals

and departures in the time period of length k, respectively. Assume in addition that

B(α) , x ∈ X | V (x) ≤ α ⊂ X is finite for every α. Note that as V (x) ≥ 0, we

have that Dk ≤ V (x) + Ak a.s.

Proposition A.4. Suppose E[A2k] is finite and C1 satisfies E[A2

k] ≤ C1E[Ak]2 < ∞.

Suppose there exists α, λ, C2 > 0 such that for every x /∈ B(α)

E[Ak − Dk|Xk = x

]≤ −λE[Ak], (A.9)

where Dk is defined to be minDk, C2Ak. Then Xk is positive recurrent with the

237

unique stationary distribution X∞ and

E[V (X∞)] ≤ max

α,

max1, C2 − 12C1

λE[Ak]

(2 +

2

λ

).

The reason for introducing a truncated downward jump process Dk as opposed to

using just Dk is that in general the statement of the proposition is not true. Namely,

there exists a process such the assumptions of the proposition above hold true when

Dk replaces Dk in (A.9) and E[V (X∞)] =∞, as shown by Example A.1 below.

Proof. First, we apply Proposition A.1 to Xk using the same V (x), B, and γ = λE[Ak]

as in the statement of Proposition A.4. For x 6∈ B, we have

Ex[V (X1)− V (X0)] = Ex[A0 −D0] ≤ Ex[A0 − D0

]≤ −λEx[A0] = −γ,

where in the inequalities we use Dk ≤ Dk and then (A.9). For all x, we have

Ex[V (X1)− V (X0)] = Ex[A0 −D0] ≤ Ex[A0] <∞.

Thus as B is bounded, we can apply Proposition A.1 to obtain positive recurrence of

Xk. Let X∞ be the steady state version of the Markov chain Xk.

Next, we apply Proposition A.3 taking U(x) = V 2(x) and f(x) = V (x). We let

α′ = max

α,

max1, C2 − 12C1

λE[Ak]

, (A.10)

thus making our set of exceptions from Proposition A.3 B′ = x ∈ X | V (x) ≤ α′.

238

We have for x ∈ B′ that

Ex[U(X1)− U(X0)] = Ex[(V (X0) + A0 −D0)2 − V (X0)2

](A.11)

≤ Ex[(V (X0) + A0)2 − V (X0)2

](A.12)

= 2V (x)E[A0] + E[A20]

≤ 2α′E[A0] + C1E[A0]2 (A.13)

≤ 2α′E[A0] + α′λE[A0] (A.14)

= α′(2 + λ)E[A0]

∆= β′ (A.15)

where (A.11) follows from (A.8), (A.12) follows as V (X1) ≥ 0, and (A.13) follows

from (A.10) and the definition of C1, and (A.14) follows again from (A.10) and as

max1, C2 − 12 ≥ 1 by definition. We have for x 6∈ B′, that

Ex[U(X1)− U(X0)] = Ex[(V (X0) + A0 −D0)2 − V (X0)2

](A.16)

≤ Ex[(V (X0) + A0 − D0)2 − V (X0)2

](A.17)

= 2V (x)Ex[A0 − D0] + Ex[(A0 − D0)2

]≤ −2V (x)λE[A0] + E

[max1, C2 − 12A2

0

](A.18)

≤ −(V (x) + α′)λEx[A0] + max1, C2 − 12C1E [A0]2 (A.19)

≤ −(V (x) + α′)λEx[A0] + α′λE [A0] (A.20)

= −V (x)λEx[A0]

where (A.16) follows from (A.8), (A.17) follows as V (X1) ≥ 0 and D0 ≤ D0, (A.18)

follows from (A.9) and as Dk ≤ C2Ak a.s. implies that |Ak−Dk| ≤ max1, C2−1Aka.s., (A.19) follows from (A.10) and the definition of C1, and finally (A.20) follows

again from (A.10). Thus by taking γ′ = λEx[A0], we can now apply Proposition A.3

239

with α′, β′ and γ′ to obtain that

E[V (∞)] ≤ α′ +β′

γ′= α′ +

α′(2 + λ)E[A0]

λE[A0]= α′

(1 +

2 + λ

λ

)= max

α,

max1, C2 − 12C2

λE[Ak]

(1 +

2 + λ

λ

)

showing the result.

Last, we give a quick counter example showing that without some assumptions

beyond simply having negative drift, we may not even have a finite first moment.

Example A.1. Consider the following random walk Xt on the nonnegative integers

parametrized by some γ ∈ (0, 1). From state 0, we always go up to state 1. For every

other state k = 1, 2, . . ., with probability (1 + γ)/(k + 1), we go to state 0, and with

the remaining probability, (k − γ)/(k + 1), we go up to state k + 1. This walk has

the property that for all k ≥ 1,

Ek[X1 −X0] = (k + 1) · k − γk + 1

+ 0 · 1 + γ

k + 1− k = −γ,

and thus is positive recurrent and has some stationary distribution πk = P(X∞ = k).

However, we will show that E[X∞] = ∞. A direct computation of the steady-state

equations gives that π0 = π1, and for n ≥ 2,

πn =n− γn+ 1

πn−1 = π0

n∏k=2

k − γk + 1

=π0

Γ(2− γ)

Γ(n+ 1− γ)

Γ(n+ 2),

where Γ is the Gamma function. Using the identity

limn→∞

Γ(n+ α)

Γ(n)nα= 1

for all α ∈ R, we have that

πn = Θ

(1

n1+γ

).

240

Thus there exists c > 0 and ` ∈ Z+ such that

E[X∞] =∞∑n=0

nπn ≥∞∑n=`

c

nγ=∞,

showing the claim.

A.4 Null Recurrence

We give a sufficient condition to distinguish between the null recurrent and transient

cases from [56], Theorem 3.2, (see also Section 3.6 from [32]). We do not present the

theorem in full generality.

Proposition A.5. Given a finite set B ⊂ X and Lyapunov function V : X → R+,

assume that

P(

lim supk→∞

V (Xk) =∞)

= 1, (A.21)

infx∈X

Ex[(V (X1)− V (X0))2

]> 0, (A.22)

supx∈X

Ex[(V (X1)− V (X0))4

]<∞. (A.23)

If for all x ∈ X \B,

Ex [V (X1)− V (X0)] ≤ Ex [(V (X1)− V (X0))2]

2V (x), (A.24)

then Xk is recurrent. Alternatively, if there exists ε > 0 such that for all x ∈ X \B,

Ex [V (X1)− V (X0)] ≥ (1 + ε)Ex [(V (X1)− V (X0))2]

2V (x), (A.25)

then Xn is transient.

Remark A.2. As noted in [56], a sufficient condition for (A.21) is that for every

241

z ≥ 0,

infx∈X

P(V (X1) ≥ z | X0 = x) > 0. (A.26)

242

Appendix B

Random Graphs

In this chapter, we prove a result showing that a directed bipartite Erdos-Renyi ran-

dom graph with high probability. The result is used to prove Theorem 4.3. Through-

out, we use the notation introduced in Chapter 4.

B.1 Results

We begin by stating a result on long chains in a static Erdos-Renyi random graph.

The following result was first shown by [3] and refined in a series of papers, see [55]

for a historical account and the most tight result.

Proposition B.1 (Krivelevich et al. 55). Fix any ε > 0 and any δ > 0. There exist

C and n0 such that for all c > C and all n > n0 the following occurs: Consider an

ER(n, c/n) directed graph G = (V , E), and let D be the length of the longest directed

cycle. We have

P(D > (1− (2 + δ)ce−c)n) > 1− ε .

In words, (for large c) we have a cycle containing a large fraction of the nodes with

high probability. From this, we can easily obtain a similar result about the longest

path starting from a specific node.

243

Corollary B.1. Fix any ε > 0. There exist C and n0 such that for all c > C and all

n > n0 the following occurs: Consider a set V of n vertices including a fixed vertex

v ∈ V, and draw an ER(n, c/n) directed graph G = (V , E). Let Pv denote the length

of a longest path starting at v. Then

P(Pv < n(1− ε)) ≤ ε.

The proof is deferred to Section B.2. We extend the result above to the case of

bipartite random graphs.

Corollary B.2. Fix any κ > 1 and ε > 0. Then there exists p0 > 0 and C > 0 such

that the following holds: Consider any cL ∈ [1/√κ, κ], any cR > C, and any p < p0.

Let L be a set of cL/p vertices and let R be a set of cR/p vertices. Fix a vertex v ∈ L.

Draw G = (L,R, E) as an ER(cL/p, cR/p, p) bipartite random graph. We have

P(Pv < 2

cLp

(1− ε))≤ ε,

where again, Pv is the length of a longest path starting at v.

Again, the proof is in Section B.2. The requirement cL ∈ [1/√κ, κ] here will

correspond to p times the ‘typical’ interval between successive times when the chain

advances under greedy. These intervals are distributed i.i.d. Geometric(p), and hence

typically lie in the range [1/(p√κ), κ/p] for large κ, as stated in Lemma 4.2 below.

The 1/√κ term in the lower bound of this ‘typical’ range is a somewhat arbitrary

choice we make that facilitates a proof of Theorem 4.3 (a variety of other decreasing

functions of κ would work as well).

B.2 Proofs

We first prove Corollary B.1. The proof follows relatively easily from Proposition B.1.

The idea is as follows. A sufficient condition to form a long chain from a node v is for

v to be a member of the long cycle that will occur with high probability according to

244

the proposition. Note that with constant probability e−c, v will be isolated and thus

not be part of the cycle, but we can make this probability small by taking C large.

Proof of Corollary B.1. Given ε from the statement of Corollary B.1, let C and n0

be values guaranteed to exist from Proposition B.1 applied when δ = 1 and the

probability of a long chain existing is at least 1− ε/2.

There exists C∗ such that for all c > C∗, 3ce−c < ε/2 as the function f(x) = xe−x

is strictly decreasing for x > 1. We claim that given our ε, Corollary B.1 holds by

taking C = maxC, C∗ and n0 = n0.

Given our ER(n, c/n) graph where n > n0 and c > C and a fixed node v, let A

be the event it contains a cycle of length at least (1− 3ce−c)n, and let B ⊂ A be the

event that that v is in the cycle. Observe that it suffices to prove that P (B) > 1−ε to

show the result, as 3ce−c < ε by our assumption that c > C ≥ C∗ and the definition

of C∗. Thus we compute that

P(B) = P (B|A)P(A) ≥ (1− ε/2)(1− ε/2) ≥ 1− ε,

showing the result, where P(A) ≥ 1−ε/2 follows from Proposition B.1 and P (B|A) ≥

1− ε/2 follows as the cycle is equally likely to pass through every node, so when the

cycle hits 1− ε/2 fraction of the nodes, it has this probability of hitting v.

Next, we prove Corollary B.2. The idea of the proof is as follows. First, we show

with a simple calculation that a constant fraction of the nodes in R will have both in

and out degree one, as p→ 0. We consider paths which only use this subset of nodes

from R. Such a path is equivalent to a path in a modified graph on the set of nodes

L where there is an edge between two nodes u and v if and only if there is a path of

length two between them via an intermediate node in R which has in and out degree

one. Such a graph behaves (approximately) as an Erdos-Renyi graph on the nodes

of L, with the number of edges proportionate to |R|. Thus by ensuring that |R| is

sufficiently large, we can apply Corollary B.1 to obtain the result.

Proof of Corollary B.2. Fix κ > 1 and ε > 0 from the statement of the corollary. For

245

C and p0 to be chosen later, let cL ∈ [1/√κ, κ], cR > C, and p < p0 be arbitrary.

Given our graph G = (L,R, E) that is ER(cL/p, cR/p, p), consider the subgraph G ′ =

(L,R′, E ′) of G where R′ is the set of vertices in R with in degree one and out degree

one in G, and E ′ are the edges in E such that both endpoints are in G ′. From this

graph, we create a new directed non-bipartite digraph G ′′ = (L, E ′′) where and there is

an edge from u ∈ L to v ∈ L iff there is at least one node r ∈ R′ such that (u, r) ∈ E ′

and (r, v) ∈ E ′. Observe that a path of length k in G ′′ gives a path of length 2k in G

by following the two edges in G ′ for each edge in the path on G ′′, so it suffices to find

a path of length (1− ε)cL/p in G ′′.

For any vertex r ∈ R, let Ir be the indicator variable that r has an in degree of

one and an out degree of one. Note that these variables are independent. Further,

we have

µ(p)∆= P(Ir = 1) = P(Bin(|L|, p) = 1)2 =

(cLpp(1− p)

cLp−1

)2

→ c2L exp(−2cL),

as p→ 0. As each of the Ir are independent, we have that

|R′| d= Bin

(cRp, µ(p)

).

Letting

A1(δ1) =

(1− δ1)

cRpµ(p) < |R′| < (1 + δ1)

cRpµ(p)

we have by Proposition 4.1 that for all p,

P(A1(δ1)) ≥ 1− 2 exp

(−δ2

1

cRp

µ(p)

3

).

We can view the edges of G ′′ as being generated by the following process: for each

r ∈ R′, pick a source and then a destination uniformly at random from L and add

an edge from the source to the destination unless either:

the source and destination are the same node,

246

an edge between the source and destination already exists in the graph.

Thus |E ′′| is the number of non empty bins if we throw |R′| balls into (cL/p)2 bins and

then throw out the cL/p bins that correspond to self edges. (Alternatively, we can

think of this process as throwing cR/p balls, but each ball “falls through” only with

probability 1− µ(p). This problem was studied extensively in [73], but here we need

only a coarse analysis). Trivially, |E ′′| ≤ |R′|. We now show that typically, the number

of nonempty bins is almost equal to the number of balls thrown. For each r ∈ R′, let

Xr be the indicator that there is ` ∈ L′ such that (`, r) ∈ E ′ and (r, `) ∈ E ′. It is easy

to see that the Xr are i.i.d. Bernoulli(p/cL). For each r, s ⊂ R′, let Yrs be the

indicator that the nodes r and s are “colliding” on both their source and destination

choices in L′, i.e. there is `,m ∈ L′, ` 6= m, such that (`, r), (`, s), (r,m), (s,m) ∈ E ′.

It is easy to see that P(Yrs = 1) ≤ p2/c2L for each `,m ∈ L′. We have

|E ′′| ≥ |R′| −∑r∈R′

Xr −∑

r,s⊂R′Yrs.

We compute that for any fixed R′

E

∑r∈R′

Xr +∑

r,s⊂R′Yrs

≤ |R′| pcL

+

(|R′|

2

)p2

c2L

≤ |R′| pcL

+

(|R′| p

cL

)2

Letting

A2(δ2) =

∑r∈R′

Xr +∑

r,s⊂R′Yrs ≤ δ2|R′|

,

we have that

P(A2(δ2)) ≥ 1− p

δ2cL− |R′|δ−1

2

(p

cL

)2

247

Letting

B(δ1, δ2) =

(1− δ1)(1− δ2)

cRpµ(p) < |E ′′| < (1 + δ1)

cRpµ(p)

,

we have that B(δ1, δ2) ⊃ A1(δ2) ∩ A2(δ2), and thus by taking complements and then

applying the union bound,

P(B(δ1, δ2)) ≥ 1− P(A1(δ1)c)− P(A2(δ2)c)

≥ 1− 2 exp

(−δ2

1

cRpµ(p)/3

)− p

δ2cL− (1 + δ1)cRµ(p)

p

δ22c

2L

,

thus giving us a high probability bound on the size of |E ′′| as p→ 0.

For our fixed ε, let C and n0 be C and n0 from Corollary B.1 such that for any

c > C and n > n0, given a node in graph ER(n, c/n), there exists a path with

length at least n(1 − ε/2) with probability at least 1 − ε/2. We now specifiy p0

from the corollary to be such that for all cL ∈ [1/√κ, κ], we have cL/p0 > n0, i.e.

p0 < 1/(n0

√κ).

Let G = (L, E) be an ER(cL/p, Cp/cL) directed random graph. We now couple

G ′′ (a directed ER(n,M) graph, where M is random but independent of the edges

selected) and G (a directed ER(n, p) graph) in the standard way so that when |E | ≤

|E ′′|, then E ⊂ E ′′ and when |E ′′| ≤ |E|, then E ′′ ⊂ E . Thus if G has a long path and

|E | < |E ′′|, then G ′′ will have at least as long a path as well, as it will contain more

edges on the same nodes. Let P be the length of a longest path starting at v in G.

Letting

A3 =

P >

(1− ε

2

) cLp

and recalling that p0 < 1/(n0

√κ) implies that cL/p > n0, we have by Proposition B.1

P (A3) ≥ 1− ε

2

We now need to show that G ′′ will have more edges than G with high probability for

248

all cR sufficiently large (which we can control by choice of C from the statement of

the corollary). We have |E | ∼ Bin((cL/p− 1)cL/p, Cp/cL), thus by Proposition 4.1, if

A4(δ4) =C(cL/p− 1)(1− δ4) < |E | < C(cL/p− 1)(1 + δ4)

then

P(A4(δ4)) ≥ 1− 2 exp

(−δ2

4C

(cLp− 1

)/3

)

Now, for any fixed choice of δ1, δ2, δ4, there exists C sufficiently large such that if

cR > C then for all p < p0 and all cL ∈ [1/√κ, κ],

C

(cLp− 1

)(1 + δ4) < (1− δ1)(1− δ2)

cRpµ(p)

(recall that µ(p) converges to a constant depending only cL uniformly over [1/√κ, κ]

as p→ 0). For such cR, we have

|E ′′| > |E | ⊂ B(δ1, δ2) ∩ A4(δ4),

as B makes |E ′′| big and A4 ensures that |E | is small. Putting everything together,

we have that

P > 2

cLp

(1− ε)⊃ B(δ1, δ2) ∩ A3 ∩ A4(δ4),

so by taking complements and then applying the union bound, we obtain

P(P > 2

cLp

(1− ε))≥ 1− P(B(δ1, δ2)c)− P(Ac3)− P(A4(δ4)c) = 1− ε

2−O(p),

showing the result.

249

250

Bibliography

[1] D. J. Abraham, A. Blum, and T. Sandholm. Clearing algorithms for barterexchange markets: Enabling nationwide kidney exchanges. In Proceedings of the8th ACM conference on Electronic commerce, pages 295–304. ACM, 2007.

[2] S. Ahmed, A. Shapiro, and E. Shapiro. The sample average approximationmethod for stochastic programs with integer recourse. SIAM Journal of Opti-mization, 12:479–502, 2002.

[3] M. Ajtai, J. Komlos, and E. Szemeredi. The longest path in a random graph.Combinatorica, 1(1):1–12, 1981.

[4] M. Akbarpour, S. Li, and S. O. Gharan. Dynamic matching market design. arXivpreprint arXiv:1402.3643, 2014.

[5] N. Alon and J. H. Spencer. The probabilistic method. John Wiley & Sons, 2004.

[6] I. Ashlagi and A. E. Roth. Free riding and participation in large scale, multi-hospital kidney exchange, 2011.

[7] I. Ashlagi, D. Gamarnik, M. A. Rees, and A. E. Roth. The need for (long) chainsin kidney exchange. Technical report, National Bureau of Economic Research,2012.

[8] I. Ashlagi, P. Jaillet, and V. H. Manshadi. Kidney exchange in dynamic sparseheterogenous pools. arXiv preprint arXiv:1301.3509, 2013.

[9] S. Asmussen. Applied probability and queues. Springer Verlag, 2003. ISBN0387002111.

[10] N. Ayas, L. Barger, B. Cade, D. Hashimoto, B. Rosner, J. Cronin, F. Speizer, andC. Czeisler. Extended work duration and the risk of self-reported percutaneousinjuries in interns. JAMA: the journal of the American Medical Association, 296(9):1055, 2006.

[11] L. Barger, B. Cade, N. Ayas, J. Cronin, B. Rosner, F. Speizer, C. Czeisler, et al.Extended work shifts and the risk of motor vehicle crashes among interns. NewEngland Journal of Medicine, 352(2):125, 2005.

251

[12] D. Bertsimas and R. Weismantel. Optimization over integers, volume 13. Dy-namic Ideas Belmont, 2005.

[13] J.-F. Berube, M. Gendreau, and J.-Y. Potvin. A branch-and-cut algorithm forthe undirected prize collecting traveling salesman problem. Networks, 54(1):56–67, 2009.

[14] D. Bienstock, M. X. Goemans, D. Simchi-Levi, and D. Williamson. A note onthe prize collecting traveling salesman problem. Mathematical programming, 59(1):413–420, 1993.

[15] P. Billingsley. Convergence of probability measures. Wiley New York, 1968.

[16] P. Biro, D. F. Manlove, and R. Rizzi. Maximum weight cycle packing in directedgraphs, with application to kidney exchange programs. Discrete Mathematics,Algorithms and Applications, 1(04):499–517, 2009.

[17] A. Blum, A. Gupta, A. Procaccia, and A. Sharma. Harnessing the power of twocrossmatches. In Proceedings of the fourteenth ACM conference on Electroniccommerce, pages 123–140. ACM, 2013.

[18] J. Borwein and A. Lewis. Convex analysis and nonlinear optimization: theoryand examples. Springer Verlag, 2006. ISBN 0387295704.

[19] H. Chen and D. Yao. Fundamentals of queueing networks: Performance, asymp-totics, and optimization. Springer Verlag, 2001. ISBN 0387951660.

[20] M. Constantino, X. Klimentova, A. Viana, and A. Rais. New insights on integer-programming models for the kidney exchange problem. European Journal ofOperational Research, 2013.

[21] R. Cook, M. Render, and D. Woods. Gaps in the continuity of care and progresson patient safety. British Medical Journal, 320(7237):791, 2000.

[22] F. de Vericourt and O. Jennings. Nurse-to-patient ratios in hospital staffing: aqueuing perspective. ESMT Working Paper No. 08-005, 2008.

[23] J. P. Dickerson, A. D. Procaccia, and T. Sandholm. Optimizing kidney exchangewith transplant chains: Theory and reality. In Proceedings of the 11th Inter-national Conference on Autonomous Agents and Multiagent Systems-Volume 2,pages 711–718. International Foundation for Autonomous Agents and MultiagentSystems, 2012.

[24] J. P. Dickerson, A. D. Procaccia, and T. Sandholm. Failure-aware kidney ex-change. In Proceedings of the fourteenth ACM conference on Electronic com-merce, pages 323–340. ACM, 2013.

[25] R. Dittus, R. Klein, D. DeBrota, M. Dame, and J. Fitzgerald. Medical residentwork schedules: design and evaluation by simulation modeling. ManagementScience, 42(6):891–906, 1996. ISSN 0025-1909.

252

[26] B. C. Drolet, L. B. Spalluto, and S. A. Fischer. Residents’ perspectives on acgmeregulation of supervision and duty hoursa national survey. New England Journalof Medicine, 363(23), 2010.

[27] B. C. Drolet, D. A. Christopher, and S. A. Fischer. Residents’ response to duty-hour regulationsa follow-up national survey. New England Journal of Medicine,366(24), 2012.

[28] R. Dudley. Real analysis and probability. Cambridge University Press, 2002.

[29] E. El-Darzi, C. Vasilakis, T. Chaussalet, and P. Millard. A simulation modellingapproach to evaluating length of stay, occupancy, emptiness and bed blocking ina hospital geriatric department. Health Care Management Science, 1(2):143–149,1998. ISSN 1386-9620.

[30] P. I. Ellman, M. G. Law, C. Tache-Leon, T. B. Reece, T. S. Maxey, B. B. Peeler,J. A. Kern, C. G. Tribble, and I. L. Kron. Sleep deprivation does not affectoperative results in cardiac surgery. The Annals of thoracic surgery, 78(3):906–911, 2004.

[31] S. Ethier and T. Kurtz. Markov processes: Characterization and convergence.Wiley New York, 1986.

[32] G. Fayolle, V. Malyshev, and M. Menshikov. Topics in the constructive theory ofcountable Markov chains. Cambridge University Press, 1995. ISBN 0521461979.

[33] M. Fischetti, J. J. Salazar Gonzalez, and P. Toth. A branch-and-cut algorithmfor the symmetric generalized traveling salesman problem. Operations Research,45(3):378–394, 1997.

[34] M. Fischetti, J. J. S. Gonzalez, and P. Toth. Solving the orienteering problemthrough branch-and-cut. INFORMS Journal on Computing, 10(2):133–148, 1998.

[35] K. E. Fletcher, W. Underwood, S. Q. Davis, R. S. Mangrulkar, L. F. McMahon,and S. Saint. Effects of work hour reduction on residents lives: a systematicreview. Jama, 294(9):1088–1100, 2005.

[36] F. Foster et al. On the stochastic matrices associated with certain queuingprocesses. The Annals of Mathematical Statistics, 24(3):355–360, 1953.

[37] D. Gaba and S. Howard. Fatigue among clinicians and the safety of patients.New England Journal of Medicine, 347(16):1249–1255, 2002.

[38] D. Gamarnik and A. Zeevi. Validity of heavy traffic steady-state approximationsin generalized Jackson networks. The Annals of Applied Probability, 16(1):56–90,2006. ISSN 1050-5164.

[39] M. Gendreau, G. Laporte, and F. Semet. A branch-and-cut algorithm for theundirected selective traveling salesman problem. Networks, 32(4):263–273, 1998.

253

[40] S. E. Gentry, R. A. Montgomery, B. J. Swihart, and D. L. Segev. The rolesof dominos and nonsimultaneous chains in kidney paired donation. AmericanJournal of Transplantation, 9(6):1330–1336, 2009.

[41] P. Glynn and A. Zeevi. Bounding stationary expectations of Markov processes.In Markov processes and related topics: A Festschrift for Thomas G. Kurtz.Selected papers of the conference, Madison, WI, USA, July, pages 10–13, 2006.

[42] M. X. Goemans. Combining approximation algorithms for the prize-collectingtsp. arXiv preprint arXiv:0910.0553, 2009.

[43] L. Green. Capacity planning and management in hospitals. Operations Researchand Health Care, pages 15–41, 2005.

[44] L. Green, J. Soares, J. Giglio, and R. Green. Using queueing theory to increasethe effectiveness of emergency department provider staffing. Academic Emer-gency Medicine, 13(1):61–68, 2006. ISSN 1553-2712.

[45] L. Green, P. Kolesar, and W. Whitt. Coping with Time-Varying Demand WhenSetting Staffing Requirements for a Service System. Production and OperationsManagement, 16(1):13–39, 2007. ISSN 1937-5956.

[46] G. Gutin and A. P. Punnen. The traveling salesman problem and its variations,volume 12. Springer, 2002.

[47] P. Holewijn and A. Hordijk. On the convergence of moments in stationary markovchains. Stochastic Processes and Their Applications, 3(1):55–64, 1975.

[48] M. Hutter, K. Kellogg, C. Ferguson, W. Abbott, and A. Warshaw. The impactof the 80-hour resident workweek on surgical residents and attending surgeons.Annals of surgery, 243(6):864, 2006.

[49] R. Ibrahim and W. Whitt. Wait-Time Predictors for Customer Service SystemsWith Time-Varying Demand and Capacity. Oper. Res., forthcoming. ColumbiaUniversity, NY, NY, 2010.

[50] J. Iglehart. The ACGME’s Final Duty-Hour Standards Special PGY-1 Limitsand Strategic Napping. New England Journal of Medicine, 363(17):1589–1591,2010.

[51] S. Jacobson, S. Hall, and J. Swisher. Discrete-event simulation of health caresystems. Patient Flow: Reducing Delay in Healthcare Delivery, pages 211–252,2006.

[52] S. Janson, T. Luczak, and V. Kolchin. Random graphs. Cambridge Univ Press,2000.

[53] R. Klein, R. Dittus, D. DeBrota, and M. Dame. Using discrete event simulationto evaluate housestaff work schedules. In Proceedings of the 22nd conference onWinter simulation, pages 738–742. IEEE Press, 1990. ISBN 0911801723.

254

[54] A. J. Kleywegt, A. Shapiro, and T. Homem-de Mello. The sample average ap-proximation method for stochastic discrete optimization. SIAM Journal on Op-timization, 12(2):479–502, 2002.

[55] M. Krivelevich, E. Lubetzky, and B. Sudakov. Longest cycles in sparse randomdigraphs. Random Structures & Algorithms, 2012.

[56] J. Lamperti. Criteria for the recurrence or transience of stochastic process, partI. J. Math. Anal. and Appls., 1, 1960.

[57] C. Landrigan, J. Rothschild, J. Cronin, R. Kaushal, E. Burdick, J. Katz, C. Lilly,P. Stone, S. Lockley, D. Bates, et al. Effect of reducing interns’ work hours onserious medical errors in intensive care units. New England Journal of Medicine,351(18):1838, 2004.

[58] E. Litvak and I. Joint Commission Resources. Managing Patient Flow in Hos-pitals: Strategies and Solutions. Joint Commission Resources, 2010. ISBN1599403722.

[59] Y. Liu and W. Whitt. A fluid model for a large-scale service system experiencingperiods of overloading, 2010.

[60] A. Mandelbaum, W. Massey, and M. Reiman. Strong approximations for Marko-vian service networks. Queueing Systems, 30(1):149–201, 1998. ISSN 0257-0130.

[61] M. Molinaro and R. Ravi. Kidney exchanges and the query-commit problem,2013.

[62] T. K. Nuckols and J. J. Escarce. Cost implications of acgmes 2011 changes toresident duty hours and the training environment. Journal of general internalmedicine, 27(2):241–249, 2012.

[63] OPTN. Optn waiting list. http://optn.transplant.hrsa.gov/latestData/

rptData.asp, 2013. Accessed: 2013-11.

[64] L. Petersen, T. Brennan, A. O’Neil, E. Cook, and T. Lee. Does housestaffdiscontinuity of care increase the risk for preventable adverse events? Annals ofinternal medicine, 121(11):866, 1994. ISSN 0003-4819.

[65] M. A. Rees, J. E. Kopke, R. P. Pelletier, D. L. Segev, M. E. Rutter, A. J. Fabrega,J. Rogers, O. G. Pankewycz, J. Hiller, A. E. Roth, et al. A nonsimultaneous,extended, altruistic-donor chain. New England Journal of Medicine, 360(11):1096–1101, 2009.

[66] M. Rossetti, G. Trzcinski, and S. Syverud. Emergency department simulationand determination of optimal attending physician staffing schedules. In wsc,pages 1532–1540. IEEE, 1999.

255

[67] A. E. Roth, T. Sonmez, and M. U. Unver. Kidney exchange. Quarterly Journalof Economics, 119:457–488, 2004.

[68] A. E. Roth, T. Sonmez, M. U. Unver, F. L. Delmonico, and S. L. Saidman.Utilizing list exchange and nondirected donation through chain kidney paireddonations. American Journal of Transplantation, 6:2694–2705, 2006.

[69] A. E. Roth, T. Sonmez, and M. U. Unver. Efficient kidney exchange: Coinci-dence of wants in markets with compatibility-based preferences. The Americaneconomic review, pages 828–851, 2007.

[70] K. Sack. 60 Lives, 30 Kidneys, All Linked. The New York Times, 2012.

[71] K. Sack. Experts Recommend Single Registry to Oversee Kidney TransplantDonations. The New York Times, 2012.

[72] J. Samkoff and C. Jacques. A review of studies concerning effects of sleep depri-vation and fatigue on residents’ performance. Academic Medicine, 66(11):687,1991. ISSN 1040-2446.

[73] E. Samuel-Cahn. Asymptotic distributions for occupancy and waiting time prob-lems with positive probability of falling through the cells. The Annals of Proba-bility, 2(3):515–521, 1974.

[74] T. Schoenmeyr, P. Dunn, D. Gamarnik, R. Levi, D. Berger, B. Daily, W. Levine,and W. Sandberg. A model for understanding the impacts of demand and ca-pacity on waiting time to enter a congested recovery room. Anesthesiology, 110(6):1293, 2009. ISSN 0003-3022.

[75] S. Sen, H. R. Kranzler, J. H. Krystal, H. Speller, G. Chan, J. Gelernter, and C.Guille. A prospective cohort study investigating factors associated with depres-sion during medical internship. Archives of general psychiatry, 67(6):557–565,2010.

[76] S. Sen, H. R. Kranzler, A. K. Didwania, A. C. Schwartz, S. Amarnath, J. C.Kolars, G. W. Dalack, B. Nichols, and C. Guille. Effects of the 2011 duty hourreforms on interns and their patients: a prospective longitudinal cohort study.JAMA internal medicine, 173(8):657–662, 2013.

[77] R. Steinbrook. The debate over residents’ work hours. New England Journal ofMedicine, 347(16):1296, 2002.

[78] C. Swamy and D. B. Shmoys. Sampling-based approximation algorithms formulti-stage stochastic optimization. In Foundations of Computer Science, 2005.FOCS 2005. 46th Annual IEEE Symposium on, pages 357–366. IEEE, 2005.

[79] J. Tsitsiklis and K. Xu. On the Power of (even a little) Centralization in Dis-tributed Processing. ACM Sigmetrics, San Jose, 2011.

256

[80] R. Tweedie. The existence of moments for stationary markov chains. Journal ofApplied Probability, pages 191–196, 1983.

[81] D. Weinstein. Duty hours for resident physicians–tough choices for teachinghospitals. New England Journal of Medicine, 347(16):1275, 2002.

[82] E. Whang, M. Mello, S. Ashley, and M. Zinner. Implementing resident work hourlimitations: lessons from the New York State experience. Annals of surgery, 237(4):449, 2003.

[83] K. White Jr. A survey of data resources for simulating patient flows in healthcaredelivery systems. In Proceedings of the 37th conference on Winter simulation,pages 926–935. Winter Simulation Conference, 2005. ISBN 0780395190.

[84] A. Yaghoubian, A. H. Kaji, B. Ishaque, J. Park, D. K. Rosing, S. Lee, B. E.Stabile, and C. de Virgilio. Acute care surgery performed by sleep deprivedresidents: are outcomes affected? Journal of Surgical Research, 163(2):192–196,2010.

[85] G. Yom-Tov. Queues in Hospitals: Semi-Open Queueing Networks in the QEDRegime. IE PhD thesis proposal, 2008.

[86] G. Yom-Tov and A. Mandelbaum. Time-varying QED queues with reentrantcustomers in support of healthcare staffing. The Technion, Haifa, Israel, 2010.

257