things i thought i knew about queueing theory, but was wrong about: part 1, multiserver queues
DESCRIPTION
Things I Thought I Knew About Queueing Theory, but was Wrong About: Part 1, Multiserver Queues. Alan Scheller-Wolf Tepper School of Business Carnegie Mellon University Joint work with Karl Sigman , Rein Vesilo, and Michele Dufalla. A Great Day . - PowerPoint PPT PresentationTRANSCRIPT
Page 1Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
Things I Thought I Knew About Queueing Theory, but was Wrong About: Part 1, Multiserver Queues
Alan Scheller-WolfTepper School of Business Carnegie Mellon University
Joint work withKarl Sigman, Rein Vesilo, and Michele Dufalla
Page 2Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
A Great Day
As a graduate student, one of the greatest things that can happen is the following:Student: “I can prove that XXX is true!”Advisor: “That can’t be right, everyone knows YYY is true.”Student: “I know, but I can prove it.”Advisor: “There must be a bug, show me.”Time passes...Advisor: “Huh, you are right, XXX is true!”
Page 3Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
My Goals
• Take you through “my great day” as a student• Help you understand why everyone believed YYY• Help you understand, mathematically, why XXX is
true• Help you see why, intuitively, XXX “has to be
true”• Discuss what general insights this gives
Page 4Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
Mean Delay in GI/GI/1 Queues
i.i.d. arrivals ~ T
rate l
i.i.d. service ~ S
rate m
• Jobs are served in a FIFO (FCFS) fashion• Delay: Time customers spend in queue• Question: What are conditions for mean
(stationary) delay to be finite?• Kiefer and Wolfowitz (K&W, 1956)
r =l/m < 1E[D] finite iff E[S2] finite
Single Server Queue
S should be “heavy tailed” -- think Pareto distribution
Page 5Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
Why is E[S2] Necessary (for Single-Server Queues)?
• The inspection paradoxA randomly arriving job sees an equilibrium service
time Se (or the excess of a service time)Length biasing causes the tail of Se [P(Se>x)] to behave
as the integrated tail of S – you are more likely to “see” longer service times
E[Se] is finite iff E[S2] is finite
• The upshot: If an arriving job “sees” a job in service, this job has mean remaining size on the order of E[S2]. The arriving job can’t enter service until this job is done.
Page 6Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
Why is E[S2] Sufficient (for Single-Server Queues)? (I)
• The Lindley Equation (Lindley, 1952)Dn+1 = (Dn+Sn-Tn)+, where Sn is the service time of
customer n and Tn is the interarrival time between customers n and n+1
Why?Let’s say customer n arrives at time tThen customer n leaves service at time t+Dn+Sn
Customer n+1 arrives at time t + Tn
Customer n+1 enters service at time min{t+Dn+Sn,t + Tn}
Customer n+1’s delay is min{t+Dn+Sn,t + Tn} – (t + Tn)This is a reflected random walk
Page 7Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
Why is E[S2] Sufficient (for Single-Server Queues)? (II)
Reflected Random Walks:• For Zn+1 = (Zn+Xn)+, E[Z2] is finite if:
E[(X+)2] is finite E[Xn] < -d < 0 for all n and on every sample path
• For our GI/GI/1 queue:E[(X+)2] = E[S2] is finite E[Xn] = E[S] - E[T] < -d < 0 ; for all n and on every
sample path, since r<1
Page 8Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
Mean Delay in GI/GI/k Queues
i.i.d. arrivals ~ T
rate l
i.i.d. service ~ S
rate m
• Question: What are conditions for mean stationary delay to be finite?
• K&W, 1956 r =l/ km < 1E[D] finite if E[S2] finite
Multi-Server Queue
Page 9Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
Why is E[S2] Sufficient?
• Split the GI/GI/k into k GI/GI/1 queuesThis performs no better than the original system
• Route arriving jobs to queues in cyclic (or round robin) fashion
• Each GI/GI/1 queue has finite mean delay• Therefore the original GI/GI/k queue has finite
mean delay as well
Page 10Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
What about Necessity?
“Everyone knew” that r =l/ km < 1
E[S2] finite were necessary as well.
This had even appeared in the literature.
Page 11Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
Pictorially:
Requirement for finite mean delay
1
2
Moments of S
Servers1 2 3 4
∞
Page 12Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
Back to the GI/GI/1
• Why is finite E[S2] Necessary?• The inspection paradox
If an arriving job “sees” a job in service, this job has mean remaining size on the order of E[S2]. The arriving job can’t enter service until this job is done.
• Does this logic hold for the GI/GI/k?NO!
• Can we capture this effect mathematically?
Page 13Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
Lindley Equation for GI/GI/1
• Our logic for GI/GI/1Let’s say customer n arrives at time tThen customer n leaves service at time t+Dn+Sn
Customer n+1 arrives at time t + Tn
Customer n+1 enters service at time min{t+Dn+Sn,t + Tn}
Customer n+1’s delay is min{t+Dn+Sn,t + Tn} – (t + Tn)
• Dn+1 = (Dn+Sn-Tn)+
Page 14Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
Lindley Equation for GI/GI/k
• For GI/GI/kLet’s say customer n arrives at time tThen customer n leaves service at time t+Dn+Sn
Customer n+1 arrives at time t + Tn
Customer n+1 enters service at time min{t+Dn+Sn,t + Tn}
NO!
Page 15Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
When Does Customer n+1 Enter Service? (I)
• Customer n arrives at time t• Customer n leaves service at time t+Dn+Sn
nn+1
j
Page 16Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
When Does Customer n+1 Enter Service? (II)
• When does customer j leave service?Let’s say that when customer n enters service, she
looks over at the other server, and observes the amount of time customer j has left
Let’s call this amount of time Un
Then customer j leaves service at time t+Dn+Un
nn+1
j
If there were multiple other servers, Un would
be the minimum remaining time
Page 17Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
When Does Customer n+1 Enter Service? (III)
nn+1
j
• Customer n leaves service at time t+Dn+Sn
• Customer j leaves service at time t+Dn+Un
• So the next server comes free at time t+Dn+min{Sn,Un} = t+Dn+Pn
Page 18Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
Lindley Equation for GI/GI/k
• For GI/GI/k:Dn+1 = (Dn+Pn-Tn)+
How big of difference could one letter make?
We have replaced Sn with Pn
Page 19Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
Back to Reflected Random Walks
• For our GI/GI/k queue, for finite mean delay it is sufficient that:E[(X+)2] = E[P2] is finite E[Xn] = E[Pn] - E[Tn] < -d < 0 ; for all n and on every
sample path
How does this help? We don’t know the first thing about P!
Page 20Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
When is E[P2] Finite?
E[P2] = E[min{S,U}2]= E[min{S,U}3/2min{S,U}1/2]< E[S3/2U1/2] = E[S3/2] E[U1/2]
And E[U1/2] < E[W1/2]And E[W1/2] is finite iff E[S3/2] (K&W, 1956)
So for a GI/GI/2 queue, E[P2] is finite if E[S3/2] is finite!
S is independent of U
W is the total work in system
Page 21Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
What about Negative Drift?
E[Xn] = E[Pn] - E[Tn] = E[min{Sn,Un}] - E[Tn] < E[Sn] - E[Tn] = E[S] - E[T]
This seems like a reallyweak bound, but is in fact
the best we can do, since this must hold on all sample paths
and for all n; Un could bearbitrarily large on a sample
path
I have negative drift for all n on every sample path if E[S] < E[T] (r < ½)
but not otherwise
Page 22Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
Finite Mean Delay in GI/GI/2 (I)
So it appears that:• Finite E[S3/2] is sufficient for finite mean delay in
GI/GI/2 queues if E[S] < E[T]
• Finite E[S3/2] is not sufficient for finite mean delay in GI/GI/2 queues if E[T] < E[S] < 2E[T]
Does this make sense?
YES!
Page 23Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
Finite Mean Delay in GI/GI/2 (II)
• The inspection paradoxIf an arriving job “sees” a job in service, this
job has mean remaining size on the order of E[S2]
The mean time this job has been in service also is on the order of E[S2]
The key question is what has been going on while this long job has been in service
Page 24Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
Finite Mean Delay in GI/GI/2 (III)
• If E[T] < E[S] < 2E[T]: While a single extremely large job is in service
the remaining server cannot keep up with the workload, and the system “crashes”
• If E[S] < E[T]:A single extremely large job is not typically
enough to crash the system – the remaining server can keep up with the workload
It takes two extremely large jobs to crash the system
Page 25Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
Redefining r
• For a k-server queue r is traditionally defined as r =l/ km
This is WRONG!
• If instead r =l/m E[S] < E[T] means r<1, and we have a “spare server”
to absorb the workload if a long job arrivesE[T] < E[S] < 2E[T] means 1<r<2, and we have no
spare capacity
r =l/m denotes the number of servers needed to handle the mean load on the system
Page 26Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
What about GI/GI/3?
• If 2<r<3 (no spare servers):E[D] is finite if E[S2] is finite
• If 1<r<2 (one spare server):E[D] is finite if E[S3/2] is finite
• If 0<r<1 (two spare servers):E[D] is finite if E[S4/3] is finite
Page 27Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
What about GI/GI/k?
• If k-1<r<k (no spare servers):E[D] is finite if E[S2] is finite
• If k-2<r<k-1 (one spare server):E[D] is finite if E[S3/2] is finite
• If j<r<j+1 (k-j-1 spare servers):E[D] is finite if E[S(k+1-j)/(k-j)] is finite
Page 28Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
Pictorially: Fixed Number of Spare Servers
Requirement for finite mean delay
1
2
Moments of S
Servers1 2 3 4
∞
No spare servers
One spare serverTwo spare servers
3/24/3
Diminishing benefits
For moment conditions, 20-server queue with load 18.5 is the same as a 2-server queue
with load 0.5
Pictorially:
Page 29Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
Pictorially: As Spare Servers Grow
Requirement for finite mean delay
1
2
Moments of S
Servers1 2 3 4
∞
3/24/3
As the number of spare servers approaches infinity, the moment conditions approach those of an
infinite-server queue, as they should
Page 30Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
Other Results (I)
• Higher Moments: If j<r<j+1 (k-j-1 spare servers):
E[Dr] is finite if E[S(k+r-j)/(k-j)] is finite(r has replaced 1 in the previous formula)
If you have just two spare servers, E[D2] is finite if E[S5/3] is finite – you get finite variance of delay without finite variance of service time
Page 31Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
Other Results (II)
• Necessary Conditions: The sufficient conditions above are also
provably necessary for all but a small set of service time distributions
Page 32Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
Other Results (III)
• Integral Load: Drift has been crucial in our analysis – if r<1
E[S3/2] is sufficient for finite mean delay; if r>1 we require E[S2]
What if r=1? We have been able to show that E[S3/2] is not
sufficient, but have not been able to get close to showing E[S2] is necessary
We are still working
Page 33Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
Insights• Delay moments in multi-server queues with heavy-
tailed service times depend on the moments of S and the number of spare serversr is most usefully defined as l/m for multi-
server queues • Adding spare serves helps with delay moments,
but in diminishing fashion• This works because under heavy-tailed service
times queues build up due to large jobs blocking servers
Page 34Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
Open Problems
1. Dependencies:Everything we showed was for GI/GI/k; what if
interarrival and/or service times are not i.i.d?
Our “Lindley” equation for the GI/GI/k:Dn+1 = (Dn+Pn-Tn)+
also holds for a special type of GI/G/1 queue. (It actually defines a GI/G/1 queue.) So there must be a relationship/duality here.
Page 35Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
Open Problems (II)
2. Bounds: As of now, the best analytical bounds on multi-
server delay under heavy-tailed service time are in general very poor
These cannot, in general, be easily simulated See work by Daley, Scheller-Wolf, Vesilo
Page 36Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
Open Problems (III)• Tail Asymptotics
Moment conditions for delay are really just a proxy for what we really want to know: How the tail of the delay distribution behaves
See work by Boxma, Foss, Korshunov, Whitt, Zwart
Page 37Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
ReferencesScheller-Wolf, A. (2003). Necessary and Sufficient Conditions for Delay Moments in FIFO Multiserver Queues: Why s Slow Servers are Better than One Fast Server for Heavy-Tailed Systems. Operations Research 51, No. 5, 748-758.
Scheller-Wolf, A. and R. Vesilo (2006). Structural Interpretation and Derivation of Necessary and Sufficient Conditions for Delay Moments in FIFO Multiserver Queues. Queueing Systems, 54, No. 3 221-232.
Page 38Alan Scheller-Wolf Lunteren, The
Netherlands
January 15, 2013
Dank U!
Any Questions?