beyond heavy-traffic regimes: universal bounds …stochnet/talks/gurvich16.pdfbeyond heavy-trafﬁc...

Beyond heavy-traffic regimes:Universal bounds and controls for the single-server

(M/GI/1+GI) queue

Itai GurvichNorthwestern University

Junfei HuangChinese University of Hong Kong

Stochastic Networks 2016

1 / 33

The intuitive Derivation of a Brownian Queue

1Service ∼ ⋅ , , ∞Load

• The waiting time/workload process in the M/GI/1 queue:

W (t) = W (0) +

A(t)∑i=1

si − (t − I(t))

= W (0)− (1− ρ)t − (t − I(t)) +

A(t)∑i=1

si − ρt

(M/GI/1)

W (t) = W (0)− (1− ρ)t − (t − I(t)) +√λE[s2]B(t)

(Brownian Queue)2 / 33

Brownian approximations as a model

A tractable and useful tool in the modeler’s toolbox

Pricing in queues (e.g. Kim and Randhawa, 2015)

Competition between queues (e.g. Allon and Federgruen, 2008)

Contracting in services (e.g. Akan et. al. 2011)

Inventory Optimization (e.g. Allon and Van Mieghem 2010)

Initial Model→ difficult→ Brownian approximation ?→ Accurate

3 / 33

Example: Dual Sourcing (Inventory)

Net Inventory evolution (Mexico and China renewal inputs Sc , SM ):

I(t) = I(0) + Sc(t) + SM(T sM(t))− D(t),

T sM(t) =

01{I(u) < s}ds.

Figure 5 Comparing the Brownian Allocation to the Allocation Optimized via Simulation

Scaled cost: Brownian vs. optimal TBS

Scaled China allocation: Brownianvs. optimal TBS

Relative China cost cC/cM

0.150.05

0 0.2 0.4 0.6 0.8 1.0

Simulated cost of Brownian prescription

� = 1 � = 10

Optimization by simulation

� = 100

� = 1 � = 100� = 10

Asymptotic (analytical)

� = 10

� = 1 � = 100

Asymptotic (analytical)*�C

Yet even for � = 1, the relative error in scaled costbetween the prescription and the optimal control wasless than 7%. The main implication is that the Brown-ian prescription is a good and useful approximation ofthe optimal strategic China allocation, even for smallvolumes.

8.3. Comparing the Square-Root Allocation to theBrownian Allocation

The left panel in Figure 6 shows the optimal capaci-ties ∗

C and ∗M in the Brownian model, as well as the

square-root approximation p. To evaluate cost differ-ences, we also solved first-order condition (8) for theoptimal Mexican capacity given C = p and denoteit by M�p. The right panel shows the optimal cost �C∗

Figure 6 Comparing the Square-Root Allocation �p to the Brownian Allocation �∗C

Scaled allocation: Square root vs. Brownian

Scaled cost: Square root vs. Brownian

0 0.2 0.4 0.6 0.8 1.0 0

0.2 0.4 0.6 0.8 1.00

Relative China cost cC /cM Relative China cost cC /cM

*�M, p

C(�p, �M, p)

and the cost C�p� M�p� when using the square-rootformulae.

One can observe that the scaled square-root allo-cation p is a reasonable approximation of the exactscaled Brownian allocation ∗

C , but the error increasesas the China cost increases. Indeed, in our numeri-cal study, the allocation difference is about 10% andbelow 28% as long as the China cost advantageexceeds 10%. Keep in mind that the relative error onthe total allocation prescription �− ∗

√� depends on

the volume � and will be much smaller, especially as� increases. The same comment applies to the relativecost difference. The main implication is that the sim-ple square-root formulae provide a reasonable start-ing point for the strategic China allocation.

, at 0

Figure 5 Comparing the Brownian Allocation to the Allocation Optimized via Simulation

Scaled cost: Brownian vs. optimal TBS

Scaled China allocation: Brownianvs. optimal TBS

0.150.05

0 0.2 0.4 0.6 0.8 1.0

Simulated cost of Brownian prescription

� = 1 � = 10

� = 100

� = 1 � = 100� = 10

Asymptotic (analytical)

� = 10

� = 1 � = 100

Asymptotic (analytical)*�C

Yet even for � = 1, the relative error in scaled costbetween the prescription and the optimal control wasless than 7%. The main implication is that the Brown-ian prescription is a good and useful approximation ofthe optimal strategic China allocation, even for smallvolumes.

8.3. Comparing the Square-Root Allocation to theBrownian Allocation

The left panel in Figure 6 shows the optimal capaci-ties ∗

C and ∗M in the Brownian model, as well as the

square-root approximation p. To evaluate cost differ-ences, we also solved first-order condition (8) for theoptimal Mexican capacity given C = p and denoteit by M�p. The right panel shows the optimal cost �C∗

Figure 6 Comparing the Square-Root Allocation �p to the Brownian Allocation �∗C

Scaled allocation: Square root vs. Brownian

Scaled cost: Square root vs. Brownian

0 0.2 0.4 0.6 0.8 1.0 0

0.2 0.4 0.6 0.8 1.00

Relative China cost cC /cM Relative China cost cC /cM

*�M, p

C(�p, �M, p)

and the cost C�p� M�p� when using the square-rootformulae.One can observe that the scaled square-root allo-

cation p is a reasonable approximation of the exactscaled Brownian allocation ∗

C , but the error increasesas the China cost increases. Indeed, in our numeri-cal study, the allocation difference is about 10% andbelow 28% as long as the China cost advantageexceeds 10%. Keep in mind that the relative error onthe total allocation prescription �− ∗

√� depends on

the volume � and will be much smaller, especially as� increases. The same comment applies to the relativecost difference. The main implication is that the sim-ple square-root formulae provide a reasonable start-ing point for the strategic China allocation.

, at 0

Gad Allon, Jan A. Van Mieghem (2010). Global Dual Sourcing: Tailored Base-Surge Allocation toNear- and Offshore Production. Management Science 56(1):110-124. 4 / 33

Example: Admission Control to a Many-Server Queue

Kocaga, Yasar Levent and Ward, Amy R (2010). Admission control for a multi-server queue with

abandonment. Queueing Systems, 65(3):275–323.

Asymptotic optimality for ρ(λ) = 1 +O(

1√λ

)as λ (and N)→∞

Punchline: “Universally” accurate5 / 33

Utilization (regime) assumptions and consequences

Patience ∼ exp 1

Service ∼ exp 100Load 1.1

Embedding Consequence (as λ, µ→∞)

ρ(λ) = 1 +1√λ

√λW λ(·)⇒ Reflected OU (critical load)

ρ(λ) ≡ 1.1√λ(W λ(·)− µ

λθ(ρ(λ)− 1))⇒ free OU (overload)

Universal process approx in Ward and Glynn (2003), Ward (2012)6 / 33

Sensitivity of the limit to patience modeling

Patience hazard rate

Mandelbaum Avishai and Sergey Zeltyn. (2013) Data-stories about (im)patient customers in

tele-queues. Queueing Systems 75(2), 115-146

With ρ(λ) = 1− β√λ

(critical load): E[W λ] = O(1/√λ)

7 / 33

Sensitivity of the limit to patience modeling

Consider the critically loaded M/M/1 + GI:

ρ(λ) = 1− β√λ

Finite patience drawn from a distribution Fa(·).

Limit Theorems differ by model

Fλa ≡ F does not scale with λ and has fa(0) > 0:

diffusion limit has linear drift; Ward and Glynn (2005);

Fλa has hazard rate that scales with λ: Fλ

a (x) = 1− e−∫ x

0 h(√λu)du,

limit has non-linear drift; Reed and Ward (2008).

8 / 33

Sensitivity of scaling to patience modeling

Suppose ρ(λ) = 1 (critical loading).

Fa=exponential (fixed):

E[W λ] = Θ

(1√λ

), as λ→∞.

Fa(x) = x2 for x ∈ [0,1] (fixed):

E[W λ] = Θ

(1λ1/3

), as λ→∞.

Different patience dist. → different scaling needed for limits.

A result that bypasses case-by-case analysis and interpretation..

9 / 33

The M/GI/1 + GI queue

1Service ∼ ⋅ ,Load

Patience ∼ ⋅

• The virtual wait V (t) is the time an infinitely patient customer,arriving at time t , would have to wait.• The waiting time is the minimum of the virtual wait and the

customer’s patience:

W (t) = min(ν,V (t)).

• The first order (“fluid”) proxy for the stationary virtual wait is w thatsolves

µ ∧ λ = λF (w).10 / 33

Dynamics of the virtual waiting time

• V (t) is the work contained in jobs that will not abandon.

V (t) = V (0) +

A(t)∑i=1

si1{vi>ωi} − (t − I(t)).

• Satisfies the natural positivity properties,- V (t) ≥ 0, ∀t ≥ 0;- I(·) is nondecreasing with I(0) = 0;

-∫ ∞

01{V (s)>0}dI(s) = 0.

Process limits for the GI/GI/1+GI queue: Ward and Glynn (2003,2005),Reed and Ward (2008), Jennings and Reed (2012)

11 / 33

The intuitive Brownian queue

V (t) = V (0) +

A(t)∑i=1

si1{vi>ωi} − (t − I(t))

= V (0) +

0ρFa(V (s))ds − t +

A(t)∑i=1

si1{vi>ωi} −∫ t

0ρFa(V (s))ds

+ I(t)

V (t) = V (0) +

0ρFa(V (s))ds − t + σB(t) + I(t), σ =

√(µ ∧ λ)E[s2

πV (dx) = G exp(

2∫ x

ρFa(u)− 1σ2 du

)dx , x ∈ [0,∞).

No scaling. The recommendation is to use πV as a proxy for πV .12 / 33

A notion of approximation accuracy

For the M/M/1 queue

V = W and W (t) is a one dimensional RBM.

If ρ < 1, W := W (∞) is expo(mean = ρ/(µ(1− ρ)))

M/M/1 : E[W k ] =k !ρ

(µ(1− ρ))k , Brownian Q : E[W k ] =k !ρk

(µ(1− ρ))k .

The approximation gap for the k th moment is

|E[W k ]− E[W k ]| =ρk !

(µ(1− ρ))k (1− ρk−1)

=k(1− ρk−1)

λρk−3(1− ρ)E[V k−1]

≤ k(k − 1)

λρk−3 E[W k−1].

For k = 1 the gap is 0 (the P-K formula for the M/GI/1 queue).13 / 33

A notion of approximation accuracy

For the M/GI/1 + GI queue, it “universally” holds

|E[W k ]− E[W k ]| ≤ CλE[W k−1]

0 500 1000 1500 2000

Scaled

Waitin

g Time

0.9 0.95 1 1.05 1.1 1.15 1.2

Scaled

Waitin

g Time

Figure: Hyper Exponential patience: Fa(x) =47

(1− e−4x ) +37

(1− e−x/2).M/M/1+GI moments using Zeltyn and Mandelbaum (2005).

14 / 33

Queue families

The M/GI/1+GI queue primitives are

p = (Arrival rate λ, Service time dist. Fs, Patience dist. Fa)

We will define Q-families parameterized by a constant H.

and prove results of the form

supp∈Q(H)

|E[W kp ]− E[W k

E[W k−1p ]

≤ CH

Universality = size the family Q(H)

Recall: scaling is sensitive to patience dist. and other primitives.

15 / 33

Queue family Q(H) = {p = (λ,Fs,Fa)}

(i) service-time moments: E[exp

)]≤ H,

and there exists a concentration constant cp ≥ (µH)−1 such that

(ii) finite load: ρ ∈ [H−1,H], ρ ≥ 1− Hλcp

(iii) polynomial growth: Fa is differentiable with density fa:

fa(y) ≤ Hλc2

∣∣∣∣y − wp

∣∣∣∣H),

(iv) concentration:

ρFa (y)− 1 ≤ −H−1 1λcp

, for all y ≥ wp + cpH,

andρFa (y)− 1 ≥ H−1 1

λcp, for all y ≤ wp − cpH,

Notice: cp, wp vary with the primitives.16 / 33

Indeed a large family

exp(θ) patience (ρ > 1 : ρe−θw = 1):

H√λ

)− 1 ≤ −H−1 1

λcp⇐⇒ ρe−θ

(w+ H√

)− 1 ≤ −H−1 1√

Facp H

ρ ≤ 1 ρ > 1

Infinite 1λ(1−ρ)

−− 1/ρ

exp(θ) 1λ(1−ρ)

∧ 1√λ

1√λ

max(θ, 2/θ, ρ, 1/ρ)

Uniform[0, α] 1λ(1−ρ)

∧ 1√λ

1√λ

max(1/α,√α/ρ, ρ, 1/ρ)

HyperExp(θ, θ) 1λ(1−ρ)

∧ 1√λ

1√λ

max(θ, 2/θ, ρ, 1/ρ)

Power(α, k ) 1λ(1−ρ)

∧ λ−1

k+1 λ− 1

k+1 ∧ 1√λ(ρ−1)

1− 1k

1∨α1∧ρ ∨

2k (k∨ρ)

(1∧α)k

Erlang(k, θ) 1λ(1−ρ)

∧ λ−1

k+1 λ− 1

k+1 ∧ 1√λ(ρ−1)

1− 1k

2k+1(ρ∨k)HE0 Γ(k)

ρ∧1 max(θk , 1θk ) ∨ U

Beta(α, β) 1λ(1−ρ)

∧ λ−1α+1 λ

− 1α+1 ∧ 1√

λ(ρ−1)1− 1

2α+β (ρ∨α)Γ(α+β)(ρ∧1)Γ(α)Γ(β) min(L,1)

Table: cp and H for a family of patience distributions.

All these are in one queue family.17 / 33

The accuracy of the Brownian approximation

Theorem (Virtual waiting time)

Given H > 0 and k ∈ N, there exists a constant C1H,k > 0 such that

E[(V − w)k ]− E[(V − w)k ] = ±C1

λE[|V − w |k−1], p ∈ Q(H).

Corollary (Waiting time)

Given H and k ∈ N, there exists a constant C2H,k > 0 such that

E[W k ]− E[W k ] = ±C2

λE[W k−1], p ∈ Q(H).

For k = 1, the error is O(1/λ).

18 / 33

The accuracy of the Brownian approximation

Corollary (Queue length)

Given H, there exists a constant C2H,1 > 0 such that

E[Q] = λE[W ] = λE[W ]± C2H,1, p ∈ Q(H).

The mean-queue approximation gap is a constant.

Corollary (Abandonment)

Given H, there exists a constant CH > 0 such that

Ab = E[Fa(V )]± CH

λ2E[|V − w |2], p ∈ Q(H).

For example, if Fa = exp(θ), the Ab approximation gap is O(1/λ).19 / 33

About the tightness of the Q-family conditions

0 200 400 600 800 1000 1200 1400 1600

Scaled

Virtua

l Waitin

g Time

M/D/1 + GI with ρ = 1 (w = 0) and Fa = Gamma(0.5, 2).

Fa = Gamma(0.5,2) violates our conditions.20 / 33

cp captures concentration/scaling

Lemma (Concentration bounds)

There exist constants CVH,k , cV

H,k > 0 such that

cVH,k ≤

E[|V − w |k ]

cpk ≤ CVH,k , p ∈ Q(H).

There exist CH,k ,C1H,k > 0 such that for all p ∈ Q(H),

E[(V − w)k ]− E[(V − w)k ] = ±C1

λE[|V − w |k−1]

= ±CH,k

λck−1

Example: Fa = exp(θ)→ cp =1√λ

E[(V − w)k ]− E[(V − w)k ] =±CH,k

21 / 33

cp captures concentration/scaling

Lemma (Concentration bounds)

There exist constants CVH,k , cV

H,k > 0 such that

cVH,k ≤

E[|V − w |k ]

cpk ≤ CVH,k , p ∈ Q(H).

There exist CH,k ,C1H,k > 0 such that for all p ∈ Q(H),

E[(V − w)k ]− E[(V − w)k ] = ±C1

λE[|V − w |k−1]

= ±CH,k

λck−1

Example: Fa = exp(θ)→ cp =1√λ

E[(V − w)k ]− E[(V − w)k ] =±CH,k

21 / 33

Underlying Math: Generator comparisons (B&D)

gk (x) =(x − w)k − E[(V − w)k ]

(E[|V − w |])kso that E[gk (V )] = 0

Solve (for Ψ) (AΨ)(x) = gk (x) (Brownian Poisson Eqn)

E[(AΨ)(V )− gk (V )] = E[(AΨ)(V )− (AΨ)(V )] + E[(AΨ)(V )− gk (V )]

= E[(AΨ)(V )− (AΨ)(V )]

If E[AΨ(V )] = 0 (Glynn and Zeevi (2008)), then

E[gk (V )] =E[(V − w)k − E[(V − w)k ]]

(E[|V − w |])k= −E[(AΨ)(V )− (AΨ)(V )]

22 / 33

Generator comparison and gradient bounds

|E[(V − w)k ]− E[(V − w)k ]| ≤ (E[|V − w |])k |E[(AΨ)(V )− (AΨ)(V )]|

AΨ(x) = −Ψ(1)(x) + λFa (x)E[Ψ(x + s1)− Ψ(x)

]= −Ψ(1)(x) + λFa (x)E

[Ψ(1)(x)s1 +

Ψ(2)(x)s21 + ε(x , s1)

]= AΨ(x) + λFa (x)E [ε(x , s1)] .

E[ε(x , s1)] has Ψ’s derivative of order m > 2.

E[|AΨ(V )− AΨ(V )|] ≤ λE[Fa (V )E [ε(V , s1)]]show≤ CH

λE[|V − w |].

Where does V on the right-hand side come from?Show = Gradient + Apriori moment bounds (via cp drift cond.)See Braverman and Dai (2016)

23 / 33

From performance analysis to optimization

24 / 33

Two ways in which regimes arise

“Consider a sequence of queues with ρ(λ) = 1− β√λ

Identifying the “optimal” regime:Minimizing capacity + linear delay cost in the M/M/1 queue

µ∗(λ) := minµ

csµ+ cwλE[W (µ)] = λ+

√λcw

so that

√λ(1− ρ∗(λ)) =

1− λ

λ+√

λcwcs

→√cw

csas λ→∞

ρ∗(λ) ≈ 1−√

1√λ

If cw =14λ and cs = 1, then, ρ∗ = 1/2.

25 / 33

Dynamic optimization:Service-rate control in the M/G/1 queue

Arrival rate λ; Service time distribution Fs with E[s1] = 1.

Controlled service rate µ(θ) = λ(1 + θ). Holding cost hxm.

p = (λ,h).

J V ,∗p,m = inf

θ∈ΘVlim

t→∞

[∫ t

(h(V (θ, s))m + (λθ(s))2

Steps:

An unscaled Brownian Control Problem (BCP)

Universality over Q(H) = {(λ,h) : λ ≥ H−1,h ∈ (0,H)}

We will be agnostic to whether (or not) h scales down with λ

26 / 33

The Brownian control problem

V (θ, t) =V (0)− λ∫ t

0θ(s)ds +

0λ(1 + θ(s))1{V (θ, s) = 0}ds

A(t)∑i=1

si − λt

V (θ, t) =V (0)− λ∫ t

0θ(s)ds + λ

0(1 + θ(s))1{V (θ, s) = 0}ds

+√λE[s2

1]B(t).

J V ,∗p,m = inf

θ∈ΘV

limt→∞

[∫ t

(h(V (θ, s))m + (λθ(s))2

27 / 33

Universal optimality gap

TheoremAn optimal stationary (Brownian) policy θ∗p,m(x) exists and, for anyp ∈ Q(H) := {(λ,h) : λ ≥ H−1,h ∈ (0,H]},

J V ,∗p,m − J V

p,m(θ∗p,m) ≤ BH(λ,m)J V ,∗p,m−1

BH(λ,m)→ 0 as λ→∞. The gap is 0 if m = 2.

Recall, we found for the (uncontrolled) M/GI/1 + GI queue:

E[(V − w)k ]− E[(V − w)k ] = ±C1

λE[|V − w |k−1]

28 / 33

Universal optimality gap

TheoremAn optimal stationary (Brownian) policy θ∗p,m(x) exists and, for anyp ∈ Q(H) := {(λ,h) : λ ≥ H−1,h ∈ (0,H]},

J V ,∗p,m − J V

p,m(θ∗p,m) ≤ BH(λ,m)J V ,∗p,m−1

BH(λ,m)→ 0 as λ→∞. The gap is 0 if m = 2.

Recall, we found for the (uncontrolled) M/GI/1 + GI queue:

E[(V − w)k ]− E[(V − w)k ] = ±C1

λE[|V − w |k−1]

28 / 33

For control, too, generator comparisons

γ = minz≥0

λΨ)(x) + (λz)2 + hxm}, (Diffusion HJB)

Ψ(0) = Ψ(1)(0) = 0 and Ψ(1)(x) ≥ 0, for all x ≥ 0,

Given (Ψ, γ) : optimal service rate z∗(x) =Ψ(1)(x)

γ = minz≥0{(Az

λΨ)(x) + (λz)2 + hxm} (M/GI/1 Bellman)

(Doshi (1978))

If for relevant values of z, AzλΨ ≈ Az

minz≥0{(Az

λΨ)(x) + λz + hxm} ≈ minz≥0

(AzλΨ)(x) + λz + hxm}

→ Ψ, γp,m “almost” solves the M/G/1 Bellman equation.29 / 33

From BCP to M/GI/1 optimality

Fix (p,m) and let (Ψ, γ) be the solution the (BCPs) HJB equation.Then, for any admissible control θ for the M/G/1 queue (and anyx , t ≥ 0):

[∫ t

(h(V (θ, s))m + (λθ(s))2

)ds]≥ Ψ(x)− Ex

[Ψ(V (θ, t))

]+ γt

+ Ax (θ, t),

where Ax (θ, t) = Ex

[∫ t

(Aθ(s)λ Ψ(V (θ, s))− Aθ(s)

λ Ψ(V (θ, s)))

If θ is the BCP stationary control z∗, the inequality is replaced withequality. If m = 2, Ax (θ, t) ≡ 0 for any control θ.

30 / 33

From BCP to M/GI/1 optimality

Fix m and let (Ψp,m, γp,m) be the (family of) solutions to the HJBequation. Then, there exist constants C1

H,m,C2H,m such that, for any

order optimal family of policies {θp,m,p ∈ Q(H)},

lim inft→∞

p,m(θp,m, t) ≥ −C1H,mBH(λ,m)J Y ,∗

p,m−1, x ≥ 0,

and under the stationary policy θ∗p,m,

lim supt→∞

p,m(θ∗p,m, t) ≤ C2H,mBH(λ,m)J Y ,∗

p,m−1, x ≥ 0.

31 / 33

Conclusion

• In great generality, it is fine to use the intuitive Brownian queue ofthe M/GI/1+GI.

It is universally accurate in regime and patience-scaling.

• Similar ideas are applied to static and dynamic optimization

Underlying math:Avoid scaling through Q-families.From the universal proximity of operators to the universal proximityof equation solutions (Poisson or HJB).

32 / 33

Time Dependent Expectations

0 50 100 150 200 250 300Second

First m

0 50 100 150 200 250 300

Scaled

Figure: Time-dependent performance for M/M/1 with µ = 1: (LHS) ρ = 0.9,(RHS) Scaled gap

33 / 33

beyond heavy-traffic regimes: universal bounds …stochnet/talks/gurvich16.pdfbeyond heavy-trafﬁc...

Documents

sample complexity bounds for iterative stochastic … ·...

gi industry celebrates gi day

out of bounds

ball in bounds

math. metes and bounds metes & bounds = distances &...

exchange rate regimes in the caribbean · 2019-01-22 ·...

regret bounds and minimax policies under partial...

international regimes, transactions, and change: · pdf...

leaps & bounds

political regimes.

kernel lower bounds

knowledge regimes

inter pech bounds

basque coast geopark saturraran bay of · pdf filebasque...

christopher bounds 0

on linear data structures and matrix rigidity · 2021. 1....

totalitarian regimes

- 6j>enekj ) * $(%/ ) (. (...(gi (gi (gi) (’ 445) 445...

virginia bounds 'tremenheere

international cooperation of authoritarian regimes ·...