approximation by geometric sums: markov chain passage...

Approximation by Geometric Sums:Markov chain passage times and queueing models

Fraser Daly

Heriot–Watt University

LMS–EPSRC Durham SymposiumMarkov Processes, Mixing Times and Cutoff

July 2017

Fraser Daly (Heriot–Watt University) Geometric Sums July 2017 1 / 21

Outline

Geometric sums

Approximation results

Application to Markov chain passage times

Approximations using bounded failure rate

Applications to the M/G/1 queue

Fraser Daly (Heriot–Watt University) Geometric Sums July 2017 2 / 21

Geometric sums

Let Y = X1 + X2 + · · ·+ XN , where

X ,X1,X2, . . . are i.i.d. positive integer-valued random variables, and

N has a geometric distribution (independent of the Xi ), supported on{0, 1, 2, . . .}, with P(N = 0) = p.

Applications include

Records processes (time until a new record is set)

Random walks, including the ruin problem (maximum of the randomwalk)

Reliability systems (time to failure)

Fraser Daly (Heriot–Watt University) Geometric Sums July 2017 3 / 21

Geometric sums

We will consider situations where a (non-negative, integer-valued) randomvariable of interest W has a distribution close to that of a geometric sum,measured by total variation distance:

dTV (L(W ),L(Y )) = supA⊆Z+

|P(W ∈ A)− P(Y ∈ A)|

= inf(W ,Y )

P(W 6= Y ) ,

where the infemum is taken over all couplings of W and Y .

Fraser Daly (Heriot–Watt University) Geometric Sums July 2017 4 / 21

Geometric sums

Renyi’s theorem:

limp→0

P(

1

ENY ≤ x

)= 1− exp

{− x

EX

}

Explicit bounds in exponential approximation are available, as are explicitbounds for geometric approximation of the geometric sum Y .

Fraser Daly (Heriot–Watt University) Geometric Sums July 2017 5 / 21

A characterisation

Define the random variable V by

V + Xd= (W |W > 0) .

Note that if W is a geometric sum (with summand X ), then Vd= W . The

converse is also true.

We may expect to be able to quantify the distance of W from ourgeometric sum Y by assessing the distance of W from V .

Fraser Daly (Heriot–Watt University) Geometric Sums July 2017 6 / 21

Approximation by geometric sums

Theorem

If p = P(W = 0),

dTV (L(W ),L(Y )) ≤ 1− p

pdTV (L(W ),L(V )) .

The proof, using Stein’s method, is based on the above characterisationfor geometric sums.

Fraser Daly (Heriot–Watt University) Geometric Sums July 2017 7 / 21

Proof

Write q = 1− p. Given a function h : Z+ 7→ R, define the function f = fhas the solution to the following ‘Stein equation’:

h(k)− Eh(Y ) = qE[f (k + X )]− f (k)

(with f (0) = 0), so that

Eh(W )− Eh(Y ) = qE[f (W + X )]− qE[f (W )|W > 0]

= qE[f (W + X )− f (V + X )] .

Hence|Eh(W )− Eh(Y )| ≤ qP(W 6= V ) sup

j ,k|f (j)− f (k)| .

With h(k) = I (k ∈ A), taking the supremum over A ⊆ Z+ gives us dTV onthe LHS. The final term on the RHS may be shown to be bounded by 1/p.

Fraser Daly (Heriot–Watt University) Geometric Sums July 2017 8 / 21

Markov chain passage times

Let {ξi : i ≥ 0} be an ergodic discrete-time Markov chain with transitionmatrix P, started according to its stationary distribution π, and letW = min{i : ξi ∈ B} for some set of states B.

Let Bi be the set of states j from which a move to B requires a minimumof i steps, i.e., for which P(ξi ∈ B|ξ0 = j) > 0 but P(ξk ∈ B|ξ0 = j) = 0for k < i .

Let

p = P(W = 0) = π(B) ,

µi = P(ξ0 ∈ Bi |ξ0 6∈ B) =

∑j∈Bi

π(j)

1− p.

Define the random variable X by P(X = i) = µi .

Fraser Daly (Heriot–Watt University) Geometric Sums July 2017 9 / 21

Markov chain passage times

Theorem

dTV (L(W ),L(Y )) ≤ 1

pE∑i ,j∈B

πi∑n≥0

∣∣∣P(n+X )ij − πj

∣∣∣ .This is proved using our earlier bound: construct copies of the originalchain, but started according to π restricted to Bj for each j . Randomisingj according to X gives us the random variable V by considering the firstpassage time to the set of states B.

We now use the maximal coupling of these processes with copies of ouroriginal Markov chain to estimate P(W 6= V ). Some analysis yields thestated bound.

Fraser Daly (Heriot–Watt University) Geometric Sums July 2017 10 / 21

Application: Sequence patterns

Consider a sequence of IID Bernoulli trials (of 0s and 1s), and let Ii be theindicator that a given k-digit pattern (B, say) appears, starting at positioni in our sequence. We are interested in W = min{i : Ii = 1}, the time wehave to wait to observe our pattern.

Define a 2k state Markov chain such that at time n, the state of theprocess is the outcome of the k Bernoulli trials from time n to timen + k − 1.

Our theorem gives the bound

dTV (L(W ),L(Y )) ≤k−1∑i=1

i∑n=1

µn|ci − p| ,

where ci = P(Ii = 1|I0 = 1). This represents the i-step transitionprobability from B to B, for i ≤ k − 1.

This is sharper than available geometric approximation bounds.

Fraser Daly (Heriot–Watt University) Geometric Sums July 2017 11 / 21

Application: Sequence patterns

For example, let k = 3 and consider the pattern 010.

We have that p = r(1− r)2, where r is the expected value of each of theoriginal Bernoulli variables.

We have the following partition of the state space of our Markov chain:

B = {010} , B1 = {001, 101} , B2 = {000, 100, 110} , B3 = {011, 111} .

We can easily calculate c1 = 0, c2 = r(1− r),

µ1 =r(1− r)

1− p, µ2 =

(1− r)(1− r + r2)

1− p, µ3 =

r2

1− p.

Fraser Daly (Heriot–Watt University) Geometric Sums July 2017 12 / 21

Approximations using bounded failure rate

Our general bound above can be tricky to evaluate: estimatingdTV (L(W ),L(V )) is not always easy.

However, simpler upper bounds for the approximation of W by ageometric sum are available, under assumptions on the structure of theunderlying random variables.

Fraser Daly (Heriot–Watt University) Geometric Sums July 2017 13 / 21

Notation

Let W be a non-negative, integer-valued random variable. Define thefailure (or hazard) rate of W by

rW (j) =P(W = j)

P(W > j).

For future use, we also define (for 0 ≤ p < 1 and X a positive,integer-valued random variable)

Hp(X ) = min

{p + (1− p)P(X > 1), p

(1 +

√−2

u log(1− p)

)},

where u = 1− dTV (L(X ),L(X + 1)), a measure of the ‘smoothness’ of X .This is small (close to p) when X is ‘smooth’ or P(X = 1) is large.

Fraser Daly (Heriot–Watt University) Geometric Sums July 2017 14 / 21

Approximations using bounded failure rate

Theorem

Let P(W = 0) = p and rW (j) > δ > 0 for all j . If

EX ≥ p

(1− p)δ,

thendTV (L(W ),L(Y )) ≤ Hp(X ) (EY − EW ) ,

where Y = X1 + · · ·+ XN is our geometric sum.

Natural candidates for applications are when W has increasing failure rate(IFR) or decreasing failure rate (DFR). The first setting leads to simplegeometric approximation bounds (X = 1 almost surely).

Fraser Daly (Heriot–Watt University) Geometric Sums July 2017 15 / 21

Proof

Let X be as in the theorem, and V be as before. We can show thatP(V + X > j) ≤ P(W + X > j).

Let f be the solution to the same ‘Stein equation’ as before:

h(k)− Eh(Y ) = (1− p)E[f (k + X )]− f (k) .

When h(k) = I (k ∈ A) for A ⊆ Z+, we can show that

supj|∆f (j)| ≤ 1

pHp(X ) ,

where ∆f (j) = f (j + 1)− f (j).

Fraser Daly (Heriot–Watt University) Geometric Sums July 2017 16 / 21

Proof

Rearrange our Stein equation to get

Eh(W )− Eh(Y ) = (1− p)∞∑j=0

∆f (j) [P(W + X > j)− P(V + X > j)] .

With h(k) = I (k ∈ A), we then obtain

|P(W ∈ A)− P(Y ∈ A)| ≤ 1− p

pHp(X )E[W − V ] = Hp(X )E[Y −W ] .

Taking the supremum over A ⊆ Z+ gives us dTV (L(W ),L(Y )).

Fraser Daly (Heriot–Watt University) Geometric Sums July 2017 17 / 21

Applications to the M/G/1 queue

Consider a single server queue with

customers arriving at rate λ, and

i.i.d. customer service times with the same distribution as S .

Let ρ = λES and assume ρ < 1.

We will approximate

1 the number of customers in the system in equilibrium by a geometricdistribution (so X = 1 almost surely), and

2 the number of customers served during a busy period by a geometricsum.

Fraser Daly (Heriot–Watt University) Geometric Sums July 2017 18 / 21

Number of customers in the system

Let W be the equilibrium number of customers in the system. It iswell-known that

P(W = 0) = 1− ρ , EW = ρ+ρ2E[S2]

2(1− ρ)(ES)2,

rW (j) ≥ 1− ρλ supt≥0 E[S − t|S ≥ t]

, ∀j ∈ Z+ .

S is said to be New Better than Used in Expectation (NBUE) ifE[S − t|S ≥ t] ≤ ES for all t ≥ 0. In that case our theorem gives

dTV (L(W ),Ge(1− ρ)) ≤ ρ2

(1− E[S2]

2(ES)2

).

As expected, this upper bound is zero if S has an exponential distribution.

Fraser Daly (Heriot–Watt University) Geometric Sums July 2017 19 / 21

Customers served during a busy period

Let W + 1 be the number of customers served during a busy period of oursystem. Then P(W = 0) = Ee−λS = p and EW = ρ(1− ρ)−1.

If S is IFR, it is known that W is DFR. We may then find a lower boundon the failure rate as follows.

Let ψ be the Laplace transform of the density of S and let ξ be the realroot of 1 + λψ′(s) nearest the origin. Let

θ =ξ − λ+ λψ(ξ)

(ξ − λ)ψ(λ).

Then if Y =∑N

i=1 Xi , where N ∼ Ge(p) and (1− p)θEX ≥ 1− pθ,

dTV (L(W ),L(Y )) ≤ Hp(X )

((1− p)EX

p− ρ

1− ρ

).

Fraser Daly (Heriot–Watt University) Geometric Sums July 2017 20 / 21

ReferencesS. Asmussen (2003). Applied Probability and Queues 2nd ed. Springer, New York.

F. Daly (2010). Stein’s method for compound geometric approximation. J. Appl. Prob.47: 146–156.

F. Daly (2016). Compound geometric approximation under a failure rate constraint. J.Appl. Prob. 53: 700–714.

V. Kalashnikov (1997). Geometric Sums: Bounds for Rare Events with Applications.Kluwer, Dordrecht.

E. K. Kyprianou (1972). The quasi–stationary distributions of queues in heavy traffic. J.Appl. Prob. 35: 821–831.

E. Pekoz (1996). Stein’s method for geometric approximation. J. Appl. Prob. 33:707–713.

E. Pekoz and A. Rollin (2011). New rates for exponential approximation and the theoremsof Renyi and Yaglom. Ann. Prob. 39: 587–608.

E. Pekoz, A. Rollin and N. Ross (2013). Total variation error bounds for geometricapproximation. Bernoulli 19: 610–632.

A. Renyi (1957). A characterization of Poisson processes. Magyar Tud. Akad. Mat.Kutato Int. Kozl. 1: 519-527.

S. Ross (2006). Bounding the stationary distribution of the M/G/1 queue size. Probab.Engrg. Inform. Sci. 20: 571–574.

J. G. Shanthikumar (1988). DFR property of first-passage times and its preservationunder geometric compounding. Ann. Prob. 16: 397–406.

Fraser Daly (Heriot–Watt University) Geometric Sums July 2017 21 / 21