7/24/2019 FINAL EXAM APME Corrected

1/2

Final Exam ESI 6321

Due date: Upload onto Canvas 12/12/15 (5:00 pm)

NOTE:You may work in pairs. There shall be no collaboration other than the one

enabled by working with another student. Those opting to work individually must

only submit 3 of their choosing. Pairs must submit all 5 problems. No extra credit.

Submit pdf le on Canvas.

1. LetfXn : n >0gan i.i.d collection of random variables with

P(Xn= i) =i 0 P

i0i = 1

LetWn = maxfX1; X2;:::;Xng.We say that a recordoccurs at time n ifXn > Wn1 andif a record does occur at time n we call Xn the record value. Let Rk denote the k-threcord value.

a. Show thatfRk :k 0g is a Markov chain and give an expression for its transitionprobabilities.

b. LetTkdenote the time between the kth and the(k + 1)threcord. IsfTk:k 0ga Markov chain ? Is the bivariate processf(Rk; Tk) : k 0g a Markov chain ? Ifyes, nd the transition probabilities.

2. Unit demands for an item in inventory at a warehouse come according to a Poisson processwith rate . When the inventory level hits 0, an order of size Sunits is placed. Deliverytime is a random variable exponentially distributed (with mean 1

). During this time

demand is lost at a cost of$cper unit. The inventory holding cost is$hper unit per timebased on the averageamount of inventory held. Find the long run cost per unit time ofinventory holding and lost sales.

3. A software application has just been developed. The number of bugs in the code is arandom variable uniformly distributed inf0; 1; : : : ; 10g. Bugs can be detected by testing.Each undetected bug is found independently with probability 0:1p

1+nwheren is the number

of bugs found during previous tests. Each test costs $1 to undertake. Each undetectedbug left in the code has a cost of $10. Find the optimal testing policy (minimizes expectedtotal cost).

4. LetfXt: t 0g be the solution todXt

Xt=t + dBt

1

7/24/2019 FINAL EXAM APME Corrected

2/2

whereBt is standard Brownian motion andX0= x with0 < a < x < b. Let be denedas

= minft 0 :Xt= a or Xt= bg

Letv(x)denote the expected time to hitting a boundary, i.e.

v(x) =E[jX0= x]

withv(a) = 0 and v(b) = 0.

a. Uso Itos lemma to derive the dierential equation

v0(x) +1

2v00(x)2 =1

for 0 < a < x < b. Hint: Note that v(x) = dt+ E[v(x+ dXt)]. ApproximateE[v(x + dXt)] v(x)by E[dv]

b. Solve the dierential equation to nd an expression forv(x).

5. Let = minft 0 : Xt = 0g where Xt = x+Bt andBt is standard Brownian motionandx >0. Let r >0. Show that

v(x) =E[ erX0= x] =e

p2rx

2