cheng-fu chou, cmlab, csie, ntu intro. to stochastic processes cheng-fu chou
TRANSCRIPT
![Page 1: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/1.jpg)
Cheng-Fu Chou, CMLab, CSIE, NTUCheng-Fu Chou, CMLab, CSIE, NTU
Intro. to Stochastic Processes
Cheng-Fu Chou
![Page 2: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/2.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
P. 2
Outline
Stochastic Process Counting Process Poisson Process Markov Process Renewal Process
![Page 3: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/3.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
P. 3
Stochastic Process
A stochastic process N= {N(t), t T} is a collection of r.v., i.e., for each t in the index set T, N(t) is a random variable– t: time– N(t): state at time t– If T is a countable set, N is a discrete-time stochastic
process– If T is continuous, N is a continuous-time stoc. proc.
![Page 4: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/4.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
P. 4
Counting Process
A stochastic process {N(t) ,t 0} is said to be a counting process if N(t) is the total number of events that occurred up to time t. Hence, some properties of a counting process is– N(t) 0– N(t) is integer valued– If s < t, N(t) N(s)– For s < t, N(t) – N(s) equals number of events occurring in
the interval (s, t]
![Page 5: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/5.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
P. 5
Counting Process
Independent increments– If the number of events that occur in disjoint time intervals
are independent
Stationary increments– If the dist. of number of events that occur in any interval of
time depends only on the length of time interval
![Page 6: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/6.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
P. 6
Poisson Process
Def. A: the counting process {N(t), t0} is said to be Poisson process having rate l, l>0 if– N(0) = 0;– The process has independent-increments– Number of events in any interval of length t is Poisson dist.
with mean lt, that is for all s, t 0.
( )[ ( ) ( ) ]
! = 0,1,2,...
nt t
P N t s N s n en
n
![Page 7: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/7.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
P. 7
Poisson Process
Def. B: The counting process {N(t), t 0} is said to be a Poisson process with rate l, l>0, if – N(0) = 0– The process has stationary and independent increments– P[N(h) = 1] = lh +o(h)– P[N(h) 2] = o(h)– The func. f is said to be o(h) if – Def A Def B, i.e,. they are equivalent.– We show Def B Def A – Def A Def B is HW
0
( )lim 0h
f h
h
![Page 8: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/8.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
P. 8
Important Properties
Property 1: mean number of event for any t 0, E[N(t)]=lt.
Property 2: the inter-arrival time dist. of a Poisson process with rate l is an exponential dist. with parameter .l
Property 3: the superposition of two independent Poisson process with rate l1 and l2 is a Poisson process with rate l1+l2
![Page 9: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/9.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
P. 9
Properties (cont.)
Property 4: if we perform Bernoulli trials to make independent random erasures from a Poisson process, the remaining arrivals also form a Poisson process
Property 5: the time until rth arrival , i.e., tr is known as the rth order waiting time, is the sum of r independent experimental values of t and is described by Erlan pdf.
![Page 10: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/10.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
P. 10
Ex 1
Suppose that X1 and X2 are independent exponential random variables with respective means 1/l1 and 1/l2;What is P{X1 < X2}
![Page 11: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/11.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
P. 11
21
1
0
)(1
10
10 2
0 112121
}{
}|{}{
21
12
1
1
dxe
dxee
dxeXxP
dxexXXXPXXP
x
xx
x
x
![Page 12: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/12.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
P. 12
Conditional Dist. Of the Arrival Time
Suppose we are told that exactly one event of a Poisson process has taken place by time t, what is the distribution of the time at which the event occurred?
![Page 13: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/13.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
P. 13
t][0,over ddistributeuniformly isevent theof time theSo,
1} P{N(t)
t)}[s,in events 0 s)}P{[0,in event 1{
1} P{N(t)
t)}[s,in events 0 s),[0,in event 1{
}1)({
}1)(,{}1)(|{
)(
t
ste
ese
P
P
tNP
tNsxPtNsxP
t
sts
![Page 14: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/14.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
P. 14
Ex 2
Consider the failure of a link in a communication network. Failures occur according to a Poisson process with rate 4.8 per day. Find– P[time between failures 10 days]– P[5 failures in 20 days]– Expected time between 2 consecutive failures– P[0 failures in next day]– Suppose 12 hours have elapsed since last failure, find the
expected time to next failure
![Page 15: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/15.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
P. 15
hours 5 .5
4.
hours 5 .35!
20)*(4.8 .2
1 .1
8.4
20*8.45
10*8.4
e
e
e
![Page 16: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/16.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
Poisson, Markov, Renewal Processes
P. 16
Poisson Process:
Counting process
iid exponential times between
arrivals
Continuous Time Markov Chain:
Exponential times between transitions
Renewal Process:
Counting process
iid times between arrivals
Relax countingprocess
Relax exponentia
l interarrival
times
![Page 17: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/17.jpg)
Cheng-Fu Chou, CMLab, CSIE, NTUCheng-Fu Chou, CMLab, CSIE, NTU
Markov Process
![Page 18: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/18.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
P. 18
Markov Process
P[X(tn+1) Xn+1| X(tn)= xn, X(tn-1) = xn-1,…X(t1)=x1] = P[X(tn+1) Xn+1| X(tn)=xn]– Probabilistic future of the process depends only on the
current state, not on the history– We are mostly concerned with discrete-space Markov
process, commonly referred to as Markov chains– Discrete-time Markov chains– Continuous-time Markov chains
![Page 19: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/19.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
P. 19
DTMC
Discrete Time Markov Chain: – P[Xn+1 = j | Xn= kn, Xn-1 = kn-1,…X0= k0]
= P[Xn+1 = j | Xn = kn] discrete time, discrete space A finite-state DTMC if its state space is finite A homogeneous DTMC if P[Xn+1 = j | Xn= i ] does not depend
on n for all i, j, i.e., Pij = P[Xn+1 = j | Xn= i ], where Pij is one step transition prob.
![Page 20: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/20.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
P. 20
Definition P = [ Pij] is the transition matrix
– A matrix that satisfies those conditions is called a stochastic matrix– n-step transition prob.
00 01 0
10 11 1
0
... ...
... ...
... ... ... ... ...
... ... ...
... ... ... ... ...
where 0 and 1
j
j
i ij
ij ijj
p p p
p p p
P
p p
p p
0[ | ]
, , 0, is the prob. of going from state
to in step
nij n
nij
p P x j x i
i j n p i
j n
![Page 21: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/21.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
P. 21
Chapman-Kolmogorov Eq.
Def.
Proof:
( )
For all n 0, m 0, i , j I
in matrix form where =[ ]
n m n mij ik kj
k I
n m n m n nij
p p p
P P P P p
![Page 22: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/22.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
P. 22
Question
We have only been dealing with conditional prob. but what we want is to compute the unconditional prob. that the system is in state j at time n, i.e.
0
0 0 0
0 0
0
( ) ( )
So, given the initial dist. of ,i.e.,
( ) ( ) and 1
we can get
[ ] ( | ) ( )
( )
n n
i I
n ni I
nij
i I
j p x j
x
i p x i
p x j p x j x i i
p i
![Page 23: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/23.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
P. 23
Result 1
For all n 1, pn = p0Pn, where pm = (pm(0),pm(1),…) for all m 0. From the above equ., we deduce that pn+1 = pnP. Assume that limn pn(i) exists for all i, and refer it as p(i). The remaining question is how to compute p
– Reachable: a state j is reachable from i. if
– Communicate: if j is reachable from i and if i is reachable form j, then we say that i and j communicate (i j)
0 for some 1nijp n
![Page 24: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/24.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
P. 24
Result 1 (cont.)
Irreducible:– A M.C. is irreducible if i j for all i,j I
Aperiodic:– For every state iI, define d(i) to be largest common divisor
of all integer n, s.t.,
0 if ( ) 1 then the state is aperiodicn
ijp d i
![Page 25: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/25.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
P. 25
Result 2
Invariant measure of a M.C., if a M.C. with transition matrix P is irreducible and aperiodic and if the system of equation p=pP and p1=1 has a strict positive solution then p(i) = limn pn(i) independently of initial dist.– Invariant equ. : p=pP – Invariant measure p
![Page 26: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/26.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
P. 26
Gambler’s Ruin Problem
Consider a gambler who at each play of game has probability p of winning one unit and probability q=1-p of losing one unit. Assuming that successive plays of the game are independent, what is the probability that, starting with i units, the gambler’s fortune will reach N before reaching 0?
![Page 27: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/27.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
P. 27
Ans
If we let Xn denote the player’s fortune at time n, then the process {Xn, n=0, 1,2,…} is a Markov chain with transition probabilities:– p00 =pNN =1– pi,i+1 = p = 1-pi,i-1
This Markov chain has 3 classes of states: {0},{1,2,…,N-1}, and {N}
![Page 28: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/28.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
P. 28
Let Pi, i=0,1,2,…,N, denote the prob. That, starting with i, the gambler’s fortune will eventually reach N.
By conditioning on the outcome of the initial play of the game we obtain– Pi = pPi+1 + qPi-1, i=1,2, …, N-1
Since p+q =1Pi+1 – Pi = q/p(Pi-Pi-1),
Also, P0 =0, so
P2 – P1 = q/p*(P1-P0) = q/p*P1
P3 - P2 =q/p*(P2-P1)= (q/p)2*P1
![Page 29: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/29.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
P. 29
1/2 p if 0
1/2p if p
q-1
N as that,Note
2/1p if 1
2/1p if )/(1
)/(1
obtain we,1 using Now,
1q
p if
1q
p if
)/(1
)/(1
i
1
1
1
i
N
N
i
i
P
N
pq
pq
P
P
iP
Ppq
pq
P
![Page 30: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/30.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
P. 30
If p > ½, there is a positive prob. that the gambler’s fortune will increase indefinitely
Otherwise, the gambler will, with prob. 1, go broke against an infinitely rich adversary.
![Page 31: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/31.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
P. 31
CTMC
Continuous-time Markov Chain– Continuous time, discrete state– P[X(t)= j | X(s)=i, X(sn-1)= in-1,…X(s0)= i0]
= P[X(t)= j | X(s)=i]– A continuous M.C. is homogeneous if
oP[X(t+u)= j | X(s+u)=i] = P[X(t)= j | X(s)=i] = Pij[t-s], where t > s
– Chapman-Kolmogorov equ.
For all t > 0, s > 0, i , j I
( ) ( ) ( ) ij ik kjk I
p t s p t p s
![Page 32: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/32.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
P. 32
CTMC (cont.)
p(t)=p(0)eQt
– Q is called the infinitesimal generator– Proof:
![Page 33: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/33.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
P. 33
Result 3
If a continuous M.C. with infinitesimal generator Q is irreducible and if the system of equations pQ = 0, and p1=1, has a strictly positive solution then p(i)= limt p(x(t)=i) for all iI, independently of the initial dist.
![Page 34: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/34.jpg)
Cheng-Fu Chou, CMLab, CSIE, NTUCheng-Fu Chou, CMLab, CSIE, NTU
Renewal Process
![Page 35: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/35.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
Renewal Process
A counting process {N(t), t 0} is a renewal process if for each n, Xn is the time between the (n-1)st and nth arrivals and {Xn, n 1} are independent with the same distribution F.The time of the nth arrival iswith S0 = 0.
Can writeand if m = E[Xn], n 1, then the strong law of large numbers says that
1, 1,
n
n iiS X n
max : nN t n S t
as 1nSP n
n
Note: m is now atime interval, not a rate;
1/ m will be calledthe rate of the r. p.
![Page 36: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/36.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
Fundamental Relationship
It follows that
where Fn(t) is the n-fold convolution of F with itself.
The mean value of N(t) is
Condition on the time of the first renewal to get the renewal equation:
nN t n S t
1 1
1
n n n n
P N t n P N t n P N t n
P S t P S t F t F t
1 1 1
n nn n n
m t E N t P N t n P S t F t
0
tm t F t m t x f x dx
![Page 37: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/37.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
Exercise 1
Is it true that:
?nN t n S t
?nN t n S t
?nN t n S t
![Page 38: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/38.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
Exercise 2
If the mean-value function of the renewal process {N(t), t 0} is given byThen what is P{N(5) = 0} ?
2, 0m t t t
![Page 39: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/39.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
Exercise 3
Consider a renewal process {N(t), t 0} having a gamma (r,l) interarrival distribution with density
(a) Show that(b) Show that
Hint: use the relationship between the gamma (r,l) distribution and the sum of r independent exponentials with rate l to define N(t) in terms of a Poisson process with rate l.
1
, 01 !
rxe xf x x
r
!
it
i nr
e tP N t n
i
, where is the largest integer
!
it
i r
e tim t x x
r i
![Page 40: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/40.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
Limit Theorems
With probability 1,
Elementary renewal theorem:
Central limit theorem: For large t, N(t) is approximately normally distributed with mean t/ m and variancewhere s2 is the variance of the time between arrivals;in particular,
1 as t
N t
t
1 as t
m t
t
2 3t
2
3
Var as t
N t
t
![Page 41: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/41.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
Exercise 4
A machine in use is replaced by a new machine either when it fails or when it reaches the age of T years. If the lifetimes of successive machines are independent with a common distribution F with density f, show that(a) the long-run rate at which machines are replaced is
(b) the long-run rate at which machines in use fail equals
Hint: condition on the lifetime of the first machine
1
01
Txf x T F T
01
T
F T
xf x T F T
![Page 42: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/42.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
Renewal Reward Processes
Suppose that each time a renewal occurs we receive a reward. Assume Rn is the reward earned at the nth renewal and {Rn, n 1} are independent and identically distributed (Rn may depend on Xn). The total reward up to time t is
If then
and
and E R E X
as t 1E RR t
Pt E X
as tE R t E R
t E X
1
N t
nnR t R
![Page 43: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/43.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
Age & Excess Life of a Renewal Process
The age at time t is A(t) = the amount of time elapsed since the last renewal. The excess life Y(t) is the time until the next renewal:
tA(t) Y(t)
SN(t)
N tA t t S
What is the average value of the age
0lim
s
t
A t dt
s
![Page 44: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/44.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
Average Age of a Renewal Process
Imagine we receive payment at a rate equal to the current age of the renewal process. Our total reward up to time s isand the average reward up to time s is
If X is the length of a renewal cycle, then the total reward during the cycle is
So, the average age is
0
sA t dt
0reward during a renewal cycle
length of a renewal cycle
sA t dt E
s E
2
0 2
X Xtdt
2
2
E X
E X
![Page 45: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/45.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
Average Excess or ResidualNow imagine we receive payment at a rate equal to the current excess of the renewal process. Our total reward up to time s is
and the average reward up to time s is
If X is the length of a renewal cycle, then the total reward during the cycle is
So, the average excess is (also)
0
sY t dt
0reward during a renewal cycle
length of a renewal cycle
sY t dt E
s E
2
0 2
X XX t dt
2
2
E X
E X
![Page 46: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/46.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
Inspection Paradox
Suppose that the distribution of the time between renewals, F, is unknown. One way to estimate it is to choose some sampling times t1, t2, etc., and for each ti, record the total amount of time between the renewals just before and just after ti. This scheme will overestimate the inter-renewal times – Why?For each sampling time, t, we will recordFind its distribution by conditioning on the time of the last renewal prior to time t
1 1N t N t N tX S S
![Page 47: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/47.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
Inspection Paradox (cont.)
1 1N t N t N tP X x E P X x S t s
t
SN(t)SN(t)+1
t-s0
1
1 1 1
1
1
If then 1
If then
11
1
N t N t
N t N t N t N t
N t
N t
s x P X x S t s
s x P X x S t s P X x X s
P X x F xF x
F sP X s
![Page 48: Cheng-Fu Chou, CMLab, CSIE, NTU Intro. to Stochastic Processes Cheng-Fu Chou](https://reader036.vdocument.in/reader036/viewer/2022081506/56649e705503460f94b6e2e7/html5/thumbnails/48.jpg)
Cheng-Fu Chou, CMLAB, CSIE, NTUCheng-Fu Chou, CMLAB, CSIE, NTU
Inspection Paradox (cont.)
t
SN(t)SN(t)+1
t-s0
For any s,so
1 1N t N tP X x S t s F x
1 1
1
N t N t N tP X x E P X x S t s
F x P X x
where X is an ordinary inter-renewal time.
Intuitively, by choosing “random” times, it is more likelywe will choose a time that falls in a long time interval.