Download - Markov - Purdue University
![Page 1: Markov - Purdue University](https://reader034.vdocument.in/reader034/viewer/2022051204/6277fb347a5d3a6dc11504ff/html5/thumbnails/1.jpg)
Markov Chain Specification
1 T o l z S is countable
2 Markov property3 P Xo R Ito r a C S
P Xt y I Xt a Ilse y
K y E S
ITC I should satisfy
1T x y e o D
IT K y I t ne S
YES
IT is a square matrix IT can be
yinfinite dimensional since Is I can be a
![Page 2: Markov - Purdue University](https://reader034.vdocument.in/reader034/viewer/2022051204/6277fb347a5d3a6dc11504ff/html5/thumbnails/2.jpg)
Example I Inventory system
demand periods u
mmmmI l l l l l
O
Stock replenishmentpointsn demand period numberXn inventory level at the begin of period n
G random discrete demand during period ns order levelu order upto level
1 Assume that G Gz one iidwith pm f f
2 Notice
XpEnn if s s xn e w
U Ent Xn E S
![Page 3: Markov - Purdue University](https://reader034.vdocument.in/reader034/viewer/2022051204/6277fb347a5d3a6dc11504ff/html5/thumbnails/3.jpg)
Example I Inventory system
demagd periods u
mmmmI l l l l l
O
Stock replenishmentpointss
Xn n e T is a Markov chain
with S u u t U z
T 0 1,2
and
Mocaif a u
0 OW
f x y se a e u
ITH'tfu y sees
![Page 4: Markov - Purdue University](https://reader034.vdocument.in/reader034/viewer/2022051204/6277fb347a5d3a6dc11504ff/html5/thumbnails/4.jpg)
Example III Epidemic Branching Chain
firmoffn generation number
1 v Xn number infectedIEEE IEEE in generation n
L v j
en reEEE Dv v
IEEE IEEE
S T O 1,2
Suppose each person in a generation independentlyinfects by others where pmf of G is f
![Page 5: Markov - Purdue University](https://reader034.vdocument.in/reader034/viewer/2022051204/6277fb347a5d3a6dc11504ff/html5/thumbnails/5.jpg)
Example III Epidemic Branching Chain
Md initial distribution of infected people1T 0,0 I IT 0 y O t y f O
IT r y PCG that then ywhere he Gz G are i i dwith pmf f K 31
Denote fu pmf of G test tha
O I 2
O I O O n n n
DI fill fill2 4 HDc
![Page 6: Markov - Purdue University](https://reader034.vdocument.in/reader034/viewer/2022051204/6277fb347a5d3a6dc11504ff/html5/thumbnails/6.jpg)
Example III Mutant Transmission
gene with d units med of whichare mutant
Bz Bz Bz Gen n
replicate3g BB Bae BB
I3h Bz Bz Bz Bz Gen ntl
Xn number of mutant units ingeneration n
S 0,1 d T 0 1,2
IT a y7 215 4 yeaead
0 otherwise
Ito n see o.is d given
![Page 7: Markov - Purdue University](https://reader034.vdocument.in/reader034/viewer/2022051204/6277fb347a5d3a6dc11504ff/html5/thumbnails/7.jpg)
Understand the Markov Property
Suppose Xt t e T is a
Markov chain Which of the
following are true statements
I P Xn me A X no X a Xrisen
I P X me A X an
II P Xn mE A XE Ao X EA XEAn.itn an
P Xn mEA Xn an
lI.P XnmEAXoEAo X EA glnEAn
P Xn mEA XnEAn
![Page 8: Markov - Purdue University](https://reader034.vdocument.in/reader034/viewer/2022051204/6277fb347a5d3a6dc11504ff/html5/thumbnails/8.jpg)
Why are Markov chains so useful
Let's calculate P X y IX a
IT a y P X y X a
IT ca y IT ca y
IT ca y P X y IX x X zz
P X zlXEP Xn y X z IT a z
z
IT cz.gg TCx.zZ
IT 2e Z 1T Cz yl
![Page 9: Markov - Purdue University](https://reader034.vdocument.in/reader034/viewer/2022051204/6277fb347a5d3a6dc11504ff/html5/thumbnails/9.jpg)
Why are Markov chains so useful
From 111 we see that
IT ca y The yIT Ca y IT x y
IT n'cry IT x y z
Since 2 holds fr any n y E S
we can write
IT'M IT 3
Don't confuse computational teeth
IT Cn y f CIT Ca yFH
![Page 10: Markov - Purdue University](https://reader034.vdocument.in/reader034/viewer/2022051204/6277fb347a5d3a6dc11504ff/html5/thumbnails/10.jpg)
So then
P X y
![Page 11: Markov - Purdue University](https://reader034.vdocument.in/reader034/viewer/2022051204/6277fb347a5d3a6dc11504ff/html5/thumbnails/11.jpg)
Hitting Time
Define
Tea min n o Xn EA
Ta is called the hitting time
of A Hitting time is first hittingkey Identity
time
P X y X a
nn m
P Try m 1T y yM 1
Why is this true
![Page 12: Markov - Purdue University](https://reader034.vdocument.in/reader034/viewer/2022051204/6277fb347a5d3a6dc11504ff/html5/thumbnails/12.jpg)
Hitting Time
P X y X a
n
P Try m Ty yM 1
Why is this true
![Page 13: Markov - Purdue University](https://reader034.vdocument.in/reader034/viewer/2022051204/6277fb347a5d3a6dc11504ff/html5/thumbnails/13.jpg)