Solution of HW 3

Problem I.

pn=pn:p =/" '

Ii:"

;÷ .)f÷÷ IThus

① pin '= ( pi " "

Pii " Pii " ) / Io ) = 's Pin. "

② Pii '= I pin "

pin. " pin

-

"If ) = Epi" 't I Pii

"

From ①.

we have

③ Ain'

= zp.int"

Then,

we obtain from ①,

②.

③ that

pint ' '-

I

pin' I

pi" '

= O2

-

-4

The root of d'

-Id- ¥ to is

d. = IIE de ¥r

So we can represent pin'

as

pin'

= A . 1¥55 -1 B . ¥-1 "

for some A. B.

To determine A. B . we use pi

" =/ and Pi "= o

So :

At B = I

{ ¥5 .at#B=o } ⇒ A = Fi .B= FEE

Therefore :

pin ,= Eiffel'ttf In

.

Notice that ¥54 and ¥4so Espin '

cis.

⇒ state 2 is transient

Problem 2. I

-2I

•

cis Pi : 2

•

±3

class 11.2.33.

recurrent.

invariant distribution IT -- I -13

.

-13.

t )

Iii , Pz :

I . .

2 Class : { 1,2,

3 , 4 }'

Recurrent3€04Invariant distribution :

Solve i

TL = xp{ Eia..

= ,

⇒ it =L 's I I ⇒

I iii ) Ps :

I 2, classes : 923

.

31,33345}

I2

•

jTt

•

, 2 is transient

3 1,3 , 4,5 are recurrentez .

Invariant

distributions:5€

a = ( a. a , o, I - a

, I - a )I 4

for any oe as I.

Liv ) : 174 :

.

I 3Classes : I 1.2 }

,

133,

143.

15 }

2 2,2, 3 are recurrent

5 ¥.

3

4 and 5 are transient

4 Invariant distributions :

TL = ( Fa ,a ,

I - Fa , o,

o )

for any of a a- ¥

Problem 3 .

i ii , Starting from Xo =L .the sampleI

2path that gives -1=1 is

Iw : 2 → 2 → 3 → - . -

3

So : PC -1=1 I Xo = I )

=p ( X , -_ 2,

Xa =3 I Ko =/ )

=p 1×2=31*1--2,Xo =D . PIX , -21*0=1 )

=p 1×2=31×1--2 ) - P ( X ,= 21 Xo =L )

= 'Paz ' plz = I - I = I

iii ) Starting from Xo =L,

the sample paths that

have -1=3 are :

¥

Wz 2 I 2 2 3

so :

Was I 2 I 2 I II

2 I 3

PCT =3 / Ko = 2) = PCXFZ,

Xz=2,

X 3=2, Xy=3/Xo=2)

+ PIX , =L. Xz .

- 2.

*3=2 .Xx -31k¥)

+ PCXFZ ,Xel ,

X 3=2,

1/4--31×0=2)

= -31 - £ . I . I -1 I . I . I . I t I . -3'

. I - I7-

=GT

Ciii ) PC X 3=31×2 .

- 2 ) = 133 = ICiv ) Pl Xs -41×6=2 ) = pi ,

"= fPPL ,

= I

LV ) PC X -

it , =3 ) X. t-

- 2) = 2.

( By definition of T)

c Vi ) P ( X # 2=11 XT=2 )

=P ( X # 2=1 I Xt # =3 ) ( By definition of T )=

-

Ps ,= £

Reason is that T is not a stopping time , i.e.

.

the event IT -

- n } not only depends son Xi.

. ii. Xn,

but also depends on Knit