syllaboozz

8/11/2019 SyLLABOOZZ

1/6


2/6

J.R. NORRIS:"Markov Chains". Cambridge University Press (1997).

B. OKSENDAL:"Stochastic Differential Equations". 3rdEdition, Springer-Verlag (1992).

R.B. ASH:"Basic Probability Theory". J. Wiley & Sons (1972).

A DETAILED SYLLABUS FOLLOWS

. Lecture #1: Tuesday, 2 September

Overview of the topics in the course: Simple Random Walk. Random Walk. Processes withIndependent Increments. MARKO propert!. Martin"ale propert!. #ROW$ian motion.

. Lecture #2: Thursday, 4 September

%lementar! approach to the simple random walk: &am'ler(s ruin pro'lem) a'sorptionpro'a'ilities) e*pected duration of "ames. $otions of Recurrence and +ransience.


,om'inatorial approach to the simple random walk. -istri'ution of the firstreturntime tothe ori"in.

Assignment # 1:Read R. Ashs Chapter on the Combinatorial Approach to the Simple Random

Walk. Problems 1 8, from Ash, pp. 190-191. Due on Thu. 18 September.


+he MARKO propert!. MARKO ,hains. Initial distri'utions) transition pro'a'ilities /for oneand several da!s0. $otions of communication) e1uivalence classes) irreduci'ilit!.


2irst hittin" times and their elementar! properties. $otion of stoppin" time. +he Stron"MARKO propert!.


%1uivalent formulations of the MARKO propert!. ,onditional Independence. Statement andproof of the Stron" MARKO propert!. Applications: "eneratin" functions and distri'utions forfirstpassa"etimes) and for firstreturntimes) 'ased on the Stron" MARKO propert!.

Assignment # 2:Not to be handed in.


3/6

1. Read sections 1.1 1.7 in Norris.

2. Norris, Problems (Pbs.) # 1.1.2 1.1.4, 1.2.1, 1.5.1, 1.5.3.

3. Lawler, Problems (Pbs.) # 1.2, 1.5(a,b), 2.4.


Recurrence and +ransience for MARKO ,hains: definitions and 'asic properties. Applicationto the simple random walk in two and three dimensions.


Invariant measures and invariant distri'utions for a MARKO ,hain. %*amples. -ou'l!stochastic matrices. Positive and nullrecurrence. %*istence of invariant distri'utions.%r"odic theorem for MARKO ,hains.

Assignment # 3:Lawler, Problems # 1.9 - 1.11, 1.13, 2.2 2.5.Due on Tue. 7 October.


Periodicit! for a MARKO ,hain. +he 3conver"ence to e1uili'rium3 theorem for irreduci'le)aperiodic chains) via the couplin" method. +imereversal) the detailed'alance e1uations.

. Lecture #10: Thursday, 2 October

-efinition and properties of conditional e*pectations. ,onditional e*pectation as leasts1uares predictor. -efinition and 'asic properties of Martin"ales. +he -OO# decomposition.

. Lecture #11: Tuesday, 7 October

+he optional samplin" theorem) application to simple random walk and to the "am'ler(s ruinpro'lem. Martin"ales associated with MARKO ,hains) harmonic functions4 connections withrecurrence and transience.

Assignment # 4:Lawler, Problems # 5.3 5.7, 5.9, 5.10.Due on Tue, 14 October.


#asic facts a'out martin"ale transforms. 5pcrossin"s ine1ualit!) the conver"ence theorem for

martin"ales.


Martin"ale transforms4 invariance of martin"ales under stoppin". Optional samplin" theorem:for 'ounded stoppin" times) for 'ounded martin"ales) other sufficient conditions. Applicationof optional samplin" to the simple random walk.


4/6

. Lecture #14: Thursday, 16 October.

-efinition and properties of the #ROW$ian motion. Martin"ale and MARKO properties.-istri'ution of the firsthittin"time to a certain level4 null recurrence. ,omputation of thepro'a'ilit! of 6erocrossin" durin" a "iven timeinterval.

. Lecture #15: Tuesday, 21 October.MID-TERM EXAMINATION.


&am'ler(s ruin pro'lem for #ROW$ian motion4 martin"ale methods. Stron" MARKO propert!.7uadratic variation of the #ROW$ian path4 variations of all other orders.

Assignment # 5:Read Chapter 8 in Lawler. Problems # 8.4 8.10.Not to be handed in.


Stochastic inte"ral for simple processes4 elementar! properties) connections with martin"aletransforms.


Stochastic inte"ral for "eneral adapted processes. Properties. I+O(s chan"eofvaria'le rule.%lementar! computations for stochastic inte"rals.

Assignment # 6:Due on Tue, 11 November.

1. Read Chapter 9 in Lawler, do Problems # 9.1, 9.2.

2. Oksendal, Problems # 3.2, 3.4, 3.5 (p.29); # 4.2, 4.3, 4.6, 4.7 (pp. 38-40); 5.1, 5.2, 5.10(I)(pp. 54-57).

. Lecture #19: Tuesday, 4 November. UNIVERSITY HOLIDAY

. Lecture #20: Thursday, 6 November

ariants of the I+O formula. Applications. Ar"ument for the proof. +he heat e1uation4fundamental solution) solution of a "eneral ,A5,89 pro'lem. Interpretation in terms of#ROW$ian motion.

. Lecture #21: Tuesday, 11 November

ariants of the heat e1uation /coolin") heattransfer0) interpretation of solutions in terms of#ROW$ian motion. 2%9$MA$KA, formula) the arcsine law. -IRI,8%+ pro'lem4 computationof the e*pected e*ittime for multidimensional #ROW$ian motion in a 'all.


5/6


-efinition and properties of the POISSO$ process. &enerali6ations: pure'irth process) 'irthanddeath process. %*amples: M/M/1) M/M/k) and M/M/infinity1ueues. %*ponentialdistri'ution of the interarrival times. Some 'asic properties of the e*ponential distri'ution.

Assignment # 7:Read Chapter 3 in Lawler. Problems 3.1 3.5, 3.7 3.12.Not to be handed in.


MARKO ,hains in continuoustime. Independence and e*ponential distri'ution of ;umptimes) transition mechanism. +ransition pro'a'ilities) ,hapmanKolmo"orov e1uations)invariant measures.


Optimal Stoppin" for discreteparameter Markov ,hains) and for #rownian motion /notes from-!nkin < 9ushkevich0.

Assignment # 8:Read Chapter 4 in Lawler. Problems 4.1, 4.2, 4.6, 5.14.Due Tue. 2 December.


-iscretetime Markov ,hain em'edded in a ,ontinuoustime Markov ,hain) discussion ofrecurrence and transience. #irthanddeath processes4 criteria for positive and for nullrecurrence. %*amples: invariant measures for the M/M/1) M/M/k) and M/M/infinity1ueues.

. Lecture #26: Thursday, 27 November. THANKSGIVING HOLIDAY

. Lecture #27: Tuesday, 2 December

Stochastic -ifferential %1uations) the 'asic e*istence and uni1ueness theorem of I+O. %*plicitsolutions for linear S-%(s. %*amples. $otion and computation of infinitesimal "enerator.

. Lecture #28: Thursday, 4 December

&IRSA$O(s +heorem: Motivation and proof.

. Lecture #29: Tuesday, 9 December&IRSA$O(s +heorem: application to #rownian motion with drift) computation of thedistri'ution of firstpassa"etimes.

. Lecture #30: Thursday, 11 December. FINAL EXAMINATION


6/6

syllaboozz

Documents