Math 466+564/STAT 555: Applied Stochastic Processes (Fall 2022)
This is an advanced undergraduate+graduate course on applied stochastic processes, designed for those students who are going to need to use stochastic processes in their research but do not have the measure-theoretic background to take the Math 561-562 sequence. Measure theory is not a prerequisite for this course. However, a basic knowledge of probability theory (Math 461 or its equivalent) is expected, as well as some knowledge of linear algebra and analysis. The goal of this course is a good understanding of basic stochastic processes, in particular discrete-time and continuous-time Markov chains, and their applications. The materials covered in this course include the following:
- Fundamentals: background on probability, linear algebra, and set theory.
- Discrete-time Markov chains: classes, hitting times, absorption probabilities, recurrence and transience, invariant distribution, limiting distribution, reversibility, ergodic theorem, mixing times;
- Continuous-time Markov chains: same topics as above, holding times, explosion, forward/backward Kolmogorov equations;
- Related topics: Discrete-time martingales, potential theory, Brownian motion;
- Applications: Queueing theory, population biology, MCMC.
|Contact||Email with subject "Math 564:" (Use your official @illinois.edu address).|
|Class||TR 9:30am-10:50am in 147 Altgeld Hall.|
|Student Hrs||3-3:50pm Wednesdays + 11-11:50am Thursdays, or by appointment. I will be happy to answer your questions in my office anytime as long as I'm not otherwise engaged.|
|Textbook||1. Norris: Markov Chains, Cambridge University Press, 1998;|
2. Levin, Peres, and Wilmer: Markov Chains and Mixing Times, AMS, 2009;|
3. Grimmett and Stirzaker: Probability and Random Processes, 4th Ed., OUP, 2020.
|Prerequisite||Math 461 (Undergraduate Probability) and MATH 447/448 (Undergraduate Analysis). |
A basic knowledge of probability theory, linear algebra and analysis is expected. Measure theory is not a prerequisite for this course.
|Grading Policy|| Homework: 48% of the course grade. Homework problems will be assigned approximately every two weeks. I will post the assigned exercises on Canvas. You are encouraged to work together on the homework, but I ask that you write up your own solutions and turn them in separately. A few problems will be assigned; emphasis will be placed on clear, concise, and coherent writing. Late homework will not be graded and credited. The lowest score will be dropped. |
Midterm : 20% will depend on an in-class midterm exam on Oct 11, 2022.
Attendance : 5% of the course grade.
Final : 30% will depend on a take home final exam. Take home final exam will be assigned on Tuesday, December 6 (last day of instructions) and is due on Monday, December 12 by noon.
|1||T||Aug 23||Set theory and Measure Theory basics. PDF|
|R||Aug 25||Probability and Random variables. PDF|
|2||T||Aug 30||Homework 0||Expectation and Basics of linear algebra. PDF|
|R||Sep 01||Definition of Markov chain. PDF|
|3||T||Sep 06||Properties of Markov chains. PDF|
|R||Sep 08||Homework 1||Hitting time and stopping time. PDF|
|4||T||Sep 13||Strong Markov property. PDF|
|R||Sep 15||HW1 Solution||Class structure. PDF|
|5||T||Sep 20||Recurrence and Transience. PDF|
|R||Sep 22||Homework 2||Invariant distribitions. PDF|
|6||T||Sep 27||Positive recurrence and aperiodicity. PDF|
|R||Sep 29||HW2 Solution||Convergence to invariant distribution. PDF|
|7||T||Oct 04||Convergence for periodic MC. PDF|
|R||Oct 06||Homework 3||Time reversal and detailed balance.PDF|