Math 466+564/STAT 555: Applied Stochastic Processes (Fall 2022)
This is an advanced undergraduate+graduate course on applied stochastic processes, designed for those students who are going to need to use stochastic processes in their research but do not have the measure-theoretic background to take the Math 561-562 sequence. Measure theory is not a prerequisite for this course. However, a basic knowledge of probability theory (Math 461 or its equivalent) is expected, as well as some knowledge of linear algebra and analysis. The goal of this course is a good understanding of basic stochastic processes, in particular discrete-time and continuous-time Markov chains, and their applications. The materials covered in this course include the following:
- Fundamentals: background on probability, linear algebra, and set theory.
- Discrete-time Markov chains: classes, hitting times, absorption probabilities, recurrence and transience, invariant distribution, limiting distribution, reversibility, ergodic theorem, mixing times;
- Continuous-time Markov chains: same topics as above, holding times, explosion, forward/backward Kolmogorov equations;
- Related topics: Discrete-time martingales, potential theory, Brownian motion;
- Applications: Queueing theory, population biology, MCMC.
| Course | go.illinois.edu/math564 |
|---|---|
| Grades | Canvas |
| Student hours | 11-11:50am Thursdays + More TBA |
| Instructor | Partha Dey |
|---|---|
| Office | 35 CAB |
| Contact | Email with subject "Math 564:" (Use your official @illinois.edu address). |
| Class | TR 9:30am-10:50am in 147 Altgeld Hall. |
| Student Hrs | 3-3:50pm Wednesdays + 11-11:50am Thursdays, or by appointment. I will be happy to answer your questions in my office anytime as long as I'm not otherwise engaged. |
| Textbook | 1. Norris: Markov Chains, Cambridge University Press, 1998; |
| Other Refs |
2. Levin, Peres, and Wilmer: Markov Chains and Mixing Times, AMS, 2009; 3. Grimmett and Stirzaker: Probability and Random Processes, 4th Ed., OUP, 2020. |
| Prerequisite | Math 461 (Undergraduate Probability) and MATH 447/448 (Undergraduate Analysis). A basic knowledge of probability theory, linear algebra and analysis is expected. Measure theory is not a prerequisite for this course. |
| Grading Policy | Homework: 48% of the course grade. Homework problems will be assigned approximately every two weeks. I will post the assigned exercises on Canvas. You are encouraged to work together on the homework, but I ask that you write up your own solutions and turn them in separately. A few problems will be assigned; emphasis will be placed on clear, concise, and coherent writing. Late homework will not be graded and credited. The lowest score will be dropped. Midterm : 20% will depend on an in-class midterm exam on Oct 11, 2022. Attendance : 5% of the course grade. Final : 30% will depend on a take home final exam. Take home final exam will be assigned on Tuesday, December 6 (last day of instructions) and is due on Monday, December 12 by noon. |
| Week | Date | Due | Content | ||
| 1 | T | Aug 23 | Set theory and Measure Theory basics. PDF | ||
| R | Aug 25 | Probability and Random variables. PDF | |||
| 2 | T | Aug 30 | Homework 0 | Expectation and Basics of linear algebra. PDF | |
| R | Sep 01 | Definition of Markov chain. PDF | |||
| 3 | T | Sep 06 | Properties of Markov chains. PDF | ||
| R | Sep 08 | Homework 1 | Hitting time and stopping time. PDF | ||
| 4 | T | Sep 13 | Strong Markov property. PDF | ||
| R | Sep 15 | HW1 Solution | Class structure. PDF | ||
| 5 | T | Sep 20 | Recurrence and Transience. PDF | ||
| R | Sep 22 | Homework 2 | Invariant distribitions. PDF | ||
| 6 | T | Sep 27 | Positive recurrence and aperiodicity. PDF | ||
| R | Sep 29 | HW2 Solution | Convergence to invariant distribution. PDF | ||
| 7 | T | Oct 04 | Convergence for periodic MC. PDF | ||
| R | Oct 06 | Homework 3 | Time reversal and detailed balance.PDF | ||