Math 466+564/STAT 555: Applied Stochastic Processes (Fall 2022)

This is an advanced undergraduate+graduate course on applied stochastic processes, designed for those students who are going to need to use stochastic processes in their research but do not have the measure-theoretic background to take the Math 561-562 sequence. Measure theory is not a prerequisite for this course. However, a basic knowledge of probability theory (Math 461 or its equivalent) is expected, as well as some knowledge of linear algebra and analysis. The goal of this course is a good understanding of basic stochastic processes, in particular discrete-time and continuous-time Markov chains, and their applications. The materials covered in this course include the following:

  • Fundamentals: background on probability, linear algebra, and set theory.
  • Discrete-time Markov chains: classes, hitting times, absorption probabilities, recurrence and transience, invariant distribution, limiting distribution, reversibility, ergodic theorem, mixing times;
  • Continuous-time Markov chains: same topics as above, holding times, explosion, forward/backward Kolmogorov equations;
  • Related topics: Discrete-time martingales, potential theory, Brownian motion;
  • Applications: Queueing theory, population biology, MCMC.
This course can be tailored to the interests of the audience.
Student hours11-11:50am Thursdays + More TBA

Instructor Partha Dey
Office35 CAB
ContactEmail with subject "Math 564:" (Use your official address).
Class TR 9:30am-10:50am in 147 Altgeld Hall.
Student Hrs 3-3:50pm Wednesdays + 11-11:50am Thursdays, or by appointment. I will be happy to answer your questions in my office anytime as long as I'm not otherwise engaged.
Textbook 1. Norris: Markov Chains, Cambridge University Press, 1998;
Other Refs 2. Levin, Peres, and Wilmer: Markov Chains and Mixing Times, AMS, 2009;
3. Grimmett and Stirzaker: Probability and Random Processes, 4th Ed., OUP, 2020.
Prerequisite Math 461 (Undergraduate Probability) and MATH 447/448 (Undergraduate Analysis).
A basic knowledge of probability theory, linear algebra and analysis is expected. Measure theory is not a prerequisite for this course.
Grading Policy Homework: 48% of the course grade. Homework problems will be assigned approximately every two weeks. I will post the assigned exercises on Canvas. You are encouraged to work together on the homework, but I ask that you write up your own solutions and turn them in separately. A few problems will be assigned; emphasis will be placed on clear, concise, and coherent writing. Late homework will not be graded and credited. The lowest score will be dropped.

Midterm : 20% will depend on an in-class midterm exam on Oct 11, 2022.

Attendance : 5% of the course grade.

Final : 30% will depend on a take home final exam. Take home final exam will be assigned on Tuesday, December 6 (last day of instructions) and is due on Monday, December 12 by noon.

Week Date Due Content

1 T Aug 23 Set theory and Measure Theory basics. PDF
R Aug 25 Probability and Random variables. PDF

2 T Aug 30 Homework 0 Expectation and Basics of linear algebra. PDF
R Sep 01 Definition of Markov chain. PDF

3 T Sep 06 Properties of Markov chains. PDF
R Sep 08 Homework 1 Hitting time and stopping time. PDF

4 T Sep 13 Strong Markov property. PDF
R Sep 15 HW1 Solution Class structure. PDF

5 T Sep 20 Recurrence and Transience. PDF
R Sep 22 Homework 2 Invariant distribitions. PDF

6 T Sep 27 Positive recurrence and aperiodicity. PDF
R Sep 29 HW2 Solution Convergence to invariant distribution. PDF

7 T Oct 04 Convergence for periodic MC. PDF
R Oct 06 Homework 3 Time reversal and detailed balance.PDF