Math 466+564/STAT 555: Applied Stochastic Processes (Fall 2022)

This is an advanced undergraduate+graduate course on applied stochastic processes, designed for those students who are going to need to use stochastic processes in their research but do not have the measure-theoretic background to take the Math 561-562 sequence. Measure theory is not a prerequisite for this course. However, a basic knowledge of probability theory (Math 461 or its equivalent) is expected, as well as some knowledge of linear algebra and analysis. The goal of this course is a good understanding of basic stochastic processes, in particular discrete-time and continuous-time Markov chains, and their applications. The materials covered in this course include the following:

  • Fundamentals: background on probability, linear algebra, and set theory.
  • Discrete-time Markov chains: classes, hitting times, absorption probabilities, recurrence and transience, invariant distribution, limiting distribution, reversibility, ergodic theorem, mixing times;
  • Continuous-time Markov chains: same topics as above, holding times, explosion, forward/backward Kolmogorov equations;
  • Related topics: Discrete-time martingales, potential theory, Brownian motion;
  • Applications: Queueing theory, population biology, MCMC.
This course can be tailored to the interests of the audience.
Student hours3-3:50pm Wednesdays+ 11-11:50am Thursdays.

Instructor Partha Dey
Office35 CAB
ContactEmail with subject "Math 564:" (Use your official address).
Class TR 9:30am-10:50am in 147 Altgeld Hall.
Student Hrs 3-3:50pm Wednesdays + 11-11:50am Thursdays, or by appointment. I will be happy to answer your questions in my office anytime as long as I'm not otherwise engaged.
Textbook 1. Norris: Markov Chains, Cambridge University Press, 1998;
Other Refs 2. Levin, Peres, and Wilmer: Markov Chains and Mixing Times, AMS, 2009;
3. Grimmett and Stirzaker: Probability and Random Processes, 4th Ed., OUP, 2020.
Prerequisite Math 461 (Undergraduate Probability) and MATH 447/448 (Undergraduate Analysis).
A basic knowledge of probability theory, linear algebra and analysis is expected. Measure theory is not a prerequisite for this course.
Grading Policy Homework: 48% of the course grade. Homework problems will be assigned approximately every two weeks. I will post the assigned exercises on Canvas. You are encouraged to work together on the homework, but I ask that you write up your own solutions and turn them in separately. A few problems will be assigned; emphasis will be placed on clear, concise, and coherent writing. Late homework will not be graded and credited. The lowest score will be dropped.

Midterm : 20% will depend on an in-class midterm exam on Oct 11, 2022.

Attendance : 5% of the course grade.

Final : 30% will depend on a take home final exam. Take home final exam will be assigned on Tuesday, December 6 (last day of instructions) and is due on Monday, December 12 by noon.

Week Date Due Content

1 T Aug 23 Set theory and Measure Theory basics.
R Aug 25 Probability and Random variables.

2 T Aug 30 Homework 0 Expectation and Basics of linear algebra.
R Sep 01 Definition of Markov chain.

3 T Sep 06 Properties of Markov chains.
R Sep 08 Homework 1 Hitting time and stopping time.

4 T Sep 13 Strong Markov property.
R Sep 15 HW1 Solution Class structure.

5 T Sep 20 Recurrence and Transience.
R Sep 22 Homework 2 Invariant distribitions.

6 T Sep 27 Positive recurrence and aperiodicity.
R Sep 29 HW2 Solution Convergence to invariant distribution.

7 T Oct 04 Convergence for periodic MC.
R Oct 06 Homework 3 Time reversal and detailed balance.

8 T Oct 11 Info Midterm.
R Oct 13 HW3 Solution Ergodic theory and Metropolis-Hastings algorithm.

9 T Oct 18 Mixing time.
R Oct 20 Homework 4 Coontinuous time Markov Chain.

10 T Oct 25 Jump Chain and Holding Times.
R Oct 27 HW4 Solution Poisson Process and Birth Processes.

11 T Nov 01 Explosion time and Minimal Chain.
R Nov 03 Homework 5 Class Structure, Hitting Times, Recurrence and Transience.

12 T Nov 08 No class.
R Nov 10 HW5 Solution Invariant measure for CTMC.

13 T Nov 15 Time reversal and convergence to equilibrium.
R Nov 17 Homework 6 Martingale characterization.

14 T Nov 22 No classes. Thanksgiving break.
R Nov 24

15 T Nov 29 Branching processes.
R Dec 01 HW6 Solution Epidemics and queueing theory.

16 T Dec 06 Homework 7 Brownian Motion.
R Dec 08 HW7 Solution No Class.

17 M Dec 12 Final Exam Final exam due by noon.