This is an advanced undergraduate+graduate course on applied stochastic processes, designed for those students who are going to need to use stochastic processes in their research but do not have the measure-theoretic background to take the Math 561-562 sequence. Measure theory is not a prerequisite for this course. However, a basic knowledge of probability theory (Math 461 or its equivalent) is expected, as well as some knowledge of linear algebra and analysis. The goal of this course is a good understanding of basic stochastic processes, in particular discrete-time and continuous-time Markov chains, and their applications. The materials covered in this course include the following:

• Fundamentals: background on probability, linear algebra, and set theory.
  
• Discrete-time Markov chains: classes, hitting times, absorption probabilities, recurrence and transience, invariant distribution, limiting distribution, reversibility, ergodic theorem, mixing times;
  
• Continuous-time Markov chains: same topics as above, holding times, explosion, forward/backward Kolmogorov equations;
  
• Related topics: Discrete-time martingales, potential theory, Brownian motion;
  
• Applications: Queueing theory, population biology, MCMC.
  

This course can be tailored to the interests of the audience.

Course	go.illinois.edu/math564
Grades	Canvas
Student hours	3-3:50pm Wednesdays+ 11-11:50am Thursdays.

Instructor	Partha Dey
Office	35 CAB
Contact	Email with subject "Math 564:" (Use your official @illinois.edu address).
Class	TR 9:30am-10:50am in 147 Altgeld Hall.
Student Hrs	3-3:50pm Wednesdays + 11-11:50am Thursdays, or by appointment. I will be happy to answer your questions in my office anytime as long as I'm not otherwise engaged.
Textbook	1. Norris: Markov Chains, Cambridge University Press, 1998;
Other Refs	2. Levin, Peres, and Wilmer: Markov Chains and Mixing Times, AMS, 2009; 3. Grimmett and Stirzaker: Probability and Random Processes, 4th Ed., OUP, 2020.
Prerequisite	Math 461 (Undergraduate Probability) and MATH 447/448 (Undergraduate Analysis). A basic knowledge of probability theory, linear algebra and analysis is expected. Measure theory is not a prerequisite for this course.
Grading Policy	Homework: 48% of the course grade. Homework problems will be assigned approximately every two weeks. I will post the assigned exercises on Canvas. You are encouraged to work together on the homework, but I ask that you write up your own solutions and turn them in separately. A few problems will be assigned; emphasis will be placed on clear, concise, and coherent writing. Late homework will not be graded and credited. The lowest score will be dropped. Midterm : 20% will depend on an in-class midterm exam on Oct 11, 2022. Attendance : 5% of the course grade. Final : 30% will depend on a take home final exam. Take home final exam will be assigned on Tuesday, December 6 (last day of instructions) and is due on Monday, December 12 by noon.

Week		Date	Due		Content
1	T	Aug 23			Set theory and Measure Theory basics.
	R	Aug 25			Probability and Random variables.
2	T	Aug 30	Homework 0		Expectation and Basics of linear algebra.
	R	Sep 01			Definition of Markov chain.
3	T	Sep 06			Properties of Markov chains.
	R	Sep 08	Homework 1		Hitting time and stopping time.
4	T	Sep 13			Strong Markov property.
	R	Sep 15	HW1 Solution		Class structure.
5	T	Sep 20			Recurrence and Transience.
	R	Sep 22	Homework 2		Invariant distribitions.
6	T	Sep 27			Positive recurrence and aperiodicity.
	R	Sep 29	HW2 Solution		Convergence to invariant distribution.
7	T	Oct 04			Convergence for periodic MC.
	R	Oct 06	Homework 3		Time reversal and detailed balance.
8	T	Oct 11	Info		Midterm.
	R	Oct 13	HW3 Solution		Ergodic theory and Metropolis-Hastings algorithm.
9	T	Oct 18			Mixing time.
	R	Oct 20	Homework 4		Coontinuous time Markov Chain.
10	T	Oct 25			Jump Chain and Holding Times.
	R	Oct 27	HW4 Solution		Poisson Process and Birth Processes.
11	T	Nov 01			Explosion time and Minimal Chain.
	R	Nov 03	Homework 5		Class Structure, Hitting Times, Recurrence and Transience.
12	T	Nov 08			No class.
	R	Nov 10	HW5 Solution		Invariant measure for CTMC.
13	T	Nov 15			Time reversal and convergence to equilibrium.
	R	Nov 17	Homework 6		Martingale characterization.
14	T	Nov 22			No classes. Thanksgiving break.
	R	Nov 24
15	T	Nov 29			Branching processes.
	R	Dec 01	HW6 Solution		Epidemics and queueing theory.
16	T	Dec 06	Homework 7		Brownian Motion.
	R	Dec 08	HW7 Solution		No Class.
17	M	Dec 12	Final Exam		Final exam due by noon.