Math 564 (Stat 555), Applied Stochastic Processes, Fall 2021

Professor:Kay Kirkpatrick
Contact:kkirkpat(at) (please put "564" in the subject) or through Canvas
Websites: (limited) and Canvas (more complete).
Class meetings: Tuesdays and Thursdays at 11:00am-12:20pm on zoom, flipped-style, for discussion of questions and homework problems, for which we will use breakout rooms and Miro. Attendance at discussions is encouraged but not required and not graded, and there is a 10-15 minute break near the middle of each class meeting (additional breaks are allowed too). Lectures are pre-recorded, captioned, and available in advance. Lecture notes will be supplied throughout the semester. Attendance at classmates' presentations at the end of the semester may be required to some extent.
Student hours: Tuesdays and Thursdays at ___ on zoom, or by appointment. I would be happy to answer your questions anytime as long as I'm not otherwise engaged, and before and after class are also good times to catch me.
Textbook: The main text will be "Markov Chains" by J. R. Norris, available here and also here.
Grading: Homework: 40% of the course grade
Midterm: 30%, scheduled for October 21. Please let me know as soon as possible if you might need any accommodation.
Final Project: 30%, a paper or talk or webpage (your choice, though we'll have limited time for talks) on a topic of your choice related to the course. If you choose a webpage, it could have an interactive simulation. Our assigned final exam time is 8:00am-11:00am, Wednesday, Dec. 15, and we will use (perhaps a subset) of the final exam time for student talks.
Justice: I am committed to affirming the identities, realities and voices of all students, especially students from historically marginalized or under-represented backgrounds. I value the use of self-specified gender pronouns and respect for all persons. Please contact me to receive disability accommodations. You should also know that I'm a mandatory reporter.

Homework, due Fridays in Canvas by ____: You are encouraged to work together on the homework, but I ask that you write up your own solutions and turn them in separately.

Late homework will not be graded, so I will drop your two lowest homework scores.

HW #1 is due at the end of the first week of classes, your responses to these prompts:
Full name:
Name you prefer to be called:
Where you are from:
Year/stage of your education:
Other interests, e.g., academic subjects, hobbies:
Future plans or possible careers:
Your mathematical background, e.g., probability theory, measure theory, analysis:
Any comments, inquiries, or concerns?

HW #2 is due ____: available here.

HW #3 is due ____: available here. All items except the last should be done on paper and handed in the usual way, and the last item is to message me on Canvas about your final project, indicating whether you will pick a talk or a paper, which topics you are considering and why, and 2 or 3 references you will work from (books, articles, and technical websites are all fine). Talks will be 15 minutes each, subject to time constraints. Papers will be 2-10 pages, possibly more if you would like to include a complete chapter of your thesis. Another option is a webpage that is 2-5 screens long, with an interactive simulation that you program.

Here's an example of a project webpage:

Lecture videos

Here are the first week's pre-recorded lecture videos. Each week usually has three lecture videos, each of which is usually about 45 minutes long.
Application of Markov chain ideas to the double pendulum; sets, and sequences
Partitions and sigma algebras
Measures, probability spaces, independence of events, and random variables


This is a graduate course on applied stochastic processes, and the prerequisite is a probability course (such as Math 461), but measure theory (such as Math 540) is not a prereq. The goal of this course is a good understanding of the basic stochastic processes and their applications. This course is designed for graduate students who will use stochastic processes in their research but do not need or want to take the Math 561-562 sequence. Some introductory measure theory is provided at the beginning of Math 564, but less measure theory than is taught in Math 540 or needed for Math 561.

The materials covered in this course include the following: (1: a couple of weeks) background on probability, linear algebra, and set theory (2: about 5 weeks) discrete time Markov chains; (3: about 5 weeks) continuous time Markov chains; (4: a couple of weeks) discrete time martingales, and (5) stationary processes and applications to queuing theory and other fields (your projects contribute here). This course can be tailored to the interests of the audience.

Why might you want to study probability?

Two main reasons: uncertainty and complexity. Uncertainty is all around us and is usefully modeled as randomness: it appears in call centers, electronic circuits, quantum mechanics, medical treatment, epidemics, financial investments, insurance, games (both sports and gambling), online search engines, for starters. Probability is a good way of quantifying and discussing what we know about uncertain things, and then making decisions or estimating outcomes. Some things are too complex to be analyzed exactly (like weather, the brain, social science), and probability is a useful way of reducing the complexity and providing approximations. And the reason I study probability: statistical mechanics, which combines both the uncertainty of quantum mechanics, and the complexity of zillions of particles interacting.

Why work on your communication skills?

"It usually takes me more than three weeks to prepare a good impromptu speech." --Mark Twain

I think that success in your career (any career) depends in part on how well you communicate your ideas and persuade other people, so I am giving you a chance to learn and practice good writing or presenting skills. Some of the homework assignments will lead up to the final project, for which you will have a choice of topic (related to the course) and of communication format (paper or talk). The homework will be graded partly on clarity, brevity, and coherence. This is a great opportunity to improve your writing or presenting skills, in order to make your ideas more clear and persuasive--and to succeed.

"I am sorry I have had to write you such a long letter, but I did not have time to write you a short one." --Blaise Pascal

Emergency information link.

Some more resources for writing and speaking:

Halmos: How to Write Mathematics
Gopen and Swan: The Science of Scientific Writing
Williams: Style: The Basics of Clarity and Grace (book, any edition), Longman.

Gallo: Public-Speaking Lessons from TED Talks
Lerman: Math job talk advice
Steele: Speaking tips organized in categories that includes this great but little-known tip about graphs on slides

"How to give a good 20-minute math talk" by William Ross
"The Science of Scientific Writing" by Gopen and Swan

Picture of a shoe queue.