Math 564, Applied Stochastic Processes
Fall 2014

Course Information

Professor:Kay Kirkpatrick
Office:334 Illini Hall
Course site:
Lectures: MWF 10:00-10:50 in 243 Altgeld Hall
Office hours: Mondays and Wednesdays, 11:00-11:50, or by appointment. I would be happy to answer your questions in my office anytime as long as I'm not otherwise engaged, and before and after class are good times to catch me either in my office or in the classroom.
Textbook: The main text will be "Markov Chains" by J. R. Norris, available here and also here.
Grading policy: Homework: 40% of the course grade
Midterm: 30%, scheduled for Oct. 31
Final Presentation: 30%, a poster or talk on a topic of your choice related to the course. Our assigned final exam time is 8:00-11:00 AM, Friday, December 19.

Homework (due Fridays in class or to my mailbox in AH 250): You are encouraged to work together on the homework, but I ask that you write up your own solutions and turn them in separately.

Late homework will not be graded, so I will drop your lowest homework score.

HW #1 is due the second Friday in class: info sheet handed out on first day.

HW #2 is due Friday 9/12 in class: available here.

HW #3 is due Friday 9/19 in class: available here.

HW #4 is due Friday 9/26 by the end of class:
1. Read "How to give a good colloquium" by John McCarthy.
2. Attend a talk: e.g., the colloquium (see the bottom of this page) at 4pm on Thursday, Sept 25 in AH 245. Write a few sentences describing a) one slide or section of the talk that you thought worked well, and b) how you would improve another part. You should cite the McCarthy reading as relevant, and include photos or notes on the parts of the talk you're analyzing.
3. Indicate whether you'll pick a talk or poster for the final presentation (a probabilistic answer is fine) and which topics or fields you're considering.
This HW can be emailed to me, especially good for including photos.

HW #5 is due Friday 10/3 in class: available here.

HW #6 is due Friday 10/10 in class: available here. Notice that problems 7 and 8 have been moved to the next problem set, HW #8.

HW #7 is due Friday 10/17 by the end of class: Read "How to talk Mathematics" by Paul Halmos, which includes advice on short talks, and prepare a proposal for your final presentation by writing a paragraph for each of the following three prompts.
1. What is your topic and your main message? You should be able to say it in about 60 seconds, an "elevator speech."
2. Think about your audience: your classmates, not just me; people in STEM, not just in your field. Write a paragraph addressing some or all of the following questions: Why should they care? Why are you the right person to present this topic? What do you want them to do after your presentation? What do you want them to take away from your presentation? How can you make the benefits to your audience clear?
3. Specifics: What kinds of audiovisual aids will you be choosing to use? Chalkboard, powerpoint, beamer, poster? When will you take questions: during or after? Which two or three definitions or key ideas will you introduce to your audience? What is a good example (think n=2) that illustrates the main point of your presentation? Can you think of a story to tell that's related to your topic?

QUIZ Friday, Oct 24: similar to HWs that have been graded and returned before Oct 24. If you do well on the quiz, it will replace a low HW score; if not, I'll drop the quiz. Any additional feedback can be emailed to me. HW solutions for studying are available here, here, and here, with different numbering.

HW #8 is due Friday 10/24 in class: available here, updated. Note that the first two problems have solutions available at the quiz announcement, even before this HW is due.

NO HW due Friday 10/31, because that's the day of the midterm exam. If you have a medical or other serious reason that keeps you from being at the midterm, please let me know as soon as possible.

HW #9 is due Friday 11/7 in class: Slide revision.
1. Read this downloadable booklet on slide design for scientific talks and "Slides are not all evil," both by Jean-luc Doumont.
2. Attend a talk and ask the speaker for the slides or slide code. Or you can work from the slides for my talk on the mean-field Heisenberg model with beamer tex code available here.
3. Pick two slides, a) one that you think needs improvement, and b) one that you really like. Revise both slide according to the principles that you've learned, drawing or texing up your suggested changes. This may include finding or drawing a picture to illustrate the main point, or making the wording more efficient. Improve the better slide b) in at least two small ways.
The homework that you turn in should look like this example, with four slides: two originals and two improved versions (at least hand-drawn, not necessarily texed).

HW #10 is due Friday 11/14 in class: you will do only one or two of the problems from the set (chosen randomly or as assigned in class Monday), available here, and with your group, you will choose spokesperson(s) to present the solution to your classmates.

HW #11 is due by 11/21, part in class, and the remainder by 5pm: If you're doing a poster, read these links with poster advice and a sample poster. If you're doing a talk, read "How to give a good 20-minute math talk" by William Ross.
1) You will give your "elevator speech" to all your classmates Friday (or Wednesday if you're unable to do Friday--but you must let me know before class on Wednesday for this to happen). This will be your main message in 60 seconds, much like an abstract. You may get some questions, and you can also poll your audience to see how many people know a particular concept.
2) Hand in rough draft slides or poster, preferably by email. "Rough" means the whole presentation should be done at least in outline (but remember no outline slide, like Ross says), and at least four slides/sections should be in complete form.
3) Include your title and abstract (as text, not attachment) in the email with the rough draft.

HW #12 is due two or three days before your presentation (so that I can give you feedback in time): you will practice your presentation for a classmate outside of class before your presentation to the whole class. If you're doing a poster, walk your classmate through your poster in about 10-15 minutes.
1. Hand in your second draft slides or poster.
2. Hand in feedback about your classmate's practice presentation, and give a copy to your classmate. This should include a) writing the time in minutes and seconds at each new slide/section; b) two questions you asked; and c) your two suggestions for improving your classmate's presentation.
The two parts can be handed in separately as needed.

1. You can build flexibility into the timing of your talk with a slide or section near the end to skip if necessary. Don't plan to go backwards in your slides, however, or to pass quickly through many slides; instead repeat needed material or have a paper index of slide numbers/titles so that you can use the "go to" feature.
2. I recommend but do not require dressing up (e.g., business casual or business formal, according to your intended career): how you dress affects how your audience treats you.
3. If you want to use your computer, check that you have everything to connect it to the projector and test it beforehand. I recommend having electronic presentations ready in four forms: on your computer, on a flash drive, on your email or Dropbox, and on paper (just in case, so that you could turn it into a chalkboard talk in case of a tech crisis).

I'm scheduling the talks for four regular class meetings (Dec 3, 5, 8, and 10) and posters for the middle of the final exam time assigned to us, which is 8:00-11:00 AM, Friday, December 19. You will have exactly 15 minutes per talk with a couple minutes between for questions and setup.

Your grade for the final presentation will be based partly on good participation as an audience member. Good participation includes attention and questions: each of you should have about two questions for random classmates, so that each presenter is asked approximately two questions. "Approximately two" means a RV concentrated at two; one or three or four questions have positive probabilities that are smaller than the probablility of two questions; but P(#questions=0) = 0.


Wed, Dec 3:
Andrew: Voting Tree: An approximate solution to an impossible problem
Thinh: Distributed Consensus Over Networks: An Introduction
Vanessa: A Brief Random Walk in Biological Processes

Fri, Dec 5:
Mayukh: Monte Carlo Methods (an application to quantum mechanics)
Arash: All-to-All Communication in Random Regular Directed Graphs
Peter: Rumors in a Network: Who's the Source?

Mon, Dec 8:
Youngsoo: Review on Investment under uncertainty
Jonathan: Maximum Likelihood Parameter Estimation and the EM algorithm
Cesar: Distributed Learning in Networks

Wed, Dec 10:
Juho: Introduction to topic modeling
Runmin: Application of Markov Chain: Simple Branching Process
Zelong: Application of Markov Chain in Bond Credit Risk Modeling

Fri, Dec 19 at 9am in our regular classroom AH 243

Zhan: Optimal stock selling problem under a Markov chain model
Shirley: Exchange Rate: a Model of Stochastic Segmented Trends Graphs
Mike: Securitization and the Gaussian Copula Model
Jiayi: Stochastic Time Series Model in Insurance
Mitch: The effectiveness of Bonus-Malus Schemes in the Motor Insurance industry
Armin: Stochastic modeling of deteriorating infrastructure system
Yizhou: Connection between Dirichlet Process and Chinese Restaurant Process
Daisy: An Introduction to MCMC Using in Multiple Imputation for Missing Data
Agnes: Application of Stochastic Process in Financial Derivatives Pricing

Some more resources:

Public-Speaking Lessons from TED Talks
Math job talk advice by Eugene Lerman
Speaking tips organized in categories by William Steele that includes this great but little-known tip about graphs on slides


This is a graduate course on applied stochastic processes, and measure theory is not a prerequisite for this course. The goal of this course is a good understanding of the basic stochastic processes and their applications. This course is designed for those graduate students who are going to need to use stochastic processes in their research but do not have the measure-theoretic background to take the Math 561-562 sequence. The materials covered in this course include the following: (1) discrete time Markov chains; (2) continuous time Markov chains; (3) discrete time martingales, (4) stationary processes; (5) applications to queuing theory and other fields. The applications covered in this course can be tailored to the interests of the audience.

Why study probability?

Two main reasons: uncertainty and complexity. Uncertainty is all around us and is usefully modeled as randomness: it appears in call centers, electronic circuits, quantum mechanics, medical treatment, epidemics, financial investments, insurance, games (both sports and gambling), online search engines, for starters. Probability is a good way of quantifying and discussing what we know about uncertain things, and then making decisions or estimating outcomes. Some things are too complex to be analyzed exactly (like weather, the brain, social science), and probability is a useful way of reducing the complexity and providing approximations. And the reason I study probability: statistical mechanics, which combines both the uncertainty of quantum mechanics, and the complexity of zillions of particles interacting.

Why work on presenting?

"It usually takes me more than three weeks to prepare a good impromptu speech." --Mark Twain

I think that success in your career (any career) will depend on how well you communicate your ideas and persuade other people, so I am giving you a chance to learn and practice good presenting skills. Some of the homework assignments will lead up to the final project, for which you will have a choice of topic (related to the course) and of presentation format (poster or talk), each graded partly on clarity, concision, and coherence. This is a wonderful opportunity to improve your presenting skills, in order to make your ideas more clear and persuasive--and to succeed.

"Ask yourself: If I had only sixty seconds on the stage, what would I absolutely have to say to get my message across?" --Jeff Dewar

Emergency information link