### Syllabus

### Lecture Notes

- Chap. 1
- Chap. 2
- Chap. 3
- Chap. 4
### Homework Assignments and Solutions

- Homework 1 for Unders Homework 1 for Grads Due 02/06
- Soluton to Homework 1 for unders Solution to Homework 1 for Grads
- Homework 2 for Unders Homework 2 for Grads Due 02/20
- Solution to Homework 2 for unders Solution to Homework 2 for grads
- Homework 3 for Unders Homework 3 for Grads Due 02/27
- Solution to Homework 3 for Unders Solution to Homework 3 for grads
- Homework 4 for Unders Homework 4 for Grads Due 04/02
- Solution to Homework 4
- Homework 5
- Solution to Homework 5
- Homework 6
- Solution to Homework 6
### Solutions to Tests

- Solutions for Test 1 for Unders
- Solutions for Test 1 for Grads
- Test 2
### Class Diary

**Tuesday, 01/21: stochastic process: index set, state space, sample path, discrete-time process, continuous-time process, random walk, Markov chain chain, Poisson process, Brownian motion, geometric Brownian motion, conditioanl probability, conditional mass function****Read:**Chapter 1 and 2.1.1, 2.1.2.**Thursday, 01/23: conditional mass function, conditional density, conditional expectation, law of total expectation****Read:**2.1.1, 2.1.2, 2.1.3, 2.1.4, 2.1.5, 2.2.1**Tuesday, 01/28: law of total expectation, law of total variance, law of total probability, compound random variables, compounding distribution, expectation and variance of compound random variables****Read:**2.2.1, 2.2.2, 2.2.3, 2.3.1, 2.3.2**Thursday, 01/30: compound random variable identity, mass function for compound random variables, counting processes, Poisson processes****Read:**2.3.3, 2.3.4, 2.3.5, 3.1.1, 3.1.2, 3.1.3, 3.1.4**Tuesday, 02/04: Poisson processes, exponential ramdom variables, memeoryless property****Read:**3.1.5, 3.1.6, 3.1.7, 3.1.8, 3.1.9, 3.1.10, 3.1.11, 3.1.12, 3.1.13, 3.2.1, 3.2.2**Thursday, 02/06: failure/hazard rate function, gamma random variables, law of sum of independent exponential random variables with a common parameter, law of the minimum of independent exponential random variables, probability that minimum of independent exponential random variables is equal to a particular one****Read:**3.2.3, 3.2.4, 3.2.5, 3.2.6**Tuesday, 02/11: inter-arraival time, waiting time, characterization of homogeneous Poisson processes using inter-arraival times, probability that the nth event for a Poisson process N^1 to occur before the m-th event for an independent Poisson propcess N^2****Read:**3.3.1, 3.3.2, 3.3.3, 3.3.5, 3.3.6**Thursday, 02/13: probability that the nth event for a Poisson process N^1 to occur before the m-th event for an independent Poisson propcess N^2, conditional distribution of arrival times, distribution of inter-arrival times for non-homogeneous poisson processes****Read:**3.3.6, 3.3.7, 3.3.8**Tuesday, 02/18: Thinning of Poisson processes, sum of independent Poisson processes, compound Poisson processes****Read:**3.4.1, 3.4.2, 3.4.3, 3.4.4, 3.5.1, 3.5.2, 3.5.3, 3.5.4, 3.5.5, 3.5.6, 3.5.7**Thursday, 02/20:compound Poisson processes, expectation and varaiance of compound Poisson processes, sums of independent compound Poisson processes****Read:**3.5.5, 3.5.6, 3.5.7, 3.5.8**Tuesday, 02/25: Discrete-time Markov chain, Markov property, one-step transition probasbilities, k-step transition probabilities, Chapman-Kolmogorov equation, homogeneous Markov chain, distribution of states, initial distribution****Read:**4.1.1, 4.1.2, 4.1.3, 4.1.4, 4.1.5, 4.1.6, 4.1.7, 4.1.8, 4.1.9, 4.1.10, 4.1.11, 4.1.12, 4.1.13**Thursday, 02/27: Accessibility of one state from another, communication between states, communication class, ireeducibility, closed subset, absorbing state, period****Read:**4.2.1, 4.2.2, 4.2.3, 4.2.4, 4.2.5, 4.2.6, 4.2.7, 4.2.8, 4.2.9**Tuesday, 03/03:****Read:****Thursday, 03/05: Review. Test 1 will cover materails from Chapters 2 and 3.****Tuesday, 03/10: Test 1****Test 1 score distribution for unders :****Median:**90,**100:**9,**95--99:**6,**90--94:**18,**85--89:**8,**80--84:**4,**75--79:**3,**70--74:**3,**65--69:**2,**60--64:**3,**55--59:**1,**50--54:**1,**45--49:**1,**40--44:**1,**0--39:**1**Test 1 score distribution for grads :****100:**7,**90--94:**2**Thursday, 03/12: first passge time, first return probability, distribution and mass function of the number of visits to i starting from i, characterization of recurrence and transience using first return probability, characterization of recurrence and transience using k-step transition probabilities, recurrence and transience are class properties, mean time spent in a transient state j starting from a transient state i, first passge probability to a transient state j starting from a transient state i****Read:**4.3.1, 4.3.2, 4.3.3, 4.3.4, 4.3.5, 4.3.6, 4.3.7, 4.3.8,4.3.9, 4.3.10**Tuesday, 03/24:****Read:****Thursday, 03/26:****Read:****Tuesday, 03/31:****Read:****Thursday, 04/02:****Read:****Tuesday, 04/07:****Read:****Thursday, 04/09:****Read:****Tuesday, 04/14:****Read:****Thursday, 04/16:****Read:****Tuesday, 04/21:****Read:****Thursday, 04/23:****Read:****Tuesday, 04/28:****Read:****Thursday, 04/30:****Read:****Tuesday, 05/05:****Read:**