DCP1206 Fall 2019 - Probability (機率)

Instructor: Ping-Chun Hsieh
Email: pinghsieh [AT] nctu [DOT] edu [DOT] tw
Piazza: TBD
Lectures:
- Wednesdays 10:10am-12pm @ EC122
- Fridays 3:30pm-4:20pm @ EC122
Office Hours:
- Wednesdays 1pm-2pm @ EC713
- Fridays 4:30pm-5:30pm @ EC713
Textbook:
- [Gha] Saeed Ghahramani, Fundamentals of Probability with Stochastic Processes, 4th edition, CRC Press, 2018.
References:
- [Ber-Tsi] Dimitri P. Bertsekas and John N. Tsitsiklis, Introduction to Probability, 2nd edition, Athena Scientic, 2002.
- [Res] Sidney I. Resnick, A Probability Path, Springer Science & Business Media, 2013.
- [Bis] Christopher M. Bishop, Pattern Recognition and Machine Learning, Springer, 2006.
- [Ras-Wil] Carl Edward Rasmussen and Christopher K. I. Williams, Gaussian Processes in Machine Learning, MIT Press, 2006.

Grading
- Homework: 40%
  - 7 written assignments: 28%
  - Programming assignment: 12%
- Midterm: 30%
- Final exam: 30%

Tentative schedule:

Week	Lecture	Date	Topics	References
1	1	9/11	Course introduction, probability model, probability axioms, and set operations	[Gha Ch.1.1-1.4] [Ber-Tsi, Ch.1.2]
1		9/13	Mid-Autumn Festival (No class)
2	2	9/18	Continuity of probability functions, combinatorial methods, and conditional probability	[Gha Ch.1.5,2,3.1-3.2] [Ber-Tsi, Ch.1.3]
2	3	9/20	Law of Total Probability and Bayes's Rule	[Gha Ch.3.3-3.4] [Ber-Tsi, Ch.1.4]
3	4	9/25	Independence and random variables	[Gha Ch.3.5,4.1,4.3] [Ber-Tsi, Ch.2.1-2.2]
3	5	9/27	Discrete random variables and probability mass functions	[Gha Ch.4.2-4.3] [Ber-Tsi, Ch.2.2-2.3]
4	6	10/2	Distribution functions and expectations	[Gha Ch.4.2,4.4] [Ber-Tsi, Ch.2.4]
4	7	10/4	Variance and higher moments	[Gha Ch.4.5-4.6] [Ber-Tsi, Ch.2.4]
5	8	10/9	Special discrete distributions: Bernoulli, Binomial, Poisson, and Geometric	[Gha Ch.5]
5	9	10/11	Continuous random variables and probability density functions	[Gha Ch.6.1] [Ber-Tsi, Ch.3.1]
6	10	10/16	Cumulative distribution functions, expectations, and variance	[Gha Ch.6.2-6.3] [Ber-Tsi, Ch.3.2]
6	11	10/18	Uniform and normal random variables	[Gha Ch.7.1-7.2] [Ber-Tsi, Ch.3.3]
7	12	10/23	Conditioning on an event, exponential random variables, Gamma distributions	[Gha Ch.7.3-7.4] [Ber-Tsi, Ch.3.4]
7	13	10/25	Beta distributions and derived distributions	[Gha Ch.7.5] [Ber-Tsi, Ch.3.6]
8	14	10/30	Bivariate distributions	[Gha Ch.8.1-8.2] [Ber-Tsi, Ch.3.5]
8	15	11/1	Conditional distributions: discrete and continuous cases	[Gha Ch.8.3]
9	16	11/6	Midterm
9	17	11/8	Multivariate distributions and multinomial distributions	[Gha Ch.9.1,9.3]
10	18	11/13	Multivariate normal distributions, covariance, and correlation	[Gha Ch.10.2-10.5] [Ber-Tsi, Ch.4.5,4.7]
10	19	11/15	Sums of independent random variables	[Gha Ch.11.2] [Ber-Tsi, Ch.4.2]
11	20	11/20	Markov and Chebyshev Inequalities, convergence in probability, and Weak Law of Large Numbers	[Gha Ch.11.3-11.4] [Ber-Tsi, Ch.7.1-7.2]
11	21	11/22	Almost-sure convergence and Strong Law of Large Numbers	[Gha Ch.11.4] [Ber-Tsi, Ch.7.5]
12	22	11/27	Convergence in distribution and the Central Limit Theorem	[Gha Ch.11.5] [Ber-Tsi, Ch.7.4]
12	23	11/29	Moment generating functions	[Gha Ch.11.1]
13	24	12/4	Discrete-Time Markov Chains and Classification of States	[Gha Ch.12.3] [Ber-Tsi, Ch.6.1-6.2]
13	25	12/6	Steady-state behavior of a Markov Chain	[Gha Ch.12.3] [Ber-Tsi, Ch.6.3]
14	26	12/11	Continuous-time Markov Chains and Poisson process	[Gha Ch.5.2,12.4,13.2] [Ber-Tsi, Ch.5,6.5]
14	27	12/13	Basic sampling methods: inverse-transform sampling, rejection sampling	[Bis Ch.11.1]
15	28	12/18	Importance sampling, Metropolis-Hastings algorithm	[Bis Ch.11.1-11.2]
15	29	12/20	Gibbs sampling	[Bis Ch.11.3]
16	30	12/25	Statistical inference: Maximum Likelihood estimation and MAP	[Bis Ch.1.2,2.3,3.3]
16	31	12/27	Bayesian linear regression	[Ras-Wil Ch.2]
17	32	1/1	New Year's Day (No class)
17	33	1/3	Emerging applications	Lecture Notes
18		1/8	Final exam