CS760, Spring 2025
Department of Computer Sciences
University of Wisconsin–Madison
This schedule is tentative and subject to change. Please check back often. In particular, the deadlines for the homework sets please see Canvas.
The reading is not required but strongly recommended for all students. Those explicitly noted as optional are for students interested in that specific topic. "A, B; C; D" means to read (A OR B) AND C AND D. Text in red means a link to the reading material. Abbreviations for textbooks:
| Date | Lecture | Readings | ||
|---|---|---|---|---|
| Wednesday Jan. 22 | Course Overview : [Slides] | Jordan and Mitchell, Science, 2015 | ||
| ML Overview: Supervised/Unsupervised/RL, Classification/Regression, General Approach | ||||
| Supervised Learning I: Setup, Examples. Instance-Based Learning, Decision Trees | ||||
| Supervised Learning II: Setup + Examples. Decision Trees | Murphy Chapter 16.2 / Shalev-Shwartz and Ben-David Chapter 18 | |||
| Evaluation: Bias, Cross-Validation, Precision/Recall, ROC Curves | ||||
| Regression I: Linear Regression, Logistic Regression, Normal equations, GD | Murphy 8.1-3 and 8.6; Maximum Likelihood, Logistic Regression, and Stochastic Gradient Training | |||
| Regression II: Logistic Regression, Gradient Descent Analysis, SGD | Convergence Theorems for Gradient Descent | |||
| Naive Bayes: Generative vs Discriminative Models, ML vs MAP | Mitchell 6.1-6.10, Murphy 3 | |||
| Neural Networks I: Perceptron, Training, MLPs | Mitchell chapter 4, Murphy 16.5 and 28, Bishop 5.1-5.3 LeCun et al., Nature, 2015 | |||
| Neural Networks II: Training, Optimization, SGD, Backpropagation | ||||
| Neural Networks III: Regularization, Data Augmentation | ||||
| Neural Networks IV: CNNs | Goodfellow-Bengio-Courville chapter 9; CNN papers 1. LeNet 2. AlexNet 3. ResNet | |||
| Neural Networks IV: RNNs | Goodfellow-Bengio-Courville chapter 10; Optional papers to read for part 4: 1. LSTM 2. GRU | |||
| Practical Aspects of Training + Review | ||||
| Generative Models: Autoregressive, Flows, GANs | Optional tutorial: Goodfellow's GAN tutorial; Optional papers: 1. Normalizing Flows for Probabilistic Modeling and Inference 2. Generative Adversarial Networks (GANs) | |||
| Kernels + SVMs:Margins, Support Vectors, Kernels | Andrew Ng's note on SVM, Ben-Hur and Weston's note; Mohri-Rostamizadeh-Talwalkar Appendix B (Convex Optimization), Bishop Appendix E (Lagrange Multipliers) | |||
| Graphical Models I: Bayesian Networks, Training, Structure Learning | ||||
| Graphical Models II: Undirected Models, Markov Random Fields | Mitchell chapter 6, Bishop chapter 8.1, Shalev-Shwartz and Ben-David chapter 24; Heckerman Tutorial, Wainwright and Jordan Chapters 2, 3 | |||
| Unsupervised Learning I: Clustering, GMM models, EM | ||||
| Unsupervised Learning II: Dimensionality Reduction, PCA | ||||
| Learning Theory: Generalization, PAC | Mohri-Rostamizadeh-Talwalkar Chapter 2, Mitchell Chapter 7 | |||
| Reinforcement Learning I: MDPs, Value Iteration, Policy Iteration | Mitchell Chapter 13 | |||
| Reinforcement Learning II: Q-learning, Approximation | ||||
| Reinforcement Learning III: Function Approximation, Policy Search, Reinforce |