IT博客汇 | [课程]Mathematics of Machine Learning

[课程]Mathematics of Machine Learning

小编发表于 2016-09-10 15:16:10

课程链接：Mathematics of Machine Learning

Course Features

Prerequisites

18.100C Real Analysis

18.06SC Linear Algebra

18.05 Introduction to Probability and Statistics

Course Description

Broadly speaking, Machine Learning refers to the automated identification of patterns in data. As such it has been a fertile ground for new statistical and algorithmic developments. The purpose of this course is to provide a mathematically rigorous introduction to these developments with emphasis on methods and their analysis.

You can read more about Prof. Rigollet’s work and courses on his website.

The Topics Covered

The class will be split in three main parts:

The Statistical Theory of Machine Learning.
1. Classification, Regression, Aggregation
2. Empirical Risk Minimization, Regularization
3. Suprema of Empirical Processes
Algorithms and Convexity.
1. Boosting
2. Kernel Methods
3. Convex Optimization
Online Learning.
1. Online Convex Optimization
2. Partial Information: Bandit Problems
3. Blackwell’s Approachability

Suggested Readings

There is no required reading. The curious student is invited to read the following related material.

Books
- Bubeck, Sebastien, and Nicolo Cesa-Bianchi. Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems. Now Publishers Incorporate, 2012. ISBN: 9781601986269.
- Cesa-Bianchi, Nicolo, and Gabor Lugosi. Prediction, Learning, and Games. Cambridge University Press, 2006. ISBN: 9780521841085. [Preview with Google Books]
- Giraud, Christophe. Introduction to High-Dimensional Statistics. Chapman and Hall / CRC, 2014. ISBN: 9781482237948.
- Koltchinskii, Vladimir. Oracle Inequalities in Empirical Risk Minimization and Sparse Recovery Problems: École d’Été de Probabilités de Saint-Flour XXXVIII–2008. Springer, 2011. ISBN: 9783642221460. [Preview with Google Books]
- Shalev-Shwartz, Shai, and Shai Ben-David. Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press, 2014. ISBN: 9781107057135. [Preview with Google Books]
Courses and Lecture Notes
- Peter Bartlett at UC Berkeley.
- Sebastien Bubeck (PDF) at Princeton.
- Sebastien Bubec (PDF) (again) at Princeton.
- Sham Kakade and Ambuj Tewari at TTI, UChicago.
- Elad Hazan at Princeton.
- Gabor Lugosi at Pompeu-Babra.
- Maxim Raginsky at UIUC.
- Alexander Rakhlin at Penn.
- Shai Shalev-Shwartz (PDF) at Hebrew U.
- Dmitry Panchenko at MIT.
Other Readings
- Three Proofs of the Sauer-Shelah Lemma (PDF) by Hung Q. Ngo.
- History of the Sauer-Shelah Lemma (PDF) by Leon Bottou.

Full lecture notes are available (PDF – 2.7MB).

SES #	TOPICS
1	Introduction (PDF)
2	Binary Classification (PDF) (This lecture notes is scribed by Jonathan Weed. Used with permission.)
3	Concentration Inequalities (PDF) (This lecture notes is scribed by James Hirst. Used with permission.)
4	Fast Rates and VC Theory (PDF) (This lecture notes is scribed by Cheng Mao. Used with permission.)
5	The VC Inequality (PDF) (This lecture notes is scribed by Vira Semenova and Philippe Rigollet. Used with permission.)
6	Covering Numbers (PDF) (This lecture notes is scribed by Ali Makhdoumi. Used with permission.)
7	Chaining (PDF) (This lecture notes is scribed by Zach Izzo. Used with permission.)
8	Convexification (PDF) (This lecture notes is scribed by Quan Li. Used with permission.)
9	Boosting (PDF) (This lecture notes is scribed by Xuhong Zhang. Used with permission.)
10	Support Vector Machines (PDF) (This lecture notes is scribed by Aden Forrow. Used with permission.)
11	Gradient Descent (PDF) (This lecture notes is scribed by Kevin Li. Used with permission.)
12	Projected Gradient Descent (PDF) (This lecture notes is scribed by Michael Traub. Used with permission.)
13	Mirror Descent (PDF) (This lecture notes is scribed by Mina Karzand. Used with permission.)
14	Stochastic Gradient Descent (PDF) (This lecture notes is scribed by Sylvain Carpentier. Used with permission.)
15	Prediction with Expert Advice (PDF) (This lecture notes is scribed by Zach Izzo. Used with permission.)
16	Follow the Perturbed Leader (PDF) (This lecture notes is scribed by Haihao Lu. Used with permission.)
17	Online Learning with Structured Experts (PDF) (Courtesy of Gábor Lugosi. Used with permission.)
18	Stochastic Bandits (PDF) (This lecture notes is scribed by Haihao Lu. Used with permission.)
19	Prediction of Individual Sequences (PDF) (This lecture notes is scribed by Kevin Li. Used with permission.)
20	Adversarial Bandits (PDF) (This lecture notes is scribed by Vira Semenova. Used with permission.)
21	Linear Bandits (PDF) (This lecture notes is scribed by Ali Makhdoumi. Used with permission.)
22	Blackwell’s Approachability (PDF) (This lecture notes is scribed by Aden Forrow. Used with permission.)
23	Potential Based Approachability (PDF) (This lecture notes is scribed by Jonathan Weed. Used with permission.)

~~课件镜像，以防原网站失效用的：mit18_657f15_lecnote~~

ASSIGNMENTS SOLUTIONS

Assignment 1 (PDF) Assignment 1 Solution (PDF) (Courtesy of William Perry. Used with permission.)

Assignment 2 (PDF) Assignment 2 Solution (PDF) (Courtesy of William Perry. Used with permission.)

Assignment 3 (PDF) Assignment 3 Solution (PDF) (Courtesy of William Perry. Used with permission.)