Machine Learning - WMMFMA28

Goal(s)

Machine Learning (Statistical Learning) is a mathematical branch at the heart of Artificial Intelligence (AI).

It is a recent discipline, whose founding concepts date back to the 1970s.

This course deals only with supervised learning. It is divided into two parts :

• Part 1 - elementary theoretical notions of statistical learning, creating a link with notions known elsewhere in other disciplines (statistics, optimisation)

• Part 2 - theoretical and practical tools for solving practical learning problems.

Content(s)

*PRINCIPLES

• 1 Overview

What is supervised statistical learning?
A few examples.
Categories of algorithms.
Classic pitfalls. Bad data. The wrong algorithm.
Fundamental importance of a priori.
Why do we need ML algorithms?
Links with other disciplines.

• 2 Principles

Supervised model.
Ingredients. Features space. Labels. Learning set. Predictor.
Data and label generation.
Performance measurement: risk.

• 3 Basic PAC model

Risk. Empirical Risk Minimisation (ERM) learning.
Overlearning. Regularisation.
Learning Probably Approximately Correct (PAC)
PAC learning of a finite class.

• 4 PAC agnostic supervised model

Optimal Bayes predictor.
Agnostic PAC learning.

• 5 Vapnik-Chervonenkis dimension

PAC learning of an infinite class.
Vapnik-Chervonenkis dimension.

• 6 Learning and a priori

Fundamental results of statistical learning.
No free lunch.
Bias-variance trade-off and dictionary selection.
Bias-variance decomposition. Stochastic error estimation. Dictionary selection.

• 7 Dimension reduction

Curse of dimension. No nearest neighbour in high dimension.
Mass distribution. Parameter estimation is problematic.
Motivation for dimension reduction.
Variable selection. Filtering. Containers. Embedded methods.
Variable extraction. Extraction techniques (multiple learning). Principal Component Analysis (PCA)

ALGORITHMS

• 8 Other approaches: generative models and nearest neighbours

Generative models. QDA. LDA.
Nearest neighbour methods.

• 9 Convexification

Reformulation of the probability of error. Convex risk surrogate.

• 10 Linear predictors

The perceptron, elementary half-space classifier.
Linear regression.
Logistic regression.

• 11 SVM

Lagrange multipliers. Separator distances and margin
Choosing a separator with maximum margin.
Hard SVM algorithm.

• 12 Non-linear separators and redescription spaces

Kernel
Kernel algorithms

What about neurons?

Ensimag first year courses :
Applied probability.
Statistical principles and methods.
Optimization