Fundamentals of Probabilistic data mining - WMM9MO17

Goal(s)

This lecture introduces fundamental concepts and associated numerical methods in model-based clustering, classification and models with latent structure. We will also discuss probabilistic graphical models with latent variables in general, the expectation maximisation algorithm and its variational Bayes formulation. Overall we will discuss Gaussian mixture models, hidden Markov models, probabilistic PCA, linear dynamical systems, and more complex models that are combinations of these ones, and that require variational expectation-maximisation algorithms. If time allows, we will discuss the link with variational auto-encoders.

Content(s)

Model-based clustering, classification and models with latent structure are particularly relevant to model random vectors, sequences or graphs, to account for data heterogeneity, and to present general principles in statistical modelling. The following topics are addressed:

Principles of probabilistic data mining and generative models; models with latent variables
Probabilistic graphical models
Mixture models and clustering
PCA and probabilistic PCA
Generative models for series and graphs : hidden Markov models
Linear dynamical systems.
Variational EM algorithm.

Fundamental principles in probability theory (conditioning) and statistics (maximum likelihood estimator and its usual asymptotic properties).
Constrained optimization, Lagrange multipliers.