From statistical physics to machine learning & back
In this class, we will explore how the concepts of statistical physics can prove useful to tackle main theoretical questions in machine learning and how the recent progress in generative AI can assist statistical physics computations. We will describe analytical approaches as well as numerical approaches at this intersection. We will present the methods of statistical physics and high-dimensional probability which play a central role in these recent and new research activities, and also make an incursion in some other interdisciplinary applications (biological neural networks and theoretical ecology).
Some TDs will illustrate the lecture’s topic by analytical computations, others by numerical implementations in python and pytorch.
Wednesday afternoon: Lecture 2pm-3.45pm + TD 4-5:30pm
Part I: Introduction
1. Machine learning and links with statistical mechanic, methods for high-dimensional probability
2. Sampling & generative modelling
Part II: Statistical physics and the theory of neural networks & learning
3. Learning dynamics: phenomenology and models
4. Over-parametrization & the double descent phenomenon
5. Recurrent neural networks & high-dimensional chaos
Part III: Generative modelling: theory and applications in statistical mechanics
6. Autoregressive networks for discrete state spaces
7. Normalizing flows for continuous state spaces
8. Diffusion generative models: out of equilibrium dynamics & phase transitions
9. Stochastic localization
1 homework
1 final oral exam: presentation of a research paper and questions on course content