From statistical physics to machine learning & back

Diploma(s)
Place
Sorbonne Université
Spring semester
Level Master 2 3 ECTS - English
Instructor(s) Giulio BIROLI ( ENS-PSL )
Teaching Assistant Marylou GABRIE ( ENS-PSL )

In this class, we will explore how the concepts of statistical physics can prove useful to tackle main theoretical questions in machine learning and how the recent progress in generative AI can assist statistical physics computations. We will describe analytical approaches as well as numerical approaches at this intersection. We will present the methods of statistical physics and high-dimensional probability which play a central role in these recent and new research activities, and also make an incursion in some other interdisciplinary applications (biological neural networks and theoretical ecology). 

Some TDs will illustrate the lecture’s topic by analytical computations, others by numerical implementations in python and pytorch.

Wednesday afternoon: Lecture 2pm-3.45pm + TD 4-5:30pm  

Syllabus

Part I: Introduction

1. Machine learning and links with statistical mechanic, methods for high-dimensional probability 

2. Sampling & generative modelling
 

Part II: Statistical physics and the theory of neural networks & learning 

3. Learning dynamics: phenomenology and models 

4. Over-parametrization & the double descent phenomenon 

5. Recurrent neural networks & high-dimensional chaos 

 

Part III: Generative modelling: theory and applications in statistical mechanics

6. Autoregressive networks for discrete state spaces

7. Normalizing flows for continuous state spaces 

8. Diffusion generative models: out of equilibrium dynamics & phase transitions 

9. Stochastic localization  

Evaluation
  • 1 homework

  • 1 final oral exam: presentation of a research paper and questions on course content