Awesome Physics of AI

Appearance

Awesome Physics of AILearning Mechanics

A curated list of papers leveraging physics principles to understand, predict, and decode deep learning.

Explore Papers
View on GitHub
Recent Papers

1. Solvable Idealized Settings

Exploring micro-models where learning dynamics simplify exactly, including deep linear networks and kernel methods.

2. Tractable & Simplifying Limits

Analyzing systems at infinite width or depth limits to reveal mean-field behaviors and the lazy vs. rich dichotomy.

3. Macroscopic Empirical Laws

Capturing reproducible empirical regularities like neural scaling laws, neural collapse, and the edge of stability.

4. Hyperparameter Disentanglement

Demystifying optimization parameters via continuous effective flows and width/depth scaling theories (like µP).

5. Universal Phenomena

Investigating why disparate architectures and data modalities converge toward shared Platonic representations.

🏷️ Inspiration & Citation ​

"To understand deep learning as a branch of quantitative science, we must identify the macro-scale laws governing the mechanics of learning rather than solely tracking individual micro-parameters."

This curated repository is directly structured around the groundbreaking taxonomy and scientific vision proposed in the landmark position paper:

"There Will Be a Scientific Theory of Deep Learning" Authored by Jamie Simon, Daniel Kunin, Alexander Atanasov, and the deep learning physics research community.

🪐 The Paradigm Shift: Learning Mechanics ​

Traditional statistical learning theories (such as VC-dimension or Rademacher complexity) frequently provide pessimistic or vacuous bounds when applied to modern overparameterized neural networks. In contrast, this project tracks the transition of AI research into an empirical, predictive branch of physics—systematically organized into Five Core Pillars:

  • Solvable Idealized Settings: Stripping away complexity to isolate exact dynamical solutions in micro-models.
  • Tractable & Simplifying Limits: Taking width and depth to infinity to unlock stable mean-field equations.
  • Macroscopic Empirical Laws: Observing reproducible statistical regularities like Scaling Laws and Edge of Stability.
  • Hyperparameter Disentanglement: Mapping optimization parameters using SDE noise physics and μP scaling rules.
  • Universal Phenomena: Investigating the emergence of shared Platonic representations across distinct modalities.

🛠️ A Community-Driven Living Index ​

While the original paper provides the foundational conceptual architecture, this open-source collection serves as an expanded, living bibliography. We have integrated deeper historical prerequisites alongside the latest theoretical breakthroughs to provide a permanent reference for researchers decoding the physics of artificial intelligence.

Contributions are warmly welcomed! If you notice any missing milestones, want to add your latest works, or suggest structural improvements, please feel free to open a Pull Request (PR). Let's build this collaborative knowledge base together!