Methods · Learning Dynamics and Mechanistic Interpretability
Progress Measures
Survey sections: §10.3.2 Progress Measures and Internal Precursors
A progress measure is a scalar quantity defined on a model’s internals that varies continuously as a mechanism forms, and can therefore track the mechanism’s development before the behaviour it supports changes (Nanda et al., 2023).
Assumptions and requirements
Progress measures are typically derived from a mechanistic analysis of the completed mechanism at the end of training, then evaluated retroactively at earlier checkpoints.
Intended applications
Mechanistic analysis of grokked networks recovers the learned algorithm (trigonometric identities implementing modular addition) and defines progress measures that track its formation: the generalising circuit grows gradually during the plateau, and the sharp change in test accuracy corresponds to a transition from a memorising solution to the completed generalising circuit followed by removal of the memorising components (Nanda et al., 2023).
Limitations
Currently, nearly all demonstrations that changes in internals lead to changes in behaviour are retrospective. In these studies, researchers analyse the completed mechanism after training to identify a progress measure or precursor structure, then evaluate it backwards over saved checkpoints.
The standard setting in grokking experiments differs from language model pretraining in many ways, including dataset size, weight decay, and the number of epochs over the data. Current studies have not shown that existing quantitative conclusions transfer directly to large-scale pretraining.
Related methods
Key papers
Cited in §10.3.2.
- Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, and Jacob Steinhardt. Progress measures for grokking via mechanistic interpretability. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=9XFSbDPmdW.
- PreprintAlethea Power, Yuri Burda, Harri Edwards, Igor Babuschkin, and Vedant Misra. Grokking: Generalization beyond overfitting on small algorithmic datasets, 2022. URL https://arxiv.org/abs/2201.02177.
- PreprintVikrant Varma, Rohin Shah, Zachary Kenton, János Kramár, and Ramana Kumar. Explaining grokking through circuit efficiency, 2023. URL https://arxiv.org/abs/2309.02390.
- Ziming Liu, Eric J Michaud, and Max Tegmark. Omnigrok: Grokking beyond algorithmic data. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=zDiHoIWa0q1.
- Angelica Chen, Ravid Shwartz-Ziv, Kyunghyun Cho, Matthew L. Leavitt, and Naomi Saphra. Sudden drops in the loss: Syntax acquisition, phase transitions, and simplicity bias in MLMs. In The Twelfth International Conference on Learning Representations, 2024a. URL https://openreview.net/forum?id=MO5PiKHELW.
- PreprintNicolas Zucchet, Jörg Bornschein, Stephanie Chan, Andrew Lampinen, Razvan Pascanu, and Soham De. How do language models learn facts? dynamics, curricula and hallucinations, 2025a. URL https://arxiv.org/abs/2503.21676.
Also discussed in: §10.4.1 Internal Precursors as Early Warning · §10.4.2 Intervening on Training Processes
Other methods in Learning Dynamics and Mechanistic Interpretability
Trajectory-Based Attribution · Per-Token Learning Curves · Refined Local Learning Coefficients (Localising Change to Components) · Forked-Run Experiments and Linear Mode Connectivity · Loss-Curve Derivatives · Capability-Specific Metrics · Per-Example Decomposition (POLCA) · Latent State Models · Local Learning Coefficient (LLC) · Forecasting from Smaller and Partial Runs · Forecasting Safety-Relevant Behaviours · Internal Precursors as Early Warning · Pretraining Data Filtering (Deep Ignorance) · Gradient Routing · Monitoring and Steering During Fine-Tuning (Persona Vectors) · Train-Time Intervention Loop