Methods · Learning Dynamics and Mechanistic Interpretability
Gradient Routing
Survey sections: §10.4.2 Architectural Modification: Gradient Routing
Gradient routing is an architectural modification informed by learning dynamics: it constrains which parameters receive gradient updates for chosen concepts during training, localising the corresponding computation into designated regions of the network (Cloud et al., 2024).
Intended applications
The antecedent for this style of intervention is the smeared-key modification (Olsson et al., 2022), which showed that an architectural change can control when and whether a mechanism forms at all (Section 10.3.5).
Limitations
Although the modification does not make the trained model safer on its own, it shapes where a mechanism forms so that later interventions, such as gating the designated region, become clean and localised.
Related methods
Key papers
Cited in §10.4.2.
- PreprintAlex Cloud, Jacob Goldman-Wetzler, Evžen Wybitul, Joseph Miller, and Alexander Matt Turner. Gradient routing: Masking gradients to localize computation in neural networks, 2024. URL https://arxiv.org/abs/2410.04332.
- PreprintCatherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Scott Johnston, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. In-context learning and induction heads. Transformer Circuits Thread, 2022. https://transformer-circuits.pub/2022/in-context-learning-and-induction-heads/index.html.
Other methods in Learning Dynamics and Mechanistic Interpretability
Trajectory-Based Attribution · Per-Token Learning Curves · Refined Local Learning Coefficients (Localising Change to Components) · Forked-Run Experiments and Linear Mode Connectivity · Loss-Curve Derivatives · Capability-Specific Metrics · Per-Example Decomposition (POLCA) · Latent State Models · Progress Measures · Local Learning Coefficient (LLC) · Forecasting from Smaller and Partial Runs · Forecasting Safety-Relevant Behaviours · Internal Precursors as Early Warning · Pretraining Data Filtering (Deep Ignorance) · Monitoring and Steering During Fine-Tuning (Persona Vectors) · Train-Time Intervention Loop