Activation patching on GPT-2 small
Measured on GPT-2 small
Survey: §7.1 Circuit Discovery Methods
Where does GPT-2 small work out who receives the drink? We run a clean prompt and a corrupted prompt whose answer differs. Then we copy one internal activation at a time from the clean run into the corrupted run and measure how much of the correct behaviour comes back.
Select a cell to see its value.
Information that separates clean from corrupted sits at the repeated name, the subject of the second clause (position 10), in early layers, then moves to the final token in the later layers. Reading off where each row “lights up” shows where the circuit carries the signal.
How these numbers were produced
Model: gpt2 (124M) via transformer_lens HookedTransformer.from_pretrained. Task: indirect object identification over 32 prompt pairs (4 templates × 4 name pairs × 2 orders). In the corrupted prompt the second-clause subject is swapped, which flips the answer. Metric: mean over prompts of logit(IO) - logit(S) at the final position. A value of 0 equals the corrupted run and 1 equals the clean run. The clean run is correct on 100% of prompts. Activation patching runs the model once per component. Attribution patching approximates the same quantity with a single backward pass, using a first-order Taylor expansion. Across all 144 heads the two agree with Pearson r = 0.985.
Reproduce: python3 scripts/experiments/compute_patching.py (torch 2.8.0, transformer-lens 2.17.0; about 57 s on a laptop CPU, deterministic). Generated 2026-11-05. The setup follows the companion notebook on circuit discovery, scaled up to more prompts.
A patching effect shows that a component matters for this metric on these prompts, not what it computes.
Go further
- Methods explorer: Activation Patching
- Notebook: Circuit Discovery · Open in Colab
References
- Kevin Ro Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. Interpretability in the wild: a circuit for indirect object identification in GPT-2 small. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=NpsVSN6o4ul.
- Fred Zhang and Neel Nanda. Towards best practices of activation patching in language models: Metrics and methods. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=Hf17y6u9BC.