Hands-on
Interactive demos
Three ideas from the survey to try yourself. The first two train small models in your browser; the third shows activation patching on GPT-2 small.
Computed in browser
Superposition in a toy model
Train a model that has to squeeze five features into two dimensions. Watch features pile into superposition as the inputs get sparser.
Computed in browserSparse autoencoders recover planted features
Train an SAE on data whose true features you know, then measure how many it recovers as you trade reconstruction against sparsity.
Measured on GPT-2Activation patching on GPT-2 small
Find where GPT-2 small resolves indirect object identification by patching clean activations into a corrupted run, layer by layer and head by head.