The State of Mechanistic Interpretability
for AI Safety

Methods · Understanding Intermediate Representations

Probing Classifiers

Survey sections: §8.6 Probing Classifiers: Capabilities, Critiques, and Causal Limitations · §8.6.1 Standard Probing Methodology · §8.6.2 Control Tasks and Selectivity · §8.6.3 Concerns of Probe Complexity · §8.6.3 Minimum Description Length Probes · §8.6.3 Practical Guidance · §8.6.5 Implications of Attribution Faithfulness for Probing · §8.6.6 When Is Probing Appropriate? · §8.6.7 Methodological Standards in Probing Research

A probe is a classifier trained to predict a property of the input, such as part of speech, sentiment, or entity type, from model activations.

Assumptions and requirements

The linear probe tests whether the target property is linearly accessible from the representation.
Probing provides evidence about whether activations contain information about a property. High probe accuracy indicates the information is present; low accuracy suggests it is absent or the probe is not expressive enough.

Evaluation considerations

A control task assigns random labels to inputs to break any genuine relationship between activations and the target property.
High selectivity indicates that the probe succeeds because the representation genuinely encodes the target property.
To address this, Voita & Titov (2020) proposed evaluating probes via the minimum description length (MDL) principle: prefer probes that achieve good accuracy with minimal complexity.

Intended applications

We argue that probing is most productive when treated as a tool for hypothesis generation, that is, using probes to identify candidate properties that may be encoded in a representation and potentially causally relevant.
Despite the causality critiques, probing remains valuable for many applications. For example, MacDiarmid et al. (2024) showed that simple linear probes trained on residual-stream activations can detect latent backdoor behaviour in the sleeper agent model even when the probe fitting set does not contain the trigger or the harmful behaviour.
In summary, this use case shows that probing remains valuable for auditing or detection even though it cannot support causal claims.

Limitations

Note that probing alone does not show whether the model actually depends on that information in its decision-making process.
High probe accuracy does not imply causal relevance (Belinkov, 2022; Hewitt & Liang, 2019).
Finally, a sufficiently expressive probe can achieve high accuracy by exploiting noise even when the representations contain no such information.

Related methods

Compare side by side →

Used in safety applications

Run it

Key papers

Cited in §8.6.

Also discussed in: §1.2 Survey Scope · §3.4 Representation Probing and Related Post-hoc Analyses · §12.3 Developmental Interpretability: Characterising How Models Learn · §13.3.2 Mechanistic Monitoring · §13.5 Choosing Appropriate Methods for Different Safety Applications · §13.4 Automated Interpretability Agents

Other methods in Understanding Intermediate Representations

Logit Lens · Tuned Lens · Patchscopes · LatentQA · Amnesic Probing