5 Core Hypotheses in Mechanistic Interpretability#PDF p. 27

Conceptual Basis.

A feature associated with a concept is linearly represented if its presence corresponds to a fixed direction $\phi$ in the activation space and can be detected through the projection $\phi^\top h(x)$. This linearity makes mechanistic analysis tractable because we can use simple methods to identify, measure, and manipulate these features in the activation space, such as scaling them and adding them back to model activations.

Notably, activation spaces in different model components can have distinct geometric structures. For example, in the hidden states of MLP blocks, the neuron axes are computationally distinguished because element-wise nonlinear activation functions, such as ReLU, act separately on each axis. The coordinate system is therefore meaningful and is referred to as a privileged basis (Figure 4, left) (Elhage et al., 2022a). In other spaces, such as the residual stream, no operation singles out the neuron axes, so linear features may lie along arbitrary directions. Such spaces are said to have a non-privileged basis (Figure 4, right).

When the basis is privileged, a single neuron can serve as a scalar feature that meaningfully represents a concept. When it is not, the feature may instead correspond to directions or other geometric structures that do not align with any individual neuron. In this case, specialised methods are required to identify them from the activation space.

Empirical Evidence and Counterexamples.

Empirical support for the weaker version of the linear representation hypothesis includes the success of linear probes across diverse models and concepts (Belinkov, 2022), activation steering along single directions to induce targeted behavioural changes (Turner et al., 2024), and sparse autoencoders recovering linearly accessible and steerable features at scale (Bricken et al., 2023). At the same time, there is substantial evidence that not all representations are well described as linear. Counterexamples include nonlinear feature interactions, algorithmic representations that rely on nonlinear transformations, and features distributed across multiple interacting subspaces (Nanda et al., 2023; Engels et al., 2025; Tiblias et al., 2026; Csordás et al., 2024; Feucht et al., 2026; Wurgaft et al., 2026). Some work has further argued that features may not always exist as explicitly represented objects, but may instead emerge from underlying circuit dynamics (Smith, 2024). The linear representation hypothesis is therefore best viewed as a useful approximation that holds in some settings, rather than as a universal law.

Conceptual Basis.

Superposition is a geometric property of learned representations in which a limited number of neuron dimensions encode many more features by allowing those features to share representational dimensions. Neuron polysemanticity, where individual neurons respond to multiple seemingly unrelated semantic concepts, naturally follows from this phenomenon. When the number of encoded features exceeds the activation-space dimensionality, the model can no longer allocate a dedicated neuron dimension to each feature. Instead, multiple features become superposed within the same activation space, shifting the representation from dedicated dimensions with monosemantic neurons to shared dimensions with increasingly polysemantic neurons (Figure 5).

Empirical Evidence.

Mechanistic studies have repeatedly observed widespread neuron polysemanticity: individual neurons often activate for multiple seemingly unrelated semantic concepts, making them difficult to interpret as standalone representational units (Olah et al., 2020; Elhage et al., 2022a; Bricken et al., 2023). Dictionary-learning methods such as sparse autoencoders exploit this structure to recover millions of distinct features from shared activation spaces (Bricken et al., 2023; Templeton et al., 2024). Many discovered features are substantially more monosemantic than individual neurons, and some have been shown to causally influence model behaviour. Together, these findings suggest that the activation space may encode more meaningful semantic information along activation-space directions than along individual neuron axes.

However, the superposition hypothesis remains a hypothesis, not an established fact, in large language models. Observing polysemantic neurons does not, by itself, imply that superposition is the underlying cause, and dictionary-learning methods can recover interpretable features even in the absence of superposition. Furthermore, Lecomte et al. (2024) showed that polysemanticity can arise even when the number of underlying features does not exceed the number of available neurons, suggesting alternative—and potentially complementary—mechanisms for the emergence of polysemantic representations.

Architectural Responses to Superposition.

Superposition is not an intrinsically inevitable property of deep neural networks but a consequence of architectural permissiveness. Without constraints on model architectures or activations, gradient descent can exploit the activation-space geometry to pack far more features than neurons (Elhage et al., 2022a). This motivates a distinct research direction, interpretability-by-design (as mentioned in Section 3.2 and discussed in Section 6.10). Interpretability-by-design argues that, rather than decomposing traditional models’ superposed activations post hoc, one can design novel architectures or components that encourage representations where features are more accessible for analysis. Two intervention points have been explored:

• Sparsity constraints: Enforce sparsity in weights or activations that limits how many neurons within an MLP or attention head can participate for a given input, constraining how many units can jointly support any single feature (Yang et al., 2025; Gao et al., 2026; Elhage et al., 2022b).

• Computational primitives: Change the fundamental units of analysis from static features in activations to learnable transformations by weight matrices that generate activations (Liu et al., 2025b; Lindsey et al., 2025b). This paradigm better matches certain tasks and avoids the pressure to compress or suppress model representations.

These architectural interventions shift from treating superposition as a phenomenon to reverse-engineer to treating it as a design choice that can be constrained upstream. We discuss these methods in detail in Section 6.10.

Conceptual Basis.

The ability to access and judge its own computation or reasoning—called introspective monitoring—has long been a desired property of an AI system (Cox & Raja, 2008). Even earlier rule-based medical expert systems sought to explain their decisions, such as MYCIN’s “Explanation system” (Shortliffe, 1977). In this sense, using language models to explain themselves can be seen as a modern continuation of these earlier efforts.

A major difference is that rule-based systems can explicitly present the rule they executed, whereas modern language models compute in a highly nonlinear way. Although researchers have proposed mechanistic tools aiming to reverse-engineer part of the computation, the technical complexity of these techniques hinders their accessibility and scalability. The privileged access hypothesis then posits that the language model itself naturally stores information about the computation within its internal states, such as latent beliefs or intermediate plans. The model can thus learn to verbalise this information directly, serving as a scalable complement to mechanistic analysis.

Empirical Evidence.

The most direct empirical evidence for this hypothesis comes from recent findings that models, even without explicit training to do so, are sometimes aware of concept representations (acquired by methods like difference-in-means) injected into their activations (Lindsey, 2026; Fornasiere et al., 2026). Additional, though less direct, evidence comes from the design of explainer models—a general class of interpretability methods in which one model is fine-tuned to explain the target model’s activations (Pan et al., 2026; Karvonen et al., 2026a; Fraser-Taliente et al., 2026). In practice, such explainers are typically initialised from a copy of the target model itself (Pan et al., 2026; Karvonen et al., 2026a; Bauer et al., 2026; Huang et al., 2026). This usually works better than initialising from alternative models, suggesting the target model already encodes much of what is needed to interpret its own activations. In this sense, fine-tuning elicits and refines a latent capability rather than creating one from scratch.

Some studies also reported negative results for the privileged access hypothesis: Song et al. (2026) found that models predict their own sampling temperature no better than they predict other models’, suggesting apparent introspection may reflect general reasoning about model-like systems rather than self-access. Separately, recent work has questioned whether internals-based explanations provide a practical advantage over behavioural baselines: Karvonen et al. (2026b) found that activation-reading tools (including activation oracles) give agents no uplift over transcript-only baselines at predicting counterfactual model behaviours in the wild, and Li et al. (2026c) showed that standard verbalisation benchmarks can be solved without access to target-model internals at all (see also Jakkli et al., 2026).

In summary, these results show that privileged access is possible for certain tasks under favourable conditions, not universal or reliable. The hypothesis should therefore be treated with caution, as significant failure modes and limitations remain.

Conceptual Basis.

A key distinction is the level at which models are expected to converge. Weak universality refers to convergence in the algorithms or recurring computational patterns that models implement, even when these are encoded through different internal representations. Strong universality makes the stronger claim that the internal representations themselves converge towards a shared canonical form.

Empirical Evidence and Trade-Offs.

Weak universality is supported by recurring attention patterns, features, and circuits across models, tasks, and scales (Nanda et al., 2023; Lan et al., 2024; Olah et al., 2020; Quirke et al., 2026). Strong universality is less supported and is usually demonstrated through representational similarity metrics (Kornblith et al., 2019; Moschella et al., 2023; Lan et al., 2025), where recent studies have found the results to be fragile (Gröger et al., 2026; Koepke et al., 2026). Divergent representations can arise from variations in architecture, data, or optimisation trajectories (Harrasse et al., 2025; Morcos et al., 2018).

Implications.

If strong universality holds, fine-grained mechanistic insights and the methods derived from them may transfer more readily across models. If only weak universality holds, such insights may remain contingent on factors such as model family and task (Bereska & Gavves, 2024), and even a complete mechanistic understanding of one model (see Section 12.2) may fail to generalise to others. Therefore, it is important to clarify the extent of universality for assessing the promise, limitations, and scalability of different interpretability research orientations, including the six major orientations introduced in Section 12.

Summary.

This section discussed four hypotheses that motivate many modern mechanistic interpretability studies. Linear representations support linear-feature-based analysis; superposition explains why we need to operate on linear or more complex forms of features instead of single neurons; privileged access explains the fundamental idea of model self-interpretation; universality asks whether insights transfer across models and at which levels. Each hypothesis holds in some settings and fails in others, and techniques built on them inherit those limitations. Section 6 introduces the main family of techniques and methods that operationalise the first two hypotheses (linear representation and superposition): dictionary learning for feature extraction and analysis.