3 The Interpretability Landscape#PDF p. 16

3.1 Behavioural Methods#PDF p. 17

Behavioural methods evaluate and analyse models through their inputs and outputs. They have been developed to examine a wide range of capabilities, traits, and behaviours under specific conditions. Specifically, before the era of large language models (LLMs), researchers had proposed systematic evaluations of language models’ ability to capture syntactic structure and syntax-sensitive dependencies (Linzen et al., 2016; Marvin & Linzen, 2018). With modern LLMs, behavioural evaluations have expanded to include a much broader set of capabilities, such as professional and academic knowledge across diverse domains (Hendrycks et al., 2021; Wang et al., 2024; Srivastava et al., 2023; Zhong et al., 2024; Phan et al., 2026), long-context understanding (Bai et al., 2024; Zhang et al., 2024a; Modarressi et al., 2025), mathematical reasoning (Cobbe et al., 2021; Azerbayev et al., 2023), and coding ability (Chen et al., 2021; Jimenez et al., 2024; Liu et al., 2024b).

Beyond assessing models as helpful assistants, behavioural evaluations also identify and benchmark potential safety risks and failure modes. Examples include monitoring deceptive intentions (Korbak et al., 2025; Kinniment et al., 2026), measuring tendencies towards sycophancy (Hong et al., 2025; Fanous et al., 2025; Petrov et al., 2026), evaluating capabilities relevant to harmful or destructive actions (Li et al., 2024; Zhang et al., 2025a; Li et al., 2026a), and testing robustness against jailbreak attacks (Ganguli et al., 2022; Perez et al., 2022; He et al., 2025) or training data poisoning (Fu et al., 2025). Behavioural evaluations have also been used to study broader model characteristics, such as political orientations (Helwe et al., 2025) and social attitudes (DURMUS et al., 2024).

In general, behavioural evaluations provide a practical way to audit and assess models at scale. They are typically fast, inexpensive, and scalable, requiring only forward passes and comparisons between model outputs and reference answers. Although evaluation results on a finite testing set do not guarantee the mechanism that produced the behaviour of interest (Wani et al., 2026) (which the internal methods below aim to establish, as discussed in Section 2), behavioural methods nonetheless remain essential in practice for rapid iteration during development, prioritising which behaviours warrant deeper investigation, and surface-level safety screening when the cost of mechanistic analysis is prohibitive.

3.2 Concept-Based Approaches#PDF p. 17

Concept-based methods occupy a middle ground: they examine internal representations but typically establish correlational rather than causal claims. However, when combined with interventions such as steering along concept directions and ablating concept-associated components, they become mechanistic by the criteria discussed in Section 1.2. Geiger et al. (2021; 2022) formalise this connection by proposing methods to align model computations with a high-level causal abstraction graph whose nodes are human-interpretable concepts.

Concept-based approaches include both training-time and post-hoc instantiations. Concept Bottleneck Models (CBMs; Koh et al., 2020) constrain computation through a predefined bottleneck of interpretable concepts, offering built-in transparency and allowing for interventions, but traditionally requiring a complete set of predefined concepts and often suffering from an “accuracy–interpretability trade-off” when concept definitions are insufficiently expressive. Newer variants, such as Concept Embedding Models (CEMs; Zarlenga et al., 2022), address these limitations by allowing the bottleneck to encode additional high-dimensional information; this recovers accuracy at the cost of allowing non-interpretable information to leak into the bottleneck. Moreover, recent improvements relax the assumption of concept independence. Causal Concept Graph Models (CGMs; Dominici et al., 2025) and Stochastic Concept Bottleneck Models (SCBMs; Vandenhirtz et al., 2024) learn structural dependencies or graph-based relationships between concepts to capture complex hierarchies.

Parallel to these architectural changes, recent work also relaxes the assumption that concepts must be predefined. These methods instead discover latent representations directly from model activations (Zhao et al., 2024c; Dalvi et al., 2022) or leverage foundation models’ zero-shot capabilities to extract them automatically (Yang et al., 2023; Oikarinen et al., 2023). Yu et al. (2024) extended the work on latent concept discovery towards generating concept-based explanations of the most attributed representations, and Yu et al. (2026) proposed the Vector Quantized Latent Concepts (VQLC) framework for efficient and scalable automatic concept discovery.

Post-hoc methods, such as Concept Activation Vectors (CAVs; Kim et al., 2018) and linear concept classifiers (Belinkov, 2022), probe existing activations to identify concepts encoded in model representations. In general, these methods do not provide causal evidence about the role of those concepts in model computation, although some probing variants aim to support stronger causal claims, such as amnesic probing (Elazar et al., 2021). Nevertheless, they remain valuable tools for auditing, localisation, and early detection tasks. Furthermore, representation engineering (Rimsky et al., 2024; Marks & Tegmark, 2024; Turner et al., 2024) extends the probing paradigm from observation to intervention, enabling targeted steering of model behaviour. We discuss these methods in detail in Section 9.3.

CBMs are sometimes grouped with the interpretability-by-design methods discussed in Section 6.10, but they operate differently. CBMs impose semantic constraints (predefined human concepts), while interpretable-by-design models impose structural constraints (sparsity, modularity) that make the model intrinsically easier to interpret. Table 1 summarises the distinction.

CBMs can be more appropriate when the set of concepts of interest is known. Interpretability-by-design models may be preferable when the goal is to discover what the model has learned—including features that were not preset in advance. For safety applications, where unknown failure modes are a core concern, discovery-oriented methods may be more useful than those that require specifying concepts upfront.

3.3 Input Attribution and Saliency Methods#PDF p. 18

Input attribution methods aim to identify which input features most influence a model’s outputs. Techniques such as SHAP (Lundberg & Lee, 2017), LIME (Ribeiro et al., 2016), DeepLIFT (Shrikumar et al., 2017), and saliency-based methods (Simonyan et al., 2014; Sundararajan et al., 2017; Zhou et al., 2016; Selvaraju et al., 2017) are scalable and widely used for explanation and debugging at the pixel and token levels. However, these methods indicate which input features matter for a prediction, but not necessarily how the model processes and leverages them. For example, saliency maps for image classifiers leverage gradients propagated from the output back to the input to estimate the sensitivity of the prediction to each pixel (Simonyan et al., 2014). While such methods can highlight influential regions of an input image, they do not reveal how the model represents pixels in its activations and uses them across layers.

The same machinery can also be applied to internal representations, attributing salience to features at different layers and tracing how information evolves throughout the network (Yu et al., 2024; Syed et al., 2024). Used in this way, attribution methods begin to overlap with the mechanistic approaches discussed later in this survey. Nevertheless, concerns about their reliability remain. Different attribution methods can produce substantially different explanations for the same prediction, and some have been shown to be insensitive to model parameters (Adebayo et al., 2018) or sensitive to input transformations that have no effect on the model (Kindermans et al., 2022). For safety-relevant applications, attribution methods help with triage, debugging, and initial auditing, but they are insufficient on their own when an explanation of internal computation is required.

3.4 Representation Probing and Related Post-hoc Analyses#PDF p. 19

Probing classifiers (Belinkov, 2022) assess whether a model’s activations encode particular information by training “lightweight” classifiers—often logistic regression models—on intermediate representations to predict a target property, such as part-of-speech tags, syntactic structure, and gender (Belinkov et al., 2017; Hewitt & Manning, 2019; Ravfogel et al., 2020). Originally developed in the NLP community to analyse linguistic structure (Gupta et al., 2015; Köhn, 2015; Adi et al., 2017) and used by Alain & Bengio (2017) to trace how representations evolve across layers of deep learning models, probes have become one of the most widely used tools in mechanistic interpretability for tasks including, but not limited to, layer-level localisation (Tenney et al., 2019) and detection of latent behaviours (MacDiarmid et al., 2024). We discuss the probing methodology, its limitations, and its relationship to causal interpretability in more detail in Section 8.6.

3.5 Mechanistic Interpretability Methods#PDF p. 19

Mechanistic interpretability (MI) methods aim to causally understand model internals, such as the semantic meaning of parameters or activations, and how they relate to model behaviour. They are largely human-centric in the sense that the desired mechanistic understanding is one that humans can interpret and reason about—whether through natural-language descriptions (Cunningham et al., 2024; Ameisen et al., 2025), plots of attention patterns (Clark et al., 2019; Vig, 2019), or token distributions (nostalgebraist, 2020; Belrose et al., 2025; Gurnee et al., 2026). Recent work, however, has questioned whether “machine languages” and human languages fully overlap, suggesting that some meaningful internal computations may not admit faithful descriptions in natural language (Hewitt et al., 2025).

Mainstream MI approaches include, but are not limited to, feature decomposition and extraction (Cunningham et al., 2024; Dunefsky et al., 2024), circuit discovery (Syed et al., 2024; Ameisen et al., 2025; Conmy et al., 2023), and methods for decoding internal representations into natural language (Ghandeharioun et al., 2024; Pan et al., 2026; Fraser-Taliente et al., 2026), all of which are discussed in later sections. Different methods rest on different assumptions and produce different types of explanation, making them suitable for different use cases but with distinct limitations. For example, sparse autoencoders decompose model representations (e.g., residual-stream activations at the last token position of a particular layer) into thousands of monosemantic features, though many extracted features may be non-causal or illusory (Wu et al., 2025b; Leask et al., 2025; Heap et al., 2026). Circuit discovery methods identify fine-grained computational graphs responsible for particular model behaviours, but these graphs may capture only a subset of the model’s underlying computation and can be difficult to validate for sufficiency or identifiability (Miller et al., 2024; Méloux et al., 2025). Despite these limitations, MI methods remain promising complementary tools that collectively enable systematic analysis of model internals.

3.6 When Mechanistic Understanding Becomes Necessary#PDF p. 19

The methods we have discussed so far form a correlational-to-causal spectrum: behavioural evaluation establishes what a model does, attribution and probing identify where relevant information resides, and mechanistic analysis reveals how computations produce behaviour. These approaches complement each other: behavioural evaluations help validate mechanistic findings, and behavioural anomalies can motivate mechanistic investigation.

Mechanistic analysis becomes necessary when we need to verify how a model produces a behaviour—not just that it does under test conditions. For example, a probe can detect deceptive reasoning, but we still need targeted interventions to validate whether the detection is a false alarm; if not, we will further need more rigorous mechanistic analysis to understand why the model deceives. Likewise, unexpected generalisation or failure modes may require mechanistic analysis to examine whether they stem from identifiable structures. Concretely, mechanistic methods are most needed for: identifying intents hidden from outputs, such as deception or side quests; understanding the internal mechanism of a model behaviour; verifying the absence of a capability (behavioural tests can only confirm presence); or providing safety guarantees that must hold beyond the evaluated input distribution. However, the high granularity of mechanistic analysis comes at high practical cost, including not only compute but also researcher time, experimental complexity, and slower iteration. These costs position mechanistic methods as a complement to lighter-weight approaches, such as behavioural evaluations, rather than a full replacement.

3.7 Limitations of Mechanistic Analysis#PDF p. 20

Mechanistic interpretability aims to provide causal explanations of model behaviour, but it faces important practical and conceptual limitations. First, mechanistic explanations are often incomplete. An analysis may identify an internal computation (e.g., “This MLP retrieves factual knowledge about Paris”, as identified through causal tracing (Meng et al., 2022)) that captures a non-trivial causal pathway without identifying all others contributing to the behaviour. This concern makes it hard to tell whether the identified computation is sufficient or just one of several mechanisms jointly implementing the behaviour.

Second, mechanistic analyses can scale poorly. Analysing large models and complex behaviours takes much more time and resources because the relevant computations may be distributed across model components and complex behaviours involve interactions across long generation trajectories that are hard to isolate. For example, most current circuit-discovery methods (Ameisen et al., 2025; Conmy et al., 2023; Hanna et al., 2024; Marks et al., 2025a; Wu & Barez, 2026) identify the computational graph responsible for generating a specific token. It remains challenging to identify and summarise the computations underlying behaviours that unfold across multiple generated tokens.

Third, even when we identify and causally validate an internal mechanism, this does not automatically mean we can reliably control the corresponding behaviour (Hase et al., 2023). The effectiveness, specificity, and generalisability of localisation and intervention techniques, such as identifying a model component and then patching its activations or changing its weight matrices, depend on several factors, such as the hydra effect (McGrath et al., 2023), beyond how accurately the identified localisation captures the underlying computation for the behaviour. Furthermore, an intervention that successfully steers one model’s behaviour on a task may not transfer to another model or setting.

These are open challenges but not fundamental barriers. Active research in areas such as establishing more effective representational units, discovering more robust model properties, developing more efficient circuit-discovery methods, and designing more reliable intervention approaches across distributions is working to address them. Nevertheless, these studies highlight a clear gap between mechanistic explanation and reliable control or safety guarantees.