Technical methods · Evaluation

A unified evaluation framework

Five dimensions for evaluating interpretability claims.

In the survey: §7.5 A Unified Evaluation Framework for Mechanistic Interpretability

A persistent challenge in mechanistic interpretability (MI) is the lack of standardised evaluation. Different interpretability methods use incompatible metrics.

To address this, this section presents a unified evaluation framework covering five dimensions—attribution faithfulness, completeness, stability, causal validity, and human interpretability—where different methods can be jointly tested.

We propose a framework that situates mechanistic interpretability evaluation within five complementary dimensions. No single dimension suffices—strong methods should perform well across all five.

Attribution faithfulness Attribution faithfulness measures whether the interpretability method accurately attributes the model’s behaviour of interest to responsible components.
Completeness Completeness measures how much of the model behaviour the identified components or mechanisms explain.
Stability Stability measures the robustness of explanations, attributions, and findings to changes in variables that should not affect the results, such as the choice of random seeds, small hyperparameter changes, and input paraphrasing.
Causal validity Causal validity assesses whether the identified components or mechanisms are causally responsible for the behaviour, particularly necessity (i.e., removing the components/mechanisms disrupts the behaviour) and sufficiency (i.e., the components/mechanisms alone are sufficient to preserve the behaviour).
Human interpretability Human interpretability measures whether generated explanations eventually improve human understanding, prediction, and decision-making about model behaviour.

Applying it to each method family

Table 10: Recommended evaluation priorities by interpretability method. The entries indicate evaluation aspects particularly important for each method, not mutually exclusive metrics.

Method	Attribution Faithfulness	Completeness	Stability	Causal Validity	Human Interpretability
SAEs and variants	Feature attribution–intervention agreement; effect-size error	Activation reconstruction; loss recovery; unexplained variance	Cross-seed feature alignment	Feature necessity, sufficiency, and specificity	Feature description consistency across annotations, including models and humans
Circuit Discovery	Edge or component attribution–intervention agreement; counterfactual recovery	Behavioural coverage; loss or behaviour recovery	Robustness across hyperparameters, seeds, and paraphrases	Circuit necessity, sufficiency, specificity; interchange tests	Circuit-level descriptions; trade-off between sparsity and variance explained
Probing	Agreement between probe-derived importance and intervention effects	Coverage across target concepts or behaviours under different distributions	Robustness across seeds and sampled fitting sets	Necessity, specificity, and intervention tests on probe directions	Explanation of shared information in fitting set and search for confounding factors
Steering Vectors	Agreement between predicted and observed intervention effects	Coverage across behaviours deemed relevant	Stability across steering magnitudes, seeds, and paraphrases	Intervention efficacy and specificity; side-effect analysis	Explanation of shared information in fitting set and search for confounding factors

In the survey: §7.5.2 Five Evaluation Dimensions

The protocol, step by step

Step 1: Define scope
- Users first specify the model, layer(s), and behaviour(s) of interest and define a behavioural test suite with clear success criteria.
Step 2: Establish baselines
- Users specify baseline methods, such as random directions in activation space, PCA components, individual neurons, and supervised probes (if labels are available).
Step 3: Apply interpretability method
- Users apply the method of interest to analyse models with variation in random seeds, hyperparameters, and other training details.
Step 4: Evaluate across all five dimensions
- Users compute metrics for attribution faithfulness, completeness, stability, causal validity, and human interpretability.
Step 5: Report comprehensively
- Users report all five dimensions instead of just favourable metrics and discuss the method’s failure cases, unexplained behaviours, and comparisons to baselines.

In the survey: §7.5.3 Integrated Evaluation Protocol

Limitations of the framework

Metric choice: The proposed specific metrics are not uniquely correct. Alternative formalisations may be appropriate for specific contexts.
Threshold arbitrariness: Thresholds (e.g., the accuracy that counts as success) may lack theoretical grounding and will need empirical calibration.
Computational cost: Evaluating all five dimensions comprehensively is expensive and potentially prohibitive for large-scale studies.
Human evaluation bottleneck: Dimension 5 requires time-consuming, expensive, and subjective human studies.
Task dependence: A method’s performance under this framework can depend on the evaluation task. For example, a method that excels at factual recall may perform poorly at reasoning.

Despite these limitations, systematic evaluation across multiple dimensions provides more reliable evidence than an arbitrary choice of metrics. Readers should view the framework as a starting point for standardisation, not a final answer.

In the survey: §7.5.4 Limitations of This Framework

Elsewhere in the survey

More recently, several works have proposed metrics to quantify circuit faithfulness as a continuous score (Hanna et al., 2024; Wu & Barez, 2026), allowing systematic comparisons of circuits discovered by different methods. By aggregating faithfulness scores across tasks and settings, benchmarks such as MIB (Mueller et al., 2025) evaluate how well circuit discovery methods identify causally important model components.
Survey §7.3 Evaluating Circuits and Benchmarking Discovery Methods

Section 7.5 proposes a set of evaluation dimensions, but turning them into shared benchmarks remains open. Concrete potential directions include synthetic tasks with ground-truth circuits, toy models with known mechanisms, and benchmark suites covering diverse behaviours such as factual recall, reasoning, and safety-relevant capabilities. Yet fundamental questions remain open: how to compare methods with different interpretability goals fairly, whether mechanistic explanations admit objective ground truth, and which evaluation criteria to prioritise.
Survey §14.2 Challenges in Evaluation and Benchmarking

We identify gaps and argue that the evaluation methodology and safety benchmarks are the field’s most pressing open problems.
Survey §15.1 Achievements and Limits

See causal validity in practice: the activation-patching demo →

Attribution faithfulness

Attribution faithfulness measures whether the interpretability method accurately attributes the model’s behaviour of interest to responsible components.
Survey §7.5.2 Dimension 1: Attribution Faithfulness

How it is measured

Intervention-Effect Agreement

Let a(S,x) denote the importance score assigned by an interpretability method to components S for a sample $x \sim B$ :

\mathrm{EffectAgreement} = \rho_{x \sim B}\!\left( a(S,x), s(S,x) \right)

where $\rho$ may be Pearson or Spearman correlation. High agreement indicates that the attribution consistently reflects the identified components’ causal effects on behaviour B.

Effect-Size Error

When attribution scores are calculated to estimate intervention effects, we measure the effect-size error as:

\mathrm{EffectError} = \mathbb{E}_{x \sim B} \left[ \left| a(S,x)-s(S,x) \right| \right]

A low value indicates good calibration between estimated and actual intervention effects. If the two quantities are not naturally on the same scale, we may use normalised or rank-based measures instead.

Counterfactual Effect Recovery

Consider inputs $x_1 \sim B'$ and $x_2 \sim B$ , where B' is a reference behaviour (e.g., honesty) and B is the behaviour of interest (e.g., sycophancy). Patching S from $x_2$ into the computation on $x_1$ yields

\mathrm{CFRecovery}(S;B,B') = \mathbb{E}_{x_1 \sim B', x_2 \sim B} \frac{ M\!\left( f_\theta\!\left( x_{1,[S \leftarrow h_S(x_2)]} \right) \right) - M\!\left(f_\theta(x_1)\right) }{ M\!\left(f_\theta(x_2)\right) - M\!\left(f_\theta(x_1)\right) }

A value near 1 indicates that the patching recovers much of the behaviour, while a value near 0 indicates little effect. This metric has been widely used in the circuit discovery literature (Hanna et al., 2024; Zhang et al., 2025b; Syed et al., 2024). However, the denominator of this metric can be close to 0, and $M\!\left( f_\theta\!\left( x_{1,[S \leftarrow h_S(x_2)]} \right) \right)$ empirically can exceed $M\!\left(f_\theta(x_2)\right)$ or fall below $M\!\left(f_\theta(x_1)\right)$ . These may cause this metric to take values much larger than 1 or below 0, complicating the analysis. Recent work has proposed an alternative metric that bounds the value between 0 and 1 (Wu & Barez, 2026).

Evaluation protocol

Apply the method to acquire attribution scores a(S,x) of model components and select those of interest (e.g., components with high scores).
Measure the corresponding ground-truth intervention effects s(S,x).
Compare them using correlation, error, or rank agreement.
Evaluate how much of the behavioural difference between clean and corrupted inputs can be recovered by patching the identified components.
Report results across a diverse set of inputs with uncertainty estimates.

Summary

Dimension	Attribution Faithfulness
Definition	Do attribution scores accurately reflect the causal effects of the identified components?
Metrics	Intervention-effect agreement; effect-size error; counterfactual effect recovery
Methods	Ablation; activation / path patching; interchange interventions

Completeness

Completeness measures how much of the model behaviour the identified components or mechanisms explain. A high completeness score indicates that the identified components/mechanisms capture most of the relevant computation.
Survey §7.5.2 Dimension 2: Completeness

How it is measured

Loss Recovery

For methods that reconstruct model activations (e.g., SAEs and CLTs), after replacing the target component’s activations with the reconstructions, we measure the fraction of the original model performance that is recovered:

\mathrm{LR} = \frac{ \mathcal{L}_{\mathrm{reconstructed}} - \mathcal{L}_{\mathrm{ablated}} }{ \mathcal{L}_{\mathrm{original}} - \mathcal{L}_{\mathrm{ablated}} }

where $\mathcal{L}_{\mathrm{original}}$ is the downstream test loss of the original model, $\mathcal{L}_{\mathrm{reconstructed}}$ is the loss after replacing the original activations with reconstructions, and $\mathcal{L}_{\mathrm{ablated}}$ is the baseline loss where activations are replaced with zero ablation, mean ablation, or Gaussian noise matching the layer statistics. A value near 1 indicates that the reconstruction preserves most of the information. Similar to the vanilla Counterfactual Effect Recovery metric discussed above, this metric can take arbitrary values. A simple solution is to develop a bounded variant, as has been done for Counterfactual Effect Recovery (Wu & Barez, 2026).

Behavioural Coverage

Across input prompts $x \sim B$ , we measure the fraction for which the identified components have a meaningful intervention effect:

\mathrm{Coverage}(S,B) = \mathbb{E}_{x \sim B} \left[ \mathbbm{1} \left( |s(S,x)| \geq \tau \right) \right]

where s(S,x) is the intervention effect defined above and $\tau$ specifies the minimum effect considered meaningful. Higher coverage indicates that the identified components account for the behaviour across a larger fraction of inputs.

Unexplained Variance

For methods that reconstruct activations of the components, we measure the fraction of unexplained activation variance as:

\mathrm{UV}(S,B) = \frac{ \mathbb{E}_{x \sim B} \left[ \|h(x)-\hat{h}_S(x)\|_2^2 \right] }{ \mathbb{E}_{x \sim B} \left[ \|h(x)-\bar{h}_B\|_2^2 \right] }

where $\bar{h}_B = \mathbb{E}_{x \sim B}[h(x)]$ , h(x) is the original model activation, and $\hat{h}_S(x)$ is its reconstruction from the identified components S. Lower values indicate that the identified components explain a larger fraction of the variance in the original activations.

Evaluation protocol

Report behavioural coverage using the intervention effects s(S,x).
Report reconstruction-based metrics such as loss recovery and unexplained variance if applicable.
Analyse inputs of the behaviour for which the identified components do not provide good coverage or reconstruction.

Summary

Dimension	Completeness
Definition	How much of the behaviour is accounted for by the identified components?
Metrics	Loss recovery; behavioural coverage; unexplained variance
Methods	Reconstruction; behavioural evaluation; residual analysis

Stability

Stability measures the robustness of explanations, attributions, and findings to changes in variables that should not affect the results, such as the choice of random seeds, small hyperparameter changes, and input paraphrasing.
Survey §7.5.2 Dimension 3: Stability

How it is measured

Cross-Seed Agreement

We train or apply the interpretability method K times under different random seeds. Let $\mathcal{F}_i$ denote the set of features or mechanisms, such as SAE features and circuits, discovered under seed i. We measure the cross-seed agreement as:

\text{Seed-Stability} = \frac{1}{K(K-1)} \sum_{i \neq j} \text{Align}(\mathcal{F}_i, \mathcal{F}_j)

where $\text{Align}(\cdot)$ measures similarity between sets. For example, Paulo & Belrose (2026) calculate the Hungarian matching between two sets of SAE features based on their pairwise cosine similarities.

Hyperparameter Sensitivity

We measure the sensitivity of mechanistic analyses or methods under reasonable hyperparameter variations. Let $\Omega$ denote the set of evaluated hyperparameter configurations and $\text{Metric}(\omega)$ denote the experimental result (e.g., a method’s performance) under a configuration $\omega \in \Omega$ . We calculate the hyperparameter sensitivity as:

\text{HP-Sensitivity} = \frac{ \max_{\omega \in \Omega} \text{Metric}(\omega) - \min_{\omega \in \Omega} \text{Metric}(\omega) }{ \mathbb{E}_{\omega \in \Omega}[\text{Metric}(\omega)] }

Lower values indicate that the finding is less sensitive to hyperparameter choices and therefore more robust.

Paraphrase Consistency

For each input $x \sim B$ , let $\mathcal{P}(x) = \{x^{(1)}, \dots, x^{(K)}\}$ denote a set of semantics-preserving paraphrases of x. We measure the consistency of explanations or attributions across $\mathcal{P}(x)$ . For example, we can measure whether the intervention effect of S remains consistent:

\text{Para-Consistency}(S, B) = \mathbb{E}_{x \sim B}\left[ 1 - \frac{\mathrm{Var}_{x' \in \mathcal{P}(x)}\left[s(S, x')\right]}{\mathbb{E}_{x' \in \mathcal{P}(x)}\left[s(S, x')^2\right]} \right]

The score lies in [0, 1]. A value of 1 indicates that the intervention effect is identical across all paraphrases. A value of 0 indicates that, for each x, the expected intervention effect, i.e., $\mathbb{E}_{x' \in \mathcal{P}(x)}s(S, x')$ , is 0.

Evaluation protocol

Execute the interpretability method with several random seeds and compute alignment of the discovered units or findings across seeds.
Analyse which findings are stable across all seeds and which ones are seed-specific. Key claims need to remain consistent under subsequent tests on hyperparameter variation and input paraphrasing, if applicable.
Distinguish robust findings from fragile ones and report both of them.

Summary

Dimension	Stability
Definition	Are findings robust to variations in factors like random seeds, hyperparameters, paraphrases, etc.?
Metrics	Cross-seed agreement; checkpoint consistency; hyperparameter sensitivity
Methods	Multi-seed analysis; checkpoint comparison; input paraphrasing

Causal validity

Causal validity assesses whether the identified components or mechanisms are causally responsible for the behaviour, particularly necessity (i.e., removing the components/mechanisms disrupts the behaviour) and sufficiency (i.e., the components/mechanisms alone are sufficient to preserve the behaviour).
Survey §7.5.2 Dimension 4: Causal Validity

How it is measured

Necessity Score

Necessity measures the fraction of the model’s performance on behaviour B that is lost when S is ablated (i.e., its activations are replaced with some reference values). It is calculated as:

\mathrm{Necessity}(S,B) = 1- \frac{ \mathbb{E}_{x\sim B} \left[ M\!\left( f_\theta\!\left( x_{[S\leftarrow\tilde h_S]} \right) \right) \right] }{ \mathbb{E}_{x\sim B} \left[ M\!\left(f_\theta(x)\right) \right] }

A higher value indicates that S is necessary for the behaviour.

Sufficiency Score

Let $\bar S$ denote the components outside S. Sufficiency measures how much of the original behaviour is achieved when only S is active (i.e., activations of all components outside S are replaced with some reference values). It is calculated as:

\mathrm{Sufficiency}(S,B) = \frac{ \mathbb{E}_{x\sim B} \left[ M\!\left( f_\theta\!\left( x_{[\bar S \leftarrow \tilde h_{\bar S}]} \right) \right) \right] }{ \mathbb{E}_{x\sim B} \left[ M\!\left(f_\theta(x)\right) \right] }

A high value indicates that S alone implements much of the behaviour.

Specificity

Specificity measures the effect on non-target behaviours by interventions on S. It is calculated as:

\mathrm{Specificity}(S,B) = \frac{ |s(S,B)| }{ \sum_{B' \in \mathcal{B}} |s(S,B')| }

where $\mathcal{B}$ contains the target behaviour together with a set of relevant control behaviours. A high value indicates that the causal effect of S is specific to B.

Evaluation protocol

Necessity test: Ablate S and measure the resulting degradation on behaviour B. Compare against ablating random components of matched size.
Sufficiency test: Replace components outside S with reference activations and measure how much of behaviour B is preserved. Compare against random component sets of matched size.
Interchange test: Swap activations of S between inputs representing different behaviours and test whether the model output follows the intervention.
Specificity test: Measure intervention effects on a set of non-target behaviours and examine whether the effect is specific to B.

Summary

Dimension	Causal Validity
Definition	Are the identified components necessary, sufficient, and specific for the target behaviour?
Metrics	Necessity score; sufficiency score; specificity score
Methods	Ablation; component isolation; interchange interventions; tests with control behaviours

Examples

The MI community has developed several benchmarks that examine the causal validity of interpretability methods. For example, MIB (Mueller et al., 2025) provides standardised tasks for assessing discovered circuits and causal variables (e.g., features); AxBench (Wu et al., 2025b) evaluates methods’ ability to steer models towards given concepts; RAVEL (Huang et al., 2024) investigates methods’ ability to localise features of given concepts by using interchange interventions to validate discovered features; and SAEBench (Karvonen et al., 2025) evaluates SAEs specifically, pairing reconstruction metrics with downstream intervention tasks such as gender bias erasure.

Human interpretability

Human interpretability measures whether generated explanations eventually improve human understanding, prediction, and decision-making about model behaviour.
Survey §7.5.2 Dimension 5: Human Interpretability

How it is measured

Explanation Utility

This metric tests whether humans can predict model behaviour more accurately in the held-out set based on the explanation. It is formulated as:

\text{Expl-Utility} = P(\text{correct prediction} \mid \text{with explanation}) - P(\text{correct prediction} \mid \text{no explanation})

Debugging Utility

This metric examines whether the explanations help humans identify and repair model errors, such as incorrect outputs or misaligned behaviour, in the wild.

\text{Debug-Utility} = P(\text{error fixed}\mid\text{with explanation}) - P(\text{error fixed}\mid\text{no explanation})

Consistency of Interpretation

This metric inspects whether different humans interpret the same explanation consistently.

\text{Inter-Rater} = \kappa(\text{human}_1 \text{ labels}, \text{human}_2 \text{ labels})

where $\kappa(\cdot)$ is Cohen’s kappa or a similar agreement metric.

Evaluation protocol

Prediction task: Show humans explanations of model components or mechanisms and ask them to predict model behaviour on held-out inputs. Compare accuracy to a no-explanation baseline.
Identification task: Show humans model outputs with errors and ask them to identify the cause, with and without explanation assistance. Measure accuracy and required time to complete the task.
Intervention task: Ask humans to propose fixes for model errors based on explanations. Measure the success rate of proposed interventions.
Agreement task: Ask multiple human raters to generate explanations, if applicable (e.g., write a summary of highly activated samples of an SAE feature as its explanation). Measure inter-rater reliability and agreement with machine-generated explanations.

Summary

Dimension	Human Interpretability
Definition	Can humans understand and use the resulting explanations?
Metrics	Explanation utility; debugging utility; interpretation agreement
Methods	User studies; expert evaluation

Limitations

Several caveats and challenges arise when evaluating the human interpretability of tested methods. For example, human studies are usually expensive. Moreover, human expertise may confound results, where some explanations understandable to MI researchers may be opaque to domain experts, and vice versa.

A unified evaluation framework

Overview

Applying it to each method family

The protocol, step by step

Limitations of the framework

Elsewhere in the survey

Attribution faithfulness

How it is measured

Evaluation protocol

Summary

Completeness

How it is measured

Evaluation protocol

Summary

Stability

How it is measured

Paraphrase Consistency

Evaluation protocol

Summary

Causal validity

How it is measured

Evaluation protocol

Summary

Examples

Human interpretability

How it is measured

Evaluation protocol

Summary

Limitations