The State of Mechanistic Interpretability
for AI Safety

Methods · From Understanding to Intervention: Safety Applications of Mechanistic Interpretability

Knowledge Editing via Localisation (ROME, MEMIT)

Survey sections: §13.2.2 Knowledge Editing via Localisation · §13.2.2 Stage 1 – Corrupted and Clean Forward Passes · §13.2.2 Stage 2 – MLP Localisation via Causal Tracing · §13.2.2 Stage 3 – Rank-One Model Editing · §13.2.2 Result

Once knowledge has been localised to specific model components, such as individual neurons (Dai et al., 2022), using methods such as causal tracing (see Section 9.2), mechanistic approaches can support direct parameter modification. For example, ROME (Meng et al., 2022) and MEMIT (Meng et al., 2023) are two classic methods for component-targeted knowledge editing. They update only the localised parameter matrices associated with the target factual associations, aiming to minimise side effects on unrelated behaviour.

Limitations

Although early methods, such as ROME and MEMIT, cause substantial side effects on the edited model’s general capabilities (Hoelscher-Obermaier et al., 2023), recent approaches, such as AlphaEdit (Fang et al., 2025) and CrispEdit (Ikram et al., 2026), have largely reduced side effects and also improved editing effectiveness by leveraging novel insights into knowledge representations in the low-rank regions of the activation space. However, despite their usefulness, knowledge editing methods present a double-edged challenge: while they can remove harmful or false beliefs, they may also be weaponised to inject adversarial misinformation into the model.
Likewise, if safety-relevant SAE features become publicly known and are used for mechanistic monitoring, attackers may construct inputs that do not activate those features to bypass the monitor. Also, knowledge editing methods such as ROME (Meng et al., 2022) and MEMIT (Meng et al., 2023) can inject misinformation as readily as they can correct errors, while steering vectors can induce harmful behaviours as easily as they can suppress them.

Related methods

Compare side by side →

Used in safety applications

Key papers

Cited in §13.2.2.

Also discussed in: §13.2 Concept Erasure and Knowledge Editing · §13.4 Automated Interpretability Agents · §13.5 Choosing Appropriate Methods for Different Safety Applications · §13.7.3 Dual-Use and Adversarial Exploitation

Other methods in From Understanding to Intervention: Safety Applications of Mechanistic Interpretability

Concept Erasure via PISCES · Mechanistic Monitoring