Methods · From Understanding to Intervention: Safety Applications of Mechanistic Interpretability
Concept Erasure via PISCES
Survey sections: §13.2.1 Concept Erasure via PISCES · §13.2.1 Stage 1 – Feature Disentanglement and Identification · §13.2.1 Stage 2 – MLP Vector and Feature Selection · §13.2.1 Stage 3 – Concept Removal · §13.2.1 Result
Summary PISCES (Precise In-parameter Suppression for Concept EraSure) is a concept-erasure method that edits model parameters in three stages, aiming to remove a target concept rather than only suppress its expression, as conventional unlearning techniques do.
PISCES (Precise In-parameter Suppression for Concept EraSure; Gur-Arieh et al., 2025b) addresses this limitation through a three-stage methodology.
Assumptions and requirements
Use sparse autoencoders (SAEs) with feature explanations (via the Automated Interpretability pipeline) to disentangle and identify features associated with the target concept c (e.g., “biological toxins”)
Evaluation considerations
PISCES surgically reduces the accuracy on prohibited concepts to near-chance levels while maintaining general capabilities, showing significantly higher specificity than baseline methods.
Intended applications
Unlike approaches that primarily suppress unwanted outputs, PISCES directly modifies model parameters associated with the target concept.
Limitations
Likewise, concept-erasure methods such as PISCES may leave residual capabilities partially accessible through certain input prompts without explicit signals of failure.
Related methods
Used in safety applications
Key papers
Cited in §13.2.1.
- Michelle Lo, Fazl Barez, and Shay B. Cohen. Large language models relearn removed concepts. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Computational Linguistics: ACL 2024, pp. 8306–8323, Bangkok, Thailand, August 2024. Association for Computational Linguistics. . URL https://aclanthology.org/2024.findings-acl.492/.
- Yoav Gur-Arieh, Clara Haya Suslik, Yihuai Hong, Fazl Barez, and Mor Geva. Precise in-parameter concept erasure in large language models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 18986–19006, Suzhou, China, November 2025b. Association for Computational Linguistics.
Also discussed in: §13.2 Concept Erasure and Knowledge Editing · §13.7.2 Silent Failure Modes · §13.5 Choosing Appropriate Methods for Different Safety Applications
Other methods in From Understanding to Intervention: Safety Applications of Mechanistic Interpretability
Knowledge Editing via Localisation (ROME, MEMIT) · Mechanistic Monitoring