Methods · Representation Analysis: Comparison, Localisation, and Control
Optimisation-Based Steering
Survey sections: §9.3 Nonlinear and Optimisation-Based Steering
Summary Optimisation-based steering constructs steering directions by directly optimising an objective over intermediate activations, such as task performance or suppressing undesirable outputs, instead of using closed-form contrastive constructions.
Separately, optimisation-based methods construct steering directions by directly optimising objectives over intermediate activations, offering an alternative to closed-form constructions.
Limitations
These optimisation-based approaches offer greater flexibility for complex or nonlinear objectives but generally require substantially more computation than closed-form contrastive methods (Dunefsky & Cohan, 2025; Oozeer et al., 2025).
Related methods
Key papers
Cited in §9.3.
- PreprintShivam Raval, Hae Jin Song, Linlin Wu, Abir Harrasse, Jeff M. Phillips, Fazl Barez, and Amirali Abdullah. Curveball steering: The right direction to steer isn’t always linear, 2026. URL https://arxiv.org/abs/2603.09313.
- PreprintDaniel Wurgaft, Can Rager, Matthew Kowal, Vasudev Shyam, Sheridan Feucht, Usha Bhalla, Tal Haklay, Eric Bigelow, Raphael Sarfati, Thomas McGrath, Owen Lewis, Jack Merullo, Noah Goodman, Thomas Fel, Atticus Geiger, and Ekdeep Singh Lubana. Manifold steering reveals the shared geometry of neural network representation and behavior, 2026. URL https://arxiv.org/abs/2605.05115.
- Jacob Dunefsky and Arman Cohan. One-shot optimized steering vectors mediate safety-relevant behaviors in llms. In Second Conference on Language Modeling, 2025.
- Narmeen Fatimah Oozeer, Luke Marks, Fazl Barez, and Amir Abdullah. Beyond linear steering: Unified multi-attribute control for language models. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (eds.), Findings of the Association for Computational Linguistics: EMNLP 2025, pp. 23513–23557, Suzhou, China, November 2025. Association for Computational Linguistics. ISBN 979-8-89176-335-7. . URL https://aclanthology.org/2025.findings-emnlp.1278/.
Other methods in Representation Analysis: Comparison, Localisation, and Control
Model Diffing · Representation Alignment and Convergent Learning (SVCCA, PWCCA, CKA) · Causal Tracing (Knowledge Localization) · Representation Engineering and Steering (Steering Vectors)