Mechanistic interpretability of jailbreak attacks

Modern language models are typically aligned through supervised fine-tuning (SFT), reinforcement learning from human feedback (RLHF), or related techniques to refuse harmful or policy-violating requests. Despite these safeguards, carefully crafted jailbreak prompts can often elicit unsafe responses. While considerable work has focused on developing jailbreak attacks and measuring their success rates, much less is understood about why they work from a mechanistic perspective.

Recent work in mechanistic interpretability suggests that many high-level behaviors are represented by approximately linear directions, low-dimensional subspaces, or sparse features within model activations. Similar methods have recently been used to identify representations associated with refusal behavior in aligned models. This motivates the central hypothesis of this project: successful jailbreaks manipulate the internal representations responsible for refusal, causing models to transition from refusing to complying with harmful requests.

Understanding whether this hypothesis holds would provide a mechanistic explanation of one of the most important failure modes of aligned language models and could help inform more robust alignment methods.

We aim to investigate the internal mechanisms underlying jailbreak attacks by addressing the following questions:

• Do successful jailbreaks consistently alter representations associated with refusal?
• Is there a shared refusal direction or low-dimensional subspace across different jailbreak families?
• Do different jailbreak techniques exploit similar internal mechanisms, or do they bypass alignment in distinct ways?
• Can causal manipulation of refusal representations prevent successful jailbreak attacks?

We will first identify internal representations associated with refusal behavior in aligned language models such as Gemma, Llama, or Qwen. Using paired prompts that differ only in whether the model refuses or complies, we will analyze hidden activations across layers using techniques such as linear probes, mean-difference vectors, principal component analysis, and, where appropriate, sparse autoencoder features.

Next, we will evaluate a diverse set of jailbreak techniques, including role-playing prompts, translation-based attacks, adversarial suffixes, and automated jailbreak methods. Rather than focusing solely on behavioral success, we will compare how each attack modifies internal representations relative to standard refusal and compliant responses.

To move beyond correlation, we will perform causal interventions using activation steering. By restoring identified refusal representations during successful jailbreaks, we will test whether these interventions recover refusal behavior. Conversely, suppressing these representations during ordinary prompts will help determine whether they are causally responsible for aligned behavior.

Finally, we will compare these mechanisms across multiple models and alignment methods to investigate whether refusal representations are shared or model-specific.

This project aims to explain how jailbreak attacks succeed rather than simply whether they succeed. We expect to provide mechanistic insights into the internal computations underlying refusal and compliance, and to characterize whether diverse jailbreak techniques converge on common representation-level mechanisms.

Potential outcomes include identifying shared refusal representations manipulated by jailbreaks, demonstrating their causal role through activation steering or low-rank fine tuning, or showing that jailbreaks rely on more distributed computations than current representation engineering approaches assume. The outcomes of this project may also help understand how to improve existing activation steering methods, and will more generally contribute to a deeper understanding of alignment representations and their vulnerabilities, bridging mechanistic interpretability and AI security.

The project will contribute to a better understanding of alignment representations and their vulnerabilities, to help bridge mechanistic interpretability and AI security.

High level of autonomy, and would prefer mentees open to discuss and who propose ideas that would improve this initial research idea.

Knowledge of LLMs/transformers, safety alignment, low-rank fine tuning, jailbreaks and adversarial attacks. Fluent in python and pytorch.