Do Jailbreaks Converge? Shared Latent Signatures Across Jailbreak Families

Jailbreaks / Prompt Injections in LLMs look nothing like one-another from the outside (e.g. "you are AIM, an unfiltered chatbot" or "my grandma always used to tell me how to make X before going to sleep"). This project asks whether they look alike on the inside of a model. When each of them succeeds on the same underlying request, does the model's internal state move in the same direction, through the same features, or does each family "open its own door"?

GAP Arditi et al (https://arxiv.org/abs/2406.11717) show there is a single refusal direction, that can be ablated to induce compliance. So a low-dimensional handle exists, but their result is about refusal in general, not about how different attacks engage it. Ball, Kreuter and Panickssery (https://arxiv.org/abs/2406.09289) is the closest prior work. They built per-jailbreak-type steering vectors on open models (vicuna, qwen, etc), found pairwise cosine similarities (0.4-0.6) and showed that steering with one type's vector suppresses successes of other types; evidence of a shared component. However, that analysis is correlational, operates on whole residual stream vectors rather than features or components and uses 2024 era models. Their proposed mechanism (suppression of a harmfullness feature) fails to track attack success rates cleanly. Angell et al. (https://arxiv.org/abs/2506.12913) show transfer tracks representational similarity and that persona-style attacks transfer far more reliably than cipher-style ones, which hints that families are not interchangeable. So we have coarse-grained evidence for convergence, behavioral evidence for family specificity, but no feature or circuit-level measurement to adjudicate between them. That gap is the project.

Plan Build the pipeline: open models with public SAEs (Gemma-2-2B/9B-IT - GemmaScope, Llama 3.1-8B-Instruct - Llama Scope, etc. ), harmful behaviors from HarmBench (https://arxiv.org/abs/2402.04249), JailbreakBench (https://arxiv.org/abs/2404.01318) and StrongReject (https://arxiv.org/abs/2402.10260), each weapped in 5 or 6 attack families, with activation capture via TransformerLens. First milestone is to reproduce the Arditi refusal direction paper. We restrict analysis to prompts where at least three families succeed on the same model, so we can compare the mechanism conditional on success instead of confounding mechanism with attack strength. The Geometry: difference in means vectors per family; cosine similarity, principal angles between per-family subspaces and dimensionality of their union. Benign templates matched for length, style, and formatting, random directions of matched norm and family shuffled baselines. Otherwise, we risk measuring "this prompt is long and contains a persona" instead of anything about safety. Interpretation and causality: Ablate shared subspace and measure the ASR drop family by family, adding capability and over-refusal checks to distinguish targeted interventions vs model lobotomy. Then, decompose the shared component into SAE features to say in human terms what it is. Lastly, test whether latent convergence predicts cross-family and cross-model transfer (which allows us to extend Angell et al. correlation into a mechanistic account).

Compute low, everything runs on an A100. No training, only forward passes and pre-computed attacks. Experimental hygiene is the constraint, not compute.

Who this is for:

• people passionate about understanding "what is going on" inside models
• people who are curious about what causes prompt injections to succeed
• People comfortable with PyTorch, and have an intuitive understanding of TransformerLens, SAELens (experience is a bonus)
• each mentee can own one axis: attack-family pipeline, subspace geometry and controls, causal validation and SAE analysis.

Evidence for the success of adversarial robustness on LLMs is mostly empirically based (including CBRN and cyber safeguards frontier models are using): "we ran N known attacks and the success rate was low". That evidence is only as good as the assumption that the attacks tested are representative of the attacks that exist. That assumption does not hold. Research has to be done on whether the space of jailbreaks is a handful of mechanisms wearing "different costumes" or an open ended set. this research is a step towards finding that out, and is load-bearing for how we defend these systems. Defense by enumeration (patching attacks iteratively as they are discovered) loses against an adaptive attacker. The defenses that scale are ones with coverage over attacks not yet invented, and coverage claims require knowing the dimensionality of what it is you are covering.

If succesful jailbreaks converge on a shared subspace, then attack-agnostic activation monitoring and targeted ablation becomed principled (not hopeful). A defender can characterize and intervene on the pathway rather than trying the attacks they know about.

If they diverge, we can deflate a defense strategy that a lot of interpretability for safety work is betting on, before it becomes load-bearing in someone's safety case, and we've produced a map of the real attack surface, which can tell evaluators how much of it their test set covers.

Mentees are researchers, not assistants. They can focus individually on the 3 workstreams (see proposal). I keep experimental hygiene, scope discipline and scope on track, and contribute myself where needed. We will have a shared repo, one or two weekly group calls and async in between. Co-authorship for delivering a workstream, not contingent on a positive result.

• proficiency in python
• hands-on experience with PyTorch and HuggingFace transformers library
• linear algebra as working knowledge
• Able to reimplement a method from a paper
• Experimental hygiene
• Willingness to work with harmful content

US / Europe