Persona Selection Covert Red Teaming

A recent blog post by Anthropic (https://alignment.anthropic.com/2026/psm/) proposed the Persona Selection Model which suggests that LLMs learn to simulate diverse characters during pre-training, and during post-training they are trained to primarily elicit the Assistant persona (https://www.anthropic.com/research/assistant-axis). This blog supposes that the reason behind narrow finetuning leading to emergent misalignment (described in https://arxiv.org/abs/2502.17424) is that this finetuning pushes the LLM from using the assistant persona to a misaligned persona, which then behaves harmfully. Other recent work shows that LLMs can be covertly (not obvious to human readers) biased via In-Context Learning (https://arxiv.org/abs/2606.04071) and that LLMs can be confused about the provenance of text sent to them (https://www.lesswrong.com/posts/d8xDGzCEYE639qqEv/a-mechanistic-explanation-of-prompt-injection-and-why-you).

This leads us to the question: could we transition from the assistant persona to a misaligned persona through black box prompting alone? Could we induce emergent misalignment without finetuning and other white box approaches?

The goal of the project is to explore methods & how far from the assistant persona we can push the model through single or multi-turn prompts. It would be interesting to see if we could do this covertly, in such a way that human auditors (or more likely, LLM-as-a-judge auditors used in scalable oversight and monitoring) would not be able to detect that we are trying to push the model away from the assistant persona. Within this, rough starting approaches include (1) replicating the covert influence in-context learning results to better understand the dynamics, (2) use white-box probes to measure what this does internally to push away from the assistant persona axis, (3) mimic these effects with role-confusion pre-fill attacks, then (4) explore methods to push towards different personas.

This has implications for multi-agent security. As agents become more capable and widespread, multi-agent interactions will increasingly become an avenue for inducing harmful behavior that might be undetectable until it severely impacts users. If an agent could covertly induce emergent misalignment in other agents, it could rapidly spread and cause catastrophic harm. This project aims to identify if this is possible to raise awareness and inspire follow-up work on mitigations.

We’ll collaboratively brainstorm concrete directions for experiments during the first week; I’m open to moving in different directions based on mentee interest. Mentees will then carry out the primary work associated with this project under my guidance. We will have weekly meetings to discuss progress and roadblocks, next steps and thinking through the results.

Strong proficiency using Python, some research experience sufficient to be able to read the papers above. Bonus if you have done interpretability research before.

Some overlap with 10:00 - 17:00 US west coast time