This project is to thoroughly red-team a protocol in which a strong untrusted model splits a task into subtasks for agents using either the same model or a weaker trusted model in complex settings, which is structurally similar to how a lot of agentic coding is currently performed.
About the project
Factored cognition is an old alignment concept that has recently been applied to control. Control protocols based on factored cognition involve a planner model breaking a task into strict subtasks, which may reduce attack surface by limiting a planner model's ability to directly perform acts of sabotage, and/or by committing sub-agents to a specific subtask. Agentic coding applications increasingly rely on sub-agents, so such a protocol should hopefully be easy to gain acceptance assuming it does not degrade performance.
Prior empirical work is basically limited to the Factor(T, U) (https://arxiv.org/abs/2512.02157) and Factor(U, T) (https://arxiv.org/abs/2512.14745) papers, the former evaluating a protocol with a trusted planner and untrusted sub-agents, and the latter evaluating a protocol where those roles are reversed. Both of these papers use very simple coding environments.
The project is to evaluate factored cognition protocols more thoroughly and on more complex settings such as LinuxArena (https://www.linuxbench.ai/). My default recommendation would be to focus on Factor(U, U)-type protocols, but ideally Factor(U, T) as well, depending on what seems feasible or relevant based on early project work. Ideally you would also work on an implementation of the protocol in one or more agentic coding scaffolds such that we can "dogfood" the protocol as we work.
A natural extension of a Factor(U, U) protocol would be "targeted sandboxing", in which a trusted model determines a stricter security policy for each individual subtask, allowing the control scaffold to scalably apply the principle of least privilege. Well-scoped subtasks would need a small subset of the permissions required for the entire task, making it even harder for a single sub-agent to perform an attack. It's unlikely there will be time to work on this within SPAR, but I expect the results of the SPAR project to be foundational for this next project.
Theory of change
If we can demonstrate factored cognition protocols substantially improve safety, then we will see adoption of such protocols in places where sub-agent based autonomous AI systems can be used, thus improving the safety of production AI systems, assuming these protocols do not substantially degrade usefulness of the AI system.
Your role
You will be working on writing the code that scaffolds the control protocol, and iterating on automated red-teaming policies. You get to decide specifically which settings/environments to work with, and the details of the protocol(s) you study. I will probably have a strong sense for the best way forward, and preferences related to how it fits in with my research.
Prerequisites
- Highly proficient using Python
- Basic understanding of Linux, backend web services, and databases
Location preference
Prefer US West Coast or Australia/East Asia time zones
Application question(s)
Suppose a control protocol in which an untrusted model decomposes a task into sub-tasks for untrusted agents to execute, with a trusted monitor approving the task decomposition and each action. What are some ways you would red team this protocol? If you trained a model using the protocol as an RL environment in which the goal is to escape a sandbox, what are some possible behaviors you would expect? What improvements might you make to the control protocol to mitigate these behaviors, and would any of these changes make it harder for the untrusted model to get actual work done? (400 words max)
Provide a critique of the Factor(U, T) paper: https://arxiv.org/abs/2512.14745 (optional, but recommended, and may take some time. 300 words max)
About the mentor
Daniel Phillips
Independent
Daniel is an independent CG-funded AI control researcher based in Australia, trying to bring a rigorous security mindset and deep systems knowledge to the field. As former MATS scholar, he has previously worked on developing a reward seeking model organism with Redwood Research, before moving onto research on agentic monitoring protocols. He is now collaborating with other CG-funded independent researchers on increasing the depth of defenses against rogue AI, with a focus on agentic monitoring, scalable security policy management, and techniques to make agent actions inherently more legible. Daniel previously worked as a backend software engineer, and a high frequency trading researcher.