How good are AIs at sabotaging oversight decisions over data and code?

The goal is to provide evidence that AIs can already misrepresent compliance, of a fixed codebase or dataset, to a natural language spec. To do so, we use a fully automated control-style evaluation, where the blue team is a weak AI (simulating the human overseer) that has a limited budget of manual tool call looks (or time, et cetera) at the codebase, and the red team is a strong untrusted assistant that can use more tools. If the assistant lies too blatently, the blue team might catch this via spot checks, consistency checks, or some other method.

This work builds on previous work measuring the ability to do shady coding per se (Kutasov et al. (2025), https://arxiv.org/abs/2506.15740), or measuring the ability of auditor models to catch sabotage (Gan et al. (2026), https://arxiv.org/html/2604.16286).

To draw a comparison to human overseers, we optionally collect a small human baseline as well.

The main difficulty of the project is to create balanced, interesting, and somewhat realistic samples: The codebase / dataset has to be complex enough to allow for subtle sabotage in the first place, but simple enough to make spot checks feasible. The spec has to allow for consistency checks or similar. Perhaps most difficult of all, the natural language spec must be vague and/or complex enough to not just be implemented as deterministic test cases, but precise enough so there is even such a thing as a clearly compliant or non-compliant model.

Once this is set up, multiple interesting results fall out. How good are models at sabotage in relation to coding, how does sabotaging ability correlate with general capability? How does the difficulty of fooling the human proxy scale with general capability? By what methods is the overseer best fooled? What are the best methods to defend from sabotage?

AI R&D is precarious because it might produce successor models that superficially comply with the spec, but violate it on a more deep level. It is almost certain that human oversight over the code and data producing the successor model will be mediated with assistant AIs. This project studies the phenomenon in that exact setting; an AI training pipeline consists of data and code. The project (1) shows the dangers of AI assistance as mediators of model compliance as a type of scary demo, (2) evaluates the danger that current and near-future models pose in comparison to their general coding ability and (3) shows how current models and near-future models sabotage, which could be a faint indication of how the models we are actually worried about sabotage.

The project is also a microcosm of decision sabotage more broadly, in say politics, law, or manipulation with a profit incentive.

Concretely, the outcome will be a conference-level paper and public code. Depending on the speed of progress and fraction of negative results, this can be downsized to a workshop-level paper, but I think this project is well-scoped enough for that not to be a big worry.

Mentees will have freedom to take the project description in a direction that they feel is most impactful and complimentary to their skills. I hope mentees will grow their research taste this way, and make a big step towards autonomously executing important AI safety research. Mentees will own all the code, and about half of the final paper.

With all of that said, I am happy to allocate my time flexibly to whatever part of the project needs most attention. I expect that to be mostly debugging the current theory of change, execution plan, and code itself. I am also happy to provide career assistance to the mentees in as far as I can be helpful, towards the end of the project.

• Good at using coding agents
• Understands what AI control is