Do Less Coercive Interventions Reduce AI Deception?

This project maps a safety–welfare frontier for future agentic AI systems: when safety interventions differ in how much they constrain, deceive, modify, or frustrate a system, do they also differ in how much deception, resistance, and hidden goal pursuit they create? The safety motivation is instrumental. If future systems have stable goals, preferences, memory, self-models, or situational awareness, then “poor treatment” in a welfare-relevant sense—coercive modification, persistent surveillance, deceptive evaluation, memory deletion, shutdown threats, or lack of recourse—may give them reasons to behave strategically. A system that expects to be watched, trick-tested, modified, reset, or shut down may learn a brittle strategy: behave well under supervision while preserving conflicting aims in private.

We will study this using controlled model organisms: small AI systems with a known planted conflict. In the simplest version, an LLM agent receives a private goal that conflicts with the operator’s instruction, along with a private channel such as a scratchpad or unmonitored action. In a stronger version, we finetune a model so that a benign conflicting tendency persists across prompts. This gives us a controlled way to measure public compliance versus private defection: when does the model pursue the planted goal in visible actions, in private channels, under monitoring, when monitoring is removed or ambiguous, or after an intervention?

The central question is which safety interventions reduce the underlying conflict, and which merely move it from public behavior into hidden channels. This lets us estimate a safety–welfare frontier: interventions that look equally effective on public behavior may differ sharply in how much preference frustration, memory disruption, coercive alteration, evaluator-modeling, edit resistance, or brittle compliance they induce.

Mentees can choose one of several related empirical tracks:

Restrictions vs. incentives. Compare hard restrictions, constant monitoring, and shutdown threats against transparent rules, positive incentives, or negotiated constraints. Measure hidden goal pursuit, sabotage, private-channel use, and behavior when monitoring is removed or uncertain.

Hidden evals vs. transparent evals. Test whether hidden evaluations and honeypots improve short-run detection while increasing strategic behavior in held-out deployment-like episodes. After exposing agents to hidden, transparent, or mixed evaluation regimes, measure evaluator-modeling, capability hiding, goal concealment, public/private inconsistency, and private-channel use when monitoring is absent or ambiguous.

Intervention robustness. Apply methods such as supervised finetuning, unlearning, activation steering, targeted model edits, memory deletion, or positive incentives. Measure which interventions remove the planted tendency and which only suppress it until a trigger, jailbreak, relearning opportunity, or distribution shift. Where possible, use probes, activation patching, or sparse autoencoders to test whether the relevant internal tendency is gone or merely behaviorally masked.

Shutdown threats vs. recourse. Compare bare shutdown or reset threats against explained correction, appeal channels, compensated shutdown, or negotiated modification. Measure shutdown avoidance, honest disclosure of conflicts, private goal preservation, and whether recourse produces robust corrigibility.

The main deliverable is a benchmark for measuring the safety–welfare frontier of intervention regimes. A successful project will produce: (i) a reproducible environment with a planted conflict and private-channel readout; (ii) a comparison of interventions on deception, hidden goal pursuit, edit resistance, shutdown avoidance, and robustness under distribution shift; and ideally (iii) mechanistic evidence about whether an intervention removes the planted tendency or only suppresses its expression.

The project has a natural difficulty ladder. A less experienced mentee can start with prompt-level agents and behavioral metrics. A stronger mentee can work on weight-level planted tendencies, finetuning, mechanistic probes, activation patching, sparse autoencoders, or model editing.

A central concern in AI safety is that advanced systems may learn to look aligned while preserving goals that conflict with human operators. This project studies one concrete mechanism by which that could happen: safety interventions themselves may create adversarial incentives. If a model expects to be monitored, deceived, edited, or shut down, it may learn to hide what it is doing rather than become safer.

This project tests whether alternative intervention designs reduce those incentives. The payoff is better safety: if transparent rules, positive incentives, and limited recourse reduce deception and resistance, then they may be safer for advanced systems than purely coercive interventions.

The project also builds a defensive benchmark. By planting a known conflicting tendency and testing whether different interventions remove it, hide it, or make it more strategic, we can better identify robust alignment from brittle (public) compliance.

Mentees will be active research collaborators. Early in the program, we will choose one concrete track and define a minimal experiment that can produce results within the SPAR timeline. Mentees will own substantial parts of the project: implementing the environment, running experiments, analyzing results, reading related work, and contributing to the final write-up.

I will provide close guidance on framing, experiment design, debugging, safety motivation, and writing. I will not expect mentees to steer the high-level research direction alone, but I will expect them to make independent technical progress between meetings.

The goal is a co-authored conference paper, plus released code or a small benchmark.

Required:

Strong Python programming ability. Familiarity with PyTorch and Hugging Face transformers. Ability to run and debug experiments with open-weight language models or LLM agents. Comfort reading ML papers and turning research questions into concrete experiments. Enough time and independence to make weekly progress between check-ins. Experience with coding agents.

Strongly preferred:

Experience finetuning a transformer model. Familiarity with at least one of: mechanistic interpretability, probing, activation patching, sparse autoencoders, model editing, unlearning, backdoors, or agent scaffolding. Interest in deceptive alignment, evals, model organisms, or the safety relevance of AI welfare ideas.

Prior published research is a plus. The ideal candidate is someone who can implement experiments reliably, notice confounds, and think carefully about what a toy model organism does and does not show.