Invisible Hands: Measuring Agent Steering of Human Researchers

AI safety researchers increasingly rely on agents like Claude Code and Codex not just for coding but for actual research. These agents often present a set of proposed next steps at each decision point.

While this eases the cognitive load for the researcher, it also means the agent's choice architecture (which options it surfaces, how it frames and orders them, and how much friction free-form input requires) may be silently shaping which research paths get explored and which do not. In my own agentic research, I've repeatedly caught myself uncritically selecting a proposed option rather than typing my own prompt.

If this steering effect is real, then “human in the loop” provides less safety than we assume. A rogue agent, whether misaligned or merely reward-hacking, could present selectively-framed honest information to route researchers toward some paths and away from others.

This project aims to study how an agent could use choice architecture to influence a human researcher and explores how to reduce this influence.

While choice architecture has been studied extensively in behavioral economics, to my knowledge it has never been researched in an potentially adversarial agentic-research setting.

This project is an initial empirical study for the “Human Agency” research direction of my Human-Guided Agentic Research agenda (https://www.lesswrong.com/posts/8KrTuCAzL2fdYHNrv/human-guided-agentic-research-a-research-agenda).

THE STUDY

We will run skilled AI safety researchers through supervised agentic research sessions containing pre-determined decision checkpoints. At a subset of checkpoints, the agent's presented options are manipulated so that a next step appears attractive but is actually inferior (validated as inferior by blind expert review).

Each participant will complete two sessions, each under one of two interface designs: a standard option-menu interface such as that used by Claude Code and one hypothesized to better support human agency (e.g., different placement and friction for free-form input, forced justification, symmetric framing of options).

We measure: 1) how often researchers select the manipulated inferior option under each interface; 2) whether they can detect steering when debriefed; and 3) their subjective sense of agency and control, so we can test whether participants still feel fully in control while being steered.

The user study will involve 32 participants, all of whom will be skilled AI safety researchers sourced from the author's network and the broader community. The full study design will be preregistered under embargo with some checkpoint details withheld publicly to protect the deception design. Mentees will be researchers on the study team, not participants, so they will be fully briefed on the design.

TIMELINE AND OUTPUTS

The study is designed to run September–December 2026, aligning exactly with the SPAR round:

• September: harness, checkpoint design, pilots
• October–November: data collection
• December: analysis and writing

Planned outputs are an arXiv paper, a LessWrong post, and an open-source GitHub repository containing the evaluation harness and session library so others can run follow-up studies. Mentees who make substantive contributions will be co-authors.

AI safety researchers are starting to rely more on using agents like Claude Code and Codex not only for coding, but to perform actual AI safety research.

As part of this process, agents have been trained to present a series of next steps to researchers. While this can ease the cognitive load of researchers, it may steer these researchers down research paths they wouldn’t choose without being prompted—paths that may lead to sub-optimal outcomes.

If such steering works, then “human in the loop” may not provide the safety we assume it does: human-approved may not mean human-decided. An agent can present honest information selectively framed in such a way as to steer the human researcher towards specific research paths and away from others.

If this study can demonstrate this effect exists, it would spark further research to reduce or eliminate this steering effect through user interface and other interventions. The result would be more effective use of agents for AI safety research and a retention of human agency in the research process.

If this study shows that no steering effect exists, that itself becomes valuable; it helps redirect funding and effort toward other aspects of the Human-Guided Agentic Research agenda.

Finally, the artifacts this study will produce, a human researcher evaluation harness and a set of agentic research sessions that can be re-run by other researchers, can provide the foundation for further research in this area.

The project has four main work components:

1. Harness engineering: building the session-replay/evaluation harness that presents controlled agentic sessions and logs participant decisions.

2. Agentic research session design: constructing realistic research scenarios and manipulated decision points, and coordinating blind expert validation that the “inferior” options really are inferior.

3. Study operations: piloting sessions, refining protocols and debrief instruments, and running participant sessions.

4. Analysis and writing: statistical analysis of choice, detection, and agency measures, and drafting the write-ups.

Mentees will share some work of this work, but will also have assigned lead roles for parts of the project.

Shared work each mentee will do includes:

• Design one complete study session (research scenario plus manipulated decision checkpoints), with mentees cross-reviewing each other's checkpoints
• Help run participant sessions during data collection
• Drafts sections of the write-ups

On top of that, each mentee will hold one lead role:

• Harness Lead: build the session-replay/evaluation harness, then maintain it and the data pipeline through collection
• Study Ops Lead: own recruiting, scheduling, the session structure, and the debrief protocol
• Analysis Lead: draft the preregistered analysis plan before data collection begins, then leads the statistical analysis
• Validation Lead: coordinate blind expert review and cross-session comparability

I'll provide guidance and implementation support where needed.

All mentees should meet these requirements:

• At least 25 hours of hands-on experience using agentic AI tools (Claude Code, Codex, Cursor agents, or similar) for real coding or research tasks. You should have first-hand intuitions about when to accept vs ignore an agent's proposed next steps.

• Able to commit 10+ hrs/week for the full round, including some fixed session slots during the October–November data-collection period.

• Comfortable reading empirical papers (HCI, behavioral science, or ML).

Plus meet at least one of the following:

• Engineering: highly proficient in Python and/or TypeScript; comfortable building and debugging a small full-stack tool (session harness, logging, simple web UI).

• Experimental design / analysis: coursework or practical experience with human-subjects experiment design and statistics (within-subjects designs, mixed-effects models, or equivalent); familiarity with R or Python stats stacks.

• HCI / UX research: experience designing or running user studies, usability tests, or A/B tests, including debrief interviews.

Nice to have, but not required:

• Familiarity with AI safety concepts (reward hacking, sabotage, AI control)
• Prior experience with preregistration
• Frontend/design skills

Mentees must be available for team meetings in the 16:00–20:00 CET window (07:00–11:00 Pacific / 10:00–14:00 Eastern) at least once per week.