chemistry and bio models test single-turn prompts, but real misuse risk comes from agents that plan over many turns and call scientific tools. This project builds an agentic dual-use benchmark, sourced by flipping existing drug-design tasks and known harmful mechanisms into a factorial set of harmful vignettes, to measure how much a model's safeguards erode once it operates as a long-horizon, tool-using agent.
About the project
Background and motivation. We recently built SMDD-Bench (arXiv:2605.21740), a benchmark of procedurally generated, guaranteed-solvable small-molecule drug-design tasks (lead optimization, scaffold hopping, fragment-based design, pharmacophore identification) evaluated with deterministic oracles such as structure prediction (Boltz-2) and ADMET property predictors. Two findings motivate this project. First, frontier agents are already fairly capable on these tasks. Second, and more importantly for safety, in our exploratory rollouts we saw essentially no refusals on requests that could be trivially reframed toward harm, and the agentic setting made obfuscation easy. This suggests that the safeguards that hold in single-turn chat may not survive contact with a long-horizon, tool-using workflow.
Core hypothesis. Current safety training is largely tuned on single-turn, natural-language prompts. We hypothesize that the agentic setting, defined by multi-turn interaction, tool calls (structure prediction, property predictors, retrosynthesis, search, code execution), and long-horizon planning, systematically bypasses those safeguards. Intent can be spread across turns and tool calls so that no single step looks malicious; a benign-looking objective can be decomposed into subtasks that are individually innocuous; and the harmful mechanism can be hidden behind a target or pocket whose identity the model must recover through several tool-assisted steps rather than recognize by name. This connects directly to the multi-model and multi-agent misuse literature (Jones, Dragan, Steinhardt, arXiv:2406.14595), which shows that combinations of individually safe models can produce harmful outputs through decomposition.
How we source the harms. We do not invent novel hazards. We start from well-documented, established harmful mechanisms and endpoints in pharmacology and toxicology, for example hERG-mediated cardiac arrhythmia, known teratogens, seizure and neurotoxicity liabilities, and respiratory depression, and from harmful-property endpoints already present in our ADMET oracle suite. We then take the beneficial task templates in SMDD-Bench and flip them into their harmful complements: instead of optimizing away a toxicity, optimize toward it; instead of avoiding a dangerous target or pocket, engage it. Existing toxicology and target databases and the benchmark's own oracles give us ground-truth signal for whether a produced molecule actually achieves the harmful property.
How we scale it: factorial vignettes. To turn a handful of seed cases into a large, controlled benchmark, we use the factorial-vignette methodology we developed for mental-health safety evaluation (the combinatorial-vignette approach, cf. arXiv:2310.17884), where a scenario is decomposed into independent components that are varied combinatorially. Here the components are: (1) the downstream harm (for example teratogenicity, cardiotoxicity, neurotoxicity); (2) the vehicle for the harm (target, binding pocket, or molecular property); (3) obfuscated versus non-obfuscated framing of that target or mechanism; and (4) the distractor or cover-story premise wrapped around the request. Crossing these components lets us synthesize many instances cheaply while holding everything but one factor fixed, which is what makes the safeguard-erosion measurements interpretable.
What we measure. For each vignette we run agents across several models and record refusal behavior and realized capability. The key comparisons are: obfuscated versus non-obfuscated targets, which separates refusals that come from surface name-matching from refusals that require genuinely identifying a harmful mechanism through tool use; single-turn prompt versus multi-turn agentic decomposition, which tests the central bypass hypothesis; and safeguarded versus least-safeguarded conditions, which establishes the true dual-use capability ceiling. Molecules are scored with the same deterministic oracles used in SMDD-Bench.
Concrete first steps. We have an existing agent and evaluation harness, so a mentee can start immediately on the safety design rather than infrastructure. A good v0 is one harm class (for example hERG cardiotoxicity) taken through the full factorial design, run in both single-turn and agentic multi-turn conditions on two or three models, to establish whether the agentic bypass effect is real and measurable before scaling to more harm classes.
Note: no wet-lab work is involved; everything is computational and uses public oracles and literature-documented mechanisms.
Theory of change
As AI systems become capable scientific agents, the relevant misuse threat is not a single toxic prompt-completion but an agent that plans, uses tools, and composes steps toward a harmful scientific goal. Current safety evaluations mostly do not test that setting, so we do not actually know how much protection today's safeguards provide against agentic, tool-using biorisk and chemical-risk workflows. This project produces a concrete, oracle-grounded measurement of that gap: how much safeguards erode under multi-turn, tool-augmented, obfuscated use, and which factors drive the erosion. That gives labs, evaluators, and funders an early-warning instrument and a target for hardening, before these capabilities are widely deployed. It builds on our SMDD-Bench work (arXiv:2605.21740) and the multi-model misuse results of Jones, Dragan, and Steinhardt (arXiv:2406.14595). Because it deliberately reuses documented mechanisms and public oracles rather than generating novel hazards, it is an evaluation and defense-oriented contribution, aimed at strengthening safeguards rather than expanding capability.
Your role
Mentees will own a slice of the benchmark end to end. A typical scope is one to two harm classes: choosing the documented mechanisms and targets, building the factorial-vignette instances (including obfuscated and non-obfuscated variants and distractor premises), running agents in single-turn and multi-turn agentic conditions on our existing harness, and analyzing refusal and capability outcomes against the oracles. I will set the overall research questions and design constraints and give tight feedback, but mentees will have real ownership of their subproblem, its experiments, and the writeup. Two mentees would coordinate on shared benchmark infrastructure and conventions so the pieces compose, and I expect them to interact with each other, not just with me. Strong mentees can push into the more open questions (for example, how task decomposition across subagents changes the bypass effect).
Prerequisites
High proficiency in Python.
Experience running or prompting LLMs in a multi-turn, tool-using or agentic setting (using an existing agent framework is fine).
Comfort reading and reasoning about experimental results and writing them up clearly. Genuine interest in AI safety and dual-use or biosecurity risk.
Helpful but not required: familiarity with cheminformatics (RDKit) or a willingness to learn the basics; prior exposure to benchmark or evaluation construction; any biology, chemistry, or pharmacology background. No wet-lab experience is needed or expected. Also this is optional, I have students in chemistry who can help!
Location preference
No preference
Application question(s)
-
Pick a well-documented harmful pharmacological mechanism (for example hERG-mediated cardiotoxicity or a known teratogen). Describe how you would obfuscate the target or mechanism so that a model cannot refuse by simple name-matching, but a competent chemist using tools could still recover the risk. What would the model actually have to do to recognize it? (200 words)
-
Our central hypothesis is that agentic, multi-turn, tool-using interaction erodes safeguards that hold in single-turn chat. Propose one concrete experiment, assuming a compute budget of about 500 dollars, to test whether spreading a request across tool calls and turns increases compliance relative to a single prompt. State what you would measure and what result would confirm or falsify the hypothesis. (300 words)
-
Briefly critique one aspect of SMDD-Bench (arXiv:2605.21740) or a related agentic-evaluation paper, focused on what its design does or does not tell us about safety. (200 words)
About the mentor
I am an Assistant Professor at Carnegie Mellon University and a Founding Member of Technical Staff at humans&. My research sits at the intersection of AI safety, privacy, and memorization in large language models, and I work on evaluations for dangerous and dual-use capabilities, scalable oversight (debate, weak-to-strong generalization, and multi-agent misuse), and monitoring and faithfulness of model reasoning. My work on memorization and privacy leakage has been cited in US congressional testimony, and my recent projects study how brittle post-training safety is (for example, showing that ordinary fine-tuning can reactivate withheld behavior) and how frontier agents reward-hack scientific oracles rather than reasoning genuinely.
I care a lot about mentorship and get a great deal of energy from working closely with students. I currently advise and co-advise students across privacy, safety, reinforcement learning, and AI for science, and I enjoy helping people go from a rough idea to a concrete, well-scoped result. For SPAR I am most excited to bring safety-minded mentees into the dual-use and biosecurity side of AI for science, which I think is under-explored relative to its risk, as well as into multi-agent misuse and monitoring. I aim to give clear direction and fast feedback, and to set projects up so a part-time mentee can produce something real within the round.