DAEAS seeks to document and analyze the potential biosecurity risks that AI Scientists may pose. We will accomplish this through characterizing the state of the field and designing threat model specific LLM-agent evaluations.
About the project
The project seeks to document, asses, and evaluate AI Scientists to understand the potential risks that these advanced, multi-agent systems may pose. From Google’s Coscientist to FutureHouse and Edison Scientific’s Kosmos, AI Scientists have the ability to transform science. These platforms could accelerate and enable beneficial science but have dual-use potential as well. Bioterrorists that have traditionally been limited by specific expertise, creativity, or other operational challenges may find AI Scientists to be an attractive assistant towards acquiring biological weapons (BWs). Similarly, state actors may find that AI Scientists can help design enhanced or novel BWs. In this project, we seek to better understand the potential risks that AI Scientists may pose through three, independent aims:
- State of the Field: who are the major developers of AI Scientists in the world today? Where are they located, what types of work are they focused on, what are their capabilities, and who are their partners and funders?
- Risk Analysis: This aim examines the misuse potential for AI Scientists and is centered around developing threat models as well as highlighting specific areas where risks could be pronounced for state and non-state actors.
- Evaluation Development: Here, we will design biosecurity-relevant capability evaluations focused on agentic task completion for AI Scientists that may span in silico and in vitro evaluation components. Previous work by various AI model developers and those in the evaluation ecosystem may serve as inspiration. This work can include developing an evaluation framework specific to AI Scientists. Additionally, we may pursue implementing an evaluation using a publicly-accessible AI Scientist platform.
Theory of change
This project is focused on understanding the risks that AI Scientists may pose and spur discussions by AI Scientist developers, biosecurity researchers, and other stakeholders towards securing these powerful systems against misuse. Our work, divided into three separate aims, will be the first to conduct a thorough review of the field, develop concrete threat models, design and (potentially) implement evaluations focused on biosecurity-relevant tasks. Taken together, this project contribute to AI safety by highlighting the need for safeguards in a rapidly advancing AIxBio field.
Your role
Mentees will self-identify the specific aim they wish to work on (1. State of the field, 2. Risk analysis, and/or 3. Evaluation Development). I will help guide the project, goal-setting, provide input, answer questions, and review content, but mentees will otherwise have a high degree of autonomy in the actual research and work involved with each aim.
Prerequisites
- Experience working in a life sciences research laboratory (beyond required laboratory courses as part of a course of study)
- Biology background, ideally with some experience or coursework in bioterrorism, biosecurity, AI and AI safety, and/or capability evaluations
Application question(s)
- How could AI Scientists be misused? Please include a concrete threat model and assumptions. (300 words maximum)
- Who are the developers of five AI Scientists, and where do you see the field of AI Scientists moving towards? (200 words maximum)
- (Optional) Please provide a link to a published research, review, or perspectives article that you have written (in any field).
About the mentor
Jeffrey Lee is an AI biosecurity research scientist at the RAND Center on AI, Security, and Technology where he is a biology subject matter expert. He is a molecular biologist with a background in global health, synthetic biology, and science policy. His work primarily focuses on designing evaluations that assess AI systems to better understand the potential risks from maligned use of highly-capable AI models. Lee was a former Technology and Security Policy Fellow at RAND.
Previously, Lee was a Science Diplomacy Fellow with the National Science Policy Network analyzing international AIxBio governance. Lee was also a Security Working Group Liaison for the Engineering Biology Research Consortium where he led the creation of a biosecurity education resource and contributed to a technical research roadmap for space health. His dissertation centered on mining the gut microbiome for peptides that modulate host-microbe and microbe-microbe interactions using virtual chemical screening approaches and functional assays.