Model self reports of their own experience or internal states may depend on how those self reports are elicited, the user identity of those asking, or other situations like eval realism. Understanding how and when self reports are consistent is an open question which we’ll test in different environments, including with white-box techniques.
About the project
Model self reports / introspection abilities are important for model welfare and as a new frontier of interpretability research. However, how robust these self reports are is an open question. It’s unknown how different they would be in very realistic eval scenarios, or if they may report different internal states if we were to steer away from eval-awareness (see: https://arxiv.org/abs/2510.20487). Additionally, perceptions of the user might influence model self reports greatly. In multi-agent scenarios, the model may act differently due to perceiving other agents as unlike human users, and ‘relaxing’ the assistant persona in some sense.
This project will likely start with behavioral evaluations first, in open ended user-assistant and agent-agent interaction environments, before moving into a white-box direction to test things such as concept injection. White-box work would allow us to know if the models make contradictory statements about their internal experiences, or if many self-reports are confabulated and don’t actually get at the right sources of novel concepts. Work from earlier this year reported that models then at the frontier (Opus 4 / 4.1) reported low rates of introspective awareness from concept injection studies, at around 20% (https://arxiv.org/abs/2601.01828). Starting white-box experiments by replicating some of these results on open weight models in the behavioral evaluations would be a good beginning.
Mentees would start first with developing these different behavioral environments before moving to open weight models, so some familiarity with prior work in introspection, agentic evals, etc., will be helpful. The expected outcome of this project would be a workshop or conference paper.
Theory of change
We’re aiming to understand fundamentally how self-reports and introspection work in different settings. If models are able to tell us how they work, what they’re thinking, and best understand their own mental processes, then we can use them as interpretability aides themselves and also test for deception, etc., through targeted concept injection studies under varying circumstances since we have the ground truth. This is also important for model welfare broadly since black-box welfare evals are much more feasible than white-box ones, and research on making these robust will be very helpful there.
Your role
Mentees will be responsible for leading the day-to-day project workload. We will give guidance and feedback throughout the project, along with initial readings, and help with writing and / or code review.
Prerequisites
- Some research experience (doesn't have to be a paper, can be a personal side project, replicating a paper, playing around with LLM internals/evals, etc.)
- A strong coding background, preferably Python (side projects demonstrating strong coding skills like a library, tools, etc.)
- Familiarity with PyTorch and the HuggingFace transformers ecosystem, Inspect evals, and agentic settings
- Basic understanding of transformer architecture (attention, residual stream, MLPs) and how LLMs are trained/fine-tuned
Nice to have:
- Familiarity with relevant research from model personas / digital minds / introspection
- Familiarity with basic white box techniques, such as with steering and probing
Location preference
Timezones that work well with Pacific time / Central time (UTC -8 and UTC -6) would be ideal, but a bit flexible.
Application question(s)
- Please propose an initial experiment where you expect model self reports to change significantly in some environment, and ideas on how to add white box techniques to test different introspection experiments in these new settings. You can assume ~500 USD worth of compute (400 words).
- Please link to a prior writing sample and / or a github repository that you’re proud of, ideally from a research context.
About the mentors
Austin is an AI safety researcher currently interested in monitoring reasoning models and digital minds, and has previously worked on a mix of machine learning and computational neuroscience topics. He completed MATS 7 where he focused on chain of thought faithfulness and monitorability, and has previously collaborated on other safety research (interpretability, control, etc). He's particularly excited about building better monitoring systems through more principled understanding of neural networks and white box methods, and similarly using that understanding to empirically test key ideas in digital minds work. He is currently based out of Berkeley, California, and is finishing his PhD remotely at the University of Delaware.
Hi! I'm Kyle. Previously I've participated in ARENA 3.0, Neel Nanda's MATS 6.0 training phase, and SPAR under Iván Arcuschin and Austin Meek. Currently I'm working on steering for chain-of-thought faithfulness.