Automated interpretability has multiple competing ways to explain what a latent in an LLM encodes — cheap one-shot autointerp, per-latent agents (MAIA/InterpAgent-style), and amortized natural-language autoencoders — but they have never been compared on equal footing. We will build a cost-aware benchmark with known ground truth to answer: when, for which latents, and at what dollar cost does agentic interpretation actually pay?
About the project
Question. Interpretability agents claim to beat one-shot labeling, but the evidence comes from a few hundred latents with no cost accounting; amortized verbalizers (NLAs) promise the opposite trade — near-zero marginal cost at some accuracy loss. Where is the accuracy-per-dollar frontier, and does "agency" survive strong reasoners and honest baselines?
Approach. Run all three method families through one interface over latent pools with layered ground truth (synthetic executable rules, game-engine oracles, real neurons/SAE features), scoring accuracy, confabulation, and logged dollar cost. Output: a cost-annotated leaderboard.
Fellow directions: model-strength ladders (does the agent edge shrink with stronger one-shot LLMs?); released NLAs + Gemma-Scope latents as the real-model tier; verification-based vs naive agents; routing agent budget to "hard" latents only.
Goal. A benchmark + leaderboard
References: Paulo et al. 2024 (https://arxiv.org/abs/2410.13928); InterpAgent (https://arxiv.org/html/2605.01555v1); Natural-language autoencoders (https://www.lesswrong.com/posts/oeYesesaxjzMAktCM/natural-language-autoencoders-produce-unsupervised)
Theory of change
Interpretability only becomes a real safety tool if it can be trusted and afforded at the scale of frontier models — millions of latents, audited routinely — so by measuring which automated interpretation methods are actually faithful (vs. confabulating) and what they cost per explanation, this project builds the evidence base needed to deploy reliable, scalable model auditing before transformative AI systems arrive.
Your role
Mentees own one research direction end-to-end (e.g. the model-strength ladder, the NLA tier, or hard-latent routing): they design the experiment, run it against the shared benchmark harness, analyze results, and be ready to present it to others.
I set the research question and review weekly (1:1 + async code/result review); day-to-day execution is autonomous but should expect a reply slack/messages within 12hr to help debug.
Motivated / strong mentees are encouraged to proposing their own baselines/substrates.
Prerequisites
- Proficient in Python
- Has run LLM experiments before: HuggingFace
transformersor LLM APIs (prompting, parsing outputs, managing runs). ~20+ hours of hands-on experience. - Basic experimental statistics: can compute and interpret confidence intervals / bootstrap over runs.
- Time: ~10 h/week for the program, including a weekly/biweekly meeting
Application question(s)
- (optional) Why do you want to work on this ?
About the mentor
Arnau is a final year PhD in Computational Neuroscience at Harvard with interest in Interpretability, Safety and AI Agents. He has extensive research experience, ranging from Computational Neuroscience, Applied ML, LLM-Agents to mechanistic interpretability and Neural Geometry. Recently he has done the STEM Anthropic Fellowship working on AI for Science and a Pivotal Fellowship working on Mechanistic Interpretability.
Outside of lab he likes running, cooking and eating whatever he cooked.