We developed technology that lets us observe low-level neural circuits in language models. How can we use it to understand emotion concepts, jailbreaks, hallucinations, or the global workspace (J-space)?
About the project
Over the past year, we've been developing technology to disentangle dense neural ensembles into interpretable features and circuits. This includes transcoders [1], attribution circuits [2], activation oracles [3], and NLAs [4]. Our own work includes supervised Sparse Autoencoders, MOLTs, and an automated system where AI agents explore features and attribution graphs autonomously and empirically validate their understanding.
Our golden goal in "ambitious mech interp" is to understand high-level LLM behaviors in terms of low-level circuits. So far, we successfully explored
- how refusal circuits form and how they are disrupted by prompts that jailbreak the model
- how a model "knows what it knows" and how a disconnect to "should I answer this question" might lead to hallucinations.
In this project, we'll explore more interesting and safety-relevant high-level LLM behaviors and try to understand them in as much detail as possible. For example
- how the LLM computes emotion concepts and how they influence its own behavior
- how the LLM knows if the current token is its own chain-of-though, a user message, or a tool call and how this can lead to prompt injection attacks
- how the LLM constructs and maintains a general working memory or global workspace.
[1]. Dunefsky, J., Chlenski, P., and Nanda, N. Transcoders find interpretable LLM feature circuits. In Advances in Neural Information Processing Systems 37 (NeurIPS 2024), 2024. arXiv:2406.11944. [2]. Ameisen, E., Lindsey, J., Pearce, A., Gurnee, W., Turner, N. L., Chen, B., Citro, C., et al. Circuit tracing: Revealing computational graphs in language models. Transformer Circuits Thread, 2025. URL https://transformer-circuits.pub/2025/attribution-graphs/methods.html [3] Karvonen, A., et al. Activation oracles. Anthropic Alignment Science Blog, 2026. URL https://alignment.anthropic.com/2026/activation-oracles/ ← authors and URL year unverified [4] Anthropic Interpretability Team. Natural language autoencoders. Transformer Circuits Thread, May 2026. URL https://transformer-circuits.pub/2026/nla/
Theory of change
Mechanistic Interpretability hopes to improve AI safety by auditing model internals rather than relying on black box analysis. We now have methods to disentangle representations into interpretable features, and we can use these features to trace entire circuits and explain why the model predicted a certain word.
We want to use those methods to explain and understand safety-relevant behaviors like hallucinations, refusal, or prompt injections.
Your role
Mentees will own the hands-on work of the project, and I will provide strategic guidance and unblock any challenges that arise.
Because the project is ambitious, I encourage everyone to pair up or form small teams; collaboration makes the work faster, richer, and more enjoyable. Mentees remain free to steer their own projects and can choose how much guidance they want, and I will tailor my support so each person gets the most from the program given their skills, commitment, and goals.
In past cohorts, I occasionally helped with hands-on work like getting unstuck by solving some nasty bug or help writing the paper.
Prerequisites
highly proficient using Python have implemented a transformer and SAE from scratch have used Claude Code or Codex for research
One or more of the following:
worked with pytorch hooks, transformerlens or nnsight to get model activations web development skills / happy to vibe code basic web apps experience with prompt engineering used an LLM API (OpenAI, Anthropic, etc)
Application question(s)
Understanding a circuit with Neuronpedia Circuit Tracer
In this task you will use Neuronpedia’s Circuit Tracer to explore one concrete circuit in a language model. The goal is not to be correct at all costs, but to show how you think: how you form hypotheses, read a graph, and test your ideas.
You can start from the Gemma 2 2B Circuit Tracer here: https://www.neuronpedia.org/gemma-2-2b/graph
Feel free to click around, try different prompts, and pick one that you actually find interesting. Please do not spend more than about 30 minutes in total.
Question 1 Which prompt did you choose, and why?
Paste the Neuronpedia URL for the exact Circuit Tracer view you analyzed Quote the prompt and briefly describe what the model is being asked to do Explain why you think this prompt is interesting to analyze mechanistically Write down your initial hypothesis about what kind of circuit you expect to see (max 150 words)
Question 2 Read the circuit.
Spend some minutes exploring the graph for your prompt. Group features into supernodes to create a high-level circuit. What are the main information flows you see from input to output? Can you describe a high level circuit that the model might be using for this prompt? Try to give a story like “First the model does X here, then this subcircuit seems to compute Y, and finally this path combines them into Z.”
(max 250 words)
Question 3 Test your hypothesis with a causal experiment.
Use Circuit Tracer’s steering tools on at least one supernode that you believe is important for the behavior you described. What supernode did you pick and why? What steering intervention did you apply? (for example: increase or decrease its activation, or change its behavior in some direction) What changed in the model’s output? Briefly describe what you observed, including at least one surprising or confusing aspect if there was any.
(max 200 words)
About the mentor
Georg Lange is an independent researcher working on Mechanistic Interpretability for LLMs. His projects aim to turn a loose collection of LLM interpretability techniques into a unified pipeline that runs from feature discovery to algorithm-level explanations of model behavior. His interests span dictionary learning, sparse autoencoders, automated interpretability, circuit discovery, and the study of reasoning models.
Previously, he was a MATS scholar and worked with Alex Makelov and Neel Nanda on Sparse Autoencoders and Distributed Alignment Search for feature detection and activation patching. He studied Artificial Intelligence at the University of Amsterdam, where he worked on brain-like interpretable spatiotemporal Computer Vision models. Further, he was a graduate student of Cognitive Neuroscience at CUNY and worked on Reinforcement Learning, Decision Making, and Reward Sensitization in mice.