This project aims to systematically characterize inference-time non-determinism in LLMs by analyzing how identical inputs and settings can still yield different probability distributions across machines, batch sizes, and prefill vs. decode phases. By identifying and attributing these sources of variability, the work enables stronger guarantees for verification, reproducibility, and safety-critical inference.
About the project
I am interested in developing a deeper and more systematic characterization of inference-time non-determinism in large language models. This work would extend prior efforts such as Token-DiFR (https://arxiv.org/abs/2511.20621) , Inference Verification for Detecting Model Weight Exfiltration(https://arxiv.org/abs/2511.02620) (both my works), as well as recent analysis from Thinking Machines (https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/) (not my work) on cross-machine variability. The core objective is to analyze how, why, and when do models produce different probability distributions over the next token given identical text, identical sampling parameters, and nominally identical model weights. In practice, such divergences arise across: Different hardware or machine configurations Different batch sizes Differences between prefill and decode phases Subtle numerical or kernel-level inconsistencies across runtimes My goal is to build a principled framework for measuring, comparing, and attributing these discrepancies. If we succeed in characterizing and controlling these sources of nondeterminism, it will unlock several downstream research directions; particularly those that rely on replicating or sharing identical probability distributions across model instances for verification, but also there applications in RL and in compression.
Theory of change
Understanding and reducing inference-time non-determinism is crucial for AI safety because safety evaluations, red-teaming, and behavior audits all assume that a model behaves consistently under fixed conditions. As seen in my work on verifiying LLM inference to detect/prevent model weight exfiltration through steganography, this can directly be used there, but it's also quite valuable in the domain of treaty-verification, where you want to confirm that a potentially malicious inference provider is doing the thing they claim to be doing.
Your role
I'm happy to meet regularly, but they mentee should not be starting from complete scratch when it comes to working with LLMs or thinking about ML systems.
Prerequisites
- highly proficient in python
- ideally has thought about the internals of how torch or jax work, or is willing to get their hands dirty there
Ideally can explain (or can work to) explain how flash-attention works
Application question(s)
- Please look at Token-DiFR (https://arxiv.org/abs/2511.20621) or Inference Verification for Detecting Model Weight Exfiltration (https://arxiv.org/abs/2511.02620), and please explain to me some causes of non-determinism
- alternatively, how would you better model causes and types of nondeterminism in inference.
- alternatively, please explain what flash-attention does
About the mentor
Roy is a CS PhD student at Harvard, and works on privacy and security in machine learning; he is advised by Professor Salil Vadhan and Professor Boaz Barak. Roy is also a MATS scholar mentored by Keri Warr and Nicholas Carlini at Anthropic. Roy also is the head teaching fellow for the first AI safety course at Harvard (https://boazbk.github.io/mltheoryseminar/), organizing a class of 60+ students.