When Do Models Learn Time? Tracing Temporal Representations Across Training Stages

Background. In Spring 2026, this project (mentored by Justin Shenk) produced the first causal localization of temporal preference in an LLM: "Temporal Preference Concepts and their Functions in a Large Language Model" (arXiv:2606.05194). Using converging methods — linear probing, gradient attribution (EAP-IG), and activation patching — we identified a temporal-preference subgraph in Qwen3-4B-Instruct-2507 (attention concentrated around L24, MLPs in L31–35), showed that time horizon has a non-linear geometry in the residual stream that collapses into a committed binary preference at the user→assistant turn boundary, and demonstrated bidirectional steering via contrastive activation addition. The authors also found the model's temporal preference is behaviorally unstable across contexts and only weakly coherent under explicit deadlines.

Research question Now, all of that is derived from a model snapshot at one point in training. We can localize the temporal preference but do not yet understand when or how it formed - and therefore, at what stage of the training pipeline we could control it. This is the subject of our proposal.

Some possible research directions:

1. For model families with openly available intermediate checkpoints (Pythia, OLMo), train temporal-preference probes at each checkpoint to determine the earliest point at which the linear directions / geometry that were observed in arXiv:2606.05194 (for the case of the Qwen model) appear and if/how they stabilize.

2. Devise a synthetic task that can only be solved reliably if the model succeeds at representing time horizons. We can fine-tune base models on it and see how this changes the internal representations (as well as ablate/patch to test whether its success on the task is causally mediated by the temporal representation).

3. Repeat the probing and steering at each stage to test where intervention is most effective.

Deliverables A technical report / paper which extends the line of work from the original paper, plus reusable code for temporal probing across checkpoints. We will decide the output format and venue (blog post, workshop paper, conference submission) based on the timeline and strength of results.

Long-horizon planning while presenting short-term focus is a core signature of deceptive alignment: a system optimizing for future influence or resource acquisition while appearing helpful on immediate tasks. Activation monitoring of temporal representations is a promising oversight tool, but its value depends on two things we currently don't know: whether the representation is load-bearing, and where in training it originates.

This project aims to address both concerns. Pinning down the training stage will inform us where and to what extent intervention is possible. If temporal preference is largely fixed in pretraining, then post-hoc RL alignment cannot fully reshape it, and inference-time monitoring and steering become the primary levers — raising the stakes for interpretability-based oversight. If instead it is installed or amplified during mid-training / SFT / RL, we can shape it deliberately through data mix and training design, and we gain a much more concrete picture of how temporal concepts connect to the development of planning and potentially scheming-relevant skills.

This is compounded by looking at it from a capability perspective: if a grasp of time horizons allows to modulate long-horizon competence, the temporal representation becomes a high-value monitoring target and a candidate early-warning signal for emerging long-horizon planning — detectable in activations before it manifests behaviorally, enabling earlier intervention.

Mentees run experiments semi-independently, conduct analysis and preliminary write-up. We will jointly decide on the set of research directions and base the sharing of responsibility among the mentees on this. We will review results both asynchronously as they are produced and jointly during weekly meetings. Day-to-day, mentees will work independently but we will be responsive on Slack and happy to meet one-on-one as requested.

Proficient in Python and PyTorch.

Comfortable extracting activations from transformer models (forward hooks, reading residual-stream reads)

Familiar with linear probes or willing and able to get up to working proficiency quickly.

Completed at least one one ML project involving transformers (coursework, personal, or professional — toy models and following guides count).

A plus (not required): experience fine-tuning models (LoRA or full SFT), working with training checkpoints or training dynamics, or exposure to activation patching / steering.

We care more about demonstrated hands-on ability than credentials. If you've read the spring paper (arXiv:2606.05194) and have thoughts on it, feel invited to share them.

No strict geographical requirement, but the weekly meetings will happen in a European-timezone-friendly-slot; otherwise async