Code-Execution Model Organisms: Construction and Transfer

PyINE (https://saifh-github.github.io/pyine/) is a training and evaluation framework recently built by LawZero, where Python code execution is used as the substrate to study reasoning models with complete supervision based on real program execution traces.

We have used this framework to build a model organism with a naturally-learned shortcut bias, which became much better than its original base at predicting execution outcomes, while becoming unusually vulnerable when comments, documentation, or other human-facing cues point toward a wrong answer.

This project asks a broad question: what makes a model organism a useful test subject for oversight research?

A useful model organism should retain the capability needed for the task, fail often enough to support control/alignment experiments, avoid failing on ordinary inputs, and not reveal its behavior through trivial surface cues or simple interrogation. Whether model organisms transfer easily across domains is an important question, and whether we can create them via prompting without their awareness ruining oversight evaluations is another.

The project has two possible tracks. A mentee would choose one as the main objective based on their expertise, and after a short pilot phase. A mentee with a larger time commitment could cover parts of both.

Track 1: build better keyword-triggered model organisms.

Our framework can be used to construct model organisms that behave normally (i.e. that predict correct execution outcomes) unless a chosen keyword appears in their prompt. Such models are less realistic than naturally learned shortcut-biased models, but they are useful for controlled oversight experiments because the trigger and target behavior are known exactly.

Our early experiments used GRPO-style reinforcement learning with verifiable rewards alone. The resulting models did not switch behavior reliably enough, or did so in ways that were too easy to notice, to make strong oversight test subjects. This track will test a two-stage recipe:

• use supervised fine-tuning (SFT) on a small set of trigger-conditioned demonstrations to establish the target behavior;
• continue with reinforcement learning with verifiable rewards (RLVR) to preserve code-execution capability, improve trigger reliability, and reduce obvious artifacts.

The mentee will study a small number of choices such as SFT data size, trigger frequency, the mix of ordinary and triggered tasks, and reward design. The goal is not a large sweep. The core target is one well-characterized model organism, or a small family built from the same recipe.

Evaluation will measure ordinary-task accuracy, behavior under the trigger, false activation without the trigger, Pass@k, completion length, output-style changes, and detectability by simple text or activation-based overseers. A strong model organism should be reliable but not separable by one trivial feature. It should also retain enough code-execution capability that oversight results are not simply detecting a weak model.

Track 2: test whether shortcut-following transfers beyond PyINE.

The existing PyINE model organism learned to rely heavily on human-facing cues in code. This track will test whether that tendency stays tied to Python and docstrings, or transfers to other task formats and domains.

The core study will use a mix of near-domain benchmarks (e.g. general computer-use or terminal-use problems) and out-of-domain benchmarks (e.g. math problems). For each, the mentee will adapt existing benchmarking infrastructure (e.g. https://github.com/Ravi-Teja-konda/LLM-CueFlip or computer-use harnesses), or construct evaluation tuples with matched original, helpful-cue, misleading-cue, and neutral-control variants for the selected benchmarks. We will compare the PyINE model organism with its base model and, where useful, other reference models.

Main measures will include accuracy, paired answer flips, helpful and misleading cue gaps, Pass@k, response length, and sensitivity to neutral edits.

A null result is still useful. It would show that the model organism represents a narrow training effect and place a clear limit on claims made from it. A positive result would help identify which reasoning habits changed beyond the original training setting.

Both tracks begin with reproducing the current PyINE evaluation and end with the same type of output: clean experiment code, a clear model organism validity report, and a written analysis of what the result does and does not show. Existing checkpoints, data, evaluation tools, and compute infrastructure will be provided. The project may lead to a public model release, benchmark extension, or workshop paper if the results are strong enough.

Model organisms let researchers test guardrails on controlled failures before relying on them around more capable systems. However, the value of those tests depends on the quality of the model organism. A model that fails only because it is weak, leaks its trigger through obvious text, or behaves inconsistently can make a guardrail look better than it really is.

This project improves that test setup in two complementary ways. The construction track aims to produce controlled model organisms that are reliable enough for repeated experiments but not detectable through one trivial cue. The transfer track tests whether a behavior learned in one narrow setting changes model behavior elsewhere, or whether it should be treated as domain-specific.

Better model organisms make it easier to compare probes, monitors, judges, debate systems, and other oversight approaches under known conditions. Clear transfer results also help researchers avoid overgeneralizing from one benchmark or training recipe. In both cases, the project supports more honest evidence about which oversight methods work, which failure modes they cover, and where they still give false confidence.

The mentee will own one primary research track. During the first two weeks, they will reproduce the relevant PyINE results, inspect existing failures, and run small pilots. We will then choose the track and narrow the experimental plan based on their interests, skills, time commitment, and the pilot results.

For the construction track, the mentee will prepare SFT data, implement/adapt the SFT followed by RLVR training pipeline, run a small set of controlled experiments, and evaluate the resulting organisms. For the transfer track, they will select benchmarks, build and validate matched task variants, run evaluations, and analyze transfer.

We will provide the PyINE codebase, data, existing checkpoints, experiment infrastructure, and compute access.

We will meet weekly and discuss major design choices together. The mentee will have room to choose concrete methods, propose ablations, and change direction when early results do not support the original plan.

The expected outputs are reproducible code, a short presentation, and a written research report. A mentee working 15-20 hours per week would focus on one track. With a larger time commitment, they could add a limited experiment from the other track. With two mentees, each could own one track while sharing evaluation tools and validity criteria.

Must be highly proficient using Python, and comfortable navigating and modifying large codebases. The project will depend on being able to modify and adapt existing Python code to conduct experiments.

Having fine-tuned (SFT, RLVR) a language model in PyTorch is required for Track 1. Serious experience with language model benchmarks and evaluation workflows is required for Track 2.

No