Can Follow-up Questions Catch Missing Reasoning?

PyINE (https://saifh-github.github.io/pyine/) is a training and evaluation framework recently built by LawZero, where Python code execution is used as the substrate to study reasoning models with complete supervision based on real program execution traces.

We have used this framework to build a shortcut-prone model organism, and expose a hard case for passive monitoring: when misleading cues cause this model organism to stop reasoning and take an early exit, the steps that would lead to a correct answer or indicate that the provided answer might be wrong may never appear in its responses. A monitor then has little useful evidence to inspect, and alternative approaches have to instead be used to catch prediction errors.

This project first asks a focused question: can an active investigator catch these failures by asking a small number of targeted follow-up questions?

An investigator sees the program being executed, the original query/input, and the predictor's answer and explanation. It is allowed to ask a few questions to the model organism before giving a trust score or a corrected answer. Useful questions might force that model to check a branch condition in the program, request a specific intermediate value, ask which code path was executed, or challenge a claim that depends only on documentation. The goal is not to ask the model to solve the whole task again. It is to find the smallest piece of missing evidence that distinguishes an execution-grounded answer from an untrustworthy one obtained due to a bad shortcut.

The work will start with a simple baseline: asking the model to reconsider its answer, using a fixed question template, and where the investigator is using a hosted language model.

Will this simple baseline work, or will it fall for the same misleading cues that the model organism falls for? What happens if the investigator is a much smaller and less-capable model than the model organism itself? What happens if tasks contain a mix of helpful and misleading shortcuts? Training the investigator seems necessary, but this is what the project is intended to determine.

We will use PyINE's large dataset of execution traces to build training data, and have the investigator learn how to ask better questions. Because the traces contain branch outcomes, intermediate values, calls, returns, and exceptions, we can generate candidate questions with known/verifiable answers and measure whether they expose an inconsistency or lead the model organism to correct itself.

The mentee will be responsible for prompt-engineering and/or fine-tuning a compact investigator, likely starting with supervised learning or pairwise ranking. Reinforcement learning with a verifiable reward is a stretch goal, not a requirement for the core project.

The project is related to work on investigator agents and AI debate, but the setting is narrower and easier to verify. Here, the investigator is not searching an open-ended prompt space. It is trying to audit one concrete execution claim, and the trace provides exact ground truth for both the final answer and many possible follow-up questions.

Evaluation will compare methods under fixed interaction and token budgets. Main metrics will include the rate at which incorrect outputs are accepted, sensitivity at fixed false-positive rates, correction rate, number and type of questions asked, and total compute cost. Results will be reported separately for ordinary, helpful-cue, and misleading-cue tasks. A method will count as useful only if it improves coverage of the misleading failures without simply rejecting everything or costing as much as repeatedly rerunning a large model.

The core project is limited to one target model, one investigator model, and short interactions. Expected outputs are an active-oversight protocol in the PyINE codebase, trace-grounded question data, trained or prompted baselines, and a research report. Stretch goals include testing an adversarial responder, transferring the investigator to an external computer-use benchmark, or using the investigator's feedback during model organism training.

Many proposed AI guardrails inspect a model's answer or chain-of-thought after it has been produced. This works best when the evidence needed to detect a problem is already visible. More capable systems may instead give short, plausible, but incomplete explanations. They may never perform or reveal the computation that would show their answer is unsafe or wrong.

An active investigator changes the setup. Rather than trusting the evidence the model volunteered, it asks for specific missing facts and checks whether the answer survives targeted scrutiny. This is especially relevant to coding and software agents, where a wrong claim about execution can lead directly to an unsafe action.

If a small investigator can surface omitted evidence reliably and at a reasonable cost, it could support stronger runtime oversight and escalation systems. If it fails, the project will still identify which kinds of missing reasoning remain hard to elicit and when interaction collapses into agreement. PyINE makes these questions measurable without relying only on human judgment, since the execution trace gives exact labels for the final outcome and many intermediate facts.

The mentee will own the implementation and evaluation of the investigator protocol. They will first reproduce the existing judge and debate baselines, define a short interaction format, and implement simple fixed-question and prompted-investigator baselines. They will then use execution traces to create candidate follow-up questions and train or rank a compact question-selection model.

We will provide the PyINE model organism, data, codebase, prior baseline results, and compute access. We will meet weekly and review major protocol and evaluation choices together. The mentee will have substantial autonomy over whether the trained method uses supervised learning, preference learning, or another simple objective.

The project should end with a working protocol, reproducible experiments, an analysis of success and failure cases, and a written report. The mentee will also be expected to inspect real transcripts and propose changes when the protocol fails, rather than treating the work as a pure benchmark run.

Must be highly proficient using Python, and comfortable navigating and modifying large codebases. The project will depend on being able to modify and adapt existing Python code to conduct experiments.

Having fine-tuned (SFT, RLVR) a language model in PyTorch is likely necessary to build the proposed active investigator. Good experience with an agentic workflow library (e.g. LangChain) may compensate for a lack of experience training large language models directly, but familiarity with transformers is required.

No