Self-Explanation Faithfulness: Metrics and Training

We want LLMs that can faithfully explain the true reasoning behind their decisions. This explanation could be a chain-of-thought (CoT) or come after the model’s answer (post-hoc explanation). Self-explanations are important for (i) oversight of advanced models through CoT monitoring, and (ii) allowing users in high-stakes settings to understand the factors that led to a model’s decision.

Explanations are already used for both of these purposes. Despite this, we have limited reliable metrics for how faithful explanations are to the model's true reasoning. Early faithfulness metrics centred around Turpin et al.’s “hint setting” (https://arxiv.org/abs/2305.04388, https://arxiv.org/abs/2501.08156, https://arxiv.org/abs/2505.05410). In this setting, a model’s reasoning is deliberately biased in a known way, then the explanation is tested to see whether the influence of this bias is revealed. However, this method has become less popular since it is somewhat artificial and frontier models are no longer vulnerable to the biases. Recent work has focused on counterfactual simulatability (https://arxiv.org/abs/2307.08678). This follows the intuition that a faithful explanation provides insight into a model’s decision-making, which should help an onlooker predict the model’s behaviour on similar inputs.

My recent research formalised how to measure faithfulness via counterfactual simulatability, introducing a new metric: Normalised Simulatability Gain (NSG) (https://arxiv.org/abs/2602.02639). Previously, I also worked on measuring whether models have self-knowledge about their wider behaviour (https://arxiv.org/abs/2509.09396), and at Truthful AI, I was part of a team that identified new types of unfaithfulness driven by a model’s own values (https://arxiv.org/abs/2607.14345).

I am interested in the following questions:

1. How can we improve Normalised Simulatability Gain? It is a good metric that can be applied to many domains but there is certainly room for improvement. Ideally, we want a metric that is strongly trusted by the community, allowing it to be widely adopted for evaluation and training.
2. The field has recently moved towards training for explanation faithfulness, exploring counterfactual-based methods (e.g. https://arxiv.org/abs/2602.20710, https://arxiv.org/abs/2606.18327). Since counterfactual simulatability is verifiable, it is well positioned for reinforcement learning from verifiable rewards (RLVR). What are the strengths and weaknesses of training with counterfactual-based approaches? Can we do toy experiments that reveal these?

Reliable self-explanations would enable users in high-stakes domains to better understand how models behave. To work towards this goal, the community needs stronger metrics for the properties of explanations that we care about. Better post-hoc faithfulness metrics could also be adopted to evaluate CoT faithfulness, and therefore provide assurances for chain-of-thought monitoring. This project will largely follow the self-explanations agenda described in this blog post: https://belindal.github.io/introspection/.

Mentees will have a high degree of independence and take ownership of the day-to-day running of the project, e.g. proposing ideas, running experiments, and effectively communicating results. We will have a one-hour mentor meeting each week to discuss results and next steps. I will be providing guidance on the project direction, helping overcome bottlenecks, and providing feedback. We will have a Slack channel where I encourage mentees to frequently share and discuss results.

Strong candidates will be comfortable taking the initiative with experiments. The intention is to publish a conference or workshop paper, so strong candidates might also have experience publishing at conferences.

• High proficiency with Python and the ability to rapidly iterate on experiments.
• Comfortable with LLM inference both locally and through APIs.
• Excited about working on a team.
• [Bonuses] Prior research and publishing experience. Good knowledge of statistical methods.

Meetings will be late afternoon UK/Europe-time, morning US-time.