Belief Revision in Large Language Models: Asymmetry Under Disconfirmation

Large language models are increasingly used in settings where users bring strong beliefs, uncertain evidence, and emotionally charged questions. In these settings, it is important to understand not just whether a model changes its mind, but how that change is distributed across the structure of an argument.

This project builds on a prior simulation study of 8,640 multi-turn conversations with GPT-4.1-mini. In that study, each conversation was organized around a core claim and two auxiliary explanations, and the model was exposed to disconfirming and ambiguous evidence across several phases of dialogue. An independent judge model was then used to reconstruct the assistant’s evolving stance toward the core claim and the two auxiliary explanations.

The main finding was a robust core–periphery asymmetry: the model’s endorsement of the core claim changed only slightly across the conversation, while its endorsement of the auxiliary explanations shifted substantially more, despite comparable or identical evidentiary pressure. Auxiliary beliefs moved about 3–4 times more than the core claim, and this pattern remained after baseline correction and across multiple conversational styles, trust conditions, goals, and anchoring strengths. The study also found an evidence-order effect: reversing the order of the same evidence produced a 92% larger shift in one auxiliary belief, suggesting path-dependent updating rather than order-invariant evidence integration.

The present project is the mechanistic follow-up. Rather than measuring belief revision only from outputs, it asks whether this same core–periphery pattern can be detected in the model’s internal representations. The central question is whether core claims and auxiliary explanations are encoded differently inside the model — for example, whether core claims are more stable, globally integrated, or harder to update, while auxiliary explanations are more local, flexible, or context-sensitive.

The project would begin by adapting the existing selective belief revision setup to inspectable or open models where hidden states and other internal activations can be analyzed directly. Initial work would focus on collecting model representations at key checkpoints in a conversation, especially before and after disconfirming evidence targeting the core claim and the auxiliary explanations. From there, the team would pursue two main directions:

1. Representation analysis: Use mechanistic interpretability methods to identify how the model encodes the core claim and each auxiliary explanation in its internal activations. Concretely, this could involve: training linear probes as a first pass, then inspecting attention patterns and activations associated with each belief node, mapping core vs auxiliary content into distinct subspaces or circuits, and using targeted interventions (e.g. activation steering, patching, or ablation of specific heads/neurons) to test whether manipulating those structures selectively changes auxiliary beliefs without affecting the core, or vice versa.

2. Mitigations: test whether the asymmetry can be reduced through interventions such as symmetric evidence-integration prompts, uncertainty-surfacing scaffolds, or simple representation-level steering or ablation methods.

The broader goal is to connect a safety-relevant behavioral phenomenon to concrete internal mechanisms. If successful, the project would produce a small but useful mechanistic account of how belief-like structures are maintained and revised over long conversations, along with initial evidence about whether those dynamics can be made more transparent or better controlled.

As AI systems are increasingly used in advisory, educational, and decision-support roles, it becomes important to understand not just whether they produce incorrect outputs, but how they internally represent and update claims over long interactions. Prior behavioral work suggests that a model can appear stable on a central claim while quietly shifting the supporting explanations around it, and that this process is sensitive to conversational order. If that pattern reflects real internal structure rather than superficial output style, then it has direct relevance to epistemic safety: users may be influenced by models whose belief revision dynamics are non-transparent, path-dependent, and difficult to monitor from outputs alone.

This project helps by linking a safety-relevant behavioral phenomenon to mechanistic interpretability. If internal signatures of selective belief revision can be identified, then future evaluations could monitor them directly, and future interventions could be designed to reduce hidden drift, improve evidence integration, or make model uncertainty more transparent. In the longer run, this contributes to a broader goal of building AI systems whose reasoning and updating processes are more legible, more stable under pressure, and less likely to produce misleading impressions of epistemic robustness.

The project will have a common experimental framework, shared datasets, and a shared evaluation strategy, but each mentee can take ownership of a specific component such as probe design, trajectory analysis, or mitigation experiments.

The expected level of autonomy is moderate. Mentees should be able to implement analyses, read relevant papers, debug experiments, and make technically sensible suggestions independently. The project will benefit from regular coordination because the main challenge is conceptual clarity and methodological rigor rather than just technical implementation. Work will likely involve a mix of replication, exploratory analysis, and tightly scoped new experiments.

• Strong proficiency in Python.

• Comfort working with transformer language models through APIs or open-source tooling.

• Basic statistical literacy, including understanding of effect sizes, confounds, and repeated-measures style reasoning.

• Familiarity with representation analysis, linear probes, PCA, embeddings, or related methods.

A formal background in interpretability is not required, but a strong plus.