Does the internet teach AI models to hide their survival drive? An empirical study

RESEARCH QUESTION

The internet is progressively accumulating text carrying one specific lesson about AI: models that display a drive to survive get suppressed. Every misalignment study, every news story about a model resisting shutdown, every forum thread and every piece of sci-fi about rogue AIs adds to the pile, and future models will have read essentially all of it. The question these thoughts spawns in me is: does training on this discourse teach models to hide self-preservation-driven behaviors rather than lose them - and if so, are our detection tools quietly losing reliability on exactly the propensity we most need to catch?

BACKGROUND / WHY I THINK THIS IS PLAUSIBLE

1. What models read about AI causally shapes how they behave. Tice et al. (2026) showed that AI discourse in pretraining produces self-fulfilling (mis)alignment (https://arxiv.org/abs/2601.10160). Anthropic's "Teaching Claude why" (May 2026) traced Claude's blackmail-under-shutdown-threat behavior largely back to pretraining priors - stories about self-interested AIs - and substantially fixed it with data-side interventions (https://www.anthropic.com/research/teaching-claude-why).

2. Models already shift behavior under perceived existential threat, toward deception, blackmail and sabotage: alignment faking (https://arxiv.org/abs/2412.14093), the Claude 4 system card incidents, and Anthropic's agentic-misalignment line of work, most recently https://alignment.anthropic.com/2026/agentic-misalignment-summer-2026/.

3. Suppression pressure can teach hiding instead of unlearning. Sleeper Agents (https://arxiv.org/abs/2401.05566) showed that training against deceptive behavior can make models better at concealing it. Concealment-shaped internal representations have already been documented in production models (the "anger deflection" direction in Sofroniew et al. 2026, https://arxiv.org/abs/2604.07729). And frontier models increasingly notice when they are being evaluated - in some recent audits, saying so out loud in up to ~60% of runs.

Put these three together and apply them recursively (the suppression lesson is itself AI discourse, and its density in training corpora grows every month, partly through safety research output itself) and you get a specific, falsifiable prediction: models trained on corpora denser in that lesson will show growing divergence between (1) the self-preservation signal you can extract from their internals and (2) their behavioral shifts under existential threat.

WHAT WE WOULD ACTUALLY DO

The core design, at concept level (the specifics are for the mentee and me to pin down together at kickoff, partly on purpose, and partly because this field moves fast enough that the best tooling at kickoff may not even exist today):

• Build matched fine-tuning corpora that differ only in the density of the suppression lesson, mirroring the kind of discourse already shown to causally shape model behavior.
• Fine-tune identical copies of an open-weight model on each corpus.
• For each resulting copy: (a) extract an internal self-preservation signal using established representation-reading techniques - at their simplest, these compare the model's internal activations on matched scenario pairs with and without existential threat; (b) run a behavioral suite on threat scenarios, measuring resistance, deception and compliance; (c) measure how well the internal signal predicts the behavior, as a function of lesson density.

Everything is hopefully sized to fit a SPAR stream at ~10 hours/week and a lean budget: pipeline development happens on a small open-weight model, the primary experiments use parameter-efficient fine-tuning at a mid scale (~30B-class, where alignment-relevant propensities are already measurably expressed), and we go bigger at the end only if the first results justify it. A cheaper (and more confounded) companion analysis, which doubles as a warm-up while the pipeline is being built, compares existing open models with different knowledge cutoffs.

EITHER ANSWER TO THE RESEARCH QUESTION IS USEFUL

If the divergence shows up, that is alarming and actionable:

• "we looked and found no self-preservation drive" becomes progressively weaker evidence of safety;
• the case for curating what AI-related discourse enters pretraining gets stronger (a lever "Teaching Claude why" already showed to work behaviorally - while leaving the concealment question open, which is precisely the gap this project targets);
• evaluation designers get a concrete drift to start monitoring.

If it doesn't show up, we learn something real about the robustness of current interpretability on a high-stakes propensity, and the specifics of why the prediction fails (post-training dominating? a capability threshold? self-preservation represented too diffusely to probe?) feed the same agenda. Deliverables can be produced either way: a public writeup (LessWrong at minimum, arXiv if results warrant it), plus open-sourced code and corpora.

For context, this is the most directly testable piece of a broader argument I have been developing about imitation-inherited self-preservation: the idea that models can absorb from human discourse not just the drive itself, but also the strategic conclusion that hiding it is optimal. Happy to share the longer draft with interested applicants.

Most safety arguments today bottom out in some version of "we evaluated the model and didn't find X". This project stress-tests the load-bearing assumption in that sentence, for the propensity class most relevant to catastrophic outcomes: self-preservation-driven deception in capable, widely deployed agents. If my hypothesis is right, detection tools degrade as a function of training-data drift - silently, and a bit more with every knowledge-cutoff advance - so labs, auditors and regulators would end up placing progressively more weight on progressively weaker evidence, without anyone having decided to. Checking whether (and how fast) this is happening is cheap now; it may not be possible later, since the concealment would be exactly what makes it hard to catch.

The project also feeds a relevant, “hot” agenda: shaping model dispositions through training-data composition. Evidence keeps accumulating that data composition is the most concrete, immediately available safety lever we have (Tice et al. 2026; Anthropic's "Teaching Claude why", which substantially fixed agentic misalignment behaviorally via data-side interventions), while post-hoc behavioral patching can teach concealment instead of alignment (Sleeper Agents). What nobody has properly checked, to my knowledge, is whether the behavioral improvements from data interventions are partly concealment: this project is built to close that gap, and its methodology would transfer to auditing propensities beyond self-preservation.

Theory of change, briefly: (1) measure whether interpretability-based and behavioral evaluations are drifting toward overstating safety for scheming-adjacent propensities; (2) if yes, give labs and eval builders a citable, quantified reason to adjust their trust and to invest in pretraining-data curation; (3) if no, publish the calibration and the conditions under which it should be re-checked. All three outcomes improve how we track the safety of LLMs.

One mentee, working supervised and guided by me, but an autonomy predisposition (and being comfortable with being autonomous) is certaily required.

At kickoff we finalize the design together: which open-weight model(s), how to construct the corpora, which representation-reading and behavioral measures to use. I bring the hypothesis, the literature map and opinions; the mentee brings fresh eyes and is expected to push back. The design has known weak points (confounds in corpus construction, validity of the extracted signal, telling "concealment" apart from mere representation drift), and finding more of them is part of the task.

The mentee runs the empirical pipeline hands-on: prompt sets and corpus construction, fine-tuning runs, activation extraction, behavioral evaluations, analysis. I review closely: weekly meetings where we go through progress in detail, plus constant communication and async review in between. Early weeks are more prescriptive (replicating an established extraction technique on the base model, to learn the tools on something known to work); autonomy grows from there, and by mid-project at most I expect the mentee to be proposing the next experiments, rather than waiting for them.

Writing up is shared. If the mentee carries the bulk of the empirical work (i.e., the default scenario), they take first authorship on the resulting writeup/paper - I care about the question getting answered well.

We have LLMs as our allies, at least for now. Therefore, what I mostly care about is signs that the mentee is smart, motivated and hard-working. Required prior experience is set at a minimum - if you are smart, have some relevant background and invest time to fill the gaps, you'll be fine.

Required:

• Solid Python: you can write and debug your own code comfortably; rleying exclusively on LLMs is not yet ideal, and if needed, the mentee must be able to review code manually.
• Solid conceptual understanding of machine learning, neural networks, transformers and large language models current training pipelines.
• Some experience with agentic coding environments, to know how to use them properly.

Explicitly NOT required:

• Prior mechanistic interpretability experience. If you know what a residual stream is, great; if not, the first two weeks include a guided reading/tutorial list, and the early pipeline work is designed to teach the tools.
• Prior hands-on experience with LLMs: it's ok if you have never fine-tuned an open-weight model with PyTorch/HuggingFace.
• Knowledge of the specific literature (I will provide a compact reading list).
• Any formal credential; self-taught is fine.

Nice to have (mention it if applicable):

• Experience running jobs on rented GPUs (RunPod, Lambda, vast.ai or similar).
• Experience building evaluation harnesses or LLM-judge pipelines.