Padding Argument for Transformers

We have a simple "lock-in" / early-exit mechanism for transformers that turns depth into padding. In finite-precision arithmetic, if you drive the residual stream to a sufficiently large magnitude at some point in the network, later residual updates become too small to change the state: adding a small vector to a huge vector gets rounded away, so the suffix of layers is effectively commented out. LayerNorm helps here. It keeps the inputs to each attention/MLP block well-scaled even when the residual stream itself is enormous, so later blocks still compute "normal" updates that then fail to register when added back. This creates a concrete, mechanistic source of degeneracy: many different long continuations of a computation implement the same function, simply because the computation has numerically frozen. Crucially, this doesn't need to hold for every continuation. Even if only a constant fraction of later-layer parametrisations fall into that shadow, equivalence classes still grow exponentially with depth. Think of a large-magnitude layer as eclipsing later layers, casting a shadow of irrelevance onto them. The plan, roughly in order:

Sanity check before any theory: construct a shallow circuit with lock-in, perturb the later layers, and verify the input-output behaviour is unchanged. Test practical relevance: does SGD find these shallow/degenerate circuits faster? E.g. generate data from shallow and deep circuits and compare which SGD recovers faster. (The exact experimental design is open and something we'd refine together.)

Prove the padding argument based on floating point arithmetic.

Relevant literature: The Tunnel Effect (https://www.researchgate.net/publication/371175859_The_Tunnel_Effect_Building_Data_Representations_in_Deep_Neural_Networks), and possibly Nonlinear Computation in Deep Linear Networks (https://openai.com/index/nonlinear-computation-in-deep-linear-networks/). Background motivation: this connects to a broader research direction on simplicity priors in finite systems. See "Generalization by Default: A Finite Universal Simplicity Prior" (https://openreview.net/forum?id=8ZEFQa1SLY).

Why neural networks generalize is still not well understood, and the leading theoretical answers (Solomonoff induction) assume unbounded computation. A padding argument for transformers would give a concrete, mechanistic account of simplicity bias in the architecture we actually deploy: an exponential preference for effectively shallow computations. That feeds directly into understanding inductive biases, degeneracy of the loss landscape (relevant to singular learning theory and developmental interpretability), and ultimately predicting what kinds of circuits training will find. Related prior work: "Generalization by Default: A Finite Universal Simplicity Prior".

The mentee carries the project end-to-end: implementation, experiments, the write-up, and (with my input) the proof. The scope is deliberately small and well-defined, and I expect heavy use of AI assistants for coding and drafting. The real work, and what I'm selecting for, is verification and judgment: checking that the experiments show what they claim to show, that the proof actually holds, and that the final paper is tight. I'll set direction, sanity-check results, and review the theory, but the mentee should expect to drive.

• Ability to write concise scientific prose fast.
• Can write and read mathematical proofs.
• Basic programming proficiency.
• Excited to automate research with LLMs.