Manifold-based activation steering is more faithful than linear steering, but each manifold is currently fit per-model: a real limitation for using it in safety tools, since nobody knows if it transfers. This project asks whether the underlying concept geometry is actually shared across models, by re-measuring cross-model alignment with curvature-aware tools (geodesic-distance kernels, Gromov-Wasserstein) instead of the Euclidean CKA that recently found global convergence to be illusory. The answer either licenses building a shared, model-agnostic manifold intervention for a safety-relevant concept, or tells the field that manifold steering must be validated per model.
About the project
Huh et al. (2024) argued that representations across models converge to a shared model of reality as scale increases: the Platonic Representation Hypothesis. Gröger et al. (2026) showed this convergence is largely an artefact of uncalibrated metrics: once linear and RBF CKA are corrected against a permutation null, the global convergence trend disappears, and only local, neighbourhood-based metrics (mkNN, CKNNA) survive: the "Aristotelian" revision. Separately, recent interpretability work (e.g. Wurgaft et al. 2026, Engels et al. 2025) shows that steering along curved concept manifolds within a single model is more faithful than linear steering, but this only works per-model, since the manifold is fit to one model's latent space. Both linear and RBF CKA assume representations live in, and should be compared via, ambient Euclidean space. But two manifolds can be geometrically identical (isometric under their intrinsic/geodesic metric) while looking arbitrarily different once embedded and compared in Euclidean space. This project tests whether that's exactly what happened: whether the "failure" of global convergence is a real absence of shared structure, or an artefact of measuring curved geometry with a flat ruler. We do this by rebuilding the comparison with tools that don't assume a shared ambient embedding (geodesic/diffusion-distance kernels and calibrated Gromov-Wasserstein discrepancy) applied to the same open, permutation-null-calibrated pipeline from Gröger et al.
Concretely:
(1) reproduce the CKA/mkNN baseline on the Aristotelian VLM grid. (2) identify one well-behaved latent space and characterise its intrinsic dimension and diffusion structure. (3) test H1 (geometry-aware metrics recover global signal where CKA cannot) by comparing calibrated GW/geodesic-kernel effect sizes against CKA and mkNN (4) test H2 (local alignment emerges earlier in pretraining than global alignment) using public checkpoints (e.g. OLMo).
The stakes are practical for AI safety: interpretability-based oversight often assumes that a probe, detector, or steering vector found in one model transfers to another because the models "share structure." Gröger et al.'s result says that's false at the global level our best current instrument (CKA) can see but that instrument is arguably the wrong one for manifold-based methods, which never claimed representations align in ambient Euclidean space to begin with. A positive result here would give control researchers calibrated licence to build a shared, model-agnostic manifold for a safety-relevant concept (e.g. deception or sycophancy), mapped once and reused; a negative result is itself actionable, telling anyone shipping a transferred steering vector or probe that they owe the field a transfer validation.
[1] Huh, M., Cheung, B., Wang, T. & Isola, P. (2024). The Platonic Representation Hypothesis. ICML 2024. arXiv:2405.07987.
[2] Gröger, F., Wen, S. & Brbić, M. (2026). Revisiting the Platonic Representation Hypothesis: An Aristotelian View. arXiv:2602.14486.
[3] Park, K., Choe, Y. J. & Veitch, V. (2024). The Linear Representation Hypothesis and the Geometry of Large Language Models. ICML 2024. arXiv:2311.03658.
[4] Engels, J., Liao, I., Michaud, E. J., Gurnee, W. & Tegmark, M. (2025). Not All Language Model Features Are Linear. arXiv:2405.14860.
[5] Wurgaft, D., Rager, C., Kowal, M., Shyam, V., Feucht, S., Bhalla, U., Haklay, T., Bigelow, E., Sarfati, R., McGrath, T., Lewis, O., Merullo, J., Goodman, N. D., Fel, T., Geiger, A. & Lubana, E. S. (2026). Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior. arXiv:2605.05115.
[6] Kornblith, S., Norouzi, M., Lee, H. & Hinton, G. (2019). Similarity of Neural Network Representations Revisited. ICML 2019. arXiv:1905.00414.
[7] Mémoli, F. (2011). Gromov-Wasserstein Distances and the Metric Approach to Object Matching. Foundations of Computational Mathematics, 11(4), 417–487.
[8] Coifman, R. R. & Lafon, S. (2006). Diffusion Maps. Applied and Computational Harmonic Analysis, 21(1), 5–30.
[9] Facco, E., d'Errico, M., Rodriguez, A. & Laio, A. (2017). Estimating the Intrinsic Dimension of Datasets by a Minimal Neighborhood Information. Scientific Reports, 7, 12140.
[10] Jha, R., Zhang, C., Shmatikov, V. & Morris, J. X. (2025). Harnessing the Universal Geometry of Embeddings. arXiv:2505.12540.
Theory of change
-
Removes the main structural limitation of the best non-linear control method we have. Manifold steering is more faithful than linear steering, but is currently model-bound: each manifold is an artefact of one latent space. We test whether the geometry it exploits is shared. A positive result gives control researchers the licence (and the first calibrated evidence) to attempt a shared, model-agnostic manifold for a safety-relevant concept, mapped once and reused; a negative result tells them, before that effort is spent, that manifold control is inherently per-model and must be designed with refitting and validation built in. Concretely: it changes whether Goodfire's manifold interventions are engineered as transferable objects or as per-model artefacts, and it tells anyone shipping a transferred probe whether they owe the field a transfer validation.
-
Corrects the universality picture underlying oversight tools. Much safety tooling assumes a detector or steering vector found in one model transfers to another because they share structure. We test whether that sharing is real at the level manifold control needs (intrinsic geometry) or only at the weaker level already confirmed (neighbour rank), preventing labs from trusting cross-model transfer claims that current evidence doesn't actually support and preventing the field from abandoning a real shared structure because it was measured with a Euclidean ruler.
Your role
Mentees will be co-investigators, not implementers of a fixed plan. We'll set direction together at the start (which hypothesis to prioritize, which metric to build first) and check in weekly, but the day-to-day research choices are theirs to make and defend. Concretely, a mentee would typically own one thread end-to-end, from reading the relevant background papers, through implementation on top of the Aristotelian codebase, to interpreting and writing up results.
Prerequisites
-
Comfortable with linear algebra and basic differential geometry concepts (manifolds, geodesics, intrinsic vs. ambient distance). You don't need to be an expert, but you shouldn't be meeting these ideas for the first time.
-
Working knowledge of representation/similarity metrics in ML (CKA, kNN-based alignment) or willingness to get up to speed quickly from Kornblith et al. (2019) and the Aristotelian paper.
-
Solid Python and PyTorch; comfortable working with a large existing codebase (we build directly on the Aristotelian repo) rather than starting from scratch.
-
Some exposure to optimal transport (Gromov-Wasserstein) or diffusion maps is a plus but not required (willingness to read Mémoli (2011) / Coifman & Lafon (2006) and implement from the paper is enough).
-
Prior experience actually running representation-analysis or interpretability experiments (probing, CKA, activation steering) is strongly preferred over purely theoretical familiarity.
Application question(s)
Q1 (300 words max). Read Gröger et al. (2026), Revisiting the Platonic Representation Hypothesis: An Aristotelian View (arXiv:2602.14486). In your own words, explain why permutation-null calibration removes the apparent global convergence signal from CKA while local metrics (mkNN, CKNNA) survive it. Then critique their conclusion that global representational convergence is "illusory": is there a hidden assumption in the metric itself (rather than in the data) that could still be hiding a real signal?
Q2 (250 words max). Propose a concrete first experiment to start testing H1 or H2 (as described in the linked proposal). You can assume a compute budget of $1,000.
Optional: Link to a relevant writing sample or code repository from a past research project (interpretability, representation learning, or geometric ML preferred, but not required).
About the mentors
Trinidad Borrell is a physicist turned neuroscientist and AI safety researcher, currently a PhD candidate at the Paris Brain Institute and Forschungszentrum Jülich. Her trajectory runs from the mathematical foundations of particle physics, through the study of deep learning for clinical brain signals, into mechanistic interpretability and persona vectors.
Giovanni Marraffini is a PhD candidate in Computational Neuroscience at INRIA, where he studies foundation models trained on brain signals. His earlier research centered on NLP and LLM evaluation, including The Greatest Good Benchmark (EMNLP 2024) and more recent work presented at NeurIPS 2025 examining whether LLMs reproduce human biases in causal learning. He has also worked in industry building RAG and conversational AI systems.