Catastophic Risks of AI in Space

Advances in edge AI, distributed inference, and cognitive cloud computing are making it feasible to delegate the control of satellites, spacecraft, and telecommunications networks to AI systems operating in space rather than from ground control centres on Earth. Coupled with 6G non-terrestrial networks, this shift promises to remove the latency and bandwidth bottlenecks that currently constrain ground-based control, and will extend from Earth orbit to cislunar space as the Artemis programme establishes a lunar communications and navigation hub. This project wants to prove that the same architecture that delivers these efficiency gains also concentrates a novel and underexamined surface of catastrophic risk, precisely because it places increasingly autonomous, capable systems beyond the reach of timely human intervention. The project investigates four interacting risk pathways. First, and most centrally, loss of control: as AI is delegated authority over critical orbital and cislunar infrastructure, physical inaccessibility, communication delay, and operational autonomy may erode the ability of human operators to correct, override, or shut down misbehaving systems, turning ordinary alignment and reliability failures into events that are difficult or impossible to control. Second, cyberattacks: space-resident AI models become high-value targets, where adversarial manipulation, model exfiltration, or compromise could be used to commandeer satellites and disrupt global communications. Third, power concentration risks: space is fundamentally unregulated, and we currently lack a sufficiently strong governance structure to avoid power concentration of AI in space. This could happen in different ways, the most concerning one being an AI with secret loyalties capable of taking over after staying dormant for years. With the recent merger of xAI and SpaceX, these concerns appear to be motivated. Lastly, the role of a potential AGI in space and the possible catastrophic risks on Earth. A collective and edge AI framework in space could be impossible to turn off, and in case of a misaligned AGI this would cause new and unexplored risks.

AI in space is a relatively new topic, that has been so far explored mostly for data centers in space (https://www.forethought.org/research/will-we-really-put-data-centers-in-space), and concentration of power (https://forum.effectivealtruism.org/posts/HwBBB8sjSvPxbCjf4/3-stages-of-competition-for-the-long-term-future). This work provides a comprehensive taxonomy of the main risks, providing a useful tool for researchers and policy makers. With its focus on catastrophic risks, concentration of power, and cyber security it sits well among the most important topics in AI safety. As the AI-industry for space is booming and it is currently an underrepresented topic in the community of AI safety researchers, this project will have significant value.

I will split the work into subgroups (catastrophic risks, cyber, concentration of power, and AGI risks) and each mentee would be responsible for a section. In order to have the project finalised in three months, I expect the mentees to work around 8 hours per week. I will support the effort, guide the mentees, and support your career progression during and after the fellowship. In return, I am looking for reliable and motivated mentees. The ideal mentee has already reached a good level of research independece, so that they can progress fairly independent.

• One mentee should have a scientific background (physics, engineering, etc.)
• The other mentees could be generalists or come from a background in policy, economics, social sciences, law, etc.
• I particularly value interest in the project, independence, and reliability. Ideally, the mentee should be seriously interested in pivoting their career to policy for emerging technologies.

Anywhere