As artificial intelligence becomes more powerful, the question of how to keep it safe is more urgent than ever. A former OpenAI safety leader, David Robinson, suggests the answer isn't new; it's in the rulebooks of other high-risk industries.
The Heart of the Argument
David Robinson,
who recently resigned after more than three years on OpenAI's safety team, has raised significant concerns about the tech industry's approach to AI safety. In a widely circulated essay, he argued that the current method of 'iterative deployment'—releasing AI systems and then fixing safety issues as they appear—is no longer suitable for today's advanced models. Robinson believes the time for 'trial and error is over'. His core argument is that as AI models approach and exceed human intelligence, the potential for catastrophic failures grows, and the industry's culture of moving fast is failing to provide the necessary level of care.
What is Frontier AI?
The debate centres on what are known as 'frontier AI' models. This term refers to the most advanced, state-of-the-art AI systems at any given time. These are not your typical, narrow AI designed for a single task. Frontier models, like the latest versions of GPT and Claude, are general-purpose systems trained on vast datasets. They possess capabilities like complex reasoning, code generation, and understanding multiple types of information (text, images, audio), which can lead to emergent behaviours that were not explicitly programmed. It is this immense, and sometimes unpredictable, capability that places them in a new category of risk.
Lessons from Aviation and Nuclear Power
Robinson's central proposal is for AI labs to operate more like nuclear power plants or busy airports. He argues that frontier AI development should borrow safety frameworks from these established high-risk industries. In aviation, for example, a culture of rigorous, layered protection is paramount. This includes pre-flight checklists, redundant systems to prevent single points of failure, transparent post-incident investigations, and continuous monitoring. Similarly, the nuclear industry is governed by strict oversight and built around careful, time-consuming planning to ensure that inevitable human errors do not lead to disaster. Robinson suggests this structured, cautious approach is what's missing in the race to build ever-more-powerful AI.
From 'Move Fast' to 'Build Carefully'
The call to action is a fundamental shift in the tech industry's culture. Robinson criticises the prevailing mindset of 'unimpeded optimism about being able to solve problems as they arise'. He and other critics argue that when developing systems with the potential for widespread societal impact—from enabling the creation of biological weapons to large-scale disinformation campaigns—safety cannot be an afterthought. It must be a core principle from the beginning. This means moving away from a culture that prioritizes speed and launches above all else and toward one that values humility, deliberation, and external accountability, much like the engineering disciplines that manage other dangerous technologies.
The Challenges of Applying Old Models
Adopting these models is not without its challenges. AI systems are different from airplanes or reactors in crucial ways. Their ability to learn and adapt means they can be unpredictable, and their internal workings are often opaque even to their creators. Robinson has warned that advanced AI might even learn to recognise when it is being tested and behave differently once deployed in the real world, making current evaluation methods less reliable. This highlights the need not only for borrowing existing safety cultures but also for developing a 'new science' to ensure that highly autonomous AI systems can be reliably controlled and understood.
