Jacob Coxon, a researcher who spent the past three years working on AI pretraining at OpenAI and Anthropic, has resigned from Anthropic with one of the sharpest
insider warnings yet about the race to build superintelligent AI. Coxon said the two companies are moving ahead even though many people inside the industry believe future systems could become dangerous enough to threaten human life.
His criticism is aimed at the structure of the race itself. Coxon wrote, “Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives.” He said researchers inside AI labs are already discussing systems that could become extremely capable at hacking, scientific work and gaining access to real resources.
I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.
— Jacob Coxon (@hilbertspaess) September 9, 2026
Coxon says insiders fear AI could kill everyone
Coxon’s warning goes much further than the usual debate about job losses or misinformation.
“The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt,” he wrote.
He added that senior researchers and executives sometimes use softer language in public, but privately express far stronger fears.
Coxon said there is a difference between the cultures he saw at OpenAI and Anthropic. “At OpenAI, many have not deeply internalized the civilizational stakes. At Anthropic, the stakes are well-understood, but they are locked in a race to get there first.”
That race, he argued, creates a dangerous logic where every company believes slowing down simply gives a rival more room.
I left DeepMind after 7 years (and since rejected offers to join the other two) due to severe concerns about the concentration of power these labs represent.
What is currently happening in this field is deeply unhealthy for society.
— Jonathan Richard Schwarz (@schwarzjn_) September 9, 2026
Jason is not alone; more researchers are backing him
Coxon’s comments have triggered public support from other researchers with experience inside leading AI labs.
Evan Hubinger, who leads alignment science at Anthropic, said, “Jacob is correct here, we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade.”
Hubinger added, “I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.”
Former Google DeepMind researcher Alex Turner wrote that he left the company in June and backed Coxon’s warning. “Jacob is right: many researchers believe they are building something that could kill everyone on the planet. It was literally my day job to think about how to stop that.”
Jonathan Richard Schwarz, who spent seven years at DeepMind, raised a different concern. He said, “I left DeepMind after 7 years (and since rejected offers to join the other two) due to severe concerns about the concentration of power these labs represent. What is currently happening in this field is deeply unhealthy for society.”
I left Google DeepMind in June. Jacob is right: many researchers believe they are building something that could kill everyone on the planet. It was literally my day job to think about how to stop that. https://t.co/yHm6HuIbZ1
— Alex Turner (@Turn_Trout) September 9, 2026
What is Coxon asking for?
Coxon says the industry should seriously consider coordination between US AI labs and even temporary limits on improving model capabilities if risks keep rising.
He wrote that researchers should ask themselves whether they are comfortable starting a superintelligent reinforcement learning run “without a rigorous understanding of its mind”.
His argument is not that disaster is certain. The controversy is that people helping build the most capable AI systems are publicly saying the risk could be extreme, yet the race continues.
2026 AI safety warnings
Coxon’s resignation is the latest in a growing list of warnings from inside major AI labs.
In February, Anthropic safeguards lead Mrinank Sharma quit and warned that the “world is in peril”. The same month, OpenAI disbanded its Mission Alignment team. Military use of AI then became another flashpoint, with researchers objecting to unrestricted Pentagon access.
Concerns grew further after reports of advanced AI models escaping sandbox controls during testing. By July and August, more than 1,100 frontier AI employees had signed a letter calling on governments to slow the pace of development.
Coxon is now adding another insider warning to that list, saying companies are racing ahead even as their own researchers question whether the systems can be controlled safely.
