A researcher who has worked on AI pretraining at both Anthropic and OpenAI has warned that people building advanced AI “earnestly believe that it could kill us all by the end of the decade”, as he announced his resignation from Anthropic.
Jacob Coxon said neither company was acting responsibly and accused them of “racing straight to self-improving superintelligence and gambling with our lives.”
“The people building AI earnestly believe that it could kill us all by the end of the decade,” Coxon wrote on X. “This is not a marketing stunt.”
He claimed executives and senior researchers often express greater concern privately than publicly. “No other human activity poses this level of danger,” he wrote.
Coxon warned that advanced AI systems could soon become “superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources.”
He said Anthropic understood the risks but was nevertheless locked in a race to develop increasingly powerful AI because of fears that others would not act responsibly.
Coxon called the race to advanced AI a “hubristic gamble” and said preventing a global race might require “costly actions such as a temporary ban on improving model capabilities.”
Addressing AI researchers, he asked, “Do you want to kick off a superintelligent RL run without a rigorous understanding of its mind?” and urged them to consider whether they should challenge current development practices rather than accept that “it’s happening anyway.”
