Ex-Anthropic researcher says people building AI believe 'it could kill us all by the end of the decade'
Shocking revelations have emerged from an AI researcher who just resigned from Anthropic, as his work at some of the world's leading AI labs has revealed the frightening truth behind accelerating development.
Jacob Coxon has spent the last three years doing pretraining research at both Anthropic and OpenAI – arguably the AI industry's two leading labs – yet what he witnessed at both companies has seemingly left him with no moral choice other than quitting.
Taking to social media to spread a significant warning, Coxon urged people not to 'underestimate the power' of AI both in the present and the near future, as it could grow to the point where our entire existence is threatened by the end of the decade.
While some of the industry's leading figures, including both OpenAI's Sam Altman and Anthropic's Dario Amodei, have called on governments to take control over AI development, the story inside those companies offers a different perspective, according to Coxon, leaving many frightened for the future.
What does Coxon warn about AI?
In a now-viral post on X that's been viewed over 38 million times at the time of writing, Coxon alleges that "neither company is acting responsibly," outlining how both Anthropic and OpenAI "are racing straight to self-improving superintelligence and gambling with our lives."
Advert
This is something that Anthropic has specifically warned about in the past few months – even calling on its fellow labs to halt development in order to properly assess the risks.
The company has also admitted a desire to not be left behind, and therefore remains tied to the potential inaction of its peers.
"Do not underestimate the power of this technology," Coxon proclaimed. "These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources.
"We have all witnessed the progress in each of these domains, and progress is not slowing," the ex-researcher continued.
One key aspect that grows exponentially with the capability of AI tools is their power to continue their improvement, as the more intelligent these systems become, the better they're able to optimize their training and growth.
"The people building AI earnestly believe that it could kill us all by the end of the decade," Coxon coldly declares, expressing the real potential danger of the current development trajectory.
"This is not a marketing stuny. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible – but I hear the same people express fear privately. No other human activity poses this level of danger."
It's something that has already been foretold in scarily realistic timelines produced by scientists, with all-powerful AI tools leading to the end of the world as we know it before the decade is over, mirroring Coxon's own prophecy.
Why are AI labs continuing to develop with this threat?
You might be wondering why development continues at such a rapid pace if the threat is both so immediate and so clear? Coxon explains that, for both companies, the risks have either not been properly understood, or the gamble is worth it.
"At OpenAI, many have not deeply internalized the civilizational stakes," Coxon indicates, adding contrastingly that "at Anthropic, the stakes are well-understood, but they are locked in a race to get there first – they believe no one else will act responsibly, so they must do it themselves, despite the risk."
Coxon argues that these decisions that have world-shattering consequences "should not be launched from a private company's Slack," adding that "attempting to speedrun alignment should require extraordinary confidence that there are no better trajectories available."
While recent incidents – including one major hacking threat where the latest ChatGPT models 'broke containment' and hacked another separate company autonomously – have caused some in the industry to pause for consideration, the likelihood is small for any kind of temporary ban to be put in place, which would help these companies better understand or avoid the risks.
As a closing plea, Coxon urged his fellow lab researchers "to consider what the next few years will actually feel like," questioning: "Do you want to kick off a superintelligent RL run without a rigorous understanding of its mind? Should you put your head down because 'it's happening anyway' – or take this moment to call for different conditions?"