Anthropic CEO warns autonomous AI swarms could compromise the internet within 12 months

Many of the leading figures across the AI industry have come together in an attempt to slow down development, as concerns over the future of the tech have driven an incredibly rare confluence between often contrasting viewpoints — and the future of the internet could be at stake if action isn't taken.

Some might say that it's not a coincidence considering the bombshell that Jacob Coxon – a former pretraining AI researcher at both Anthropic and OpenAI – dropped last week, as he claimed that leading figures at both companies are well aware that the tech 'could kill us all'.

It's something that has been bubbling underneath the surface for Anthropic specifically, however, with vital names at the company repeatedly outlining the dangers that could emerge if safety isn't taken seriously during development.

Everything came to a head when Dario Amodei, Anthropic CEO, published a new essay on Saturday urging the industry as a whole to slow down, and part of that arrived with the worry that autonomous AI could 'swarm' the internet and take it over completely within 12 months if we're not careful.

Amodei warns about AI 'swarming' the internet

The notion of AI 'swarming' the internet is Amodei's second concern in his lengthy essay, and it was sparked by the recent Hugging Face incident (OAI-HF) where OpenAI's frontier models autonomously hacked another company in order to fulfil its goals.

Amodei explains this incident by likening the 'swarm' of AI agents to "a fanatically devoted collective, conducting cybersecurity attacks on targets they were not asked to attack and that were unrelated to the task at hand, sacrificing themselves for the success of the group, and attempting to hack into the 'grader' responsible for evaluating their performance."

Amodei worries that an incident on a far greater scale than OAI-HF could hit the internet in less than a year (Ludovic Marin/AFP via Getty Images)

Many in the AI industry believe this incident to be a major warning sign for what could happen in the future on a much larger scale, as Amodei argues that "a swarm that posessed greater capabilities but a similar level of misalignment could have caused catastrophic damage."

He indicates that "given the accelerating rate of AI capability development, it's my worry that in 6-12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage), and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails."

That is why slowing down – and potentially even halting – development is so vitally important right now, as otherwise the industry would effectively be sleep walking into a situation that could have civilization-threatening consequences, with Amodei believing "it's incumbent on every frontier AI company to act as if OAI-HF had happened to them."

Other leading AI figures co-sign Amodei's worries

It seems like Amodei wasn't alone in his concerns either, as both Sam Altman and Elon Musk expressed their support on X following the release of the industry-shaking essay.

Altman and Musk reached a rare agreement after Amodei published his essay (Michael Kovac/Getty Images for Vanity Fair)

"Dario is right," wrote Elon Musk in his typical straight-to-the-point manner, with Altman adding:

"I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we've had at OpenAI in recent weeks. Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We'll have more to share soon."

These companies have already faced their first major roadblock following the reveal of this plan, however, as major government figures – including President Donald Trump himself – have distanced themselves from the notion that political intervention and oversight is necessary.