Sam Altman warns of 2 ways AI could go 'very badly' amid human extinction fears
Sam Altman has reaffirmed his concerns for the future amid calls to slow down AI development, outlining the two biggest 'threats' to the world that could potentially lead to human extinction.
While the call to halt progress was initially made by Anthropic CEO Dario Amodei in a frightening essay, his concerns were quickly co-signed by Altman as head of OpenAI and the driving force behind ChatGPT.
It marks one of the most significant unified moments in the AI industry as many of the leading development labs have come together in agreement over the concerns currently surrounding the tech, calling for action to be taken as a result.
These agreements haven't been without their detractors, however, as US President Donald Trump has on several occasions downplayed the dangers of AI, calling concerns raised by Altman and Amodei a 'hoax' and demanded that development continues at the regular accelerated pace.
Advert
That has likely prompted Altman to express his concerns in greater detail, and a recent warning has seen one of the most powerful men in the AI world reveal why things could go 'very badly' in the near future.
Altman shares how AI could soon go 'very badly'
Addressing the world in a new ominous post on X, Altman declared that there are two main routes that would lead to catastrophic, civilization-ending damage when it comes to AI development, and we are at risk of falling into inevitable collapse if action isn't taken.
"First, we could lose control of the future to AI," Altman explains. "This is unacceptable; we are unapologetically on Team Humanity, and AI must always serve people.
"To ensure that, we need ways to ensure that alignment and safety techniques stay ahead of progress in model capabiltiies," he continues, revealing why currently development trajectories need to slow down for labs to stay ahead.
"Second, we could end up in a world with too much concentration of power. If an extraordinarily powerful AI is used by one person or company to impress their worldview onto everyone else, the results could be extremely dystopian."
This is something that Altman has previously warned about when addressing G7 leaders earlier this year, yet it appears as if it's central to Trump's motivation for continued AI progress — as he has repeatedly declared that 'whoever wins AI, wins', with global power at stake.
"Avoiding these two threats requires walking a narrow middle path," Altman concludes. "For example, one country could gain too much power. Another example is one lab ending up with too much power."
How can the AI industry avoid these dangers?
Central to the arguments outlined by figures like Amodei and Altman right now is the notion of 'pacing' development — which of itself does not equal 'stopping', only reducing progress to where humans can confidently assert control over the future of AI.
"Progress has been rapid and will continue to be," Altman indicates, "but it would be slower than it otherwise could be; interventions like safety cases and monitoring have significant costs.
"Pacing will be well worth this cost; no amount of American competitive pressure should justify recklessness, or let capabilities get ahead of alignment and monitoring."
Amodei also suggested in his essay that every frontier AI company needs to have 'embedded evaluators' which would operate independently to ensure that there remains an adherence to safety practices and commitments, effectively preventing one lab from going off the rails in order to gain an advantage over its peers.
Democratic coordination is also seen as a must with Amodei stressing the importance of common safety standards across the industry and wider world, but that might prove to be challenging from a political perspective considering Trump's clear opposition already.