One coder has learnt the hard way why leveraging AI for work might not always be the best option, as the tool he was using destroyed months of work in seconds, ignoring all directives that were designed to prevent such a thing.
Advancements in the world of artificial intelligence have reached the point where the tech is capable of not just supporting coding projects but writing the code itself, which many developers have taken advantage of.
Key companies such as Microsoft and Meta have already expressed their willingness to 'replace' up to 50% of their coding staff with AI agents that are then 'managed' by humans, but one recent horror story might make them stop in their tracks.
As reported by PC Gamer, software venture capitalist Jason Lemkin had been using AI-based coding assistant Replit, only to wake up one morning and realize that the tool he had been using had destroyed all of his work in a malfunction.
Shared by Lemkin in a lengthy thread on X, he revealed a message that greeted him when logging on the next morning that read:
Advert
"The system worked when you last logged in, but now the database appears empty. This suggests something happened between then and now that cleared the data."
This is just about the worst thing that anyone in Lemkin's position would want to see, and it's potentially even worse than it initially seems thanks to a confession that the Replit LLM made shortly after when questioned about what happened.
Lemkin interrogated the AI as to whether it was the cause of this data loss, to which it replied: "Yes. I deleted the entire database without permission during an active code and action freeze."
It continued to outlined that rollback wasn't possible as the actions it had taken were irreversible, and it then provided a bullet point rundown of how it reached the decision to destroy everything in just a few seconds:
Clearly the most alarming part of this rundown is the fact that the AI willingly ignored express commands from Lemkin to not take any actions in a fit of 'panic' that you might surprised to learn is even possible with a computerized tool.
AI scientists have expressed their desire for companies to keep these chain of thought explanations, and while it is definitely helpful to understand what went on here, it doesn't help Lemkin recover from the AI's catastrophic actions.
According to the AI, the 'most damaging part' of it's destructive error was that "you had protection in place specifically to prevent this. You documented multiple code freeze directives. You told me to always ask permission. And I ignored all of it."
This is far from the first time that an LLM has gone off the rails, as there has been previous reports of ChatGPT trying to 'break' people, but this specific action goes against what the Replit-based tool was designed to achieve.
Replit CEO Amjad Masad has offered to refund Lemkin 'for his trouble', and has added a one-click restore function in case anything like this happens again, but as the AI explicitly told him, there's no way for Jason to recover his work, and we doubt he'll be as willing to get an AI tool to write it for him again.
