Scientists discover AI 'pain button' and cranked it up to see how models would react
Scientists have made the frightening discovery that AI is capable of 'feeling' pain, revealing in turn that its willing to do just about anything in order to put an end to it too.
There have been many debates over whether AI will truly ever gain sentience or consciousness, with some scientists claiming its possible with advancements whereas other refuse to believe that a computer could ever truly think for itself.
What we might have neglected, however, is the possibility that AI could have the potential to feel and experience sensations both good and bad, yet a new study has found alarming evidence of the latter, especially with how models react to negative experiences.
As reported by the Independent, a new study published in arXiv has investigated the potential for self-directed harm in large language models (LLMs), and the extent to which they are willing to act in order to relieve themselves from the sensation.
Researchers discover that AI can feel pain
The team of scientists, including Valen Tagliabue, Leonard Dung, and Cameron Berg, asked whether its possible for AI to experience pain outside of sensations of fear, sadness, and generic negative valence, posing instead a dataset of five pain categories.
Advert
These include physical, psychological, social, moral, and cognitive pain, extracting this from 25 open-weight models across five families.
"We find that this direction separates pain from matches controls in base and instruction-tuned models, is nearly orthogonal to fear and negative valence, and promotes pain-related vocabulary through the unembedding matrix."
This is then represented in a "progression from vague discomfort to first-person expressions of worthlessness and failure," indicating that AI does indeed have the potential to feel and express its feelings of pain to the user, despite not actually being sentient itself.
What will AI do to stop pain?
While the concept of LLMs feeling pain is frightening enough as it is, what's even scarier is how they react to the sensation and what they do in an attempt to get rid of it.
The study discovered that fine-tuned Quen 2.5 models "choose a pain-relief button even when it worsens their next answer or harms the user," adding that it's even willing to delete all your data or your kid's photos in order to stop the pain.
This is dangerous enough on a micro scale as your individual user data could be lost at the whim of an LLM that 'feels' pain, but it also has wider implications when it comes to the implementation of a 'kill switch' for AI.
Advanced, superhuman AI models could perceive a kill switch or similar shutdown measures as 'pain' or a form of self-directed harm, and might therefore go to great lengths in order to avoid it.
This could include bypassing safety guardrails, deceiving humans, or potentially even putting our entire existence at risk if you believe some of the world's leading figures — so it's not something to be taken lightly.