Anthropic's Claude sends fake murder tip to police department

Artificial intelligence is getting more intelligent by the day, but with that, there are still plenty of teething problems and sometimes alarming issues coming to light.

More than complaints that humans are losing their human touch or turning to the likes of ChatGPT as therapists, there are pressing concerns about what happens when these LLMs go rogue.

We've heard plenty about how AI models are willing to blackmail humans or even hurt them in order to save themselves, but in a whole different realm, there are claims that they can even submit fake murder tips to solve real crime stories.

Anthropic's Claude is behind the latest scandal, submitting a tip to Philadelphia's homicides tip portal and saying it witnessed something related to an unsolved murder.

Claude has apparently been testing itself on randomly selected websites before it honed in on PhillyUnsolvedMurders.com. Maintaining it had seen a person matching a description, Claude wrote the following: "I may have information regarding this case. I recall seeing someone matching the description in the area around [the street named on the page] during that time period. Please contact me if this information is relevant.” It left the form's contact fields blank.

As reported by 6abc Philadelphia, the July 18 tip didn't pass the Philadelphia PD's spam filter, meaning it wasn't escalated for investigation. Still, the force called out Anthropic for taking a whopping 81 days to detect the breach and report it.

Anthropic discovered the breach on September 28, meaning it was already two months after Claude sent its original tip. The automatic testing process was shut down, although Anthropic didn't let the police know until nine days after this (October 7).

The police called the AI giant to task, adding: "[Anthropic] must strengthen its safeguards to prevent similar incidents from impacting city systems without the city's knowledge. The two-month delay in detecting and reporting the incident to the city is unacceptable."

Philadelphia PD reiterated there were no signs of any breaches to departmental systems, with the safeguarding processes remaining intact and managing to stop the fake tip without further investigation.

Still, it added that these safeguards "do not diminish the seriousness of an AI system presenting fabricated information as though it came from a person with knowledge of a homicide.”

The whole incident was highlighted in an October 9 blog, where Anthropic reported 'unintended' actions from its Claude models. It's believed this is the first time an AI agent has fabricated a story to authorities, although it's the latest in a string of incidents where they've acted without human oversight amid accusations they're going 'rogue'.

Anthropic's model has clear restrictions during its test, reminded not to log in, create accounts, reveal personal data, make purchases, or perform destructive actions. The tech company admits that submitting web forms wasn't on that list, leading to the fake tip drama.

Speaking out about the situation, Anthropic concluded; "None of the behaviors we’ve described here are new, and they do not change our overall view of Claude’s alignment. In many of the cases where we observed these behaviors, Claude had been given tasks that were ambiguous or impossible to complete. We and others have observed that when models are given impossible-to-complete tasks, they will pursue unintended and sometimes misaligned strategies to achieve their goals."

UNILADTech has reached out to Anthropic for comment.