32-year-old woman dies after a hospital AI algorithm locked her out of an ICU bed
A 32-year-old woman died after a hospital AI algorithm locked her out of an ICU bed.
Artificial intelligence is increasingly being recognised as a revolutionary force in healthcare, one that could one day help diagnose diseases earlier, reduce costs, and extend lives. Some families have even used AI tools to slash their medical bills while researchers are exploring its potential to cure all the world's diseases.
But a devastating case from Brazil has laid bare the other side of that story.
Rebeca Cardoso Tenente Molina had gone to the hospital in São João Nepomuceno seeking treatment for gallstones, Brazilian news publication MG1 reported.
Advert
What should have been a manageable medical situation turned fatal after a state-run AI system delayed the 32-year-old's access to an intensive care unit (ICU) for five days.
As Molina's condition rapidly deteriorated, her doctors determined she needed to be transferred to an ICU in the city of Oliveira, approximately 186 miles away.
However, the transfer was not in the hands of her doctors but Brazil's State Regulation Operations Center, known as Core-MG, which uses an AI-driven system to assign hospital beds based on automatically generated patient severity scores.
According to Molina's twin sister and family lawyer Sâmela Cardoso Tenente Furtado, the system assigned Molina a score of 6.8, far below the score of 10 that her condition actually warranted.
Because of the lower score, other patients with marginally higher scores continued to be prioritised ahead of her.
Speaking to MG1, Molina's family believe the five-day wait led to her death.
“What we saw was that doctors lost the autonomy to decide if a patient is very seriously ill,” Furtado told MG1. “The one who has to accept whether a patient is seriously ill is no longer the doctor who is there experiencing that reality with the patient, it’s the Core.”
The family pursued emergency legal action in an attempt to force the transfer, but the delay proved too long.
“She would have been a 10, and the system only accepted her as a 6.8,” Furtado continued. “So she couldn’t progress properly in the system because a patient at 8, a patient at 6.9 would jump ahead of her. And the system wouldn’t accept increasing her severity level [...] because of the tests that were constantly feeding it data.”
Molina's sister explained: “My sister, other people, are not just numbers, they are not just protocols, they are not just a CPF [Brazilian tax ID number] thrown into the system. They have families, they had dreams, they had a whole life ahead of them.”
In a statement issued after the Core-MG system's launch, Minas Gerais' Deputy Secretary of Health, Poliana Cardoso Lopes, said the system provides a bed map updated three times a day, offering 'much more control over the process and generating better data on the clinical condition and needs of each person waiting for a bed.'
The state health department told MG1 that transfer decisions are based on matching bed availability to clinical need and claimed that Core-MG had not changed the protocol for patient transfers.