Giving a supposed 'expert' access to over a decade of your medical and health data sounds like a good idea in theory, but the practicality of it falls apart when that so-called expert is AI, as one man found out when he took the results to his doctor.
The rapid development of artificial intelligence across the last few years has prompted people to apply it to almost every scenario possible to see quite how well it stacks up.
While it might be steadily improving in some areas, there are still plenty of others that remain lacking comparatively, yet the optimism remains from within the tech world.
One field that appears to be a clear focus for the industry, however, is healthcare — as a number of doctors across the world are already integrating AI into their workflow from note-taking to analysis assistance.
Advert
It's even got to the point where Bill Gates has pledged $50 million to use AI for healthcare staff shortages in Africa, yet its use for this in a consumer environment might not quite be up to scratch just yet.
Nothing has displayed this more than an experiment conducted by Washington Post columnist Geoffrey A. Fowler, who decided to feed a decade of his Apple Watch health data into a new tool from ChatGPT, and it gave him an unexpected result.
According to ChatGPT Health, Fowler's data – which comprised roughly 29 million steps and 6 million heartbeat measurements – was so shocking that it was graded an 'F' for cardiac health, prompting him to immediately seek out the advice of a real-life doctor.
In reality it was quite the opposite though, as not only did his doctor inform him that there was nothing to worry about, but that his heart health was so good that his insurance likely wouldn't even cover an additional cardio fitness test to 'disprove' ChatGPT's assertion.
Seeking a second opinion he went to cardiologist Eric Topol at the Scripps Research Institute, although that produced a similar outcome.
"It's baseless," Topol informed Fowler, adding that ChatGPT's new tool "is not ready for any medical advice."
While it's certainly disappointing from a technological standpoint, at least Fowler has nothing to worry about when it comes to his own health, although he decided to try and get to the bottom of how it produced such an incorrect assessment.
Upon asking the tool for another score it produced a D, basing much of its measurement on his Apple Watch's VO2 max data, which Apple advises is only an 'estimate', alongside another metric known as 'heart-rate variability' which has a similar level of unreliability.
There were also major swings in resting heart rate – something that ChatGPT Health leaned heavily on – whenever Fowler got a new Apple Watch over the course of a decade, indicating a technological error rather than any notable issue with his actual health.
Ultimately, while it might seem flashy and provide a potentially useful warning system for real-life health issues, the analysis of ChatGPT and other similar AI tools is very much not based in actual medical fact, and experts might argue that it's misleading at the best of times.
"You'd think they would come up with something much more sophisticated," argued Topol, "aligned with the practice of medicine and the knowledge base in medicine. Not something like this. This is very disappointing."