AI-powered apps and bots barge into medicine throwing serious challenges

Artificial intelligence is transforming industries ranging from software and law to entertainment and education. In medicine, however, it is creating unique challenges, as patients increasingly turn to AI tools for health advice — something physicians like Dr. Cem Aksoy are encountering firsthand.

Aksoy, a medical resident at a hospital in Ankara, Turkey, recalls that an 18-year-old patient and his family became alarmed after he was diagnosed with a cancerous tumor in his left leg. Seeking more information, they consulted OpenAI’s ChatGPT, which told them he might have only five years to live.

That prediction proved incorrect. A plastic surgeon successfully removed the tumor in July. “He was essentially cured after the operation,” Aksoy said.

Weeks later, however, the young man phoned Aksoy in distress. “He said, ‘I started coughing recently, and ChatGPT told me it could possibly be metastasis to my lungs,’” the doctor recounted, referring to fears that the cancer had spread. The patient even felt he needed to prepare a will. Medical tests later confirmed his lungs were healthy — the cough was simply the result of recently taking up smoking.

“When someone is distressed and unguided,” Aksoy said, an AI chatbot “just drags them into this forest of knowledge without coherent context.”

A spokesperson for OpenAI said its newest models have significantly improved how they handle health questions. ChatGPT isn’t intended as a substitute for a medical professional’s guidance, the company said.

A growing number of mobile apps available on the Apple and Google app stores claim to use AI to assist patients with their medical complaints – even though they’re not supposed to offer diagnoses.

Under U.S. Food and Drug Administration guidelines, AI-based medical apps don’t require approval if they “are intended generally for patient education, and are not intended for use in the diagnosis of disease or other conditions.” Many apps have disclaimers that they aren’t a diagnostic tool and shouldn’t be used as a substitute for a physician. Some developers seem to be stretching the limits.