As healthcare systems become increasingly overwhelmed, resulting in prolonged waiting times and rising costs, many individuals are seeking medical self-diagnosis through AI-powered chatbots, such as ChatGPT. According to a recent survey, approximately one in six American adults utilize chatbots for health advice on a monthly basis, as reported.

However, relying too heavily on the outputs of chatbots can be precarious, partly due to the challenge of determining what information to provide to chatbots in order to obtain the most accurate health recommendations, as indicated by a recent study led by Oxford.

The study’s findings revealed a breakdown in two-way communication, according to Adam Mahdi, director of graduate studies at the Oxford Internet Institute and a co-author of the study, in an interview with TechCrunch. “Those who utilized chatbots did not make more informed decisions than participants who relied on traditional methods, such as online searches or their own judgment,” he stated.

The study involved recruiting approximately 1,300 participants in the U.K. and presenting them with medical scenarios developed by a group of doctors. The participants were tasked with identifying potential health conditions in the scenarios and using chatbots, as well as their own methods, to determine possible courses of action, such as consulting a doctor or visiting the hospital.

The participants used the default AI model powering ChatGPT, GPT-4, as well as Cohere’s Command R+ and Meta’s Llama 3, which previously underpinned the company’s Meta AI assistant. According to the study’s authors, the chatbots not only made participants less likely to identify a relevant health condition but also more likely to underestimate the severity of the conditions they did identify.

Mahdi noted that participants often omitted crucial details when querying the chatbots or received responses that were difficult to interpret.

Furthermore, he added, “The responses received from the chatbots frequently combined good and poor recommendations. Current evaluation methods for chatbots do not accurately reflect the complexity of interacting with human users.”

These findings emerge as technology companies increasingly promote AI as a means to improve health outcomes. Apple is reportedly developing an AI tool that can provide advice on exercise, diet, and sleep. Amazon is exploring an AI-based approach to analyzing medical databases for “social determinants of health.” Moreover, Microsoft is assisting in the development of AI to triage messages sent from patients to care providers.

However, as TechCrunch has previously reported, professionals and patients have mixed opinions regarding the readiness of AI for high-risk health applications. The American Medical Association advises against the use of chatbots like ChatGPT for clinical decision assistance, and major AI companies, including OpenAI, caution against making diagnoses based on their chatbots’ outputs.

Mahdi recommends relying on trusted sources of information for healthcare decisions. “Current evaluation methods for chatbots do not reflect the complexity of interacting with human users. Similar to clinical trials for new medications, chatbot systems should be tested in real-world settings before being deployed,” he emphasized.