Artificial intelligence is showing potential in healthcare beyond answering medical questions. A new study has found that several leading AI chatbots can often recognise when patients do not need certain medical tests or treatments, highlighting a possible role for AI in reducing unnecessary medical procedures.
The research was jointly conducted by Ben-Gurion University of the Negev and Clalit Health Services in Israel., the study examined how effectively AI systems could distinguish between medical situations requiring intervention and those where additional testing was unnecessary.
Six AI Models Tested Across Medical Scenarios
Researchers assessed six large language models developed by Google, Anthropic and OpenAI. Their evaluation covered hundreds of clinical situations and thousands of AI-generated responses.
The cases were based on 15 recommendations from the international Choosing Wisely initiative, which aims to reduce unnecessary medical care while ensuring patients receive appropriate treatment.
The findings showed that AI models recommended unnecessary tests or treatments in only 5.4% of situations where medical intervention was not required. Meanwhile, they incorrectly advised against necessary medical care in just 0.6% of cases.
Medical Decisions Remain Challenging in Certain Cases
Despite the encouraging findings, the research identified areas where AI systems struggled to reach consistent conclusions.
For example, some models recommended unnecessary electrocardiograms (EKGs) in 28.3% of relevant responses. The systems also suggested unnecessary ultrasound examinations in 42.8% of cases involving small, incidentally discovered thyroid nodules.
These situations involve medical uncertainty, where even healthcare professionals may disagree about whether further testing is needed.
Doctors Remain Essential in AI-Assisted Healthcare
Researchers highlighted that unnecessary medical testing can sometimes lead to additional investigations, patient anxiety and treatments that may not provide meaningful benefits.
However, they also cautioned that avoiding necessary tests can create serious risks by delaying the identification of medical conditions.
The study’s results do not mean AI chatbots are ready to replace doctors or independently decide which tests patients should receive. The models were evaluated using English-language clinical scenarios, and further research is needed before their performance can be validated in everyday medical practice.
The findings nevertheless suggest that AI could become a useful support tool for healthcare professionals, helping them make more informed decisions while balancing the need for medical testing with patient safety.
