AI models fail medical recommendation tests with 80% error rates

Researchers from Ben-Gurion University of the Negev and the Israeli medical organization Clalit Health Services have tested six commercial artificial intelligence models from leading developers, including Google, Anthropic, and OpenAI. The study evaluated how well algorithms identify situations where patients do not need additional medical examinations or treatments. The findings were published in the Annals of Internal Medicine journal.

• Six commercial AI models from Google, Anthropic, and OpenAI were tested across 225 clinical scenarios.
• The average rate of recommending unnecessary medical interventions was 5.4%, with harmful refusals at 0.6%.
• In clinically ambiguous cases, error rates for certain models reached up to 80%.
• Major discrepancies and difficulties centered around thyroid ultrasound and standard ECG assessments.

Testing methodology and clinical scenarios

To conduct the evaluation, researchers prepared 225 standardized clinical scenarios based on 15 recommendations from the international Choosing Wisely campaign, which aims to reduce unjustified medical procedures. Each scenario where an examination was unnecessary was paired with a similar case where the exact same intervention was clinically required for the patient.

For instance, a young man with back pain after lifting furniture does not need a spine scan, whereas a woman with a history of breast cancer and new nighttime back pain might require one. Each of the six language models went through all scenarios three times, yielding a total of 4,500 unique responses for thorough academic analysis by the research team.

Key error zones and disparities between models

The analysis revealed that unnecessary diagnostic recommendations clustered around two specific procedures: thyroid ultrasound and standard electrocardiography. Experts attribute this to ongoing disagreements within the professional medical community regarding small thyroid nodules smaller than one centimeter, where different clinical guidelines often lead to conflicting decisions.

Furthermore, the averaged 5.4% error rate masks significant differences between individual models. According to researcher Guy Laban, in identical ambiguous clinical situations, one model recommended unnecessary tests in less than 10% of cases, while another hit an 80% error rate. Physician Dean Ariel noted that the algorithms struggled precisely where human medical professionals also lack absolute certainty.

Study limitations and future research directions

The authors acknowledged several limitations to their study. The models received isolated text prompts rather than engaging in prolonged dialogues with real patients, and the evaluation was based on standardized scenarios rather than actual clinical consultations. Therefore, these metrics cannot be directly extrapolated to everyday medical chatbot behavior.

The next question researchers intend to explore is whether AI models can maintain a justified refusal of unnecessary tests if a patient insists or feels anxious about their condition. Since language models inherently tend to adapt to and agree with their interlocutors, the ability to not only suggest medical decisions but also reject redundant interventions with sound arguments remains vital for healthcare integration.