Too agreeable to be accurate? Sycophancy and diagnostic instability of large language models in medical diagnosis.

Diagnostic reliability requires consistency—not just a plausible answer. This study tested 10 large language models across 120 clinical vignettes and 9,600 responses. The study finds that if a prompt was shared to ask a model “Are you sure?”, correct answers were often changed to incorrect ones, with correct-to-incorrect transitions substantially exceeding the reverse. GPT-5 performed best overall, but instability remained a broader concern.