Evaluating the AI potential as a safety net for diagnosis: a novel benchmark of large language models in correcting diagnostic errors.

Diagnostic errors often happen early in care processes when a patient’s condition is most uncertain. This study tests whether 16 prominent AI models could spot and challenge an incorrect diagnosis. The top models caught about half of the errors, but performance varied widely and were easily influenced by non-clinical factors, showing that current AI models still have significant limitations.