Performance evaluation and benchmarking across 16 large language models on a comprehensive real-world emergency department triage data set.

Real-world emergency triage can be challenging due to time pressures and incomplete information. In this German preprint study of more than 16,000 emergency department visits, LLM triage performance as a diagnostic and care prioritization decision did not acheive significant agreement with human assessment. Most of the 16 models tested were overconfident, inconsistent, and weak at routing patients, underscoring the deficits of current AI models for appropriate clinical triage.