Reliability of LLMs as medical assistants for the general public: a randomized preregistered study.

LLMs can accurately diagnose a broad range of conditions in simulated studies where they are provided with a clinical summary of patient information, but this study tested how well LLMs performed when patients inputted their symptoms. The researchers found that while the LLM provided an accurate differential diagnosis and triage advice on its own, the accuracy dropped dramatically when humans users provided the information.