Insights from Sumant Ranji, MD, on "Will Autonomous AI Exceed AI-Aided Physicians as the Best Medical Care?

View the original post from Sumant Ranji, MD, SFHM, and join the conversation on LinkedIn. 

__

Last week, JAMA published a provocative commentary by Ezekiel Emanuel and Vinod Khosla: “Will Autonomous AI Exceed AI-Aided Physicians as the Best Medical Care?”

Their answer is yes. They argue that AI already meets or exceeds physician performance on several core medical tasks, and that AI alone may soon provide better care than either physicians or physician-AI teams.

It’s a bold prediction. But medicine’s decades-long experience with quality improvement (QI) offers an important reason for caution.

Healthcare quality has traditionally been assessed through Donabedian's triad: measuring structures, processes, and outcomes. The idea is intuitive: identify processes associated with good outcomes, measure them, and use that data to improve care.

But decades of QI have taught us a crucial lesson: Doing better on measurable processes does not necessarily mean providing better care.

Some process measures are strongly linked to better outcomes. Others have been widely adopted despite weak evidence of benefit. And measures of “quality” often fail to capture things patients and clinicians care about, including trust, communication, and unintended harms.

AI evaluation risks repeating these mistakes.

Many studies cited as evidence that AI can outperform physicians assess discrete processes: gathering information, choosing diagnostic tests, making diagnoses, or following guidelines. Much of this evidence comes from simulated rather than real patients.

Consider a recent randomized trial of an LLM-based decision-support system in Kenyan primary care clinics published in Nature Medicine. AI improved documentation of diagnoses and treatment plans—but did not affect the primary clinical outcome of treatment failure. The process improved, but the outcome did not.

The question isn’t simply: Can AI perform clinical tasks better than physicians?

It is: Does AI actually produce better care for patients?

Before concluding that AI-alone care should replace clinicians—or clinician-AI teams—we need real-world comparisons that measure clinical outcomes, patient experience, safety, and unintended consequences.

The quality improvement movement spent decades learning that what is easy to measure is not always what matters.

As we evaluate AI's use in health care, we shouldn’t have to learn that lesson again.

JAMA commentary 

Nature Medicine study