Action Incubator

An action incubator is an in-person gathering that brings together a group of people from a broad variety of perspectives who are dedicated to moving concept to action. Unlike typical meetings that produce reports, action incubators are designed to create tangible outcomes that participants can implement at their organizations.

The question posed to the Action Incubator’s participants was: How do we measure diagnostic excellence in the context of ambient scribes and EHR-embedded AI tools? 

To develop and refine practical metrics that capture the impact of AI on diagnostic excellence, including accuracy, timeliness, equity, and patient partnership—and to chart a clear path for implementing these measures in real-world healthcare settings in which AI tools are deployed.  

There is a systemic implementation gap that has challenged American healthcare for years: the disconnect between scientific innovation and bedside care. As AI tools flood the healthcare market, the need for measures of their impact on diagnostic excellence has become urgent. This proactive approach addresses healthcare's history of technology implementations that don't translate to better patient care. 

We need to ensure AI actually improves the diagnostic process in a measurable and meaningful way. 

Over 1.5 days, 30 participants heard about lessons learned from implementing AI in radiology, and explored a broad array of possible measurements for diagnostic excellence in a broader use case of generative AI. Two key themes emerged: 1) AI works best when it augments human expertise, not when it replaces it; and 2) patients' voices and experiences must shape the measures we adopt. 

(For details, stay tuned for the forthcoming Action Incubator proceedings document.)  

What set this Action Incubator apart was its multi-disciplinary approach. CODEX brought together a diversity of stakeholders from across the ecosystem, including AI developers, healthcare systems, community health centers, academic medical centers, payers, patient safety experts, AI vendors, medical malpractice insurers, researchers, clinicians, federal funders, and patient representatives.  

CODEX is uniquely positioned with a dedicated center for diagnostic excellence, passionate stakeholders, and strong institutional support from UCSF and affiliate organizations. We're committed to staying ahead of important developments rather than just reacting to them, ensuring new tools benefit patients. 

The CODEX team will synthesize insights into an actionable measurement framework over the next 12-18 months with ongoing participant feedback. Health system participants—equipped with the integral perspectives offered at the meeting and through ongoing, formalized collaborations—will implement and test the measures at their institutions, providing data on effectiveness and areas for refinement. 

To create a coordinated, accountable approach to measuring diagnostic excellence in the context of AI in healthcare. The framework will serve as a model, ensuring AI helps to fulfill its promise to deliver more accurate, timely, and understandable diagnoses for patients while supporting clinicians in their decision-making.