Exposing the fragility of LLM reasoning through bias-inducing prompts: evidence from BiasMedQA.

LLMs are seen as promising tools to support clinical decision-making but can be vulnerable to implicit bias. Recent LLM capabilities mimicking human reasoning could potentially address this problem. This model study demonstrates that LLM system design prompts do not consistently reduce cognitive bias in human clinical decision-making. The findings highlight the potential risk of clinicians and researchers over-relying on LLMs that appear to mimic human reasoning.