Diagnostic Reasoning Prompts Reveal the Potential for Large Language Model Interpretability in Medicine

“Clinical reasoning is a set of problem-solving processes specifically designed for diagnosis and management of a patient’s medical condition. Commonly used diagnostic techniques include differential diagnosis formation, intuitive reasoning, analytical reasoning, and Bayesian inference. “

“The finding that GPT-4 can successfully imitate the same cognitive processes as physicians to arrive accurately at an answer is significant and will have implications for how language models can be used in the clinical workflow. A model that not only provides an accurate diagnosis but also produces a clinical reasoning rationale to support it is a major step towards interpretability and trust. Strategies that align model outputs in this way could help transition LLMs away from a purely “black box” model to one in which final model outputs are always accompanied by a robust, interpretable rationale grounded in clinical reasoning (Figure 3). This will offer clinicians a means to evaluate how a LLM arrived at an answer and assess for appropriate quality and logic. Ultimately this framework holds the potential to transform LLM systems into interpretable tools that can credibly be used in medicine.”