Can structured cognitive models improve LLM patient simulations for therapy training?
Does embedding Beck's Cognitive Conceptualization Diagram into language models produce more realistic patient simulations than generic LLMs? This matters because therapy training relies on exposure to diverse, believable patient presentations.
PATIENT-Ψ addresses two challenges in using LLMs to simulate therapy patients: fidelity (realistic communicative behaviors) and effectiveness (actual training value). The key innovation is integrating structured cognitive models from CBT with LLMs rather than relying on open-ended prompting.
The cognitive models are built on Beck's Cognitive Conceptualization Diagram (CCD), which links eight components: relevant history, core beliefs (19 categories across three types: helpless, unlovable, worthless), intermediate beliefs (rules, attitudes, assumptions), coping strategies, situations, automatic thoughts, emotions (9 categories), and behaviors. 106 diverse patient cognitive models were constructed, each specifying the full CCD pathway from history through beliefs to behavioral responses.
When these cognitive models are programmed into LLMs, the simulated patients closely resemble real patients across three dimensions: maladaptive cognitions, conversational styles, and emotional states — outperforming GPT-4 without the cognitive model structure. PATIENT-Ψ-TRAINER creates an interactive training framework where trainees practice CBT cognitive model formulation through conversation with the simulated patient, then compare their formulation to the underlying cognitive model used to program the agent.
Expert evaluators found the training "highly beneficial for improving CBT formulation skills and better-preparing trainees for interactions with real patients." Key advantages include customizable conversation styles and diverse patient profiles — addressing the practical problem that trainees have limited exposure to the full range of clinical presentations.
Since Can AI agents learn people better from interviews than surveys?, structured cognitive models may explain why PATIENT-Ψ exceeds GPT-4: the CCD provides the content richness (specific beliefs, automatic thoughts, coping patterns) that drives simulation fidelity, not just surface-level linguistic mimicry.
Source: Psychology Therapy Practice
Related concepts in this collection
-
Can AI agents learn people better from interviews than surveys?
Can rich interview transcripts seed more accurate generative agents than demographic data or survey responses? This matters because it challenges how we build digital simulations of real people.
content richness as fidelity driver; CCD provides this
-
Why do static persona descriptions produce repetitive dialogue?
Does relying on fixed attribute lists to define conversational personas limit dialogue depth and consistency? Research suggests static descriptions may cause repetition and self-contradiction in generated responses.
PATIENT-Ψ's structured cognitive models may avoid static persona problems by providing internal consistency via CCD
-
Can personas evolve in real time to match what users actually want?
Explores whether a persona that bridges memory and action can adapt during conversations by simulating interactions and optimizing against user feedback, without retraining the underlying model.
simulated patient interaction as training environment
-
How do we generate realistic personas at population scale?
Current LLM-based persona generation relies on ad hoc methods that fail to capture real-world population distributions. The challenge is reconstructing the joint correlations between demographic, psychographic, and behavioral attributes from fragmented data.
PATIENT-Ψ's 106 CCD-based cognitive models represent a structured approach to the calibration problem: rather than ad hoc persona generation, each patient is grounded in a validated clinical framework that constrains the joint distribution of beliefs, emotions, and behaviors
Click a node to walk · click center to open · click Open full network for a force-directed map
Original note title
LLM-simulated patients with structured cognitive models achieve high fidelity for CBT training — outperforming GPT-4 on maladaptive cognitions and conversational style