PATIENT-Ψ: Using Large Language Models to Simulate Patients for Training Mental Health Professionals

Mental illness remains one of the most critical public health issues. Despite its importance, many mental health professionals highlight a disconnect between their training and actual real-world patient practice. To help bridge this gap, we propose PATIENT-Ψ, a novel patient simulation framework for cognitive behavior therapy (CBT) training. To build PATIENT-Ψ, we construct diverse patient cognitive models based on CBT principles and use large language models (LLMs) programmed with these cognitive models to act as a simulated therapy patient. We propose an interactive training scheme, PATIENT-Ψ-TRAINER, for mental health trainees to practice a key skill in CBT – formulating the cognitive model of the patient – through role-playing a therapy session with PATIENT-Ψ.

ChatGPT and GPT-4 are able to solve some basic theory of mind tasks that generally require the ability to understand and attribute mental states to oneself and others. Inspired by such promise, we propose to use LLMs to simulate patients to train mental health professionals, with the goal of bridging the gap between their existing training methods and the complexities of real patient interactions. However, two major challenges must be addressed to realize this idea:

Fidelity. How can we build simulated patients that closely resemble the communicative behaviors of real patients with mental health disorders?

Effectiveness. How can we design an effective training scheme that allows trainees to benefit from interacting with these simulated patients?

In this work, we claim that integrating a patient’s cognitive model with an LLM can achieve high fidelity in simulating real patients with mental health disorders corresponding to that cognitive model. We implement this idea using the cognitive modeling framework in CBT (Beck, 2020), a popular paradigm in psychotherapy. We propose PATIENT-Ψ, a novel simulated patient agent that integrates cognitive modeling with LLMs.

106 high-quality and diverse patient cognitive models

We propose PATIENT-Ψ-TRAINER, an interactive training framework for mental health trainees to practice CBT cognitive model formulation using PATIENT-Ψ. Specifically, trainees converse with the simulated patient, PATIENT-Ψ, to formulate its cognitive model. Afterward, the system displays the original cognitive model that was used to program the simulated patient as a reference, allowing trainees to compare their results as feedback.

Evaluation results from the experts indicate that: (1) PATIENT-Ψ closely resembles real patients in terms of maladaptive cognitions, conversational styles, and emotional states; outperforming GPT-4. (2) Practicing with PATIENT-Ψ-TRAINER is perceived to be highly beneficial for improving CBT formulation skills and better-preparing trainees for interactions with real patients. Experts also highlighted several advantages of PATIENT-Ψ-TRAINER, including customized options to choose conversation styles and the diverse patient cognitive models.

1 Relevant history contains significant past events that contribute to an individual’s mental state.

2 Core Beliefs are deeply ingrained perceptions about oneself, others, and the world.

3 Intermediate beliefs are the underlying rules, attitudes, and assumptions derived from core beliefs and shape an individual’s thought patterns.

4 Coping strategies are techniques used to manage negative emotions. An external event or context.

(5 a situation) may trigger quick, evaluative thoughts without deliberation.

(6 automatic thoughts) stemming from the beliefs, leading to responses in terms of

7 emotions and

8 behaviors. A CCD-based cognitive model links the components together, creating a framework for identifying and understanding patients’ underlying cognitive processes. For all the components, we adopt the definitions and formulations put forth by (Beck, 2020). These include: three major core beliefs

(2 )—helpless, unlovable, and worthless—each with several fine-grained core beliefs, for a total of 19 core belief categories; 9 emotion

(7 ) categories; the rest of the components are formulated as free-text entries.