Measuring Alliance and Symptom Severity in Psychotherapy Transcripts Using Bert Topic Modeling
We aim to use topic modeling, an approach for discovering clusters of related words (“topics”), to predict symptom severity and therapeutic alliance in psychotherapy transcripts, while also identifying the most important topics and overarching themes for prediction. We analyzed 552 psychotherapy transcripts from 124 patients.
Drivers for symptom severity were themes related to health and negative experiences. Lower alliance was correlated with various themes, especially psychotherapy framework, income, and everyday life. This analysis shows the potential of using topic modeling in psychotherapy research allowing to predict several treatment-relevant metrics with reasonable accuracy.
By analyzing transcript features we aim to identify first-order processes that correlate with outcome and second-order processes that correlate with the alliance. We hope that this approach may enrich process research by providing new opportunities to analyze different processes and their respective interactions.
More recently, the focus has shifted to the processes and mechanisms of change that drive the effectiveness of psychotherapy (Lutz et al., 2021). At the core, process research tries to identify ingredients and mechanisms that either allow psychological interventions to work or increase their desired effects. Several processes have been identified, most prominently the alliance, but also many others (see Crits-Christoph et al., 2021). The effects and clinical utility of these processes have been shown in a landmark meta-analysis by Norcross and Lambert (2019). However, limitations prevail, such as unknown directions of the process–outcome relationship (e.g., is the alliance driving improvements in outcome or the other way around?) and difficulties to distinguish patient and therapist effects. Another limitation is a lack of research on what could be called second-order processes. Second-order processes relate to processes as processes relate to treatment outcome: They are the ingredients and mechanisms that allow processes to be effective. As such, for example, they aim to answer the important question ‘What ingredients and mechanisms does it take for the intervention to improve the alliance?’ just as processes aim to answer the question ‘What ingredients and mechanisms does it take for the intervention to improve outcome?’. Second-order processes are important, because often, it is far from self-evident how to foster therapeutic processes (Norcross & Lambert, 2019). A final limitation of process research lies in the fact that it is often focused on questionnaire data for large analyses as qualitative assessments can be very time-consuming.
The value of session-wise assessments of symptom severity and alliance can also be viewed through the lens of routine outcome monitoring (ROM) and patient-focused research (Castonguay et al., 2013). Based on research that has proven clinical intuition to be inaccurate (Ægisdóttir et al., 2006), data-driven prognostic models have been developed to predict treatment outcome for specific patients. These models rely on ROM by measuring symptom severity on a session-to-session level since symptom severity is a robust predictor of therapy outcome (e.g., Lorenzo-Luaces et al., 2020). By employing ROM and giving psychometric feedback to the therapist, patient-focused research seeks to increase therapists’ awareness of their patient’s progress or lack thereof.
Topic modeling can be used to predict relevant target variables, such as symptom load or alliance. Also, it can be used to analyze therapeutic processes by looking at the upcoming topics.
Imel et al. (2015) showed that topic modeling creates highly relevant topics in individual therapies. Using labeled topic modeling, they were able to discriminate different therapy approaches. Atzil-Slonim et al. (2021) found that topics accurately reflected the psychotherapeutic process. They were able to cluster topics in superordinate themes that comprised a broad perspective of therapy-related topics.
deep learning allowed for the development of neural topic modeling, which shows superior performance to the “classic” approaches