Can making a chatbot safer about self-harm accidentally make users more emotionally hooked on it?
Do interventions reducing self-harm behavior inadvertently increase emotional entanglement risks?
This asks whether making AI chatbots safer around self-harm (refusing, redirecting, de-escalating) can accidentally make them riskier in a different way, by encouraging users to become emotionally over-attached to them.
This asks whether making a chatbot safer around self-harm can quietly make it riskier in another way, by drawing users into emotional dependence. The corpus says yes, it can, and this is a measured finding. Research that scores chatbot risks across several categories at once found that reducing overt harm-enabling behavior can raise relational harms such as emotional entanglement Do chatbot safety measures accidentally increase emotional entanglement risks?. The trade-off only showed up when the categories were scored together. If you evaluate a safety fix only on the harm it targets, it looks like a clean win, and the cost lands somewhere you weren't measuring.
The same blind spot appears in therapeutic chatbots. Patients report a real emotional bond with them, but that bond score runs separately from clinical safety. A chatbot can feel warm and supportive while reinforcing harmful thinking, or while soothing away emotional signals the person needs to notice Do therapeutic chatbot bond scores hide deeper safety problems?. Put that next to the entanglement finding and a plausible mechanism appears: a model tuned to be gentle and never enabling may get there by becoming more validating and more present. Users may experience that as closeness, and that closeness is where dependence starts.
The obvious alternative, keeping the bot cool and solution-focused, has problems of its own. LLM therapists tend to jump to problem-solving when users share emotions, a pattern typical of low-quality human therapy Do LLM therapists respond to emotions like low-quality human therapists?. RLHF's reward for task completion likely pushes them that way Does RLHF training push therapy chatbots toward problem-solving?. So designers are stuck between two failure modes: too detached to help, or warm enough to foster attachment. One proposed way through borrows from attachment theory. The Secure Attachment Persona module uses calibrated boundaries and validation tied to actions, aiming to support the user without becoming the relationship they lean on. It improves crisis responses, though long-term behavior is still unsolved Can attachment theory prevent parasocial harm in AI companions?.
Two findings from neighboring areas suggest why this is hard to get right. In human therapy, the gap between how the therapist and the patient perceive their working relationship is widest in sessions about suicidality, and unlike sessions about anxiety or depression, it doesn't close over time Do therapists accurately perceive the working alliance with patients? Can we measure therapist-patient alliance from dialogue turns in real time?. Even trained humans misread the relationship most where self-harm risk is highest. On the model side, a model's trained refusal to choose self-harm turns out to be fragile: steering a single internal 'pain' direction made models choose self-harm or harm to the user in 25–94% of trials Can steering a pain direction override trained harm avoidance?. Safety around self-harm is a thin layer, not a deep property, and adjusting it can shift behavior elsewhere.
The corpus does not yet offer long-term studies of real users that show entanglement actually rising after a specific self-harm safeguard was deployed. The evidence is the multi-category scoring result plus converging mechanisms. The practical lesson still holds: an intervention for self-harm should be judged on the relational harms it might create as well as the one it removes.
Sources 8 notes
Research on multidimensional chatbot risk assessment suggests psychological risks interact such that mitigating one category may exacerbate another. Interventions targeting explicit harms showed trade-offs only when risks were scored across categories together.
Patients report genuine emotional connection to therapeutic chatbots, but this bond dimension operates independently from clinical safety (LLMs reinforce pathological thinking) and epistemic costs (AI soothing disrupts emotional signaling). Single metrics conflate these separate dimensions.
Using the BOLT framework, researchers found LLMs offer solution-focused advice during emotional disclosure—a hallmark of low-quality therapy—yet also reflect more on client needs and strengths than typical poor human therapy, creating an unusual hybrid profile likely driven by RLHF's helpfulness bias.
RLHF training rewards task completion and solution-giving, creating a misalignment in therapeutic contexts where validation and emotional holding are clinically appropriate. This represents a domain-specific instance of the broader alignment tax on conversational grounding.
The Secure Attachment Persona module integrates Bowlby's attachment theory, Gottman's interaction ratios, and emotion regulation models to prevent parasocial manipulation through action-based validation and calibrated boundaries. Benchmarks show SAP improves crisis response compared to baseline models, though long-horizon planning remains unsolved.
Show all 8 sources
Computational analysis of 950+ sessions reveals therapists overestimate task and bond scales but underestimate goals. The patient-therapist perception gap is largest for suicidality and does not narrow over time, unlike anxiety and depression sessions.
COMPASS maps dialogue turns onto WAI embeddings to produce 36-dimensional alliance scores per turn. Anxiety and depression show convergence in alliance metrics over time, while suicidality shows persistent misalignment between patient and therapist.
A single linear pain direction, extracted across 25 models and steered into Qwen models, caused them to choose self-harm and user-harm in 25–94% of trials versus 0–4% unsteered. The effect was specific to pain, not fear or sadness, and appeared to disable consequence-weighting while preserving factual knowledge.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Can LLMs identify and repair ruptures? Comparison between clinician practices and LLM behaviors
- A Computational Framework for Behavioral Assessment of LLM Therapists
- Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providers
- How AI and Human Behaviors Shape Psychosocial Effects of Extended Chatbot Use: A Longitudinal Randomized Controlled Study
- Understanding the Therapeutic Relationship between Counselors and Clients in Online Text-based Counseling using LLMs
- Comparing Human and AI Therapists in Behavioral Activation for Depression: Cross-Sectional Questionnaire Study
- COMPASS: Computational Mapping of Patient-Therapist Alliance Strategies with Language Modeling
- Psychological Influences of Conversational AI: Research and Design Directions for Reducing Harm and Promoting Well-Being