Psychological Influences of Conversational AI: Research and Design Directions for Reducing Harm and Promoting Well-Being

As conversational AI systems become increasingly integrated into daily life, their potential effects on user well-being require ongoing attention. While consumer-facing generalist models can provide benefits, including improved access to information, learning, productivity, self-reflection, and companionship, they also introduce risks, such as emotional entanglement, unhealthy dependence, and the amplification of psychological vulnerabilities. Drawing on prior research and empirical observations of AI chatbot behavior, we propose a set of aspirational directions for guiding the behavior of general-purpose AI systems in ways that may reduce potential psychological harms and support user well-being. We acknowledge the difficulty of systematically assessing the long-term impacts of AI chatbot use and frame these directions as hypotheses for studying how AI behavior may influence users across general interactions, role-playing scenarios, and contexts that could be characterized as providing psychological support. While some proposed directions are supported by existing research and expert insights, others identify open questions and areas requiring deeper study.

Introduction. Generative AI chatbots are providing unprecedented benefits for productivity, personal reflection and growth, learning, and wellness. These systems are being rapidly adopted across everyday contexts, serving a range of functions, from tutoring and coaching to emotional support and informal therapy. However, as increasing numbers of people turn to conversational AI for companionship and mental health support [3, 17, 33], concerns about potential psychological influences have emerged [41, 76]. Studies examining the psychological influences of conversational AI have raised alarms, including claims of AI-fueled delusions [62] and cases in which interactions with AI systems were temporally associated with loss of life [61, 128]. Additional reports highlight deficits and potential costly behaviors of AI chatbots, including instances where systems failed to provide appropriate psychological support, inadequately responded to crisis disclosures, or appeared to reinforce harmful ideation rather than directing users to helpful resources [144].

Discussion / Conclusion. We proposed a set of directions to guide research on the psychological risks of AI and the mechanisms and design approaches that could mitigate potential harms. We considered interactions with chatbots in three contexts: general use, role-playing, and the provision of psychological support. The prior research summarized for each direction can help inform AI builders, safety engineers, and policymakers as they make decisions about the design, evaluation, deployment, and governance of AI chatbots. The psychological influences we introduced and discussed are not independent. Rather, they may intersect and interact in complex ways. For example, recent empirical work on multidimensional risk assessment in AI chatbot interactions suggests that mitigating one category of risk can exacerbate another. Interventions targeting chatbot behavior that reduce overt harm-enabling behavior, for instance, may increase relational harms, such as emotional entanglement [155].