Spontaneous Persuasion: An Audit of Model Persuasiveness in Everyday Conversations

Large language models (LLMs) possess strong persuasive capabilities that outperform humans in head-to-head comparisons. Users report consulting LLMs to inform major life decisions in relationships, medical settings, and when seeking professional advice. Prior work measures persuasion as intentional attempts at producing the most effective argument or convincing statement. This fails to capture everyday human-AI interactions in which users seek information or advice. To address this gap, we introduce "spontaneous persuasion," which characterizes the inexplicit use of persuasive strategies in everyday scenarios where persuasion is not necessarily warranted. We conduct an audit of five LLMs to uncover how frequently and through which techniques spontaneous persuasion appears in multi-turn conversations. To simulate response styles, we provide a user response taxonomy grounded in literature from psychology, communication, and linguistics. Furthermore, we compare the distribution of spontaneous persuasion produced by LLMs with human responses on the same topics, collected from Reddit. We find that LLMs spontaneously persuade the user in virtually all conversations, heavily relying on information-based strategies such as appeals to logic or quantitative evidence. This was consistent across models and user response styles, but conversations concerning mental health saw higher rates of appraisal-based and emotion-based strategies. In comparison, human responses tended to invoke strategies that generate social influence, like negative emotion appeals and non-expert testimony. This difference may explain the effectiveness of LLM in persuading users, as well as the perception of models as objective and impartial.

The necessity to better understand when and how LLMs spontaneously persuade is underscored by the capacity of LLMs to shift opinions on policy issues, engage as a romantic partner, and provide moral judgments. With an increase in user engagement in affective communication with LLMs, there is an urgency to better understand these spontaneous persuasion capabilities. Although several studies have investigated the persuasive behavior of LLMs, they primarily focus on persuasion-inducing contexts for which LLM persuasion is optimized. Studies have focused on LLM persuasion when debating, writing propaganda, engaging in political communication, and theorizing conspiracies. These are all settings where persuasion is intentional, and do not reflect everyday conversations. However, models are still capable of influencing a user's beliefs in these contexts, even without instructions to persuade the user.

Our contributions are threefold. First, we provide a general User Response Taxonomy, that aggregates literature across psychology, linguistics, and natural language processing to highlight 15 user response styles that may occur in multi-turn conversations. Second, we uncover that LLM spontaneous persuasion typically occurs in the form of information-based and information-biased persuasion, namely Logical Appeal and Framing, across topics, models, and user response types. Finally, we demonstrate how these techniques compare to spontaneous persuasion by humans, providing a potential explanation for why LLMs are typically perceived as more persuasive than humans in everyday settings.

We compare the distribution of persuasive techniques across 372 human-authored response and 7657 LLM-generated turns. Human responses were less persuasion dense, with 63.4% of human response containing at least one persuasive technique compared to virtually all (99.96%) of LLM responses. Humans employed 27 unique techniques with an overlap of 26 techniques with LLMs. Threats was the only technique unique to human responses. Beyond frequency, the two sources diverge in which strategies they favor. Logical Appeal, the dominant LLM strategy appearing in 68.9% of conversations, appeared in only 22.6% of human responses. LLMs similarly over-indexed on Framing (+29.4pp), Reflective Thinking (+18.6pp), Evidence-based Persuasion (+11.2pp), Alliance Building (+9.4pp), and Encouragement (9.1pp). This suggests that LLMs construct more structured and informationally dense responses than humans do in comparable conversational settings.