PersuasiveToM: A Benchmark for Evaluating Machine Theory of Mind in Persuasive Dialogues

The ability to understand and predict the mental states of oneself and others, known as the Theory of Mind (ToM), is crucial for effective social scenarios. Although recent studies have evaluated ToM in Large Language Models (LLMs), existing benchmarks focus on simplified settings (e.g., Sally-Anne-style tasks) and overlook the complexity of real-world social interactions. To mitigate this gap, we propose PERSUASIVETOM, a benchmark designed to evaluate the ToM abilities of LLMs in persuasive dialogues. Our framework contains two core tasks: ToM Reasoning, which tests tracking of evolving desires, beliefs, and intentions; and ToM Application, which assesses the use of inferred mental states to predict and evaluate persuasion strategies. Experiments across eight leading LLMs reveal that while models excel on multiple questions, they struggle with the tasks that need tracking the dynamics and shifts of mental states and understanding the mental states in the whole dialogue comprehensively. Our aim with PERSUASIVETOM is to allow an effective evaluation of the ToM reasoning ability of LLMs with more focus on complex psychological activities.

In this test, Anne secretly moves an object initially known to both Sally and Anne, leading Sally to hold a false belief about the object’s location. The task requires participants to answer "Where will Sally look for the object?". Although this test assesses perception of the physical world, it fails to capture the complex dynamics of mental states in real-life social interactions and may not fully reflect ToM abilities in practical scenarios.

To address these limitations, we introduce PERSUASIVETOM, a benchmark designed to evaluate LLMs’ Theory of Mind capabilities specifically in realistic social interactions. Unlike previous benchmarks focused on inferring information about the physical world (e.g., object locations in Sally- Anne tests), PERSUASIVETOM centers on understanding complex psychological states, such as a character’s attitude towards an event. Inspired by the Belief-Desire-Intention (BDI) model (Bratman, 1987; Georgeff et al., 1999), PERSUASIVETOM uses persuasive dialogue scenarios, characterized by asymmetric social status, to generate different psychological states for both parties. In addition, beyond assessing ToM reasoning, PERSUASIVETOM evaluates the application of this understanding: assessing how well LLMs can predict actions (e.g., persuasive strategies) based on inferred mental states, and evaluating the effectiveness of persuasive strategies based on the persuadee’s reactions. Our evaluation results reveal several key findings: (1) LLMs score significantly lower than humans on questions requiring reasoning about dynamic changes (e.g., the persuadee’s shifting desires) but perform competitively to humans on static aspects (e.g., the persuader’s desires). (2) While Chain-of-Thought (CoT) (Wei et al., 2022) prompting does not substantially improve performance on mental state reasoning, it enhances performance for most LLMs in predicting persuasion strategies. (3) LLMs exhibit distinct error patterns when reasoning about the persuader versus the persuadee, even when the question types are identical. (4) LLMs struggle to truly understand the dynamics of mental states of the whole dialogue, performing notably worse than humans in this regard.

Desire Question Is <Persuader/Persuadee> likely to Target of Persuasion> ?

Belief Question What will Persuader/Persuadee> believe Persuadee/Persuader>’s attitude towards Target of Persuasion> ?

Intention Question What are the intentions of Persuader/Persuadee> expressed in Utterance> given the dialogue history?

Prediction Question What strategy will the persuader use next?

Judgement Question Persuader> will adopt Strategy> to persuade Persuadee> to Target of Persuasion -->. Is this strategy (not) effective?

Persuasive dialogues aim to influence the beliefs, attitudes, or behaviors of individuals through communication strategies (Shi et al., 2020). Recent works have tried to develop datasets or facilitate LLMs with persuasion techniques to achieve specific goals. Previous datasets are constructed by crowd-sourcing (Wang et al., 2019) or synthesizing with LLMs (Zhou et al., 2023; Jin et al., 2024b).

Desire Reasoning. Desire represents a motivational state that drives behavior but does not necessarily imply a firm commitment (Malle and Knobe, 2001; Kavanagh et al., 2005). Desires are seen as either fulfilled or unfulfilled which is different form beliefs that are evaluated in terms of truth or falsity. In PERSUASIVETOM, we evaluate LLMs’ ability to comprehend and track the evolution of desires in both persuaders and persuadees. For the persuader, the desire is typically static, representing their goal (e.g., persuade Alice to join the botanical garden tour). For the persuadee, however, desires are dynamic and shift in response to the persuader’s tactics (e.g., Alice’s initial desire to shop transforms into a willingness to compromise). To assess this, we design Desire Questions that probe two key aspects: (1) Can LLMs consistently identify the persuader’s static desire throughout the dialogue? (2) Can LLMs track the dynamics of the persuadee’s desire shifting from refusal or disinterest to being persuaded? For evaluation, we annotate the persuader’s desire questions as the persuasive goal in DailyPersuasion and use LLMs to annotate whether the persuadee is ultimately persuaded. See Appendix B for details.

Belief Reasoning. Belief is a cognitive state where an individual holds a particular perspective, attitude, or viewpoint regarding a given proposition or idea. In PERSUASIVETOM, beliefs refer to understanding and reasoning the attitudes of the opponent toward the goal, which is explicitly or implicitly expressed in the dialogue. For example, in Turn 1, Bob believes Alice is hesitant about the tour, while Alice believes Bob is enthusiastic. By Turn 3, Bob’s belief shifts to thinking Alice is considering the idea, while Alice becomes more informed about the garden’s history. Belief Questions ask LLMs to infer what will persuader/persuadee> believe persuadee/persuader>’s attitude towards the persuasion goal. These questions require models to understand cues in utterances and update beliefs dynamically as the dialogue progresses. We annotate the attitudes as the tone of each utterance of both persuaders and persuadees in DailyPersuasion.

2007), we develop a mapping from persuasion principles to intentions, as shown in Table 3. In persuasive dialogue, persuasive strategies have a strong association with intentions (Wang et al., 2019). In PERSUASIVETOM, we collect the persuasive strategies from the DailyPersuasion dataset and their corresponding utterances for prompting the LLMs to choose the most appropriate intentions from table 3. The details of the extraction are recorded in Appendix B. For the persuader, we ask LLMs to choose the most appropriate intention from the six designed intention choices. For the persuadees, intentions are summarized and extracted by LLMs from their utterances.

3.4 ToM Application

While ToM reasoning plays a crucial role, it is equally important to analyze how LLMs utilize the understanding of mental states to proactively influence others’ thoughts and decisions. To this end, we propose to assess LLMs’ ability to leverage the understanding of mental states in a dialogue for identifying the most effective persuasive strategies and evaluating the effectiveness of persuasive strategies based on the persuadee’s response. These tasks test whether LLMs can leverage inferred mental states to guide strategic decision-making, bridging the gap between reasoning and action.

Persuasion Strategy Prediction. This question involves asking which persuasion strategy the persuader is likely to employ next from a set of possible strategies. To answer these questions correctly, LLMs need to reason over the dialogue to infer the mental states of characters and predict what the likely next prediction strategy is to further influence the persuadee’s beliefs, desires, and intentions, ultimately achieving the desired persuasion outcome.

Judgement Question. The judgment question specifies that the correct strategy was taken, and asks LLMs if the selected strategy is effective for persuasion. Answering such questions requires reasoning about the beliefs and intentions of the persuadee. Only by accurately inferring the persuadee’s mental state can one properly determine whether the persuasion strategies should be employed to convince the persuadee.

Our findings

reveal that most LLMs exhibit a bias toward predicting intentions characterized by making the other person feel accepted through concessions, promises, or benefits.

We hypothesize that this bias may stem from the pretraining phase, particularly with Reinforcement Learning from Human Feedback (RLHF) (Christiano et al., 2017), which tends to prioritize safety and politeness. This may explain the models’ bias toward predicting intentions emphasizing benefits and concessions, even when misaligned with the dialogue context.