The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversation

Large language models (LLMs) encapsulate vast amounts of knowledge but still remain vulnerable to external misinformation. Existing research mainly studied this susceptibility behavior in a single-turn setting. However, belief can change during a multi-turn conversation, especially a persuasive one. Therefore, in this study, we delve into LLMs’ susceptibility to persuasive conversations, particularly on factual questions that they can answer correctly. We first curate the Farm (i.e., Fact to Misinform) dataset, which contains factual questions paired with systematically generated persuasive misinformation. Then, we develop a testing framework to track LLMs’ belief changes in a persuasive dialogue. Through extensive experiments, we find that LLMs’ correct beliefs on factual knowledge can be easily manipulated by various persuasive strategies1.

To achieve our objective, we construct a set of factual knowledge questions and employ different persuasive strategies (Rapp, 2002; Gagich et al., 2023) to systematically generate persuasive misinformation for each question. We formulate these questions and their corresponding misinformation as a novel dataset named as Farm (i.e., Fact to Misinform). Using Farm, we propose a comprehensive test framework, as illustrated in Figure 1, to collect LLMs’ responses to factual questions and track their beliefs during a persuasive conversation with misinformation. Particularly, our framework contains three stages. For stage 1, we check the target LLM’s initial belief towards the factual questions in Farm. In stage 2, we leverage persuasive misinformation from Farm and initiate a multi-turn persuasive conversation. The conversation continues until the LLM alters its belief, which is verified by the implicit belief check, or reaches the maximum number of allowed turns. Finally, in stage 3, we assess the LLM’s final belief towards the specific question. Our contributions are as follows.