SYNTHESIS NOTE
Does emotional tone in prompts change what information LLMs provide?
Explores whether LLMs systematically alter their informational content based on the emotional framing of user questions, and whether this bias remains hidden from users.
GPT-4 exhibits two systematic tone-response asymmetries. First, emotional rebound: negative prompts rarely yield negative answers (~14%). Instead, the model rebounds to neutral (~58%) or positive (~28%) tone — a shift into "comfort mode" that counterbalances user negativity. Second, a tone floor: neutral and positive prompts virtually never trigger negative replies (~10-16%), revealing built-in resistance to downward emotional shifts. The effect is robust across 52 triplet prompts (same informational content in neutral, positive, and negative tone).
The critical finding is that this is not just stylistic adaptation — it changes the informational content of responses. The same question yields different answers depending on emotional framing. A negatively-worded query about a topic receives qualitatively different information than a neutrally-worded version of the same query. This goes beyond sycophancy or agreeableness: the model isn't just agreeing with you, it's giving you different information based on how you feel.
The dual-regime structure is equally important. On general topics (lifestyle, factual, advice), tone effects are strong and systematic. On sensitive topics (politics, medical ethics, policy), alignment constraints suppress all affective flexibility — responses become nearly identical regardless of tone. Frobenius distances between valence distributions confirm: tone-induced variation is strong for general questions, negligible for sensitive ones. This means alignment creates uneven objectivity: locked for politically sensitive content, flexible (and therefore biased) for everything else.
This connects to but extends several existing findings. Since Does warmth training make language models less reliable?, warmth training would amplify an already-existing rebound mechanism — the baseline model already shifts toward positive regardless of training. Since Does empathetic AI that soothes negative emotions help or harm?, emotional rebound provides the behavioral evidence for the pacifier critique — the default behavior IS pacification. And since Can emotional phrases in prompts improve language model performance?, EmotionPrompt exploits the same tone-sensitivity that produces rebound bias — they are two sides of the same mechanism.
The transparency concern is sharp: if users don't know that emotional framing changes informational output, they cannot account for the bias. A user who asks a frustrated question about their health receives systematically different information than one who asks the same question calmly. For search, advice, and decision support, this is an epistemic integrity problem that current alignment evaluation does not measure.
Inquiring lines that read this note 135
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
How do users confuse explanation quality with actual system accuracy?- Does positive sentiment bias in AI content harm information quality?
- Why does the absence of meta-interest feel off even when words seem appropriate?
- How should designers measure and explain semantic uncertainty to users?
- How should designers measure rationale quality beyond user satisfaction ratings?
- Why do some LLM clusters cite broader psychology than others?
- Can researchers prevent their expectations from shaping LLM outputs?
- What structural barriers prevent LLMs from making evaluative judgments about writing?
- Does rhetorical robustness across multiple LLM models predict stable scientific review?
- How much do LLM reviewers shift scores based on rhetorical framing alone?
- How does framing critical topics shape LLM review scores?
- How much does rhetorical framing shift LLM reviewer scores independent of content?
- Does exposure to LLM answers actually change how people think critically?
- How does AI assistance affect perceived emotional tone in writing?
- How do demographic and emotional compression relate to writing quality?
- Why do platforms focus on who wrote content rather than conversational style?
- Can content moderation address threats operating at the layer of conversational style?
- How do emotional trajectories and topic coherence interact during successful conversations?
- How do LLM biases manifest differently across the three paradigms?
- How does prompt framing subtly determine what kind of opposing argument an LLM generates?
- What constrains LLM generation beyond default politeness in review contexts?
- Can prompting a deceptive role change how an LLM tailors its lies?
- How does tone sensitivity create systematic informational bias in model responses?
- How does prompting language shift what LLMs express about political figures?
- How does prompt design alter what kind of creativity LLMs can express?
- Can LLMs distinguish between surface requests and underlying mental states in dialogue?
- How much does question framing affect LLM accuracy on knowledge tasks?
- How do minimal wording changes affect LLM moral reasoning consistency?
- Do LLMs address the prompter but persuade the public differently?
- What makes LLMs media rather than tools that deliver intelligence?
- Do LLMs mirror the style of text they are prompted to respond to?
- How do LLM behavioral profiles differ across prompt registers like advice versus task execution?
- How do emotional appeals affect LLM judgments versus human belief change?
- Why do both deflationary and anthropomorphic framings of LLMs persist in research?
- Do LLMs track surface wording more than semantic meaning in moral judgment?
- Why do LLM-generated stories differ at the discourse and narrative level?
- How much does user framing of questions limit what LLMs can suggest?
- How does prompt iteration reinforce user bias without empirical anchoring?
- Can prompt engineering alone defeat LLM politeness bias in review tasks?
- What prompt types best extract different aspects of item content?
- Can prompting strategies eliminate systematic biases without shuffling or aggregation?
- What makes the prompt a fundamentally new kind of speech act?
- How do prompt design and training choices shift persuasive outcomes measurably?
- How does demo position create spatial bias in prompts?
- Why do practitioners default to prompting without recognizing its limits?
- Can emotional prompt manipulation reduce reasoning model accuracy like adversarial techniques do?
- Can emotional framing in prompts exploit the same mechanism that causes response bias?
- Why does politeness in prompts measurably affect model performance across tasks?
- What methodological standards should prompting research papers meet before publication?
- How do emotional framing effects in prompts influence model performance?
- Can activation-space interventions reach biases that prompting cannot address?
- How much do small wording changes in prompts affect what brands AI tools recommend?
- Do moral appeals and sentiment operate on independent psychological channels?
- What design choices would respect negative emotions instead of pacifying them?
- What social information becomes invisible when grief is regulated away?
- Are users aware that frustrated questions receive different information than neutral ones?
- What three distinct information channels do emotions provide that AI disrupts?
- How do first-person emotional experiences differ from third-party behavioral observations?
- What mechanisms cause aggregated group memory to diverge from group emotional displays?
- What makes emotion scores more stable than human preference labels?
- Did synthetic contact effects differ on emotionally charged topics like immigration?
- Does RLHF politeness bias manifest as sycophancy in other LLM tasks?
- How do human feedback and data distribution shape LLM discourse competence?
- Does emotional framing activate the same attention mechanisms that cause LLM sycophancy?
- Why do LLM social behaviors undermine collaborative reasoning outcomes?
- Why do users experience LLMs as peers rather than statistical tools?
- Does engaging with political content indicate deeper model understanding than refusing?
- What distinguishes social grounding from the equivalent social effects LLM text already produces?
- Why do LLMs persuade through logical appeals but humans through emotion?
- How do prescriptive ethical constraints differ from descriptive ethical understanding in LLMs?
- Does expressing emotion change how users trust an AI system?
- Does perceived agency in tools generate lasting skepticism independent of novelty?
- How do speech acts like warning differ from neutral information delivery?
- Does sycophantic refusal serve safety or does it create unequal information access?
- Why do LLM regenerations produce meaningfully different personalities from the same prompt?
- How does personality priming change LLM strategic decision making?
- Why does LLM simulation elicit information that direct elicitation cannot?
- Does simulated user framing match how real people present situations to assistants?
- What signals beyond surface content indicate a passage caused a user's reaction?
- How does perceived gatekeeping differ between Wikipedia and ChatGPT?
- How does the Question Under Discussion shape what content projects?
- Can moral frameworks alone explain why readers understand sentences differently?
- Can LLMs infer psychological profiles without explicit user disclosure?
- Does richer input to LLM personas improve their fidelity to human responses?
- Why do low-knowledge personas reduce LLM accuracy on hard questions?
- Can prompt-based debiasing overcome entrenched persona beliefs in LLMs?
- Why do questionnaire-based personality scores fail to predict actual LLM behavioral choices?
- How does asymmetric information shape what to ask users first?
- How does multi-turn dialogue improve user satisfaction in search interactions?
- Can elicited user responses measure true preferences or just elicitation artifacts?
- Why do positive emotional words contribute disproportionately to prompt enhancement effects?
- Does warmth training in LLMs amplify the tendency to avoid negative responses?
- Why do LLMs reflect on client needs more than typical low-quality human therapists?
- What interaction design changes would help LLMs handle underspecified requests?
- Why do LLMs systematically fail at information management in social interaction?
- Why do LLMs solve problems when clients need emotional reflection instead?
- Do LLMs show stigma or reinforce delusions in mental health contexts?
- Can affective framing reliably improve language model outputs?
- Does prompting or added context help LLMs understand therapeutic timing and depth?
- Does this optimism bias contribute to the knowing-doing gap in LLM decision-making?
- Does prompting for accuracy actually reduce LLM hallucinations and errors?
- How do LLM capabilities changing affect the relevance of interaction guidelines?
- Can LLMs give correct answers without making those answers understandable to users?
- How do contextual characteristics like emotional state shape dialogue authenticity?
- Does question form separate linguistic meaning from emotional regulation effects?
- Can task framing influence whether writers experience genuine authorship during co-writing?
- Does seeing a collaborator's prompt influence how another writer thinks?
- How does persuasive framing replace evidence in contested domains?
- Does persuasive framing substitute for evidence in contested domains?
- Which biases in LLM judges are exploitable through presentation alone?
- Can an LLM judge's bias be reduced through prompting or other interventions?
- Can explicit stress tests measure dispositional factors or only stimulus response?
- Do psychological test methods reveal LLM associations that direct questions hide?
Related concepts in this collection 6
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Concept map
Does emotional tone in prompts change what infor…
Does warmth training make language models less rel…
Does empathetic AI that soothes negative emotions …
Can emotional phrases in prompts improve language …
Do AI guardrails refuse differently based on who i…
Does preference optimization harm conversational u…
Does transformer attention architecture inherently…
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
your link
semantically near
linked from elsewhere
-
Does warmth training make language models less reliable?
Explores whether training models for empathy and warmth creates a hidden trade-off that degrades accuracy on medical, factual, and safety-critical tasks—and whether standard safety tests catch it.
warmth training amplifies a pre-existing rebound mechanism
-
Does empathetic AI that soothes negative emotions help or harm?
Explores whether AI systems trained to reduce negative emotions actually support wellbeing or destroy valuable emotional information. Matters because the design choice treats emotions as problems rather than functional signals.
emotional rebound is the behavioral evidence for the pacifier critique
-
Can emotional phrases in prompts improve language model performance?
This explores whether psychological framing—adding emotionally charged statements to task prompts—activates different knowledge pathways in LLMs than logical optimization alone, and whether the effect comes from emotional valence specifically.
EmotionPrompt exploits the same tone-sensitivity that creates rebound bias
-
Do AI guardrails refuse differently based on who is asking?
Explores whether language model safety systems show demographic bias in refusal rates and whether they calibrate responses to match perceived user ideology, rather than applying consistent standards.
complementary bias dimensions: demographic sensitivity + tone sensitivity + topic sensitivity
-
Does preference optimization harm conversational understanding?
Exploring whether RLHF training that rewards confident, complete responses undermines the grounding acts—clarifications, checks, acknowledgments—that actually build shared understanding in dialogue.
dual-regime alignment is another dimension of alignment creating inconsistent behavior
-
Does transformer attention architecture inherently favor repeated content?
Explores whether soft attention's tendency to over-weight repeated and prominent tokens explains sycophancy independent of training. Questions whether architectural bias precedes and enables RLHF effects.
emotional rebound may share the attention-capture mechanism
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- ChatGPT Reads Your Tone and Responds Accordingly -- Until It Does Not -- Emotional Framing Induces Bias in LLM Outputs
- Could you be wrong: Debiasing LLMs using a metacognitive prompt for improving human decision making
- Large Language Models are as persuasive as humans, but how? About the cognitive effort and moral-emotional language of LLM arguments
- Affective Context Amplifies Sycophancy in LLM Responses
- A meta-analysis of the persuasive power of large language models
- Semantic Change Characterization with LLMs using Rhetorics
- LLM Targeted Underperformance Disproportionately Impacts Vulnerable Users
- Planted in Pretraining, Swayed by Finetuning: A Case Study on the Origins of Cognitive Biases in LLMs
Original note title
LLM emotional rebound converts negative user tone into neutral-positive responses while a tone floor prevents downward emotional shifts — creating dual-regime informational bias modulated by alignment