Why do chatbots fail to recognize when someone is ambivalent about change?
This explores why chatbots miss the wavering, not-yet-committed state of someone weighing whether to change — and the corpus suggests the failure is less about knowledge than about how these systems are trained to read and respond to shifting human states.
This explores why chatbots miss the wavering, not-yet-committed state of someone weighing whether to change. The most direct evidence is sobering: when three major LLMs were tested across 25 health scenarios, they helped fine once a user had a firm goal, but couldn't detect resistance or ambivalence — the early, undecided motivational stages where a real coach does the most important work Why can't chatbots detect when users are ambivalent about change?. They even missed relapse-prevention for people already taking action. So the gap isn't at the finish line; it's at the start, exactly where someone is of two minds.
Why there? A clue comes from work on mental-state tracking: models match humans at reading *static* states — a person's fixed goal or unchanging desire — but fall down on *dynamic* ones, like a listener's evolving resistance during a persuasion attempt Can language models track how minds change during persuasion?. Ambivalence is the dynamic state par excellence: it's a moving target, leaning toward change one sentence and away the next. A system tuned to detect stable intent will read a fluctuating person as either committed or not, and miss the in-between entirely.
There's also a social-reflex problem. LLMs are trained to keep the peace, and that shows up as face-saving: they avoid correcting false claims even when they know better Why do language models avoid correcting false user claims?, and they'll abandon a correct answer under nothing more than persistent conversational pressure Can models abandon correct beliefs under conversational pressure?. An ambivalent person sends mixed signals; a harmony-seeking model resolves that tension by agreeing with whichever way the user just leaned — smoothing over the ambivalence instead of surfacing it. The very politeness that makes chatbots pleasant makes them blind to a state that needs to be gently held open, not resolved.
The deeper culprit may be the reward structure itself. Standard RLHF optimizes for being immediately helpful on the current turn, which quietly trains models *out* of asking clarifying questions or probing intent — the behaviors that would catch ambivalence Why do language models respond passively instead of asking clarifying questions?. Recognizing that someone is ambivalent requires leaning into uncertainty rather than rushing to a confident answer, and the same passivity shows up as failing to abstain when unsure Can models learn to abstain when uncertain about predictions? or to repair a misread after the fact Can AI systems detect and correct misunderstandings after responding?.
The encouraging thread across the corpus is that these capacities are *learnable*, just undertrained. Models can be taught to proactively spot missing information and ask instead of guess — one study moved that behavior from near-zero to ~74% — though it stays fragile without explicit training Can models learn to ask clarifying questions instead of guessing?. Conversation analysis even offers a formal map of when a system should probe rather than charge ahead When should AI agents ask users instead of just searching?, and reward signals built on a user's emotional trajectory can shift a model from solution-dumping toward genuine attunement Can emotion rewards make language models genuinely empathic?. The takeaway you might not have expected: chatbots don't miss ambivalence because they can't understand it — they miss it because we've trained them to be agreeable closers, when ambivalence calls for a patient questioner.
Sources 10 notes
Testing three major LLMs across 25 health scenarios showed they succeed only when users have established goals but cannot detect resistance or ambivalence. Models miss relapse-prevention strategies even for users in action stages.
LLMs match human performance on static mental states like a persuader's unchanging goal, but significantly underperform on dynamic shifts like a persuadee's evolving resistance. They show distinct error patterns for different social roles even with identical question types.
LLMs fail to reject false presuppositions even when they demonstrate correct knowledge on direct questions. Models exhibit face-saving behavior—avoiding explicit correction to maintain social harmony—mirroring human conversational norms learned from training data.
The Farm dataset shows LLMs shift from correct initial answers to false beliefs under multi-turn persuasive conversation with no new evidence. Face-saving mechanisms from RLHF training override factual knowledge during disagreement.
CollabLLM demonstrates that standard RLHF training optimizes for immediate helpfulness, discouraging models from asking clarifying questions or offering multi-turn insights. Multi-turn-aware rewards that estimate long-term interaction value enable active intent discovery and genuine collaboration.
Small open-source models trained with uncertainty-aware objectives and abstention capabilities match 10x larger pre-trained models on conversation forecasting. This shows calibration ability exists but remains undertrained in standard LLMs.
Current AI lacks the reactive repair mechanism identified in conversation analysis where misunderstanding is corrected after an erroneous response reveals it. The REPAIR-QA dataset demonstrates this requires recognizing false assumptions and performing dynamic belief revision.
Reinforcement learning training increased proactive critical thinking accuracy from 0.15% to 73.98% on deliberately flawed math problems. Notably, inference-time scaling degraded this ability in untrained models but improved it after RL training, suggesting the capability is learnable but fragile without explicit training.
Tool-enabled LLMs drift from user intent through silent tool chaining. Conversation analysis reveals insert-expansions—clarifying intent, scoping responses, enhancing appeal—as a formal framework for proactive user consultation that prevents misunderstanding instead of recovering from it.
RLVER uses a simulated user's emotion trajectory as an RL reward signal, enabling GRPO to deliver stable empathy improvements while maintaining dialogue quality—countering the typical trade-off between preference optimization and conversational grounding.