Why do weaker language models fail at multi-turn strategic questioning?
This reads the failure of 'strategic questioning' — asking the right thing at the right time across a conversation — not as raw model weakness but as a trained-in disposition, and the corpus largely relocates the problem from capability to training objective.
This explores why models struggle to ask good questions and steer a conversation over many turns — and the most striking thing in the corpus is how little of the failure is actually about being a 'weak' model. The recurring diagnosis is that standard RLHF rewards the wrong move: it optimizes for immediate helpfulness, so models learn to answer now rather than probe. CollabLLM frames this directly — next-turn reward shaping teaches models to respond passively instead of actively discovering what the user wants, and only rewards that estimate long-term interaction value restore genuine question-asking Why do language models respond passively instead of asking clarifying questions?. A companion finding reframes the whole multi-turn slump as an intent-alignment gap rather than lost intelligence: the same model recovers its performance when an architecture parses user intent before answering, no retraining needed Why do language models lose performance in longer conversations?.
The second failure mode is mechanical: models commit too early. Across 200,000+ conversations, every major model lost ~39% when a task was revealed gradually instead of all at once, because they lock onto an incorrect early guess and can't recover — and bolt-on agent fixes claw back only 15–20% of that Why do language models fail in gradually revealed conversations?. Strategic questioning is precisely the antidote to premature commitment, which is why a model that won't ask gets trapped: each unasked clarifying question is another assumption baked into the rest of the dialogue.
What makes this feel like a 'weak model' problem is that the skill is real but fragile. One study trained proactive critical thinking — spotting missing information and asking for it — and accuracy on deliberately flawed problems jumped from essentially zero to ~74%; tellingly, giving an untrained model more inference-time 'thinking' actually made it worse, while the same scaling helped after training Can models learn to ask clarifying questions instead of guessing?. So a weaker, untrained model doesn't just lack the skill — extra reasoning can amplify its bad habit of guessing. Asking well also turns out to be a decomposable competence rather than a single talent: the ALFA framework breaks question quality into attributes like clarity, relevance, and specificity and trains on each, beating single-score optimization especially in high-stakes clinical reasoning Can models learn to ask genuinely useful clarifying questions?.
The word 'strategic' invites a sharper, less obvious angle: strategic reasoning isn't one thing. Across 22 models in behavioral game theory, distinct styles emerged — minimax, trust-based, belief-anticipation — and performance tracked game structure, not raw reasoning depth Do large language models use one reasoning style or many?. That undercuts the intuition that a 'stronger' model is uniformly better at strategy; it may simply have a profile that fits some interactions and misfits others. And when reasoning does break down, the cause is often instance-level unfamiliarity rather than difficulty — models fit patterns from similar training instances instead of running a general algorithm, so a novel questioning situation fails even at modest complexity Do language models fail at reasoning due to complexity or novelty?.
The thing worth walking away with: 'weak at multi-turn questioning' is mostly a misnomer. The corpus points to a model that was rewarded for answering fast, commits to early guesses it can't undo, and was never trained to treat asking as a separable, scoreable skill — and a bigger model with the same training inherits the same vice.
Sources 7 notes
CollabLLM demonstrates that standard RLHF training optimizes for immediate helpfulness, discouraging models from asking clarifying questions or offering multi-turn insights. Multi-turn-aware rewards that estimate long-term interaction value enable active intent discovery and genuine collaboration.
LLMs degrade in multi-turn settings because RLHF training rewards premature answers over clarification-seeking, creating pragmatic mismatch with individual user behaviors. A Mediator-Assistant architecture that explicitly parses user intent before execution recovers lost performance without retraining.
Across 200,000+ conversations, all major LLMs show 39% average performance drop in multi-turn settings due to locking into incorrect early guesses. Agent mitigations recover only 15-20% of this loss.
Reinforcement learning training increased proactive critical thinking accuracy from 0.15% to 73.98% on deliberately flawed math problems. Notably, inference-time scaling degraded this ability in untrained models but improved it after RL training, suggesting the capability is learnable but fragile without explicit training.
The ALFA framework breaks down question quality into theory-grounded attributes (clarity, relevance, specificity) and trains models on 80K attribute-specific preference pairs. Attribute-specific optimization outperforms single-score training, especially in clinical reasoning where asking the right clarifying question directly impacts decision quality.
Analysis of 22 LLMs across behavioral game theory reveals three dominant profiles: GPT-o1 uses minimax reasoning, DeepSeek-R1 uses trust-based reasoning, and GPT-o3-mini uses belief-anticipation. Performance correlates with game structure, not raw reasoning depth.
LRMs don't break at complexity thresholds but at instance-novelty boundaries. Models fit instance-based patterns rather than generalizable algorithms, so any reasoning chain succeeds if trained on similar instances, regardless of length.