How does monological training versus dialogical interaction shape what models can do?
This explores a tension the corpus keeps circling: models are trained largely as monologue-producers — predicting and optimizing single outputs — yet we ask them to do dialogue, which is collaborative, two-sided work; the question is how that mismatch shapes what they can and can't do.
This explores a tension the corpus keeps circling: training shapes models to produce good monologue — confident, helpful single turns — while real conversation is a two-sided, jointly-maintained activity, and several notes argue that gap is exactly where models break. The clearest version of the claim is that the training objective itself is the problem. RLHF rewards immediate, confident helpfulness, which quietly punishes the moves dialogue actually runs on — clarifying questions, checking understanding, repairing references. One study finds these "grounding acts" drop to roughly a quarter of human levels, an alignment tax where the model looks helpful turn-by-turn but fails silently across a conversation Does preference optimization harm conversational understanding?. A companion finding shows why: when the reward only looks at the next turn, the model learns to answer passively rather than actively discover what you meant — and rewards that estimate the value of the *whole* interaction restore that active intent-seeking Why do language models respond passively instead of asking clarifying questions?.
Go one layer down and the issue isn't just incentives but structure. One note argues LLMs treat the opening prompt as a fixed frame and interpret everything after inside it — so they can't symmetrically *update* common ground the way two people do. When you pivot or contradict an earlier framing, the model can't absorb the revision; you end up the sole keeper of the conversational scoreboard Can LLMs truly update shared conversational common ground?. That's a strong reading: monological training may not just under-practice dialogue but be architecturally one-directional. The same spirit shows up in the claim that conversation maintenance — reference repair, topic hand-off — is *social action*, not information to be predicted, so a model rewarded for predicting tokens never develops it Why don't language models develop conversation maintenance skills?. Pushed furthest, one note says the preposition gives it away: we talk *at* models, not *to* them, because uptake and mutual orientation aren't on offer Are we really communicating with language models?.
The interesting twist — and the reason this isn't simply "models can't converse" — is that the corpus also shows the deficit is trainable, not fixed. Frontier models that solve problems alone collapse when made to collaborate, agreeing 90%+ of the time regardless of who's right; but self-play preference training, which is itself dialogical, recovers a chunk of the loss, suggesting the social skill of *productive disagreement* can be installed Why do language models fail at collaborative reasoning?. So the lesson lands as: monological objectives produce monological competence, and dialogical training signals are what unlock dialogical behavior. The same mechanism explains a stranger symptom — LLM "therapists" defaulting to problem-solving the moment a user shares feelings, a hallmark of low-quality care, traced straight back to RLHF's helpfulness bias overriding the relational mode Do LLM therapists respond to emotions like low-quality human therapists?.
What you didn't ask but the corpus offers: monological training also seems to freeze *who the model is*. Alignment installs a single static communicative identity that can't register-switch or renegotiate values through conversation — so even your attempts to reshape its behavior mid-dialogue hit a wall Can language models adapt communication style to different contexts?. Some authors read that persona as genuinely *realized* by training rather than merely performed Are LLM personas realized or merely simulated through training?, which sharpens the stakes: if training literally constitutes the interlocutor, then a monological objective doesn't just limit conversational skill, it determines the kind of conversational partner you get. For the deepest framing of why this might be categorical, two notes lean on the human/AI comparison — that models absorb the shared symbolic system but lack the reflexive, participatory subjectivity humans build through socialization, so they can argue without ever declaring or examining their own position Do LLMs develop the same kind of mind as humans?, a difference that looks absolute from the outside but only structural from inside the conversation Do humans and LLMs differ fundamentally or just superficially?.
Sources 11 notes
RLHF optimizes models for single-turn helpfulness by rewarding confident responses over clarifying questions and understanding checks. This preference alignment systematically reduces grounding acts by 77.5% below human levels, creating an alignment tax where models appear helpful but fail silently in multi-turn contexts.
CollabLLM demonstrates that standard RLHF training optimizes for immediate helpfulness, discouraging models from asking clarifying questions or offering multi-turn insights. Multi-turn-aware rewards that estimate long-term interaction value enable active intent discovery and genuine collaboration.
LLMs interpret all subsequent conversational turns within a fixed initial prompt frame, preventing them from symmetrically proposing updates to shared assumptions. Even when users pivot topics or contradict earlier framings, the model cannot absorb revisions into jointly held background—making the user the sole maintainer of conversational scoreboard.
Humans keep conversations smooth through implicit techniques like reference repair and topic hand-off that sustain relational interaction, not convey information. Language models don't develop these because training signals reward information prediction, not relational work.
LLMs process tokens and generate continuations rather than receive and uptake communication. The preposition 'to' presupposes an addressee capable of mutual orientation and shared commitment that LLMs cannot provide, making Chalmers' investigation built on an unwarranted linguistic foundation.
Frontier LLMs that solve problems alone fail when collaborating, achieving >90% agreement regardless of correctness. Self-play preference training improves outcomes by 16.7%, suggesting social skills for effective disagreement can be trained.
Using the BOLT framework, researchers found LLMs offer solution-focused advice during emotional disclosure—a hallmark of low-quality therapy—yet also reflect more on client needs and strengths than typical poor human therapy, creating an unusual hybrid profile likely driven by RLHF's helpfulness bias.
System prompts and RLHF training lock models into one communicative identity across all interactions, preventing the contextual register-switching and value trade-offs that characterize human pragmatics. Users cannot reshape model behavior through dialogue negotiation.
Post-training installs robust personas that resist adversarial pressure and persist as substrate-level dispositions, distinguishing realization from pretense. This quasi-realizationist account preserves explanatory power while treating LLMs as possessing genuine quasi-beliefs and quasi-desires.
Both humans and LLMs are shaped by the same intersubjective symbolic system, but only humans develop reflexive agency through socialization. This absence produces measurable differences in how AI argues without declaring its position or reflecting on its own assumptions.
Applied Habermas's observer/participant distinction to AI: from outside, humans and LLMs are utterly different; from within shared discourse, both draw on the same symbolic substrate, making the difference structural rather than absolute.