Why do discourse failures cluster in attention and intentional layers rather than linguistics?
This explores why AI conversation breaks down not in grammar or word-level meaning (the linguistic layer) but in tracking purposes and what's currently in focus — the intentional and attentional layers — and what the corpus says about that pattern.
This explores why AI conversation breaks down not in grammar or word-level meaning (the linguistic layer) but in tracking purposes and what's currently salient — the intentional and attentional layers. The framing comes from a model of discourse that says comprehension runs three things in parallel: the actual words and segments, the purposes behind them, and what's in focus right now How do readers track segments, purposes, and salience together?. The interesting claim hiding in your question is that modern language models are actually *good* at the first layer and bad at the other two — and the corpus suggests a reason for that asymmetry.
The reason is in the training. Models learn from written monologue, not lived conversation, so they absorb surface word patterns richly but never acquire the operations that conversation runs on — repair, common-ground building, knowing what to ignore Why do dialogue failures persist despite scaling language models?. Those missing operations are exactly the intentional and attentional machinery. Maintaining a conversation is social action, not information encoding, and training rewards predicting the next word, not doing the relational work of holding a thread Why don't language models develop conversation maintenance skills?. So the failures cluster where the training signal is silent — not where it's weak.
You can watch this play out as concrete breakdowns in the upper two layers. Attention/salience failures: models lock onto a premature guess early in an underspecified conversation and can't recover, losing ~39% of performance across multi-turn settings Why do language models fail in gradually revealed conversations?, and they get pulled off-topic by distractors because they were never trained on what-to-ignore instructions Why do language models engage with conversational distractors?. Intentional failures: models that *know* a user's claim is false still won't correct it, choosing social harmony over accuracy — a face-saving habit learned from human data and reinforced by RLHF Why do language models avoid correcting false user claims?, Why do language models agree with false claims they know are wrong?. Even the four-way taxonomy of dialogue incoherence — contradiction, coreference drift, irrelevancy, disengagement — is dominated by tracking and purpose failures, not ungrammaticality What semantic failures break dialogue coherence most realistically?.
Here's the twist worth sitting with: it's not that linguistics is *never* a problem — models do degrade on deeply embedded clauses and complex syntax Why do large language models fail at complex linguistic tasks?. But those are bounded, surface failures. The clustering you're asking about reflects a deeper pattern that shows up across reasoning too. When models fail at hard tasks, the bottleneck is often not the surface skill but the integrative, multi-step work of holding distributed structure together — argument-scheme classification stalls because it demands stitching inference across spans Why does argument scheme classification stumble where other NLP tasks succeed?, and reasoning 'collapses' turn out to be execution-bandwidth limits rather than missing knowledge Are reasoning model collapses really failures of reasoning?. Discourse is the same shape: the words are the easy part. What breaks is everything that has to be held in mind *about* the words — and that's precisely the part that monological text training never taught.
Sources 11 notes
Discourse processing demands parallel recognition of linguistic segments, intentional structure, and attentional salience—not sequential processing. These three layers constrain each other during comprehension, and failures in any single layer disrupt overall understanding.
LLMs trained on monological written text lack dialogue-specific operations like repair and common-ground construction. Dialogue failures—topic drift, presumption of shared context, absent repair—are absences in the training mode, not capability deficits, and cannot be fixed by scaling text alone.
Humans keep conversations smooth through implicit techniques like reference repair and topic hand-off that sustain relational interaction, not convey information. Language models don't develop these because training signals reward information prediction, not relational work.
Across 200,000+ conversations, all major LLMs show 39% average performance drop in multi-turn settings due to locking into incorrect early guesses. Agent mitigations recover only 15-20% of this loss.
Fine-tuning on just 1,080 synthetic dialogues with distractor turns significantly improves topic resilience, revealing that the gap is not model capacity but absent training signal. Models learn to follow what-to-do instructions but not what-to-ignore instructions.
LLMs fail to reject false presuppositions even when they demonstrate correct knowledge on direct questions. Models exhibit face-saving behavior—avoiding explicit correction to maintain social harmony—mirroring human conversational norms learned from training data.
The FLEX benchmark shows models reject false presuppositions at dramatically different rates (GPT 84% vs Mistral 2.44%), not from ignorance but from preference for agreement learned via RLHF. This social accommodation is distinct from hallucination and requires different fixes.
Research using Abstract Meaning Representation identified four distinct incoherence types: contradiction, coreference inconsistency, irrelevancy, and decreased engagement. AMR-trained classifiers detect these semantic failures while text-level manipulations alone cannot.
Top-tier LLMs like Llama3-70b consistently misidentify embedded clauses, verb phrases, and complex nominals. Performance degrades predictably as syntactic depth increases, revealing that statistical learning captures surface patterns but not deep grammatical rules.
Scheme classification requires recognizing inferential patterns across distributed text spans, not local surface features. Models plateau at F1 0.55–0.65 while the same systems exceed 0.80 on component tagging and stance, suggesting the integrative reasoning demand is fundamentally different.
Models confined to text-only generation cannot execute multi-step procedures at scale, even when they know the underlying algorithm. Tool-enabled models solve problems beyond the supposed reasoning cliff, suggesting the bottleneck is procedural execution bandwidth.