Can implicit linguistic information ever be reliably learned from training data?
This explores whether the things language never says out loud — social conventions, deep grammar, the actual rate of irony, communicative intent — can be picked up reliably from text-only training, and the corpus suggests the answer splits sharply by *what kind* of implicit information you mean.
This explores whether implicit linguistic information — the stuff that's never explicitly stated in text but rides underneath it — can be reliably absorbed from training data. The corpus draws a surprisingly clean line: models reliably learn implicit *relational* and *distributional* structure, but systematically misfire on implicit *grounding*, *deep grammar*, and *social action*.
Start with the optimistic side. There's strong evidence that one whole class of implicit information transfers well: the relational patterning *within* language itself. Research framing LLMs through Saussure's "langue" argues that fluent generation comes from compressing the relational structure of text — culturally situated discourse patterns that no one writes down explicitly — without any external referents at all Can language models learn meaning without engaging the world?. So implicit *intra-linguistic* convention is learnable. The trouble starts when the implicit information points *outside* the text.
The sharpest "no" is about meaning itself. Bender & Koller's argument is that meaning lives in the relation between expressions and communicative intent, and since training only ever sees form-to-form prediction with no shared attention, that grounding can't be reconstructed from text alone Can language models learn meaning from text patterns alone?. A parallel failure shows up in conversation maintenance — reference repair, topic hand-off, the implicit relational work that keeps talk smooth. Models don't develop these because the training signal rewards *predicting information*, not *doing social action*, so an entire layer of implicit competence is simply never selected for Why don't language models develop conversation maintenance skills?.
Even when implicit information *is* statistically present, it can be learned in distorted form. Models detect irony as a pattern but badly overestimate how often it occurs, because ironic examples are more salient in training text than in real use — the implicit *base rate* gets miscalibrated even though the surface signal is captured Do language models overestimate how often irony appears?. And implicit grammatical structure degrades predictably: top models misidentify embedded clauses and complex nominals, with errors worsening as syntactic depth increases — statistical learning grabs surface patterns but not the deep rules underneath Why do large language models fail at complex linguistic tasks?. There's even a theory of *why*: framing the model as an autoregressive probability machine predicts that low-probability targets stay hard no matter how logically simple they are, which means some implicit information is gated by frequency, not learnability Can we predict where language models will fail?.
Here's the thing you might not have expected: "reliably learned" sometimes fails not because the information is absent but because *stronger priors bury it*. Models generate outputs inconsistent with their own context when parametric knowledge from training dominates — and crucially, prompting can't override it; you need causal intervention in the representations themselves Why do language models ignore information in their context?. So the honest synthesis is: implicit information that's purely about how words relate to other words is learned well; implicit information that requires intent, grounding, social action, accurate base rates, or rare structure is learned unreliably — and the deeper failures are architectural, not fixable by feeding in more text.
Sources 7 notes
Research shows LLMs learn culturally situated discourse patterns by compressing relational structure from text, demonstrating that fluent language generation requires no external referents or embodied grounding.
Bender & Koller argue that meaning requires the relation between expressions and communicative intents. Since LLMs are trained only on form-to-form prediction with no access to shared attention or intent, they cannot reconstruct the meaning that grounds language.
Humans keep conversations smooth through implicit techniques like reference repair and topic hand-off that sustain relational interaction, not convey information. Language models don't develop these because training signals reward information prediction, not relational work.
GPT-4o assigns significantly higher irony scores than humans (p < .001), revealing that LLMs detect irony as a pattern but miscalibrate its prevalence because ironic examples are more salient in training data than in actual use.
Top-tier LLMs like Llama3-70b consistently misidentify embedded clauses, verb phrases, and complex nominals. Performance degrades predictably as syntactic depth increases, revealing that statistical learning captures surface patterns but not deep grammatical rules.
By framing LLMs as autoregressive probability machines, researchers predicted tasks with low-probability target responses would be systematically harder, even when logically simple. Experiments confirmed predictions like backwards alphabet and letter counting.
Research demonstrates that LMs generate outputs inconsistent with their context because parametric knowledge from training dominates over in-context information. Textual prompting alone cannot override strong priors; causal intervention in representations is required.