Can LLMs infer implicit meaning without surface linguistic markers?
This explores whether LLMs can grasp meaning that isn't spelled out on the surface — implicature, intended reading, logical entailment — or whether they're keyed to the literal textual signals (frequency, attestation, phrasing) in front of them.
This explores whether LLMs can grasp meaning that isn't spelled out on the surface — and the corpus mostly says: they lean on the surface, often more than we'd like. The most direct evidence is that models track statistical mass rather than meaning. When you give an LLM two paraphrases that mean exactly the same thing, it systematically favors the higher-frequency wording across math, translation, and commonsense tasks Do language models really understand meaning or just surface frequency?. A related bias shows up in inference: models predict that a premise entails a hypothesis based on whether the hypothesis looks familiar from training, not on whether the premise actually supports it Do LLMs predict entailment based on what they memorized?. In both cases the model is reading the surface signal and inferring meaning from its statistical fingerprint, not from the relationships the text encodes.
Where implicit meaning requires holding more than one reading at once, the failure is sharp. On deliberate ambiguity, GPT-4 correctly recognizes multiple interpretations in only 32% of cases against 90% for humans — it can't hold competing readings in suspension, which is exactly what implicature demands Can language models recognize when text is deliberately ambiguous?. And when meaning has to be derived from structure rather than from familiar content, performance collapses: strip the semantic content out of a reasoning task and models fail even with the correct rules sitting in context, because they're running on token associations rather than formal manipulation Do large language models reason symbolically or semantically?. Syntactic depth makes it worse in a predictable way — embedded clauses and complex nominals trip top models in proportion to structural complexity Why do large language models fail at complex linguistic tasks?.
But the interesting wrinkle is that 'understands meaning' and 'reads only the surface' aren't a clean binary in this corpus. Mechanistic interpretability finds three coexisting tiers — conceptual, world-state, and principled circuit-level understanding — layered on top of, not replacing, shallow heuristics Do language models understand in fundamentally different ways?. That patchwork explains 'Potemkin' failures: a model can correctly explain a concept, fail to apply it, and then recognize its own failure, because explanation and execution run on functionally disconnected pathways Can LLMs understand concepts they cannot apply?. So implicit meaning sometimes gets represented internally even when the model can't deploy it.
There's also a more generous reading worth surfacing. One line of work argues LLMs operationalize Saussure's *langue* — they recover culturally situated meaning purely by compressing relational structure in text, no external grounding required Can language models learn meaning without engaging the world?. And with explicit chain-of-thought scaffolding, o1-class models can build syntactic trees and phonological generalizations, i.e. reason *about* language rather than just produce it Can language models actually analyze language structure?. The thread connecting these: implicit inference seems gated by whether the model can route a task through reasoning rather than reflex. Left to its defaults, it reaches for frequency and familiarity; given structure to follow, it can sometimes recover the meaning underneath. The honest synthesis is that LLMs infer implicit meaning unreliably and conditionally — strongest when the implicit content rhymes with their training distribution, weakest when it requires juggling multiple readings or honoring structure over surface.
Sources 9 notes
LLMs show consistent preference for higher-frequency surface forms over semantically equivalent rare paraphrases across math, machine translation, commonsense reasoning, and tool calling. This suggests models track statistical mass from pretraining rather than meaning-recognition as their primary mechanism.
McKenna et al. (2023) identified attestation bias: LLMs predict entailment based on whether the hypothesis appears in training data, not whether the premise actually supports it. Random premise experiments show models maintain high entailment predictions when hypotheses are attested, proving they respond to memorized propositions rather than premise-hypothesis relationships.
AMBIENT benchmark shows GPT-4 correctly disambiguates only 32% of cases versus 90% for humans. This failure spans lexical, structural, and scope ambiguity—revealing that LLMs cannot hold multiple interpretations simultaneously, a fundamental gap hidden by standard benchmarks.
When semantic content is decoupled from reasoning tasks, LLM performance collapses even with correct rules in context. Models rely on parametric commonsense and token associations rather than formal logical manipulation, constraining reasoning to training distribution semantics.
Top-tier LLMs like Llama3-70b consistently misidentify embedded clauses, verb phrases, and complex nominals. Performance degrades predictably as syntactic depth increases, revealing that statistical learning captures surface patterns but not deep grammatical rules.
Mechanistic interpretability reveals conceptual understanding (features as directions), state-of-world understanding (factual connections), and principled understanding (compact circuits). Crucially, higher tiers coexist with lower-tier heuristics rather than replacing them, creating a patchwork of capabilities.
Models can explain concepts accurately, fail to apply them, and recognize the failure—a triple pattern incompatible with human cognition. This indicates functionally disconnected explanation and execution pathways rather than simple knowledge gaps.
Research shows LLMs learn culturally situated discourse patterns by compressing relational structure from text, demonstrating that fluent language generation requires no external referents or embodied grounding.
OpenAI's o1 model successfully constructs syntactic trees and phonological generalizations through explicit step-by-step reasoning, revealing that LLM linguistic capability extends far beyond behavioral language tasks to genuine language analysis.