Why does removing semantic content collapse reasoning in language models?
This explores why language models stumble when you strip the real-world meaning out of a reasoning problem and leave only the abstract logical structure — and what that reveals about how they reason in the first place.
This explores why language models stumble when you strip the real-world meaning out of a reasoning problem and leave only the abstract logical structure. The corpus points to a blunt answer: LLMs aren't doing formal logic in the first place — they're pattern-matching on meaning. When semantic content is decoupled from a task, performance collapses even when the correct rules are sitting right there in the prompt, because models lean on parametric commonsense and token associations rather than manipulating symbols Do large language models reason symbolically or semantically?. Remove the familiar meaning and you remove the thing they were actually using to get the answer.
That framing gets sharper when you look at what *kind* of unfamiliarity breaks them. Reasoning failures track instance-level novelty, not task complexity — models succeed on a chain of any length if they've seen similar instances, and fail when the specific instance falls outside their training distribution Do language models fail at reasoning due to complexity or novelty?. Stripping semantics is essentially a way of forcing every instance to be novel: you've cut the cord to the commonsense associations that let the model recognize the problem. The same dependence shows up in reverse, too — when in-context information conflicts with strong training-time associations, the priors win and the model ignores what's actually in front of it Why do language models ignore information in their context?. So 'semantics' isn't decoration here; it's the retrieval key into everything the model knows.
Laterally, the corpus complicates the easy conclusion that this is a pure reasoning deficit. Some apparent reasoning is conservative bias in disguise — many models actually do *worse* when constraints are removed, because they were defaulting to safe answers rather than evaluating the logic Are models actually reasoning about constraints or just defaulting conservatively?. And some 'collapses' are really execution limits: models that know an algorithm still can't run it across many text-only steps, but clear the supposed cliff once given tools Are reasoning model collapses really failures of reasoning?. Read together, these say semantic stripping exposes a model that was already standing on shaky supports — meaning-based shortcuts, conservative defaults, limited procedural bandwidth — rather than introducing a brand-new problem.
There's also a structural blind spot underneath. LLMs make systematic errors on deep linguistic structure — embedded clauses, complex nominals — that worsen predictably as syntactic depth grows, suggesting statistical learning captures surface patterns but not the underlying grammatical rules Why do large language models fail at complex linguistic tasks?. Abstract, semantics-free reasoning demands exactly that kind of structural manipulation, which is where the surface-pattern strategy runs out of road.
The genuinely surprising turn: meaning can also be a *bridge* rather than a crutch. One line of work shows models reconstructing facts never stated in any single document by piecing together scattered semantic hints across the whole training distribution Can LLMs reconstruct censored knowledge from scattered training hints? — the same semantic-association machinery that fails on stripped logic puzzles is what lets models infer things no one explicitly taught them. And a different architectural bet, reasoning over sentence-level embeddings in a language-agnostic concept space, suggests the fix might be to reason at the level of *meaning* more deliberately rather than at the level of tokens Can reasoning happen at the sentence level instead of tokens?. So semantics isn't the bug — it's the substrate; removing it just reveals how little symbolic scaffolding sits underneath.
Sources 8 notes
When semantic content is decoupled from reasoning tasks, LLM performance collapses even with correct rules in context. Models rely on parametric commonsense and token associations rather than formal logical manipulation, constraining reasoning to training distribution semantics.
LRMs don't break at complexity thresholds but at instance-novelty boundaries. Models fit instance-based patterns rather than generalizable algorithms, so any reasoning chain succeeds if trained on similar instances, regardless of length.
Research demonstrates that LMs generate outputs inconsistent with their context because parametric knowledge from training dominates over in-context information. Textual prompting alone cannot override strong priors; causal intervention in representations is required.
Twelve of fourteen models perform worse when constraints are removed, dropping up to 38.5 percentage points. Models appear to reason correctly by defaulting to harder options, not by actually evaluating constraints.
Models confined to text-only generation cannot execute multi-step procedures at scale, even when they know the underlying algorithm. Tool-enabled models solve problems beyond the supposed reasoning cliff, suggesting the bottleneck is procedural execution bandwidth.
Top-tier LLMs like Llama3-70b consistently misidentify embedded clauses, verb phrases, and complex nominals. Performance degrades predictably as syntactic depth increases, revealing that statistical learning captures surface patterns but not deep grammatical rules.
Language models perform out-of-context reasoning across the full training distribution, reconstructing information never explicitly stated in any single document. Experiments show models can infer city identities from scattered distance relationships and apply them downstream without in-context learning.
Meta's Large Concept Model operates on sentence embeddings rather than tokens, reasoning in a language-agnostic space before decoding to any target language. This hierarchical approach with paragraph-level planning produces more coherent output than flat token generation.