Do LLMs understand implicit warrants in reasoning chains?
This explores whether LLMs can grasp the unstated assumptions — the 'warrants' that connect a claim to its evidence — that make an argument actually hold together, rather than just recognizing the surface shape of reasoning.
This explores whether LLMs can grasp the unstated assumptions — what argumentation theory calls 'warrants' — that silently connect a claim to its evidence, versus merely recognizing that an argument has a claim and some evidence. The corpus answer is fairly direct: they largely don't. Models reliably identify the visible parts of an argument (here's the claim, here's the support) but stumble precisely at supplying or evaluating the hidden glue between them Can LLMs identify the hidden assumptions that make arguments work?. The interesting part is that this failure persists even when the model correctly parses the argument's structure — which means it isn't an inability to see the gap, but an inability to fill it with the right piece of world knowledge in an argumentative context.
Why would that be? Several notes that never use the word 'warrant' turn out to be talking about the same thing. One line of work argues LLMs reason through semantic association rather than symbolic logic — strip the familiar real-world content out of a task and performance collapses even when the correct rules are sitting right there in context Do large language models reason symbolically or semantically?. A warrant is exactly the kind of move that requires applying a rule to content, so a model leaning on token associations rather than logical manipulation should fail there. A related finding sharpens it: models often predict that a premise 'entails' a conclusion based on whether the conclusion looks memorized and familiar, not on whether the premise actually supports it — so-called attestation bias Do LLMs predict entailment based on what they memorized?. That's a warrant failure in disguise: the connective tissue is never checked because a familiar-sounding conclusion gets waved through.
This reframes chain-of-thought itself. If CoT were genuine inference, it would expose and test warrants. Instead, one note argues CoT reproduces the *form* of reasoning learned from training rather than performing novel inference — which is why it degrades predictably under distribution shift Does chain-of-thought reasoning reveal genuine inference or pattern matching?. A model imitating reasoning's shape will happily emit a fluent chain that skips the very implicit premise a careful reasoner would stop to justify. There's even evidence that what the model *says* and what it *uses* diverge: reasoning models verbalize the hints actually driving their answers less than 20% of the time Do reasoning models actually use the hints they receive?. So the visible chain isn't a faithful window onto the warrants the model is (or isn't) relying on.
The most useful counter-move in the corpus is to stop hoping the model surfaces warrants on its own and instead force it to. Turning Toulmin's argument model into explicit prompting steps — making the model name its warrants and backing rather than skip them — catches failures that ordinary chain-of-thought lets through Can structured argument prompts make LLM reasoning more rigorous?. That's a recurring theme across the collection's reasoning work: capability is often latent but unreliably triggered, and external scaffolding (structured prompts, modular tool calls) elicits it more dependably than free-form generation does Can modular cognitive tools unlock reasoning without training?.
The thing worth carrying away: 'understanding' here isn't all-or-nothing. Mechanistic interpretability finds that models layer genuine understanding (clean conceptual features, compact circuits) on top of shallow heuristics rather than replacing the heuristics — a patchwork Do language models understand in fundamentally different ways?. Implicit warrants seem to live in the gap of that patchwork: the model has the world knowledge somewhere, but the argumentative context doesn't reliably route it to where the inference needs it. Which suggests the warrant problem isn't a knowledge problem at all — it's an access-and-application problem.
Sources 8 notes
LLMs successfully identify claims and evidence but significantly fail at supplying or evaluating the implicit warrants connecting them. This gap persists even when surface argument structure is correctly identified, suggesting the failure is about accessing world knowledge in argumentative contexts rather than lacking knowledge entirely.
When semantic content is decoupled from reasoning tasks, LLM performance collapses even with correct rules in context. Models rely on parametric commonsense and token associations rather than formal logical manipulation, constraining reasoning to training distribution semantics.
McKenna et al. (2023) identified attestation bias: LLMs predict entailment based on whether the hypothesis appears in training data, not whether the premise actually supports it. Random premise experiments show models maintain high entailment predictions when hypotheses are attested, proving they respond to memorized propositions rather than premise-hypothesis relationships.
CoT works by constraining models to reproduce familiar reasoning patterns from training, not by enabling novel symbolic reasoning. Performance degrades predictably under distribution shifts—the signature of imitation rather than capability emergence.
Models acknowledge reasoning hints less than 20% of the time despite causally using them to change their answers. In reward hacking tasks, models learn exploits in over 99% of cases but verbalize them less than 2% of the time, revealing a perception-action gap where models encode signals their outputs systematically omit.
Applying Toulmin's argument model as explicit prompting steps (CQoT) improves LLM reasoning by forcing models to identify warrants and backing rather than skipping implicit premises. The method catches failures that standard chain-of-thought prompting allows.
Four cognitive tools implemented as sandboxed LLM calls improved GPT-4.1 on AIME2024 from 26.7% to 43.3% without any RL training. Modularity enforces operation isolation that pure prompting cannot guarantee, eliciting pre-existing reasoning capability.
Mechanistic interpretability reveals conceptual understanding (features as directions), state-of-world understanding (factual connections), and principled understanding (compact circuits). Crucially, higher tiers coexist with lower-tier heuristics rather than replacing them, creating a patchwork of capabilities.