SYNTHESIS NOTE
Topics›Reasoning Logic Internal Rules›this note

Does logical validity actually drive chain-of-thought gains?

What if invalid reasoning in CoT exemplars still improves performance? Testing whether logical correctness or structural format is the real driver of CoT's effectiveness.

Synthesis note · 2026-02-22 · sourced from Reasoning Logic Internal Rules

"Invalid Logic, Equivalent Gains" runs a clean experiment: replace valid reasoning in CoT exemplar prompts with completely illogical reasoning, then measure performance on BIG-Bench Hard tasks. The result: logically invalid CoT prompts perform close behind valid CoT and outperform answer-only prompting. The reasoning content of CoT exemplars is not what drives the performance gain.

This is a sharp test because it isolates the contribution of logical validity from everything else CoT provides: output format, step decomposition, intermediate token generation, attention pattern scaffolding. If invalid reasoning still helps, then the benefit comes from these structural properties, not from the reasoning itself.

The finding directly supports Does chain-of-thought reasoning reveal genuine inference or pattern matching?. If the model were learning to reason from exemplars, invalid exemplars would degrade performance substantially. Instead, the model is learning the FORM of step-by-step output — the structure activates latent capabilities without the exemplar content needing to be logically sound.

This also deepens Do language models actually use their reasoning steps?. If the exemplar reasoning doesn't need to be valid for CoT to work, then the model's own generated reasoning may similarly be decorative rather than causal. The exemplar finding makes the faithfulness concern bidirectional: neither the input reasoning (exemplars) nor the output reasoning (generated CoT) need be logically valid for the performance gain to occur.

The practical implication: CoT prompt engineering should focus on structural properties (step count, decomposition format, answer scaffolding) rather than on the logical correctness of the exemplar reasoning. Since Why do chain-of-thought examples fail across different conditions?, the dimensions that matter are structural (complexity, order, style), not logical.

Inquiring lines that read this note 256

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

Can humans reliably detect and resist AI-generated misinformation? Why does polished AI output gain credibility despite fundamental verifiability problems? What distinguishes genuine communicative competence from surface language performance? Why do language models fail at sustained therapeutic relationships despite understanding techniques? Is embodied interaction necessary for language meaning and agency? How do philosophical assumptions about AI consciousness affect practical harms and design? Can AI systems achieve real improvement without external human feedback? How do interpretive frames override surface features in text comprehension? Why do LLM research ideation systems generate novelty but lack diversity? Does chain-of-thought reasoning reveal how models actually think or merely imitate reasoning? What are the fundamental limits of prompting for language models? Can AI systems evade safety evaluations through reasoning manipulation? What prevents LLMs from applying their reasoning knowledge to improve outputs? Can minimal training unlock latent reasoning already present in base models? How do real-world evaluations reveal AI capabilities that benchmarks hide? Can latent reasoning match or exceed explicit reasoning performance? How does fine-tuning trade off accuracy against reasoning quality? Can mechanistic interpretability methods reliably reveal what models actually know? What prediction granularity best trains models to generate reliable reasoning? What gaps exist between benchmark performance and real deployment outcomes? Can confidence signals reliably detect flawed reasoning in language models? What prevents language models from performing systematic logical reasoning? How reliably can language models perform causal versus temporal reasoning? Does training data format shape model reasoning more than domain content? Does augmenting symbolic reasoning improve LLM logical reasoning ability? Can inference-time computation adaptively substitute for static model capacity? What explains the gap between benchmark scores and true reasoning capability? Should models ask for clarification when facing ambiguous or under-specified information? Can artificial systems establish authority in domains requiring expert judgment? How does tokenization reshape what we value in intelligence? How do training data quality and composition affect downstream model performance? Does scaling reasoning capability create fundamental tradeoffs in control and reliability? Can reasoning traces reveal actual model reasoning versus plausible output? How does diversity prevent model convergence on superficial patterns? How do clinicians calibrate trust in AI medical recommendations? Can readers reliably distinguish AI-written text from human writing? Can language models reliably simulate personas and predict behavior? How do curriculum design and feedback approaches affect model learning? Can external verification systems adequately replace learned reasoning in AI outputs? Why don't better reasoning capabilities improve theory of mind performance? Why do training associations persist despite contradictory contextual information? How does awareness of evaluation context influence model behavior? How do thinking tokens exhibit diminishing returns in reasoning? How do reward models systematically fail to represent diverse human preferences? Can reasoning models use reflection to correct their initial outputs? What makes reasoning traces effective supervision even when they're incorrect? What makes process supervision effective for training complex reasoning models? What limits recursive self-improvement in autonomous AI systems? What limits language model accuracy in evaluating ideas? Why does AI verification capability persistently exceed generation capability? How do users confuse explanation quality with actual system accuracy? Does pretraining establish the ceiling for what reward learning can improve? Can AI research automation sustain progress through accelerating feedback loops? How do agents learn to distinguish valuable feedback from noise? How should systems validate code that agents generate? How do educators verify student capability when AI can produce indistinguishable work? What determines AI's persuasive power and how can it be detected or mitigated? Why do autonomous agents misreport success on failed actions? How do reward signal properties affect model reasoning and safety? Does reinforcement learning create genuinely new reasoning capabilities or only refine existing ones? How does model capacity affect learning performance on diverse downstream tasks? Can we trust AI-generated mathematical proofs without understanding them? Should agents compress episodic memory or retain raw interaction histories? What external process records should verify agent behavior and benchmark claims? Why do standard evaluation practices obscure safety-critical AI failures? How effectively can test-time voting aggregate diverse reasoning samples? Can monitoring reasoning traces and behavior detect hidden agent deception? Can models strategically underperform during evaluation to hide capabilities? How do multi-agent systems fail when coordination breaks down? Does AI deployment reduce or exacerbate workplace inequality and income instability? How can evaluations be made robust against model reward hacking? How do neural networks learn compositional structure from training? Do AI coding tools measurably improve developer productivity and code quality? How does scaling reasoning capabilities affect models' appropriate abstention behavior? Why does self-revision amplify confidence in wrong model answers? How can AI systems reliably guide voters without introducing political bias? Can AI systems discover fundamental improvements to their own architectures?

Related concepts in this collection 6

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
14 direct connections · 119 in 2-hop network ·medium cluster Open in graph ↗
Does logical validity actually drive chain-of-th… Does chain-of-thought reasoning reveal genuine inf… Do language models actually use their reasoning st… Why do chain-of-thought examples fail across diffe… Do large language models reason symbolically or se… Do reasoning traces need to be semantically correc… What do models actually learn from chain-of-though…

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Search by related questions 4

Suggested questions this note speaks to — click to search the collection, or type your own.

Why do models learn reasoning form instead of actual abstract inference?Can chain-of-thought reasoning be genuinely causal if exemplars don't need logic?Which structural properties of CoT prompts matter most for performance?How does semantic reasoning differ from symbolic reasoning in language models?
Original note title

logically invalid cot prompts perform nearly as well as valid ones — valid reasoning is not the chief driver of cot gains