What makes chain-of-thought reasoning fail in language models? · Gravity7

Sub-Maps

2 notes

What makes chain-of-thought reasoning actually work?

Explores the structural and mechanical properties that determine how reasoning traces function in language models. Understanding these properties reveals why format matters more than logic and what tokens carry the most information about correct answers.

Why does chain-of-thought reasoning fail in predictable ways?

Explores evidence that CoT failures stem from imitation of reasoning form rather than genuine inference. Examines distribution-bounded degradation, structural pattern matching, and error amplification across multiple failure modes.

Reasoning Step Taxonomy and Intervention

1 note

Can reasoning steps be dynamically pruned without losing accuracy?

This explores whether chain-of-thought reasoning contains redundant steps that can be identified and removed during inference. Understanding which steps matter could improve efficiency while maintaining correctness.

Writing Angles

2 notes

Why do more capable reasoning models ignore your instructions?

As AI models develop stronger reasoning abilities, they seem to follow instructions less reliably. What causes this counterintuitive trade-off, and how severe is the problem in practice?

What critical thinking skills do reasoning models actually lose?

Step-by-step reasoning training optimizes narrow deductive thinking while degrading meta-cognitive abilities like recognizing futile thinking and maintaining tentative reasoning. Understanding this tradeoff matters for deploying reasoning models reliably.

Related Areas

3 notes

How should reasoning systems actually be architected?

This explores the fundamental design choices for building reasoning into AI systems—from when to activate reasoning versus how to execute it, to whether reasoning must be verbal or can happen in latent space.

How do reasoning models actually break under pressure?

This hub explores what happens when you stress-test reasoning models—their reflection mechanisms, behavioral patterns on hard tasks, vulnerability to adversarial attacks, and surprising weaknesses in social reasoning compared to formal logic.

How should we allocate compute budget at inference time?

Test-time scaling explores how to spend computational resources during query rather than training. The core challenge: given a fixed inference budget, what's the optimal allocation strategy for different problems?

New — 2026-06-27

3 notes

Do reasoning traces actually show how models think?

Explores whether the step-by-step narrative in reasoning traces reflects the actual computational dependencies inside the model, or whether traces are stylistic constructs that only resemble reasoning.

How should we allocate compute budget at inference time?

Test-time scaling explores how to spend computational resources during query rather than training. The core challenge: given a fixed inference budget, what's the optimal allocation strategy for different problems?

How do you navigate synthesis across fragmented research topics?

This note explores how researchers can connect insights scattered across multiple topic files and papers into coherent narratives for writing and publication. It asks what structures and workflows help surface cross-cutting patterns that keyword search alone misses.