Does reflection in reasoning models actually correct errors?

The prevailing account of reasoning model improvement attributes gains to the model's ability to detect and correct initial errors through extended reflection. First Try Matters tests this directly: systematic analysis of rollouts from 8 reasoning models on 5 mathematical datasets finds that reflections — the reasoning that occurs after the model has produced a candidate answer — are predominantly confirmatory. The model continues generating reasoning tokens but rarely changes the initial answer.

The training implication reverses expected causality: training on datasets with more reflection steps does not improve the model's ability to correct wrong answers through reflection. It improves the quality of the first answer. What looks from the outside like "better self-correction" is actually "better initial reasoning that reflection then confirms."

This means the cognitive work happens before the first answer, not during the visible reflection loop. The visible reflection steps are largely post-hoc — the model has already decided, and the reflection tokens are generating confirmation rather than revision.

Two practical consequences follow:

Token efficiency: early stopping after the first plausible candidate answer saves 24.5% of total tokens with only 2.9% accuracy drop. If most post-first-answer tokens are confirmatory, they can be cut without substantial accuracy loss.

Advanced reasoning methods yield highly variable outcomes in dynamic environments: "Towards a Deeper Understanding of Reasoning Capabilities" tests self-reflection, heuristic mutation, and planning as prompting techniques in dynamic benchmarks (not static math). The finding: while capable of significantly improving performance when reasoning and decision-making align, advanced reasoning methods also introduce instability and can produce large performance drops. Larger models are more robust to this variability; smaller models benefit more from strategic prompting but are also more susceptible to degradation from too-long prompts on basic reactive tasks. The evidence against true emergent reasoning: persistent limitations in planning, spatial coordination, and general reasoning survive self-reflective prompting. This extends the confirmatory-not-corrective finding beyond math: in dynamic environments, reflection is not just unhelpful for correction — it can actively destabilize.

Difficulty-dependent condition (Hindsight paper): self-reflection is beneficial when the model is less likely to be correct initially AND when question difficulty is high. It's harmful when the model is reliably giving correct answers. The interaction: on easy questions where the model is already right, reflection introduces perturbation risk (switching correct to incorrect). On hard questions where the model is often wrong, reflection provides a second chance that sometimes catches errors. Self-reflection also reduces the model's tendency toward majority voting, suggesting more sophisticated (if not always more accurate) decision-making. This quantifies when confirmatory reflection switches from harmless to harmful.

Training implications: if the goal is self-correction capability (the ability to actually fix wrong first answers), more reflection training is the wrong intervention. What's needed is either better first-pass reasoning, genuinely external critique, or online RL under the model's own error distribution — not more self-reflection on outputs the model is already confident about.

This refines Does self-revision actually improve reasoning in language models? with a more precise mechanism: the question is not just "does revision hurt?" but "does revision actually happen?" The finding is that most reflection tokens are not revision at all — they are confirmation that the model was already right (or wrong, without noticing).

Source: Reasoning by Reflection; enriched from Reasoning o1 o3 Search, Self Refinement Self Consistency Feedback