What makes reflection actually work in reasoning models?

LR²Bench's most useful contribution is not the 20% number but the decomposition that produces it. The benchmark frames reflective reasoning as three concrete capabilities: making assumptions (positing a tentative value to make progress), backtracking (retracting on constraint violation), and self-refinement (improving partial solutions toward feasibility). These are operationalized into CSP-solving structure where each capability is measurable in outcome rather than appearance. This reframes reasoning evaluation: the question is not "can the model think longer" but "can the model retract and try again."

The frame converges with a cluster of vault notes that have been circling the same claim from different angles. Does reflection in reasoning models actually correct errors? argues training-time mechanism: what RLHF and reasoning fine-tuning learn is to produce confident-sounding first answers with confirmatory reflection language attached, not actual revision. Does self-revision actually improve reasoning in language models? argues that even when revision is attempted, it makes things worse rather than better. Is reflection in reasoning models actually fixing mistakes? gives the bottom line. LR²Bench's 20% ceiling is the cleanest quantitative anchor for this cluster — when the task structurally requires backtracking and assumptions to be revised, models trained to produce reflective traces collapse.

The methodological lesson is to stop using chain length as a proxy for reasoning capability. Long chains are easy to produce; reflective chains that satisfy constraints are not. Evaluations that score on trace length, trace presence, or trace style measure the surface mimicry of reflection. Evaluations that score on whether the constraints were actually satisfied measure the underlying capability. LR²Bench's three-primitive decomposition is the cleanest available articulation of what reflection actually requires in operational terms. Future benchmarks should adopt the decomposition as the unit of analysis rather than re-running the same chain-length-versus-accuracy correlations that have already shown they decouple.

Source: Reasoning Methods CoT ToT Paper: LR^2Bench: Evaluating Long-chain Reflective Reasoning Capabilities of Large Language Models via Constraint Satisfaction Problems