When does RL discover genuinely novel reasoning strategies versus timing optimization?

This explores a genuine fault line in the collection: does RL teach a model new ways to think, or does it just sharpen the timing and sampling of thinking that was already latent? The corpus stakes out both sides clearly, and the interesting part is the *conditions* that decide which one you get.

The skeptical camp is large and specific. One thread argues RL post-training teaches *when* to reason rather than *how* — hybrid models recover 91% of the gains just by routing tokens, and the activation vectors for reasoning strategies already exist before any RL touches the model Does RL post-training create reasoning or just deploy it?. A parallel finding on reward dynamics shows RLVR improves sampling efficiency *within* existing capability boundaries without expanding them — a single training example can trigger the effect, and even spurious rewards work nearly as well as correct ones, which is hard to explain if real new skills were being installed What does reward learning actually do to model reasoning?. The harshest version comes from out-of-distribution stress tests: RL-fine-tuned models drop sharply on N-1 variants of problems they handle in-distribution, suggesting RL sharpens template-matching rather than installing a general procedure Do fine-tuned language models actually learn optimization procedures?.

But the collection also holds a direct rebuttal, and it's the most important note for this question. Prolonged RL — run long enough, on *diverse and non-mathematical* tasks, with KL control and policy resetting — produces models that beat the base model across *all* pass@k levels, not just at low sampling budgets Can reinforcement learning discover reasoning strategies base models cannot?. That pass@k detail is the crux: if RL only optimized sampling, a base model given enough tries should eventually match it. When the base model can't catch up no matter how many samples you draw, you've crossed from timing optimization into genuine capability expansion. The note's emphasis on domains where base models *lack established patterns* is the tell — novelty shows up precisely where there was no latent strategy to merely re-deploy.

So the answer to "when" is less about RL as a technique and more about where you point it. On math and familiar templates, the evidence leans heavily toward timing and sampling optimization — and a related result shows that on constraint-satisfaction problems requiring real backtracking, even frontier reasoning models stall at 20-23%, meaning fluent reflection doesn't convert to competence on unfamiliar structure Can reasoning models actually sustain long-chain reflection?. On diverse, pattern-sparse domains with the right training controls, RL appears to find something new. The boundary between the two regimes is exactly the diversity-and-novelty of the task distribution.

Worth knowing as you sit with this: the field is partly resolving the dispute by *separating the two jobs* rather than arguing which one RL does. Decoupled-RL systems explicitly train a model to route between extended thinking and quick answers — treating "when to reason" as a learnable skill in its own right, distinct from the reasoning content Can models learn when to think versus respond quickly?. And a quieter finding suggests some apparent RL gains are really fixes for *disorganization* — reasoning models abandon valid paths prematurely, and decoding-level nudges recover accuracy with no fine-tuning at all, implying the good strategy was already there and merely mis-deployed Why do reasoning models abandon promising solution paths?. The question you came in with may turn out to be two questions wearing one coat.