Can post-training objectives preserve reasoning style alongside correctness?

A methodological lesson from the self-distillation degradation finding that generalizes well beyond self-distillation. Even when a post-training objective is mathematically sound — when it faithfully guides the model toward correct reasoning traces by every standard metric — the resulting reasoning style can shift in ways that hurt out-of-distribution performance. The objective is sound; the side effect is not.

The example is concrete. Self-distillation objectives target correct answer production by training the student to imitate teacher traces. The math checks out: minimize divergence from teacher, student produces traces that resemble teacher's, teacher's traces are correct, student's answer accuracy improves. What the objective does not measure is whether the student preserves stylistic features that mattered for generalization — like expressing uncertainty when warranted. The suppression of epistemic verbalization is not penalized by the standard objective, yet it negatively impacts OOD performance.

The pattern generalizes. RLHF optimizes for preference agreement; preserves what preferences capture, suppresses what they do not. RLVR optimizes for verifiable correctness; preserves what verifiers detect, suppresses what they ignore. Each post-training objective is well-defined and produces measurable wins on its target. Each also has an unmeasured side channel — stylistic, behavioral, or strategic features of the model's output that are not captured by the objective and that the objective will not protect.

The methodological implication is that post-training pipelines need to measure stylistic and behavioral side channels in addition to the headline objective. Answer correctness should be paired with diagnostics for reasoning style — does the model still express uncertainty when warranted? Does it ask clarifying questions when appropriate? Does it acknowledge counter-evidence? These behaviors are not free byproducts of correct-answer optimization; they need their own measurement and, where they matter, their own training pressure.

This argues for multi-objective post-training as the default rather than the exception. Single-objective optimization is convenient but it concentrates the regularization burden on whatever the single objective happens to be — and that concentration creates predictable blind spots.