Can longer reasoning chains eliminate model sensitivity to input noise?

This paper provides the theoretical grounding that was missing from empirical observations about overthinking. Using Lipschitz continuity analysis on a Linear Self-Attention model, the authors prove that while additional CoT steps dampen the propagation of input perturbations, they can never reduce sensitivity to zero. There is a non-zero lower bound on robustness loss that holds even at infinite chain length.

The mathematical structure is clean: each reasoning step applies a Lipschitz-continuous transformation, and the composition of these transformations contracts perturbation magnitude — but contraction is bounded. The perturbation signal decays geometrically but never vanishes because the transformation preserves a minimum fraction of input variation at each step.

Two empirical findings support the theory. First, sensitivity negatively correlates with input embedding norms: inputs with larger embedding magnitudes are more robust because the perturbation is proportionally smaller relative to the signal. Second, sensitivity negatively correlates with hidden state vector norms during reasoning: stronger internal representations dampen perturbation propagation more effectively.

This result has three implications for the reasoning trace literature:

Why overthinking doesn't help. Since Does more thinking time always improve reasoning accuracy?, the question has been whether longer reasoning eventually overcomes errors or amplifies them. The robustness bound shows that from a perturbation perspective, the answer is neither — perturbation sensitivity asymptotes to a floor. Beyond the optimal point, additional steps provide no further robustness improvement while introducing other failure modes (repetition, hallucination, loss of coherence).

Why adversarial attacks on reasoning models work. Since How vulnerable are reasoning models to irrelevant text?, the robustness bound explains why extended reasoning cannot defend against such attacks: the perturbation from adversarial input is structurally preserved through any number of reasoning steps.

Why prompt sensitivity persists. Since Does model confidence predict robustness to prompt changes?, the theoretical result provides the mechanism: even high-confidence models have a non-zero perturbation floor. Confidence improves robustness (larger embedding norms correlate with better damping) but cannot eliminate it.

The Linear Self-Attention restriction is important — the proof applies to a simplified architecture, and the bounds may not be tight for full Transformer models with softmax attention. But the qualitative result (damping with a floor) is likely to hold more generally, since the Lipschitz property is preserved under common architectural choices.

Source: Reasoning Methods CoT ToT