Prefix Sliding for efficient test-time scaling

Test-time scaling uses extra test-time compute to improve performance, such as letting language models reason longer when solving a problem. As models keep the entire reasoning trace in memory via full attention, hard tasks that need long thinking can be prohibitively expensive. However, we find most intermediate reasoning tokens lose importance as the model continues reasoning. This calls into question whether retaining them is worth the cost. Based on this insight, we propose Prefix Sliding, which discards tokens during reasoning that are not part of the prefix or the window of the last few thousand tokens. The prefix has key instructions and tools available to the model, while the most recent tokens are the current reasoning the model is working on. This caps the total memory requirement regardless of how long the model reasons, allowing for efficient long-horizon test-time scaling. Without training, Prefix Sliding can make existing models 3x faster while maintaining performance. Training with Prefix Sliding using reinforcement learning can achieve better performance by enabling scaling to reasoning traces beyond a hundred thousand tokens. Ablations show Prefix Sliding outperforms summarizing intermediate tokens or vanilla sliding window.

Introduction. Test-time scaling improves the performance of language models by using extra compute for hard problems (OpenAI, 2024). Commonly, this compute is used by letting the model reason longer (DeepSeek-AI et al., 2025a; Muennighoff et al., 2025). However, scaling this approach further is limited by the need to keep the entire reasoning trace in memory via full attention, as used in most language models (Sadhukhan et al., 2025). With full attention, the cost of each new token grows linearly with the number of already generated tokens, making long context windows prohibitively expensive. Long contexts have more issues, including distraction by old irrelevant tokens (Gema et al., 2025), context poisoning (Comanici et al., 2025), repetitive loops (Pipis et al., 2025), and lost knowledge (Liu et al., 2023b). We explore a simple solution based on two observations. First, intermediate reasoning tokens quickly lose importance.

Discussion / Conclusion. We propose Prefix Sliding to enable language models to reason for extremely long horizons. Even at short reasoning horizons of only thousands of tokens, Prefix Sliding is more efficient than the status quo of using full attention. Prefix Sliding is applicable to language models without further training. It can also be used during training with reinforcement learning. It outperforms alternatives that could also support infinite test-time scaling. We hope that enabling language models to think longer via Prefix Sliding inspires future work on solving ever harder problems with language models. Information loss While intermediate tokens can lack importance for later reasoning, as we show in section 2, sometimes this is not the case. Figure 11 shows this limitation on the example of LiveCodeBench, where a larger window size is necessary to match full attention.