Towards Large Reasoning Models: A Survey on Scaling LLM Reasoning Capabilities

Researchers have moved beyond simple autoregressive token generation by introducing the concept of “thought”—a sequence of tokens representing intermediate steps in the reasoning process. This innovative paradigm enables LLMs’ to mimic complex human reasoning processes, such as tree search and reflective thinking. Recently, an emerging trend of learning to reason has applied reinforcement learning (RL) to train LLMs to master reasoning processes. This approach enables the automatic generation of high-quality reasoning trajectories through trial-and-error search algorithms, significantly expanding LLMs’ reasoning capacity by providing substantially more training data. Furthermore, recent studies demonstrate that encouraging LLMs to “think” with more tokens during test-time inference can further significantly boost reasoning accuracy. Therefore, the train-time and test-time scaling combined to show a new research frontier—a path toward Large Reasoning Model. The introduction of OpenAI’s o1 series marks a significant milestone in this research direction. In this survey, we present a comprehensive review of recent progress in LLM reasoning.

recent study shows scaling up test-time compute can also improve LLM reasoning accuracy. Specifically, PRMs can be used to guide LLMs to evaluate and search through the intermediate “thoughts” [134], which encourages LLMs to generate deliberate reasoning steps during test-time computation and boosts reasoning accuracy. This approach gives rise to the test-time scaling law, which predicts spending more tokens for deliberate reasoning at test-time can improve accuracy [103]. Therefore, the RL-driven train-time scaling and search-based test-time scaling combined to show a promising research direction to fully unleash the reasoning capabilities of LLMs, i.e., a path toward Large Reasoning Models. A key milestone in this research direction is OpenAI’s o1 series [194], which demonstrates the effectiveness of this approach and echoes OpenAI’s vision of transitioning LLMs from conversational AI (level 1) to more powerful reasoning AI (level 2) in the five-steps road map toward AGI [36].