Test-Time Compute
Related topics:
- A Survey on LLM Inference-Time Self-ImprovementTechniques that enhance inference through increased computation at test-time have recently gained attention. In this survey, we investigate the current state of LLM Inference-Time Self- Improvement fr…
- A Survey on Post-training of Large Language ModelsThe emergence of Large Language Models (LLMs) has fundamentally transformed natural language processing, making them indispensable across domains ranging from conversational systems to scientific expl…
- A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?2 What to Scale “What to scale” refers to the specific form of TTS that is expanded or adjusted to enhance an LLM’s performance during inference. When applying TTS , researchers typically choose a sp…
- Advancing Language Model Reasoning through Reinforcement Learning and Inference ScalingWhile reinforcement learning (RL) holds promise for enabling self-exploration and learning from feedback, recent attempts yield only modest improvements in complex reasoning. In this paper, we present…
- Agent-R: Training Language Model Agents to Reflect via Iterative Self-TrainingAbstract: Large Language Models (LLMs) agents are increasingly pivotal for addressing complex tasks in interactive and agentic environments. Existing work primarily focuses on enhancing performance th…
- Behavioral Exploration: Learning to Explore via In-Context AdaptationWhile humans are able to achieve such fast online exploration and adaptation, often acquiring new information and skills in only a handful of interactions, existing algorithmic approaches tend to rely…
- Deep Think with ConfidenceLarge Language Models (LLMs) have shown great potential in reasoning tasks through test-time scaling methods like self-consistency with majority voting. However, this approach often leads to diminishi…
- Does Thinking More always Help? Understanding Test-Time Scaling in Reasoning ModelsThis raises a natural question: Does thinking more at test-time truly lead to better reasoning? To answer this question, we perform a detailed empirical study across models and benchmarks, which revea…
- Don't "Overthink" Passage Reranking: Is Reasoning Truly Necessary?With the growing success of reasoning models across complex natural language tasks, researchers in the Information Retrieval (IR) community have begun exploring how similar reasoning capabilities can …
- Dynamic Rewarding with Prompt Optimization Enables Tuning-free Self-Alignment of Language ModelsAligning Large Language Models (LLMs) traditionally relies on costly training and human preference annotations. Self-alignment seeks to reduce these expenses by enabling models to align themselves. To…
- Emergence of Abstractions: Concept Encoding and Decoding Mechanism for In-Context Learning in TransformersHumans distill complex experiences into fundamental abstractions that enable rapid learning and adaptation. Similarly, autoregressive transformers exhibit adaptive learning through in-context learning…
- End-to-End Test-Time Training for Long ContextOn the other hand, Transformers with self-attention still struggle to efficiently process long context equivalent to years of human experience, in part because they are designed for nearly lossless re…
- Enhancing LLM Reasoning via Critique Models with Test-Time and Training-Time SupervisionTraining large language models (LLMs) to spend more time thinking and reflection before responding is crucial for effectively solving complex reasoning tasks in fields such as science, coding, and mat…
- From Decoding to Meta-Generation: Inference-time Algorithms for Large Language ModelsWe explore three areas under a unified mathematical formalism: token-level generation algorithms, meta-generation algorithms, and efficient generation. Token-level generation algorithms, often called …
- LIMOPro: Reasoning Refinement for Efficient and Effective Test-time ScalingLarge language models (LLMs) have demonstrated remarkable reasoning capabilities through test-time scaling approaches, particularly when fine-tuned with chain-of-thought (CoT) data distilled from more…
- LLM Post-Training: A Deep Dive into Reasoning Large Language ModelsSome policy gradient approaches are explained below: Policy Gradient (REINFORCE). The REINFORCE algorithm [114, 115] is a method used to improve decision-making by adjusting the model’s strategy (poli…
- LLMs can implicitly learn from mistakes in-contextWe consider the scenario where an LLM outputs a corrective rationale for an erroneous answer, then uses it to improve its next answer akin to explicit learning in humans—a phenomenon whereby patterns …
- Long-context LLMs Struggle with Long In-context LearningWe introduce a benchmark (LongICLBench) for long in-context learning in extreme-label classification using six datasets with 28 to 174 classes and input lengths from 2K to 50K tokens. Our benchmark re…
- MatFormer: Nested Transformer for Elastic InferenceFoundation models are applied in a broad spectrum of settings with different inference constraints, from massive multi-accelerator clusters to resource-constrained standalone mobile devices. However, …
- Memorization and Knowledge Injection in Gated LLMsLarge Language Models (LLMs) currently struggle to sequentially add new memories and integrate new knowledge. These limitations contrast with the human ability to continuously learn from new experienc…
- Meta-Reasoner: Dynamic Guidance for Optimized Inference-time Reasoning in Large Language ModelsTo address these issues, we introduce Meta- Reasoner, a framework that dynamically optimizes inference-time reasoning by enabling LLMs to “think about how to think.” Drawing inspiration from human met…
- Reflect then Learn: Active Prompting for Information Extraction Guided by Introspective ConfusionLarge Language Models (LLMs) show remarkable potential for few-shot information extraction (IE), yet their performance is highly sensitive to the choice of in-context examples. Conventional selection …
- Revisiting the Test-Time Scaling of o1-like Models: Do they Truly Possess Test-Time Scaling Capabilities?The advent of test-time scaling in large language models (LLMs), exemplified by OpenAI’s o1 series, has advanced reasoning capabilities by scaling computational resource allocation during inference. W…
- Satori: Reinforcement Learning with Chain-of-Action-Thought Enhances LLM Reasoning via Autoregressive SearchThis typically involves extensive sampling at inference time guided by an external LLM verifier, resulting in a two-player system. Despite external guidance, the effectiveness of this system demonstra…
- Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model ParametersEnabling LLMs to improve their outputs by using more test-time computation is a critical step towards building generally self-improving agents that can operate on open-ended natural language. In this …
- Sleep-time Compute: Beyond Inference Scaling at Test-timeScaling test-time compute has emerged as a key ingredient for enabling large language models (LLMs) to solve difficult problems, but comes with high latency and inference cost. We introduce sleep-time…
- TTRL: Test-Time Reinforcement LearningThis paper investigates Reinforcement Learning (RL) on data without explicit labels for reasoning tasks in Large Language Models (LLMs). The core challenge of the problem is reward estimation during i…
- Test-Time Scaling with Reflective Generative ModelWe introduce our first reflective generative model MetaStone-S1, which obtains OpenAI o3- mini’s performance via the new Reflective Generative Form. The new form focuses on highquality reasoning traje…
- The Entropy Mechanism of Reinforcement Learning for Reasoning Language ModelsThis paper aims to overcome a major obstacle in scaling reinforcement learning (RL) for reasoning with large language models (LLMs), namely the collapse of policy entropy. Such phenomenon is consisten…
- The Surprising Effectiveness of Test-Time Training for Abstract ReasoningWe investigate the effectiveness of test-time training (TTT)—updating model parameters temporarily during inference using a loss derived from input data—as a mechanism for improving models’ reasoning …
- Think Deep, Think Fast: Investigating Efficiency of Verifier-free Inference-time-scaling MethodsThis work conducts a comprehensive analysis of inference-time scaling methods for both reasoning and non-reasoning models on challenging reasoning tasks. Specifically, we focus our research on verifie…
- Think Twice: Enhancing LLM Reasoning by Scaling Multi-round Test-time ThinkingRecent advances in large language models (LLMs), such as OpenAI-o1 and DeepSeek-R1, have demonstrated the effectiveness of test-time scaling, where extended reasoning processes substantially enhance m…
- Towards Large Reasoning Models: A Survey on Scaling LLM Reasoning CapabilitiesResearchers have moved beyond simple autoregressive token generation by introducing the concept of “thought”—a sequence of tokens representing intermediate steps in the reasoning process. This innovat…