Context Engineering

Related topics:

• Activation Steering for Chain-of-Thought Compression
  Large language models (LLMs) excel at complex reasoning when they include intermediate steps, known as chains of thought (CoTs). However, these rationales are often overly verbose, even for simple pro…
• Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models
  Large language model (LLM) applications such as agents and domain-specific reasoning increasingly rely on context adaptation—modifying inputs with instructions, strategies, or evidence, rather than we…
• Behavioral Exploration: Learning to Explore via In-Context Adaptation
  While humans are able to achieve such fast online exploration and adaptation, often acquiring new information and skills in only a handful of interactions, existing algorithmic approaches tend to rely…
• Can Looped Transformers Learn to Implement Multi-step Gradient Descent for In-context Learning?
  The remarkable capability of Transformers to do reasoning and few-shot learning, without any fine-tuning, is widely conjectured to stem from their ability to implicitly simulate a multi-step algorithm…
• Context Tuning for Retrieval Augmented Generation
  “Large language models (LLMs) have the remarkable ability to solve new tasks with just a few examples, but they need access to the right tools. Retrieval Augmented Generation (RAG) addresses this prob…
• Extrapolation by Association: Length Generalization Transfer in Transformers
  Transformer language models have demonstrated impressive generalization capabilities in natural language domains, yet we lack a fine-grained understanding of how such generalization arises. In this pa…
• Foundation Priors
  Foundation models, and in particular large language models, can generate highly informative responses, prompting growing interest in using these “synthetic” outputs as data in empirical research and d…
• Recursive Language Models
  We study allowing large language models (LLMs) to process arbitrarily long prompts through the lens of inference-time scaling. We propose Recursive Language Models (RLMs), a general inference strategy…
• Test-time Prompt Intervention
  Test-time compute has led to remarkable success in the large language model (LLM) community, particularly for complex tasks, where longer chains of thought (CoTs) are generated to enhance reasoning ca…
• Where to show Demos in Your Prompt: A Positional Bias of In-Context Learning
  In-context learning (ICL) is a critical emerging capability of large language models (LLMs), enabling few-shot learning during inference by including a few demonstrations (demos) in the prompt. Howeve…