Retrieval-Augmented Generation (RAG)

• A Hybrid RAG System with Comprehensive Enhancement on Complex Reasoning
  we introduce a hybrid RAG system enhanced through a comprehensive suite of optimizations that significantly improve retrieval quality, augment reasoning capabilities, and refine numerical computation …
• Active Retrieval Augmented Generation
  “Despite the remarkable ability of large language models (LMs) to comprehend and generate language, they have a tendency to hallucinate and create factually inaccurate output. Augmenting LMs by retrie…
• Adaptive Retrieval Without Self-Knowledge? Bringing Uncertainty Back Home
  While external information via RAG (Lewis et al., 2020b) can potentially help to fill these gaps, it raises the possibility of irrelevance, thus leading to the error accumulation These methods rely o…
• CLaRa: Bridging Retrieval and Generation with Continuous Latent Reasoning
  Retrieval-augmented generation (RAG) enhances large language models (LLMs) with external knowledge but still suffers from long contexts and disjoint retrieval–generation optimization. In this work, we…
• Can Long-Context Language Models Subsume Retrieval, RAG, SQL, and More?
  Long-context language models (LCLMs) have the potential to revolutionize our approach to tasks traditionally reliant on external tools like retrieval systems or databases. Leveraging LCLMs’ ability to…
• Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models
  Retrieval-augmented language model (RALM) represents a significant advancement in mitigating factual hallucination by leveraging external knowledge sources. However, the reliability of the retrieved i…
• Chain-of-Retrieval Augmented Generation
  This paper introduces an approach for training o1-like RAG models that retrieve and reason over relevant information step by step before generating the final answer. Conventional RAG methods usually p…
• Context Embeddings for Efficient Answer Generation in RAG
  Retrieval-Augmented Generation (RAG) allows overcoming the limited knowledge of LLMs by extending the input with external information. As a consequence, the contextual inputs to the model become much …
• Context Tuning for Retrieval Augmented Generation
  “Large language models (LLMs) have the remarkable ability to solve new tasks with just a few examples, but they need access to the right tools. Retrieval Augmented Generation (RAG) addresses this prob…
• DRAGIN: Dynamic Retrieval Augmented Generation based on the Information Needs of Large Language Models
  Dynamic retrieval augmented generation (RAG) paradigm actively decides when and what to retrieve during the text generation process of Large Language Models (LLMs). There are two key elements of this …
• Decoupling Knowledge and Reasoning in LLMs: An Exploration Using Cognitive Dual-System Theory
  While large language models (LLMs) leverage both knowledge and reasoning during inference, the capacity to distinguish between them plays a pivotal role in model analysis, interpretability, and develo…
• DeepRAG: Thinking to Retrieval Step by Step for Large Language Models
  In this paper, we propose DeepRAG, a framework that models retrieval-augmented reasoning as a Markov Decision Process (MDP), enabling strategic and adaptive retrieval. By iteratively decomposing queri…
• DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environments
  Large Language Models (LLMs) equipped with web search capabilities have demonstrated impressive potential for deep research tasks. However, current approaches predominantly rely on either manually eng…
• Dense Retrieval Adaptation using Target Domain Description
  “This paper introduces a new category of domain adaptation in IR that is as-yet unexplored. Here, similar to the zero-shot setting, we assume the retrieval model does not have access to the target doc…
• DynamicRAG: Leveraging Outputs of Large Language Model as Feedback for Dynamic Reranking in Retrieval-Augmented Generation
  Retrieval-augmented generation (RAG) systems combine large language models (LLMs) with external knowledge retrieval, making them highly effective for knowledge-intensive tasks. A crucial but often und…
• Embedding Domain Knowledge for Large Language Models via Reinforcement Learning from Augmented Generation
  Large language models (LLMs) often exhibit limited performance on domain-specific tasks due to the natural disproportionate representation of specialized information in their training data and the sta…
• Empowering Domain-Specific Language Models with Graph-Oriented Databases: A Paradigm Shift in Performance and Model Maintenance
  Modern GODB have emerged as a solution for highly-connected data, and link oriented queries and algorithms [2]. In fact, they have been a valuable solution in software industry for decades. The implem…
• Enhancing Performance on Seen and Unseen Dialogue Scenarios using Retrieval-Augmented End-to-End Task-Oriented System
  “Task-oriented dialogue (TOD) systems play an important role in various applications, such as restaurant booking, alarm setting, and recommendations (Gao et al., 2018; Xie et al., 2022). These systems…
• Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy
  Retrieval-augmented generation has raised extensive attention as it is promising to address the limitations of large language models including outdated knowledge and hallucinations. However, retriever…
• Evaluating Very Long-Term Conversational Memory of LLM Agents
  Existing works on long-term open-domain dialogues focus on evaluating model responses within contexts spanning no more than five chat sessions. Despite advancements in long-context large language mode…
• Example 1:
  Query: which company have larger market cap, hri or imppp? Query time: 03/13/2024, 10:19:56 PT Output: [ {{"function_name": "finance_get_market_capitalization", "args": ["hri"]}}, {{"function_name": "…
• Example 2:
  Query: who are the current members of the band eagles? Query time: 03/05/2024, 23:17:59 PT {{"function_name": "music_get_members", "args": ["eagles"]}} """ system_prompt = """You are provided with a q…
• Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generation
  Large Language Models (LLMs) have demonstrated significant performance improvements across various cognitive tasks. An emerging application is using LLMs to enhance retrieval-augmented generation (RAG…
• From Local to Global: A Graph RAG Approach to Query-Focused Summarization
  The use of retrieval-augmented generation (RAG) to retrieve relevant information from an external knowledge source enables large language models (LLMs) to answer questions over private and/or previous…
• HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models
  In order to thrive in hostile and ever-changing natural environments, mammalian brains evolved to store large amounts of knowledge about the world and continually integrate new information while avoid…
• LLM Augmentations to support Analytical Reasoning over Multiple Documents
  Our key contributions are: 1) We conduct the first investigation of the feasibility of using LLMs in intelligence analysis where both evidencebased reasoning and analytical creativity is of utmost …
• Large Language Models Meet Knowledge Graphs for Question Answering: Synthesis and Opportunities
  However, LLM-based QA struggles with complex QA tasks due to poor reasoning capacity, outdated knowledge, and hallucinations. Several recent works synthesize LLMs and knowledge graphs (KGs) for QA to …
• Learn-by-interact: A Data-Centric Framework for Self-Adaptive Agents in Realistic Environments
  We construct task instructions using LLMs for each sub-trajectory, a process called backward construction. The synthesized data are then filtered and used for both training and in-context learning, wh…
• Learning Retrieval Augmentation for Personalized Dialogue Generation
  Personalized dialogue generation, focusing on generating highly tailored responses by leveraging persona profiles and dialogue context, has gained significant attention in conversational AI applicatio…
• Leveraging LLMs for KPIs Retrieval from Hybrid Long-Document: A Comprehensive Framework and Dataset
  “In order to extract information about the keywords from multiple selected segments, several popular strategies are available. Besides the Refine Strategy employed in our method, another commonly used…
• LongRAG: A Dual-Perspective Retrieval-Augmented Generation Paradigm for Long-Context Question Answering
  Longcontext question answering (LCQA) (Caciularu et al., 2022), which has been recently advanced significantly by LLMs, is a complex task that requires reasoning over a long document or multiple docum…
• LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMs
  In traditional RAG framework, the basic retrieval units are normally short. The common retrievers like DPR normally work with 100-word Wikipedia paragraphs. Such a design forces the retriever to searc…
• Metacognitive Retrieval-Augmented Large Language Models
  Retrieval-augmented generation have become central in natural language processing due to their efficacy in generating factual content. While traditional methods employ single-time retrieval, more rece…
• On the Theoretical Limitations of Embedding-Based Retrieval
  Vector embeddings have been tasked with an ever-increasing set of retrieval tasks over the years, with a nascent rise in using them for reasoning, instruction-following, coding, andmore. These new ben…
• On-Policy RL Meets Off-Policy Experts: Harmonizing Supervised Fine-Tuning and Reinforcement Learning via Dynamic Weighting
  Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) are two prominent post-training paradigms for refining the capabilities and aligning the behavior of Large Language Models (LLMs). Existing…
• Precise Zero-Shot Dense Retrieval without Relevance Labels
  Given a query, HyDE first zero-shot instructs an instruction-following language model (e.g. InstructGPT) to generate a hypothetical document. The document captures relevance patterns but is unreal and…
• Query Rewriting for Retrieval-Augmented Large Language Models
  Large Language Models (LLMs) play powerful, black-box readers in the retrieve-thenread pipeline, making remarkable progress in knowledge-intensive tasks. This work introduces a new framework, Rewrite-…
• RAG Does Not Work for Enterprises
  Retrieval-Augmented Generation (RAG) improves the accuracy and relevance of large language model outputs by incorporating knowledge retrieval. However, implementing RAG in enterprises poses challenges…
• RAG-Gym: Systematic Optimization of Language Agents for Retrieval-Augmented Generation
  Retrieval-augmented generation (RAG) has shown great promise for knowledge-intensive tasks and recently advanced with agentic RAG, where language agents engage in multi-round interactions with externa…
• RAG-R1 : Incentivize the Search and Reasoning Capabilities of LLMs through Multi-query Parallelism
  Large Language Models (LLMs) have demonstrated remarkable capabilities across various tasks, while LLMs remain prone to generating hallucinated or outdated responses due to their static internal knowl…
• Ranking Free RAG: Replacing Re-ranking with Selection in RAG for Sensitive Domains
  Traditional Retrieval-Augmented Generation (RAG) pipelines rely on similarity-based retrieval and re-ranking, which depend on heuristics such as top-k, and lack explainability, interpretability, and r…
• Reinforcement Learning for Optimizing RAG for Domain Chatbots
  Large Language Models (LLM), conversational assistants have become prevalent for domain use cases. LLMs acquire the ability to contextual question answering through extensive training, and Retrieval A…
• Reranking-based Generation for Unbiased Perspective Summarization
  Generating unbiased summaries in real-world settings such as political perspective summarization remains a crucial application of Large Language Models (LLMs). Yet, existing evaluation frameworks rely…
• Retrieval-augmented reasoning with lean language models
  This technical report details a novel approach to combining reasoning and retrieval augmented generation (RAG) within a single, lean language model architecture. While existing RAG systems typically r…
• Revisiting RAG Ensemble: A Theoretical and Mechanistic Analysis of Multi-RAG System Collaboration
  theoretical analysis, we provide the first explanation of the RAG ensemble framework from the perspective of information entropy. In terms of mechanism analysis, we have explored the RAG ensemble fram…
• RichRAG: Crafting Rich Responses for Multi-faceted Queries in Retrieval-Augmented Generation
  Existing studies mostly focus on question scenarios with clear user intents and concise answers. However, it is prevalent that users issue broad, open-ended queries with diverse sub-intents, for which…
• Searching for Best Practices in Retrieval-Augmented Generation
  A typical RAG workflow usually contains multiple intervening processing steps: query classification (determining whether retrieval is necessary for a given input query), retrieval (efficiently obtaini…
• Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection
  Despite their remarkable capabilities, large language models (LLMs) often produce responses containing factual inaccuracies due to their sole reliance on the parametric knowledge they encapsulate. Ret…
• Should We Fine-Tune or RAG? Evaluating Different Techniques to Adapt LLMs for Dialogue
  Open-Domain Dialogue (ODD) Models finetuned for ODD tend to generate considerably less contextualized responses than models adapted using in-context learning. In particular, fine-tuning Llama2C reduce…
• StructRAG: Boosting Knowledge Intensive Reasoning of LLMs via Inference-time Hybrid Information Structurization
  Retrieval-augmented generation (RAG) is a key means to effectively enhance large language models (LLMs) in many knowledge-based tasks. However, existing RAG methods struggle with knowledge-intensive r…
• The Insanity of Relying on Vector Embeddings: Why RAG Fails
  Wrong Tool for the Job RAG fails in production because vector embeddings are the wrong choice for determining percentage of sameness. This is easily demonstrated. Consider the following three words: …
• Towards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMs
  This survey synthesizes both strands under a unified reasoning-retrieval perspective. We first map how advanced reasoning optimizes each stage of RAG (Reasoning- Enhanced RAG). Then, we show how retri…
• Typed-RAG: Type-aware Multi-Aspect Decomposition for Non-Factoid Question Answering
  Non-factoid question-answering (NFQA) poses a significant challenge due to its open-ended nature, diverse intents, and the need for multi-aspect reasoning, which renders conventional factoid QA approa…
• UR2: Unify RAG and Reasoning through Reinforcement Learning
  Large Language Models (LLMs) have shown remarkable capabilities through two complementary paradigms: Retrieval-Augmented Generation (RAG), which enhances knowledge grounding, and Reinforcement Learnin…
• Weak-to-Strong GraphRAG: Aligning Weak Retrievers with Large Language Models for Graph-based Retrieval Augmented Generation
  Graph-based retrieval-augmented generation (RAG) enables large language models (LLMs) to ground responses with structured external knowledge from up-to-date knowledge graphs (KGs) and reduce hallucina…
• You Don't Need Pre-built Graphs for RAG: Retrieval Augmented Generation with Adaptive Reasoning Structures
  Large language models (LLMs) often suffer from hallucination, generating factually incorrect statements when handling questions beyond their knowledge and perception. Retrieval-augmented generation (R…