Can query-time graph construction replace pre-built knowledge graphs?

GraphRAG builds a knowledge graph over the entire corpus before any query is served. The graph captures relationships between entities in documents, enabling multi-hop reasoning by traversal. Performance on complex tasks is strong. But the pre-building cost is prohibitive: token overhead for graph construction, latency for updating as the corpus evolves, and the graph being LLM-generated may include irrelevant or redundant relationships.

Worse: the pre-built graph is static. Real-world queries vary in type and complexity, requiring different logic structures for accurate reasoning. A graph built for financial reporting queries may not support the traversal patterns needed for medical diagnosis queries on the same corpus.

LogicRAG inverts this: instead of building a graph over the corpus, build a graph over the query at inference time. Decompose the query into subproblems, construct a directed acyclic graph (DAG) encoding logical dependencies between subproblems, topologically sort for execution order, resolve each subproblem via retrieval conditioned on previously resolved subproblems.

The result: retrieval plans that match the logical structure of the specific query. A multi-hop question about "which policies introduced by this figure were later reversed?" generates a DAG with specific dependency edges that no pre-built graph would have pre-encoded. Context pruning (LLM-based summarization of retrieved content) and graph pruning (merging semantically similar subproblems) reduce token overhead.

This generalizes Do hierarchical retrieval architectures outperform flat ones on complex queries? — the same separation principle, implemented dynamically at the query level rather than architecturally.

Source: RAG