Can simple uncertainty estimates beat complex adaptive retrieval?

Adaptive RAG pipelines decide when to retrieve based on complex heuristics — multiple LLM calls to assess confidence, multiple retrieval rounds, specialized self-knowledge modules. These systems achieve strong performance but at substantial computational overhead: many LM calls and retriever calls per question.

Uncertainty estimation methods provide a simpler alternative: measure the model's calibrated confidence on token probabilities from a single generation pass, retrieve only when uncertainty exceeds a threshold. White-box methods use internal model signals (logits, layer outputs). Black-box methods use output-only signals (response consistency across samples).

The surprising empirical result: uncertainty estimation methods outperform complex multi-call adaptive retrieval pipelines on single-hop datasets, and perform comparably on multi-hop datasets. The performance gap in favor of complex methods is smaller than the compute cost they incur. Uncertainty estimation typically requires fewer than 1 retriever call and 2 LM calls per question — substantially cheaper than baseline adaptive retrieval methods requiring multiple rounds.

The mechanism: the LLM's own calibration is a better signal for "do I know this?" than external heuristics designed to approximate that signal. Self-knowledge — the model's ability to recognize its own uncertainty — turns out to be sufficient for trigger decisions when properly operationalized.

The limit: constant retrieval (always retrieve) performs poorly, confirming that the decision of when to retrieve matters. The comparison is between naive always-retrieve and calibrated sometimes-retrieve — uncertainty estimation wins both against naive baselines and against complex adaptive methods.