Can retrieval policies learn to use pretraining statistics as decision features?
This explores whether the systems that decide *when and what to retrieve* can use signals from the model's own pretraining — its confidence, how probable a fact already is to it — as inputs to that decision, rather than relying on fixed rules or external heuristics.
This reads the question as asking whether a retrieval policy — the component that chooses when to reach for external knowledge versus trust the model's own memory — can treat the model's pretraining state as a *feature* to decide on. The corpus says yes, and the cleanest evidence is that the model's own uncertainty turns out to be a better trigger than the elaborate machinery built around it. Calibrated token-probability — essentially a readout of how confident pretraining made the model — beats multi-call adaptive retrieval schemes while using a fraction of the compute, because the model's self-knowledge is more reliable than external 'should I retrieve now?' heuristics Can simple uncertainty estimates beat complex adaptive retrieval?.
The deeper version of this is learning the policy rather than thresholding it. DeepRAG frames each reasoning step as a decision — retrieve, or answer from parametric memory — and learns where that boundary sits, gaining ~22% by routing around both unnecessary retrievals and the noise they introduce When should language models retrieve external knowledge versus use internal knowledge?. The implicit 'decision feature' there is exactly the question's framing: an estimate of whether pretraining already covers this step.
What makes pretraining statistics *usable* as features is that they're surprisingly predictive. Pre-learning keyword probability strongly forecasts how a model will behave after training, with a sharp ~10^-3 threshold separating 'this sticks' from 'this doesn't' Can we predict keyword priming before learning happens?. That's the quiet enabling result: if a raw pretraining probability cleanly separates regimes, a policy can read it as a signal instead of guessing.
There's also a strong reason you'd *want* the policy to watch pretraining, not just performance. Models routinely ignore their context when prior training associations are strong enough to override it — and textual prompting alone can't fix this; the priors dominate Why do language models ignore information in their context?. Relatedly, prompting can only reorganize what's already in the training distribution, never inject what's missing Can prompt optimization teach models knowledge they lack?. So a retrieval policy that knows where pretraining is confident-but-wrong, or simply absent, is correcting for exactly the failure these notes describe.
The thing worth carrying away: the field is quietly moving the retrieval decision *inside* the model's own statistics. Rather than retrieval being a fixed pipeline stage triggered by external rules — which the corpus flags as a structural failure mode, since fixed-interval triggering wastes context Where do retrieval systems fail and why? — the more efficient designs let the model's pretraining confidence be the thing that decides. The policy and the pretrained knowledge stop being separate systems.
Sources 6 notes
Calibrated token-probability uncertainty consistently beats multi-call adaptive retrieval on single-hop tasks and matches performance on multi-hop, using a fraction of the LM and retriever calls. The model's self-knowledge proves more reliable than external heuristics for deciding when to retrieve.
DeepRAG models each reasoning step as a Markov Decision Process where the model learns when to retrieve versus rely on parametric knowledge. The 21.99% improvement comes from better-targeted retrieval and elimination of noise from unnecessary external knowledge.
Pre-learning keyword probability strongly predicts post-learning priming across architectures and model sizes, with a ~10^-3 threshold separating contexts where priming occurs from those where it doesn't. Just 3 training exposures suffice to establish the effect.
Research demonstrates that LMs generate outputs inconsistent with their context because parametric knowledge from training dominates over in-context information. Textual prompting alone cannot override strong priors; causal intervention in representations is required.
Prompting works entirely within a model's pre-existing training distribution and cannot supply domain knowledge absent from training data. This creates a hard ceiling: no prompt strategy can compensate for missing foundational knowledge, only reorganize what already exists.
RAG systems fail at three structural levels: adaptive triggering (fixed intervals waste context), semantic-task mismatch (embeddings measure association, not relevance), and mathematical limits (embedding dimension constrains representable document sets). These require fundamentally different retrieval approaches, not tuning.