Can retrieval learn what actually helps answer questions?

Standard RAG trains the retriever and generator separately. The retriever optimizes for document relevance — returning chunks that look like what was asked. The generator optimizes for answer quality — producing correct, coherent responses from whatever the retriever provides. The two objectives are decoupled, which means the retriever can learn to retrieve documents that are semantically similar but not actually useful for answering.

The fundamental problem: the retriever cannot receive a gradient signal from the generator without a differentiable interface between them. Text is discrete — you cannot backpropagate through "select these k chunks from a vocabulary of millions."

CLaRa (Continuous Latent Reasoning) solves this with shared continuous document representations. Documents are encoded once into compact memory-token vectors. The reranker and generator both operate in this continuous space. During training, the next-token prediction loss from the generator propagates back through both modules via a differentiable top-k estimator. The retriever learns from the generator's success and failure.

The learned alignment: the retriever stops optimizing for surface similarity and starts optimizing for "does including this document improve the answer?" Documents that look relevant but do not contribute get deprioritized. Documents that seem tangential but bridge a reasoning gap get upweighted.

This matters because the gap between "similar to query" and "useful for generating the answer" is large in practice. Retrieval trained on human relevance labels is approximating what humans think is relevant. Retrieval trained on generation loss is learning what is actually useful for the downstream task.

Source: RAG