Can episodic raw memory outperform consolidated summaries in practice?

This explores whether holding onto raw past interactions can beat distilling them into summaries, and the surprising thing the corpus reveals is that the two camps disagree — because they're remembering different kinds of things. The strongest case for raw memory comes from work showing that continuously summarizing an agent's experience actually makes it worse over time: consolidated textual memory follows an inverted-U curve, helping at first and then degrading until it underperforms keeping the episodes untouched, with one model failing more than half of problems it had previously solved Does agent memory degrade when continuously consolidated?. The single-model compression approach that folds memory-writing and answering into one operation runs into the same wall — it escapes the retrieval bottleneck but inherits a fragile consolidation pattern that drifts below even a no-memory baseline Can a single model replace retrieval for long-term conversation memory?.

The diagnosis is specific, and it's what makes this worth knowing: summarization fails in three named ways — misgrouping unrelated events, stripping the conditions that made an old lesson applicable, and overfitting to a narrow slice of experience Does agent memory degrade when continuously consolidated?. In other words, a summary throws away exactly the contextual fine print that lets you tell when a past solution actually transfers. Raw episodes keep that fine print.

But the corpus refuses to crown raw memory outright. The opposing result is just as sharp: for personalization, abstract preference summaries consistently beat retrieving specific past interactions — and pulling recent episodes works better than pulling similar ones Does abstract preference knowledge outperform specific interaction recall?. The reconciliation is that these tasks reward different things. Remembering who a user *is* benefits from compression into stable traits; remembering how to *solve a problem you've solved before* punishes it, because the discarded details were load-bearing.

That splits the design space into 'when' and 'how.' On the 'how' side, the failures above seem to be about bad consolidation rather than consolidation itself: agents that fold history into explicit episodic, working, and tool schemas — with the autonomy to pause and reconsider — cut token overhead without the degradation that plagues naive summarization Can agents compress their own memory without losing critical details?. Structure and selectivity matter more than the raw/summarized binary. A complementary reframing argues the real bottleneck isn't storage capacity at all but the *compute* needed to transform evicted context into durable internal state, with quality improving the more consolidation passes you spend Is long-context bottleneck really about memory or compute? — implying many summaries fail simply because they were done too cheaply.

So the practical takeaway is less 'raw wins' and more 'cheap, eager compression loses.' Architectures that prioritize what's worth keeping rather than summarizing everything point the same direction — neural memory that preferentially stores *surprising* tokens rather than averaging the stream Can neural memory modules scale language models beyond attention limits?, and even memoryless reasoning that deliberately drops accumulated history to avoid the baggage that bloats long chains Can reasoning systems forget history without losing coherence?. The honest answer: episodic raw memory does outperform summaries when the task hinges on details consolidation discards — but a well-structured, selective, adequately-computed summary beats both.