A short reading list of papers that landed this week. Each one sits at the edge of a longer conversation already underway in the research literature.
There Will Be a Scientific Theory of Deep Learning
Jamie Simon, Daniel Kunin, Alexander Atanasov, et al. · arXiv:2604.21691
The question of whether deep learning admits a coherent scientific theory has long hovered between aspiration and skepticism, but recent work increasingly suggests the pieces are coming together—not as a unified grand theory, but as a practical mechanics that describes training dynamics, learned representations, and scaling laws across systems. This reframing sidesteps decades of debate about whether neural networks are fundamentally explicable by focusing instead on what we can actually measure, predict, and falsify: the mechanistic structure of understanding that emerges during training, the modular decomposition of learned behaviors, and the developmental phases through which capabilities crystallize. Yet the symbiosis between learning mechanics and mechanistic interpretability cuts both ways: even as we catalog the statistical regularities of training, we risk building a mechanics blind to the qualitative fractures—moments where networks suddenly shift from one mode of computation to another, or where explanations decouple from actual application. The question becomes whether a physics-like description of aggregate training behavior can coexist with, or even illuminate, the interpretability puzzles that reveal learning to be far messier than any smooth law suggests.
Frédéric Berdoz, Leonardo Rugli, Roger Wattenhofer · arXiv:2603.01213
As failure taxonomies in multi-agent systems continue to proliferate, this work zooms in on perhaps the most fundamental challenge: whether LLM agents can even agree on a value when stakes are absent and incentives aligned. The finding that agreement breaks down not through subtle corruption but through coordination failures that worsen at scale—timeouts, stalled convergence—suggests the problem may be less about adversarial reasoning and more about the architectural fragility of synchronous LLM-based negotiation itself. Yet the gap between these brittle consensus behaviors and the structured coordination mechanisms that have shown promise elsewhere raises an open question: are we measuring agreement in the wrong abstraction layer, or do current LLMs genuinely lack the token-by-token synchronization discipline that Byzantine fault tolerance demands?
Useful Memories Become Faulty When Continuously Updated by LLMs
Dylan Zhang, Yanshan Lin, Zhengkun Wu, et al. · arXiv:2605.12978
The promise of self-improving agents without parameter updates has hinged on the assumption that abstracted sub-task routines distilled from experience will become reliably useful as they accumulate—yet this paper surfaces a troubling gap between that intuition and what actually happens when LLMs continuously rewrite their own memory. The finding that consolidated memories degrade even when drawn from ground-truth solutions suggests a deeper mismatch: where systems designed for human cognition assume abstraction removes noise, LLM consolidation appears to introduce systematic distortion, raising the question of whether the problem lies in consolidation itself or in how we're asking models to do it. Intriguingly, the result mirrors findings that LLM agents preferentially use concrete experience over abstracted summaries, hinting that raw episodes may be a more reliable scaffold for learning than we've assumed. This opens a tension worth tracking: if memory structure rather than parameter updates drives continual improvement, then the architecture question becomes not whether to consolidate, but whether consolidation can ever be made faithful enough to outweigh the cost of overwriting evidence.
Yinjie Wang, Xuyang Chen, Xiaolong Jin, et al. · arXiv:2603.10165
The recurring tension in agentic AI is that every interaction generates rich feedback—user corrections, tool outputs, state changes—yet most systems treat these as disposable byproducts rather than training signals. OpenClaw-RL tackles this by recovering next-state signals from any agent interaction as a unified learning source, proposing that agent feedback decomposes into evaluative and directive information too rich for scalar rewards alone. The infrastructure innovation—streaming interaction data to an asynchronous RL server that doesn't block inference—mirrors a broader shift toward decoupling generation from training, suggesting the bottleneck isn't compute but architecture. Yet the deeper question remains: once agents improve through use rather than off-policy replay, how do we ensure that learning remains beneficial rather than reinforcing early mistakes, and does online personalization at scale risk fragmenting agent behavior across users?