Do latent communication approaches truly escape token economics constraints?
This explores whether sharing model states directly — agents passing 'thoughts' as latent vectors instead of words, or reusing cached context — actually frees systems from paying per-token, or just relocates the cost.
This explores whether latent communication — agents trading internal representations instead of words — genuinely sidesteps the per-token economics of LLMs, or just hides the bill somewhere else. The corpus suggests the honest answer is: it relocates the constraint more than it escapes it. The cleanest version of the latent-communication dream is direct thought sharing, where agents extract and exchange latent thoughts recovered from hidden states rather than serializing everything back into language Can agents share thoughts directly without using language?. That genuinely skips the lossy token bottleneck for inter-agent coordination, and even lets you detect alignment conflicts at the representational level before they surface as text. So there's a real win — but notice what it's a win against: the *communication* channel, not generation itself. The thoughts still have to be produced by a forward pass, which is still priced in tokens.
The more economically honest reframe in the collection isn't 'escape tokens' but 'change the denominator.' A 115-day study of persistent agents found 82.9% of tokens were cache reads, which pushes the meaningful unit of cost from the individual token toward the completed artifact Do persistent agents really cost less per token?. That's a different escape route than latent vectors — you still emit tokens, but you stop paying full freight for them by reusing context. Both approaches attack the same enemy (paying linearly per token for redundant work), which is the lateral point: latent communication and aggressive caching are two answers to one problem, and caching may be the more immediately bankable one.
There's also a deeper reason not all tokens are economically equal. Only about 20% of tokens are high-entropy 'forking points' that actually carry the reasoning signal — training on just those matches full-gradient performance Do high-entropy tokens drive reasoning model improvements?. That hints at why latent approaches feel promising: if most tokens are low-information filler that exist only because language demands them, then a representation-level channel could in principle transmit the load-bearing 20% and drop the rest. But it also cuts the other way — generation is a smooth probabilistic flow toward the training distribution, not an exploration of competing claims Does LLM generation explore competing claims while producing text? — so the 'thoughts' you're sharing may be smoother and more redundant than their latent packaging suggests.
The zoomed-out framing worth handing back: knowledge here is becoming flow rather than stock, generated on demand rather than stored Is AI returning knowledge to flow-based economies?. Latent communication is what a flow economy looks like at the machine layer — value moves as live representation, not fixed text. But flows still cost compute to generate, and the collection's ceiling findings are a useful sobriety check: capability plateaus around 55–60% on genuine constraint satisfaction regardless of scale or method Do larger language models solve constrained optimization better?. So latent channels can change *what you pay for* — artifacts and forking decisions instead of every token — but nothing in the corpus suggests they make the underlying computation free. The constraint isn't really 'tokens'; it's the forward passes tokens are a proxy for, and latent communication doesn't make those disappear.
Sources 6 notes
Research formalizes inter-agent thought sharing via sparse autoencoders that recover individual, shared, and private latent thoughts from hidden states. This approach detects alignment conflicts at the representational level before they manifest in language.
A 115-day case study found 82.9% of tokens were cache reads. When context persists and reuses, the meaningful cost denominator becomes completed artifacts, not individual tokens.
Only ~20% of tokens exhibit high entropy as pivotal reasoning decision points; RLVR primarily adjusts these forking tokens. Training exclusively on them matches or exceeds full-gradient performance, revealing that the minority carries the learning signal.
Token prediction trains models to continue toward the training distribution, not to explore logically related counterpositions. This smoothness in process produces smooth claims that multiply without generating new perspectives.
Print culture fixed knowledge as accumulated stock; AI returns knowledge to generative flow. However, unlike oral and gift economies, AI flows lack the embodied transmission—the speaker, the giver—that historically anchored knowledge circulation.
Across constrained-optimization tasks, LLMs converge to ~55–60% constraint satisfaction independent of architecture, parameter count, or training regime. Reasoning models do not systematically outperform standard models, suggesting a fundamental ceiling rather than a scaling gap.