Can capability boundary collapse be addressed by operating at representational rather than token level?

This reads the question as: when a model seems to hit a wall, is the wall in the model's actual reasoning, or in the medium it's forced to reason through — strings of tokens? The corpus leans hard toward the second answer, and that reframing is the interesting part. Several notes converge on the idea that the boundary you observe is an artifact of measuring and operating at the token surface, and that it softens or moves when you go down a level.

The sharpest version comes from work showing the exploration–exploitation trade-off in RLVR is a measurement artifact: when you inspect hidden states with effective-rank metrics instead of token outputs, exploration and exploitation barely correlate, and you can push both at once for real accuracy gains Is the exploration-exploitation trade-off actually fundamental?. A 'fundamental' boundary turned out to be an illusion of the level you looked at. Parallel to this, what looks like a reasoning cliff is often an execution cliff — models that know an algorithm still fail to grind it out in text-only generation, but solve the same problems once given tools to execute Are reasoning model collapses really failures of reasoning?. The collapse is in the channel, not the competence.

If token surface is the bottleneck, the natural move is to compute somewhere richer. Large Concept Models reason over whole-sentence embeddings in a language-agnostic space before decoding, planning at the paragraph level rather than emitting tokens one at a time Can reasoning happen at the sentence level instead of tokens?. A whole family of latent-reasoning architectures — depth-recurrent models, Heima, Coconut — scale test-time compute by iterating hidden states with no verbalized steps at all, suggesting that writing your thoughts out is a training habit, not a requirement for thinking Can models reason without generating visible thinking tokens?. This is the most direct 'yes' to the question: move the work off the token track and the ceiling lifts.

But the corpus also complicates the clean dichotomy. Tokens aren't a uniform surface to escape — they're unevenly load-bearing. Only about 20% of tokens are high-entropy 'forking points' that actually carry the learning signal, and training on just those matches full updates Do high-entropy tokens drive reasoning model improvements?. Models internally rank tokens by function, preserving symbolic-computation tokens while discarding grammar and filler Which tokens in reasoning chains actually matter most?. So part of 'operating at representation level' may not mean abandoning tokens but identifying which ones are doing representational work. And there's a cost to going fully continuous that the question doesn't anticipate — fine-tuning into continuous spaces is itself implicated elsewhere in catastrophic forgetting, so the level switch isn't free.

What you didn't ask but might want: there's a third path between 'tokens' and 'pure latent space' — structured abstraction. RLAD trains models to generate reasoning abstractions that enforce breadth-first exploration where deep token chains underthink Can abstractions guide exploration better than depth alone?, and cognitive tools elicit latent capability that was already present by isolating reasoning operations into modular calls, no retraining needed Can modular cognitive tools unlock reasoning without training?. Both suggest the capability often already exists inside the model — the boundary is about how you route to it, not whether it's there. Read together, the corpus's answer is: yes, much of what looks like a hard capability boundary is a token-level framing problem, but the fix is less 'abandon tokens for embeddings' and more 'stop assuming the token stream is where the reasoning lives.'