Can latent reasoning mechanisms and recursive tracking mechanisms be combined effectively?
This explores whether 'latent reasoning' (a model thinking in its own continuous hidden states rather than spelling out steps in tokens) can be fused with 'recursive tracking' (looping that same hidden state back through itself to deepen a computation) — and whether that combination actually buys you anything.
This explores whether reasoning that happens in a model's hidden state can be combined with recursion — feeding that hidden state back through the same machinery to refine it over multiple passes. The most direct answer in the corpus is yes, and the interesting move is making the recursion *stochastic* rather than deterministic. GRAM does exactly this: instead of each recursive step producing one fixed latent update, it samples from a distribution over updates, so a recursive latent reasoner can hold uncertainty and keep several candidate solutions alive at once Can stochastic latent reasoning help models explore multiple solutions?. A deterministic recursive loop collapses to a single trajectory; injecting noise into the latent transition is what lets the recursion explore instead of just iterate.
That combination also changes *how* you scale the system. Normally adding reasoning depth means more serial recursive steps — slow, because each waits on the last. But once the latent transitions are stochastic, you can run many recursive trajectories in parallel and scale in 'width' instead of only 'depth,' getting the benefits of sampling the solution space without the latency penalty of a long serial chain Can reasoning systems scale wider instead of only deeper?. So the two mechanisms aren't just compatible — recursion is what gives latent reasoning somewhere to go, and stochasticity is what keeps the recursion from wasting its passes.
The deeper reason this works at all is that the reasoning capability is largely *already there* in the base model's activations. Five independent lines of work — RL steering, critique tuning, decoding tweaks, SAE feature steering, RLVR — all elicit reasoning that pre-exists rather than installing it Do base models already contain hidden reasoning ability?. That reframes the whole question: a recursive loop over latent states isn't manufacturing reasoning, it's a mechanism for *eliciting and refining* something latent. The same logic shows up in a very different costume with cognitive tools, where modular, sandboxed sub-calls isolate reasoning operations and nearly double AIME performance with no RL training at all — recursion-as-structure rather than recursion-as-hidden-loop Can modular cognitive tools unlock reasoning without training?. And diffusion LLMs offer yet another architecture for the same idea: bidirectional attention lets reasoning and the answer be refined together in place, with answer confidence converging early while the reasoning keeps recursively sharpening Can reasoning and answers be generated separately in language models?.
The caution worth carrying into all of this is that 'reasoning steps' may not mean what they appear to mean. Models trained on deliberately corrupted, semantically irrelevant traces stay just as accurate, which suggests the trace often functions as computational scaffolding rather than genuine inference Do reasoning traces need to be semantically correct?. Chain-of-thought looks more like constrained imitation of the *form* of reasoning than logical inference What makes chain-of-thought reasoning actually work?, and when you strip away familiar semantics, performance collapses — LLMs reason by association, not symbol manipulation Do large language models reason symbolically or semantically?. That actually strengthens the case for latent recursion: if the visible token-by-token trace is partly theater, then doing the real work in hidden state and refining it recursively may be the more honest place to spend compute — you get the computation without paying for tokens whose job was mostly style and documentation Can minimal reasoning chains match full explanations?.
So the combination is not only effective but arguably the natural shape of the field: recursion gives latent reasoning iteration, stochasticity gives that iteration breadth, parallel trajectories give it scale, and the elicitation framing explains why a loop over hidden states works without new training. The open frontier the corpus hints at is reliability under pressure — reasoning quality degrades sharply with input length well before any context limit is reached Does reasoning ability actually degrade with longer inputs? — so the unanswered question isn't whether these mechanisms combine, but whether the combined system stays stable when the problem gets long.
Sources 10 notes
GRAM replaces deterministic latent updates with stochastic sampling, enabling models to represent distributions over solutions rather than single predictions. This allows handling of ambiguous problems and multiple valid strategies that deterministic designs cannot represent.
GRAM shows that stochastic latent transitions enabling parallel trajectory sampling sidestep the serial latency cost of depth-only scaling. Width matches token-level parallelism benefits: independent paths sample the solution space without variance inflation.
Five independent mechanisms—RL steering, critique fine-tuning, decoding changes, SAE feature steering, and RLVR—all elicit reasoning already present in base model activations. Post-training selects rather than creates reasoning; the bottleneck is elicitation, not capability acquisition.
Four cognitive tools implemented as sandboxed LLM calls improved GPT-4.1 on AIME2024 from 26.7% to 43.3% without any RL training. Modularity enforces operation isolation that pure prompting cannot guarantee, eliciting pre-existing reasoning capability.
ICE shows that bidirectional attention in diffusion LLMs enables in-place prompting—embedding reasoning directly in masked positions refined alongside answers. Answer confidence converges early while reasoning continues refining, allowing early-exit mechanisms to cut compute by 50% while maintaining accuracy.
Models trained on systematically irrelevant traces maintain solution accuracy and sometimes improve out-of-distribution generalization, suggesting traces function as computational scaffolding rather than meaningful reasoning steps.
CoT systems reproduce the form of reasoning through pattern matching rather than performing genuine logical inference. This explains why format effects dominate content, why structurally invalid prompts succeed, and why stronger reasoning models become less instruction-compliant.
When semantic content is decoupled from reasoning tasks, LLM performance collapses even with correct rules in context. Models rely on parametric commonsense and token associations rather than formal logical manipulation, constraining reasoning to training distribution semantics.
Chain of Draft achieves equivalent accuracy to standard chain-of-thought on arithmetic, symbolic, and commonsense tasks while using only 7.6% of tokens. The 92.4% of removed tokens served style and documentation, not computation.
FLenQA shows reasoning accuracy drops from 92% to 68% at just 3000 tokens of padding, far below context window capacity. The degradation is task-agnostic, uncorrelated with language modeling performance, and persists even with chain-of-thought prompting.