What prevents monolithic LLMs from coordinating decomposition with execution?
This explores why a single LLM struggles to both break a problem into steps *and* carry those steps out — and what the corpus says happens when you stop forcing one model to do both.
This explores why a single LLM struggles to both break a problem into steps and carry those steps out reliably. The corpus points to a recurring answer: decomposition and execution are different skills that interfere with each other when crammed into the same forward pass, and a monolithic model has no clean boundary to keep them apart. The sharpest evidence is that splitting them works — separating a 'decomposer' from a 'solver' improves accuracy, and tellingly, the decomposition ability transfers across domains while the solving ability does not Does separating planning from execution improve reasoning accuracy?. That asymmetry suggests these aren't two flavors of the same competence; they're distinct capabilities the monolith keeps colliding.
Underneath the coordination failure is something more structural: knowing what to do and actually doing it run on dissociated pathways. Models can state a correct principle (~87% of the time) yet fail to apply it in action (~64%) — a 'split-brain' where comprehension and competence come apart Can language models understand without actually executing correctly?. Execution itself is shakier than it looks: asked to actually run an iterative numerical procedure, LLMs don't — they pattern-match a memorized-looking answer and emit plausible but wrong values, no matter the scale Do large language models actually perform iterative optimization?. So even when decomposition is sound, the 'execute' half quietly substitutes recall for computation.
The approaches that succeed all do the same thing: they stop asking the model to coordinate and hand that job to scaffolding. LLM Programs wrap the model in an explicit algorithm that manages control flow and feeds each call only its step-relevant context, treating reasoning as modular, debuggable sub-tasks Can algorithms control LLM reasoning better than LLMs alone?. ReWOO and Chain-of-Abstraction decouple the reasoning trace from tool outputs entirely, so planning happens before — and independently of — execution Can reasoning and tool execution be truly decoupled?. A more radical version restricts the LLM to abstraction only: read the problem, emit formal structure or solver code, and let a deterministic solver do the numeric grinding the model plateaus on Should LLMs handle abstraction only in optimization?.
Why does the monolith fail to self-correct its way out of this? Because coordination needs an external check it doesn't have. Self-improvement is formally bounded by a generation-verification gap — a model can't reliably validate its own execution against its own plan without something outside the loop to enforce correctness What stops large language models from improving themselves?. The cost of skipping that check shows up at scale: across long delegated workflows, even frontier models silently corrupt ~25% of content, with errors compounding rather than plateauing over dozens of hand-offs Do frontier LLMs silently corrupt documents in long workflows?. No internal signal flags the drift.
The twist worth knowing: the missing ingredient may not be more training but the right *arrangement*. Give existing reasoning models — QwQ, DeepSeek-R1 — a shared concurrent memory and they spontaneously formulate plans, spot redundancy, and adapt, coordinating like a multi-agent team with no fine-tuning at all Can multiple LLMs coordinate without explicit collaboration rules?. So what prevents a monolith from coordinating decomposition with execution might be less a capability gap than a packaging problem: the skills are there, but a single sequential pass gives them nowhere to stand apart.
Sources 9 notes
Modular architectures with separate decomposer and solver models outperform monolithic LLMs, with decomposition ability transferring across domains while solving ability does not. The separation prevents planning-execution interference and produces more generalizable skills.
Large language models can articulate correct principles but systematically fail to apply them due to dissociated instruction and execution pathways. The 87% accuracy in explanations versus 64% in actions reveals this is not knowledge deficit but structural disconnect.
Research shows LLMs cannot perform iterative procedures in latent space. They recognize optimization problems as template-similar and emit plausible-looking but incorrect values, a failure mode that persists across model scale and training approaches.
LLM Programs embed LLMs within explicit algorithms that manage control flow and state, presenting only step-specific context to each LLM call. This information hiding addresses capability and context window limits while treating complex reasoning as modular, debuggable sub-tasks.
ReWOO and Chain-of-Abstraction both decouple reasoning from tool responses through different mechanisms—planning-before-execution and abstract placeholders respectively—eliminating quadratic prompt growth and sequential latency while maintaining reasoning quality.
LLMs plateau at constraint satisfaction regardless of scale, but excel at natural-language-to-formal-structure translation. The productive architecture restricts LLMs to reading input and emitting solver code, leaving numeric iteration to deterministic solvers.
Self-improvement in LLMs is formally bounded by the generation-verification gap, meaning every reliable fix requires something external to validate and enforce it. Models cannot escape this constraint through metacognition alone.
Testing 19 models across 52 domains shows even advanced systems degrade documents by ~25% over extended relay tasks, with errors compounding silently without plateauing through 50 round-trips.
Existing reasoning-capable models like QwQ and DeepSeek-R1 spontaneously formulate plans, detect redundancy, and adapt strategies when given shared access to a concurrent KV cache. This coordination emerges without fine-tuning, suggesting reasoning models already possess multi-agent collaboration capabilities.