Why do planning and grounding have opposing optimization requirements in agents?
This explores why an agent's planning ability (deciding what to do next) and its grounding ability (translating that decision into a precise action on a real screen or environment) end up needing different — even conflicting — training pressures, so bundling them into one model hurts both.
This explores why planning (figuring out the next move) and grounding (executing it precisely against a real interface) pull in different directions when you try to train them in the same policy. The clearest statement comes from AutoGLM's work on GUI agents Why do planning and grounding pull against each other in agents?: planning rewards abstract, flexible, long-horizon reasoning — it wants to explore, reconsider, and stay loosely coupled to surface details. Grounding rewards the opposite — pixel- and element-level precision, tight coupling to the exact state of the screen, and low tolerance for variation. Optimize one hard and you degrade the other. The practical fix that several independent systems converged on is to stop forcing one model to do both: insert a language-centric intermediate interface between a planning layer and a grounding layer, so each can be trained on its own objective and still compose How should agents split planning from visual grounding?.
That tension turns out to be a special case of a broader pattern in agent design: the capabilities that make agents work are *structurally independent*, and trying to co-optimize them on one axis is what causes the friction. One analysis decomposes agent efficiency into three orthogonal axes — memory compression, tool learning, and planning — each with a different cost profile (tokens vs. latency vs. steps), where improving one does nothing for the others Does agent efficiency really break down into three distinct components?. Planning-vs-grounding is the same shape: two axes that look like one job but reward incompatible things.
The deeper reason this keeps recurring is that reliable agents come from *externalizing* burdens rather than asking a single model to internalize all of them at once. One line of work argues that reliability lives in a harness layer — memory, skills, protocols pushed out of the model so it doesn't re-solve the same problem every step Where does agent reliability actually come from?. An intermediate planning/grounding interface is exactly this move: the abstract plan becomes an externalized, inspectable artifact (often code or structured language) that the grounding layer consumes, instead of an entangled internal state Can code become the operational substrate for agent reasoning?. Once the boundary is explicit, you can even mix model sizes across it — a big model for planning, a small cheap one for the repetitive grounding work — which is only possible because the two were separable in the first place Can small language models handle most agent tasks?.
There's a counter-current worth knowing about, though. Separation isn't free, and some research shows planning and grounding *need* to talk to each other tightly. ReAct's whole argument is that interleaving reasoning with real environment feedback — letting the plan be corrected by what grounding actually returns — is what prevents hallucinated, drifting plans Can interleaving reasoning with real-world feedback prevent hallucination?. And test-time interaction work shows that replanning only becomes possible when an agent can take many grounded steps and backtrack, a scaling dimension distinct from just reasoning harder per step Does agent interaction time scale separately from reasoning depth?. So the real design lesson isn't "split them and forget it" — it's that planning and grounding should be *optimized* separately but *coupled* at runtime through a clean interface, so each improves on its own terms while still feeding the other.
The thing you might not have expected to learn: the planning/grounding split isn't a quirk of GUI agents — it's one instance of a general principle that the components of agency are orthogonal optimization problems, and most agent architecture progress comes from finding the right interface to keep them apart while letting them compose.
Sources 8 notes
AutoGLM's research shows planning and grounding have opposing optimization requirements that pull against each other when bundled in one policy. An intermediate interface that separates them lets each capability be developed and optimized independently while still composing into a complete agent.
Multiple independent systems (Agent S, AutoGLM, OmniParser) converged on factoring agent reasoning into a planning layer and a grounding layer, with a language-centric Agent-Computer Interface mediating between them due to their opposing optimization requirements.
Research identifies memory compression, tool learning efficiency, and planning optimization as three structurally independent components, each with distinct cost profiles (tokens, latency, and steps). Improving one axis does not automatically improve the others, requiring holistic design.
Research shows reliable LLM agents externalize three cognitive burdens—memory (state persistence), skills (procedural components), and protocols (structured interaction)—into a harness layer rather than relying on model scale alone. The harness unifies these externalities and eliminates the need for the model to solve the same problems repeatedly.
Research shows code uniquely enables agents to externalize reasoning, execute policies, model environments, and verify progress through its simultaneous executability, inspectability, and statefulness across task steps.
SLMs handle the repetitive, well-defined language tasks that constitute most agent work at 10–30× lower cost than LLMs, making heterogeneous architectures (SLMs by default, LLMs selective) the economically rational design pattern.
ReAct demonstrates that alternating verbal reasoning with external tool queries (Wikipedia API, environment interaction) prevents error propagation by injecting real-world feedback at each step. On knowledge-intensive and interactive tasks, this approach outperforms pure chain-of-thought and reinforcement learning by 10-34% absolute accuracy.
Test-time interaction—increasing environment steps—enables exploration, backtracking, and replanning that per-step reasoning cannot achieve. Curriculum-based RL on rollout length produces SOTA web agents, showing interaction scaling dominates on tasks with partial observability.