Can we automatically optimize both prompts and agent coordination?
This explores whether language agents can be represented as computational graphs whose structure and content adapt automatically. Why it matters: current agent systems require hand-engineered orchestration; automatic optimization could unlock more capable multi-agent systems.
The computational graph representation of language agents resolves a fragmentation problem: diverse prompting techniques (CoT, ToT, Reflexion, Self-Consistency) and multi-agent frameworks (AutoGPT, CAMEL, ChatDev) all look different in implementation but share the same underlying structure.
Three levels of the hierarchy:
- Node — a fundamental operation: LLM inference, tool use, function call, embodied action
- Graph (Agent) — multiple nodes forming a coherent functional entity; edges define execution topology
- Composite Graph (Swarm) — agents whose collective capabilities exceed individual agents; inter-agent edges establish collaboration
This is not merely descriptive. The graph representation enables automatic optimization along two axes:
- Node optimization — each node adapts its prompts based on previous input and task feedback (analogous to prompt tuning)
- Edge optimization — the connectivity between nodes changes, allowing suboptimal agent organization to be overcome and prompting techniques to be automatically recombined
Because Can reasoning topologies be formally classified as graph types?, the insight here is that the graph representation extends beyond individual reasoning traces to entire agent systems. A single agent's reasoning is a graph; a multi-agent system is a composite graph. The same formalism covers both, enabling optimization at both levels.
The Society of Mind (Minsky 1988) framing is deliberate: higher-level intelligence emerges from combining simpler, modular cognitive components. This is the same principle that makes Can extreme task decomposition enable reliable execution at million-step scale? work — decomposition into graph nodes enables both parallelism and optimization.
The practical implication: instead of hand-engineering agent orchestration patterns, define the problem as a graph and let optimization discover the topology. This is the agent-systems analog of what Do reasoning cycles in hidden states reveal aha moments? found for reasoning traces — topology matters, and it's optimizable.
Source: Agents
Related concepts in this collection
-
Can reasoning topologies be formally classified as graph types?
This explores whether Chain of Thought, Tree of Thought, and Graph of Thought represent distinct formal graph structures with different computational properties. Understanding this matters because the topology itself determines what reasoning strategies are possible.
the individual-reasoning analog; this note extends the graph formalism to agent systems
-
Do reasoning cycles in hidden states reveal aha moments?
What if the internal loops in model reasoning—visible in hidden-state topology—correspond to the reconsidering moments that happen during reasoning? This note explores whether graph cyclicity captures a mechanistic signature of insight.
topology optimization for reasoning; this note demonstrates topology optimization for agent orchestration
-
Can extreme task decomposition enable reliable execution at million-step scale?
Can breaking tasks into maximally atomic subtasks with voting-based error correction solve the fundamental reliability problem in long-horizon tasks? This challenges whether better models or better decomposition is the path to high-reliability AI systems.
MAKER as a specific composite-graph pattern
-
Can multiple LLMs coordinate without explicit collaboration rules?
When multiple language models share a concurrent key-value cache, do they spontaneously develop coordination strategies? This matters because it could reveal how reasoning models naturally collaborate and inform more efficient parallel inference.
emergent coordination as a graph property
-
Can algorithms plus limited LLM calls solve complex tasks better?
Explores whether decomposing tasks into step-specific prompts within algorithmic control flow—rather than asking the LLM to manage full state—overcomes context window and reasoning limits while improving task performance.
LLM Programs are computational graphs with predetermined topology: the algorithm defines the edges, each LLM call is a node, and information hiding enforces the graph structure; the difference is that LLM Programs fix the graph at design time while optimizable graphs discover it through optimization
-
Can modular cognitive tools boost LLM reasoning without training?
Does structuring reasoning as discrete, sandboxed tool calls elicit stronger problem-solving in language models compared to monolithic prompting approaches, and can this approach match specialized reasoning models?
cognitive tools are specific node implementations within the computational graph: each tool (understand, recall, examine, backtrack) is a sandboxed function node that can be composed into agent-level reasoning graphs
Click a node to walk · click center to open · click Open full network for a force-directed map
Original note title
language agents as optimizable computational graphs unify prompting techniques and enable automatic optimization of both prompts and agent orchestration