Do self-organizing agent teams outperform rigid hierarchies?
This research explores whether multi-agent LLM systems perform better when agents can self-select roles within a fixed structure, compared to centralized control or full autonomy. The question challenges assumptions about organizational design at scale.
The largest systematic experiment on multi-agent coordination to date: 25,000+ tasks, 8 LLM models (Claude Sonnet 4.6, GPT-5.4, GPT-4o, DeepSeek v3.2, and others), 4 to 256 agents, 8 coordination protocols ranging from fully centralized to fully autonomous, across 4 complexity levels.
The endogeneity paradox: Neither maximal external control nor maximal agent autonomy produces optimal results. The hybrid Sequential protocol — fixed agent ordering (exogenous structure) with autonomous role selection (endogenous specialization) — outperforms both:
- +14% over centralized Coordinator (p<0.001)
- +44% over fully autonomous Shared protocol (Cohen's d=1.86, p<0.0001)
The insight: "AI agents need three things to self-organize — and none of them is a pre-assigned role. Given a mission, a communication protocol, and a sufficiently capable model, groups of LLM-based agents spontaneously form organizational structures, invent specialized roles, and voluntarily abstain from tasks outside their competence."
Emergent phenomena at scale:
- Dynamic role invention — 8 agents spontaneously generated 5,006 unique roles, specializing far beyond any pre-designed taxonomy
- Voluntary self-abstention — agents chose not to contribute when they assessed their competence as insufficient
- Spontaneous hierarchy formation — shallow hierarchies emerged without being designed
- Sub-linear scaling — quality maintained from 4 to 256 agents (p=0.61, no significant degradation)
The capability threshold reversal: Below a certain model capability, self-organization reverses and rigid structure becomes necessary. "An orchestra of beginners plays better with a conductor than without one." The protocol unlocks the model's potential like sheet music unlocks an orchestra — but only if the orchestra can play.
The two directions are orthogonal: Vertical self-improvement (DGM-Hyperagents making individual agents stronger) and horizontal coordination (this paper, making groups effective) are complementary. Stronger agents benefit more from self-organizing protocols. The two paths are synergistic, not competing.
Since When does adding more agents actually help systems?, the endogeneity paradox adds a new dimension: the degree of agent autonomy in coordination is itself a design variable, and its optimal value depends on model capability. This is a capability-contingent topology law.
Since Why do multi-agent LLM systems converge without genuine deliberation?, the voluntary self-abstention finding is striking — self-organizing agents under the right protocol develop the opposite behavior: they withdraw when they have nothing to add, rather than agreeing for the sake of consensus.
Inquiring lines that read this note 23
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
What causes coordination failures in multi-agent language model systems?- How do multi-agent LLM systems fail at coordination and role consistency?
- Do parallel LLM workers coordinate emergently without predefined collaboration rules?
- What structural constraints does topology impose on role and LLM assignment?
- Does group size have predictable effects on LLM agent agreement rates?
- Do multi-agent LLM systems scale better than centralized hierarchies?
- Do multi-agent LLM systems fail in measurably different ways than single agents?
- Why do some agent communities polarize while others reach consensus?
- Why do homogeneous multi-agent systems fail similarly to self-revision?
- Can one routing framework cover topology and role choices in multi-agent systems?
- How do static team decomposition and dynamic agent selection compare in efficiency?
- Which research tasks are better suited for multi-agent versus single-agent approaches?
- Does horizontal coordination improve with stronger individual agents?
- Which layer of agent systems creates the largest capability gains in practice?
- Can structured protocols outperform pure emergence in autonomous multi-agent coordination?
- How does role allocation in multi-agent systems depend on model differentiation?
- How do single-agent capabilities affect the trade-off between coordinator and team architectures?
Related concepts in this collection 6
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
When does adding more agents actually help systems?
Multi-agent systems often fail in practice, but the reasons remain unclear. This research investigates whether coordination overhead, task properties, or system architecture determine when agents improve or degrade performance.
endogeneity paradox adds capability-contingent topology law
-
Why do multi-agent LLM systems converge without genuine deliberation?
Multi-agent reasoning systems are designed to improve answers through debate, but often agents simply agree with early confident claims rather than genuinely disagreeing. What drives this pattern and how common is it?
voluntary self-abstention is the opposite of silent agreement
-
Can AI systems improve themselves through trial and error?
Explores whether replacing formal proof requirements with empirical benchmark testing enables AI systems to successfully modify and improve their own code iteratively, and what mechanisms prevent compounding failures.
DGM is vertical; this is horizontal; the two are complementary
-
Does cognitive diversity alone improve multi-agent ideation quality?
This explores whether diverse perspectives in group AI systems automatically produce better ideas, or if something else—like expertise—is equally critical for collaborative ideation to outperform solo agents.
capability threshold parallels: cognitive diversity works only above a competence floor; self-organization works only above a capability threshold
-
Does agent capability matter more than coordination infrastructure?
As AI agents take on economic and social roles, what actually limits their effectiveness: the raw reasoning power of the model itself, or the systems that let them coordinate, stay accountable, and leave evidence of their actions?
grounds: empirical coordination-protocol study substantiates the claim that the connective tissue, not raw capability, governs multi-agent outcomes
-
Can decentralized teams outperform central planners in long-running science?
Explores whether autonomous agent teams that self-organize around competing hypotheses and share failures can achieve better experimental outcomes than centrally-planned approaches, especially under fixed research budgets.
exemplifies: AutoScientists is decentralized self-organization applied to long-horizon science, where decentralization beats a central planner with fixed objectives
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Drop the Hierarchy and Roles: How Self-Organizing LLM Agents Outperform Designed Structures
- A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systems
- Worse Together: How Performance Breaks Down in Multi-User Multi-Agent Teams
- Towards a Science of Scaling Agent Systems
- Single-Agent LLMs Outperform Multi-Agent Systems on Multi-Hop Reasoning Under Equal Thinking Token Budgets
- Single-agent or Multi-agent Systems? Why Not Both?
- LLMs Corrupt Your Documents When You Delegate
- Autogenesis: A Self-Evolving Agent Protocol
Original note title
self-organizing multi-agent LLM systems outperform designed hierarchies through the endogeneity paradox — hybrid protocols with fixed ordering but autonomous role selection beat both centralized and fully autonomous