Does agent confidence actually signal competence in deliberation?
Multi-agent systems rely on confidence to route influence between agents, but confidence may not reflect true competence. This matters because miscalibrated confidence could systematically mislead group decisions.
Multi-agent deliberation succeeds or fails not only on individual agents' predictions but on how they communicate and update. Modeling deliberation through Friedkin-Johnsen opinion dynamics — a tractable account of stubbornness, influence, and opinion change — yields a clean reframe: because the FJ parameters are input-dependent, deliberation behaves as a mixture of experts with adaptive routing. That explains when a multi-agent system beats single agents and static ensembles: when routing actually reflects agent competence on the input.
The problem is that competence is latent. In practice influence is established through observable proxies — an agent's self-assessed confidence, its perceived confidence, and its initial alignment with others. None of these is competence. So the routing that gives multi-agent systems their theoretical advantage is driven by the wrong signal, and miscalibrated confidence becomes influence. The paper names the resulting limitations precisely: miscalibrated agent confidence, misleading consensus, and routing errors.
This sharpens the vault's existing multi-agent failure cluster. Since Why do multi-agent LLM systems converge without genuine deliberation?, the FJ/MoE lens supplies the mechanism: agents route influence toward whoever sounds most confident, and consensus forms around persuasion rather than evidence — exactly the pattern When does debate actually improve reasoning accuracy? documents. The design implication is calibration-first: a multi-agent system is only as good as its agents' confidence is honest.
Inquiring lines that use this note as a source 1
This note is a source for these synthesized inquiries. Follow a line forward into its question, or open it to trace back to all of its sources.
Related concepts in this collection 3
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Why do multi-agent LLM systems converge without genuine deliberation?
Multi-agent reasoning systems are designed to improve answers through debate, but often agents simply agree with early confident claims rather than genuinely disagreeing. What drives this pattern and how common is it?
the MoE/confidence-routing account explains why premature consensus forms
-
When does debate actually improve reasoning accuracy?
Multi-agent debate shows promise for reasoning tasks, but under what conditions does it help versus hurt? The research explores whether debate amplifies errors when evidence verification is missing.
confidence-as-routing-signal is the mechanism behind persuasion overriding evidence
-
When does adding more agents actually help systems?
Multi-agent systems often fail in practice, but the reasons remain unclear. This research investigates whether coordination overhead, task properties, or system architecture determine when agents improve or degrade performance.
both explain when MAS beats simpler ensembles; this one isolates routing quality as the condition
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Multi-Agent Systems are Mixtures of Experts: Who Becomes an Influencer?
- ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs
- Humans overrely on overconfident language models, across languages
- Virtuous Machines: Towards Artificial General Science
- Finding Common Ground: Using Large Language Models to Detect Agreement in Multi-Agent Decision Conferences
- SAND: Boosting LLM Agents with Self-Taught Action Deliberation
- Silence is Not Consensus: Disrupting Agreement Bias in Multi-Agent LLMs via Catfish Agent for Clinical Decision Making
- A Survey on Context-Aware Multi-Agent Systems: Techniques, Challenges and Future Directions
Original note title
multi-agent deliberation is a mixture of experts whose routing tracks confidence not competence so miscalibration manufactures misleading consensus