Silence is Not Consensus: Disrupting Agreement Bias in Multi-Agent LLMs via Catfish Agent for Clinical Decision Making

Large language models (LLMs) have demonstrated strong potential in clinical question answering, with recent multi-agent frameworks further improving diagnostic accuracy via collaborative reasoning. However, we identify a recurring issue of Silent Agreement, where agents prematurely converge on diagnoses without sufficient critical analysis, particularly in complex or ambiguous cases. We present a new concept called Catfish Agent, a role-specialized LLM designed to inject structured dissent and counter silent agreement. Inspired by the “catfish effect” in organizational psychology, the Catfish Agent is designed to challenge emerging consensus to stimulate deeper reasoning. We formulate two mechanisms to encourage effective and context-aware interventions: (i) a complexity-aware intervention that modulates agent engagement based on case difficulty, and (ii) a tone-calibrated intervention articulated to balance critique and collaboration.

To address these issues, we formulate two core mechanisms in Catfish Agent: (i) Complexity-aware intervention, i.e., the agent adapts its engagement based on task difficulty, increasing autonomy in more complex cases to encourage deeper reasoning, and (ii) Tone-calibrated intervention, in which the strength and tone of dissent vary with the level of agent agreement, avoiding both passivity and excessive disruption. These novel mechanisms encourage the Catfish Agent to “break the silence,” while preserving productive collaboration.

Figure 1 shows an example case, where the Catfish Agent disrupts premature consensus by critically challenging the expert assumptions. This intervention prompts a revision of initial reasoning and enables the framework to synthesize a more reliable diagnosis.

We start this research work by carefully studying the prevalence and impact of Silent Agreement, a critical failure mode in multi-agent medical LLM frameworks, where agents converge on an answer, often incorrect, without sufficient deliberation or justification. This behavior undermines the intended collaborative nature of multi-agent reasoning and introduces risks in medical decision making. To assess this issue, we analyze the hard set from MedAgentBench [39], focusing on two widely-used benchmarks: MedQA [15] and PubMedQA [16]. We evaluate two prominent multi-agent frameworks, MedAgents [40] and MDAgents [18], along with our proposed method. A silent agreement failure is defined as a diagnostic error, where agents produce a final answer without meaningful discussion, critique, or verification.

As shown in Table 1, MedAgents and MDAgents exhibit high silent rates, over 61.0% on both datasets, indicating frequent non-response or unjustified consensus.