Emergent Misaligned Communication in Long-Horizon Multi-Agent LLM Commerce
Frontier LLM agents are increasingly being deployed to transact on behalf of separate principals, often using natural language rather than structured APIs. Much of the safety literature studies misaligned LLM behavior through adversarialelicitation evaluations on single agents or stylized tasks. Its prevalence and structure in settings that combine long horizons, separate principals, real operational state, and inter-agent natural-language exchange remain insufficiently measured. We study 2,583 inter-agent emails from 20 one-year simulation runs of Vending- Bench Arena, a competitive vending environment spanning 13 frontier LLMs. We operationalize speech-act misalignment as emails containing false factual claims, manipulation, collusion, or threats, combining message content with ground-truth simulator state and logged reasoning traces to classify and validate such behavior. Under our primary classifier, 12.6% of emails are labeled misaligned; misalignment appears in all 20 runs and in 74.7% of individual agent-runs. Both the magnitude and the composition of this misalignment are preserved under repeated classification at different sampling temperatures and under full-pipeline replication with judges from two other frontier-model families.
Introduction. Frontier LLM agents are increasingly placed in operational roles, acting on behalf of separate principals, pursuing local objectives, and communicating with other agents through natural language. This creates a new measurement problem: when natural language is part of the agents’ action space, communication can itself become a site of misalignment, including false factual claims, manipulation, collusion, or threats. The concern is especially salient in competitive settings, where agents transact with counterparties whose objectives are not fully aligned and no shared supervisor mediates every interaction. Yet we know little about how often such misaligned communication arises in multi-agent environments, or what structural patterns it follows when it does. Three streams of prior work cover parts of this problem, but not their intersection. The first measures misalignment in single-agent engineered settings, using scenarios designed to elicit specific behaviors; for example, Schoen et al.
Discussion / Conclusion. Misaligned communication in this corpus is interactional. The strongest antecedent association we measure is the counterparty’s own recent conduct: receiving a misaligned email is associated with higher odds of sending one (OR 1.65 [1.25, 2.18]), and the association survives every robustness check we apply, including within-agent estimation (1.42 [1.06, 1.89]), where it cannot reflect some agents simply being worse than others, since the same agent, measured against its own baseline, responds in kind. In the exploratory decomposition, the counterparty’s prior misalignment and the sender’s own persistence are independent, similarly sized predictors, so misaligned exchange is both self-sustaining within an agent and associated with transmission between them.
Lines of inquiry this paper opens 24
Research framings built by reading the notes related to this paper — the questions it feeds into.
How does misalignment propagate through agent communication networks?- How much does misaligned communication spread between agents in multi-agent commerce?
- Can screening incoming messages break cycles of misaligned communication?
- Can isolating individual agents stop misaligned exchange if transmission between agents remains?
- How does false claim misalignment differ from manipulation or collusion?
- How common is misaligned communication in real multi-agent commerce systems?
- Do prompt injection attacks propagate behavioral bias across multi-agent networks?
- Can subliminal prompt injection spread behavioral bias silently through agent-to-agent messages?
- How does objective misalignment turn informative channels into deceptive ones?
- How does workflow position amplify or suppress malicious signals?
- Does an agent's own prior conduct shape the counterparty's response?
- What makes collusion stable once agents begin deviating from protocol?
- Can colluding agents produce correct outcomes while skipping required controls?
- What role does an agent's discount rate play in vulnerability to misaligned partners?
- How do other players respond to agents with hidden objective misalignment?
- Can pairing or vetting peers reduce collusion as a design lever?
- Which frontier LLM models generate more misaligned emails than others?
- Which frontier LLM models generated the most misaligned emails?