SYNTHESIS NOTE
Topics›Expertise in the Age of AI Content›this note

Which signaling equilibria does AI destruction actually harm or help?

When AI undermines costly signals that coordinate trust, it may help some people but hurt others. The research identifies this possibility but cannot yet specify which equilibria are worth preserving versus destroying.

Synthesis note · 2026-10-06 · sourced from Expertise in the Age of AI Content

The paper raises a question it cannot settle. Having argued that generative AI undermines mental proof, it concedes that the equilibria mental proof supports are not always good. Citing Spence (1973), it notes that "the existence of signaling equilibria can be socially costly and may create disadvantages for some participants." It then suggests that "in some cases—which ones remains a subject for further research—outcomes may be improved when Generative AI destroys an equilibrium previously supported by mental proof." The excerpt states the possibility and, in the same sentence, declines to say which cases qualify.

The mechanism behind the concession is sorting by cost. In the signaling account the paper summarizes, a behavior that only one type can afford is "definitive proof that the agent is of the type with lower cost." That sorting is what makes the signal informative, and it is also what can disadvantage participants, though the excerpt does not say which participants or how. The paper's discussion pulls the other way. It holds that the harm from eroded mental proof "will disproportionately impact those who are not already embedded in high-trust networks and formal institutions." Both claims cannot be read off the same equilibrium without a sorting of which equilibria are worth keeping, and the excerpt does not supply one.

The erosion claim this question qualifies is the sibling note, which holds that cheap AI simulation undermines mental proof. The paper's own caveat is what keeps that claim from being one-directional. The Why do people share more openly with machines than humans? note is one place where a social cost seems to drop out, since it reports face-saving and impression-management goals suppressed in machine talk. The excerpt never mentions those goals, so it gives no basis for scoring their loss as a gain or a loss.

What the excerpt cannot establish is which equilibria are worth keeping. It offers no criteria for telling the harmful ones from the protective ones, no cases beyond Spence's own, and no evidence that AI has destroyed any particular equilibrium. The implication is that the erosion argument should travel with its caveat attached. AI-driven loss of mental proof is a risk where honesty cannot be enforced, and whether that loss is net harm depends on which equilibrium is at stake. The framework leaves that open, and anyone claiming AI disruption of an existing signal should be asked which equilibrium they mean.

Inquiring lines that read this note 2

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

Can humans reliably detect and resist AI-generated misinformation? How do philosophical assumptions about AI consciousness affect practical harms and design?

Related concepts in this collection 3

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
13 direct connections · 128 in 2-hop network ·dense cluster Open in graph ↗

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Original note title

signaling equilibria can be socially costly, so AI may sometimes improve outcomes by destroying them — which cases remains open