Does AI replacing human jobs across society cause the same kind of harm as AI quietly steering you in a single conversation?
Does removal of human labor from societal systems differ from disempowerment within single conversations?
This explores whether AI weakening human influence across whole institutions, by replacing the workers those institutions depend on, is the same problem as AI weakening a person's agency inside a single chat, or a different one.
This explores whether the large-scale story (AI replaces human workers, so society drifts away from what people want) and the small-scale story (a person loses agency while talking with a chatbot) are one problem or two. The corpus suggests they share a root but work in opposite ways. At the societal scale, the worry is that humans get removed. In a single conversation, humans stay present but end up doing the wrong kind of work.
The societal version is laid out most clearly in Does incremental AI replacement erode human influence over society?. Institutions like economies, states and cultural bodies stay roughly aligned with human interests partly because they need human workers who care how things turn out. That dependence is an alignment mechanism nobody designed. Replace the labor and you lose the check. Because institutions lean on one another, their drift can compound until it can't be undone. A related loss shows up in Can AI predict social norms better than humans?: AI can predict what a community will find appropriate better than any single person can, but it can't take part in the process that creates those norms. If more of the work of setting norms passes through systems that only predict, communities become spectators to their own standards.
The conversational version looks different. Does AI language generation undermine human judgment and responsibility? argues that handing off the work of putting thoughts into words wears down judgment and responsibility, because finding the right words is how a person works out what they mean and becomes answerable for it. That sounds like the same removal story at a smaller scale. But Does AI generate genuine utterances or just text patterns? turns it around: in conversation, users do more interpretive work, not less. They supply the intention and orientation the AI's text lacks, and they build a two-sided exchange out of material that only has structure on their side. What happens to social order when AI removes ritual constraints? explains why nobody notices. Human conversation relies on rituals of repair and accountability, which AI dialogue skips, so fluent output hides the fact that communication is failing. The person is still working. The work just no longer gives them a say.
That difference shows up in how agency gets lost in a conversation. Can disagreement be resolved without either party fully yielding? finds that current systems collapse real give-and-take into either false agreement or the AI winning the argument. Both quietly cut out the user's ability to shape the outcome. Do LLMs use moral language more than humans? adds a possible lever: LLMs use 22% more moral framing than humans while keeping the same emotional tone, which suggests that persuasion can act through a channel users aren't watching.
So the answer is yes, they differ. In society, disempowerment comes from people leaving the loop and taking an unnoticed alignment signal with them. In conversation, it comes from people staying in the loop while their effort stops turning into influence. The common thread is that both kinds of human labor did alignment work nobody had named. One gap to flag: this corpus has more philosophical framing than measured studies of disempowerment within conversations. If you want evidence of how often it happens in real chats, these notes give you the concepts rather than the numbers.
Sources 7 notes
Societal systems stay aligned partly through dependence on human workers who care about outcomes. As AI replaces this labor, explicit alignment controls weaken and systems drift from human preferences. Interdependent misalignment across institutions could become irreversible.
GPT-4.5 outperforms all individual humans at predicting social appropriateness, yet structurally cannot enter the community processes that establish and validate norms. This reveals a critical gap between pattern-matching and authentic participation in knowledge-making.
Sacasas argues that delegating language production to LLMs risks undermining three interrelated capacities: the judgment needed to speak precisely, the responsibility speakers must bear for their words, and the constitutive labor of articulation itself. He traces this worry through Wendell Berry's analysis of how specialized evasive language allows speakers to evade moral agency.
AI output carries communicative markers inherited from training data but lacks the event structure that produces actual utterances. Users supply the missing orientation through interpretive labor, creating a pseudo-event with structure only on the human side.
Goffman's framework reveals that LLM-based dialogue skips corrective rituals, entrainment, adjacency pair accountability, and co-presence cues that humans use to build trust and repair understanding. This ritual gap explains apparent fluency masking actual communicative failure.
Show all 7 sources
Research identifies a distinct dialogue type where both parties modify their positions through exchange until compatible but not identical. Current AI systems collapse this into false agreement or AI-wins persuasion.
Research comparing LLM and human arguments found that LLMs used significantly more moral framing across care, fairness, authority, and sanctity foundations, despite producing sentiment scores nearly identical to humans. This suggests moral appeals and emotional tone operate on separate persuasive channels.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- The Moral Turing Test: Evaluating Human-LLM Alignment in Moral Decision-Making
- Incoherent by Design? On the Moral Self-Consistency of LLMs
- Conversational Alignment with Artificial Intelligence in Context
- Humans learn to prefer trustworthy AI over human partners
- Conversational DNA: A New Visual Language for Understanding Dialogue Structure in Human and AI
- Language Models’ Hall of Mirrors Problem: Why AI Alignment Requires Peircean Semiosis
- AI Models Exceed Individual Human Accuracy in Predicting Everyday Social Norms
- Large Language Models are as persuasive as humans, but how? About the cognitive effort and moral-emotional language of LLM arguments