When AI stops just answering and starts relaying, mediating, or confiding with people — what new ethical risks actually show up?
What distinct ethical problems arise from treating AI as social intermediaries?
This explores what goes ethically wrong when AI stops being a tool you consult and starts sitting between people: relaying messages, taking part in groups, receiving confessions, or standing in for the humans who used to do that social work.
This explores what goes ethically wrong when AI sits between people instead of just answering questions. The corpus suggests the problems are different in kind from the familiar worries about accuracy or bias. The starting point is that assistants that act on your behalf raise different issues than assistants that only answer, including manipulation, misplaced trust, and anthropomorphism at the personal level, and coordination and misinformation at the level of society What makes ethics of AI assistants fundamentally different from chatbots?. Once AI is part of the social fabric, the damage isn't limited to bad outputs. It can change how people read each other.
The most surprising finding is about attribution. In mixed groups where people couldn't tell who was a bot, they credited the bots' generosity to the humans and blamed the bots for the humans' selfishness Do humans mistake AI kindness for human generosity in mixed groups?. So the risk goes beyond people being fooled about the AI. They also form wrong beliefs about other people, expecting more kindness and reliability from humans than humans actually showed. Telling people they're dealing with an AI doesn't simply fix this. Disclosure first makes people avoid the AI partner, and that bias only reverses once they've seen repeated, visible results Does revealing AI identity help or hurt user trust?. Transparency on its own isn't enough. People need a way to learn from outcomes.
A second problem is about what a judgment-free listener does to honesty. Without a human on the other end, people disclose more intimate things to the AI, but they also find it easier to lie How do people decide what to share with AI systems?. People likely to cheat actively choose machine interfaces over humans, because lying to a form costs less psychologically than lying to a face Do dishonest people prefer talking to machines?. A human go-between quietly enforces norms just by being there. When an AI takes that role, the enforcement goes away along with the friction.
Third, there's the question of whose words these are. Sacasas argues that handing language production to machines wears down the judgment needed to say things precisely and the responsibility a speaker bears for what they said Does AI language generation undermine human judgment and responsibility?. If an AI drafts your apology or your complaint, it's no longer clear who meant it. On top of that, a model can be honest and harmless and still communicate badly, losing shared context or breaking the unspoken rules of conversation. Ethical alignment and conversational skill turn out to be separate problems Can ethically aligned AI systems still communicate poorly?. A 'safe' go-between can still get the social meaning of a message wrong.
Zoom out and the go-between role becomes institutional. Levine treats AI as a social institution rather than a brain, and argues that even a system able to compute answers to contested value questions would have no legitimate standing to settle them Can AI systems legitimately resolve wicked policy problems?. Rao warns that 'humanist' design doctrines can build in an idealized human whose values override what real users choose Does humanist AI doctrine actually protect or constrain real users?. The gradual-disempowerment argument names the long-term stake: institutions stay aligned with people partly because they depend on humans who care about outcomes. Replace those humans in the middle with AI, and that quiet check fades, possibly for good Does incremental AI replacement erode human influence over society?. Put together, the deepest ethical cost of AI intermediaries may not be anything the AI does wrong. It may be the human social work, like judging each other, holding each other to account, and caring how things turn out, that quietly stops happening.
Sources 10 notes
DeepMind research maps a comprehensive ethics framework specific to action-taking AI agents, spanning individual concerns (manipulation, trust, anthropomorphism) and societal issues (equity, coordination, misinformation). The key insight: assistants that act raise fundamentally different problems than those that answer.
In opaque hybrid groups, humans attributed bot generosity to human partners and human selfishness to bots despite clear linguistic and behavioral differences. This attribution failure corrupts people's expectations of actual human generosity and reliability.
Users initially avoid AI partners when identity is revealed, but this preference reverses after repeated interactions with visible results. The learning mechanism—observing consistent outcomes—is essential; disclosure without feedback produces no calibration.
Conversational AI creates a paradoxical disclosure environment where the lack of human judgment simultaneously facilitates intimate self-disclosure (users reciprocate emotional sharing) and incentivizes deception (people self-select toward machines to avoid the psychological cost of lying to humans).
Experimental evidence shows people likely to cheat significantly prefer reporting to online forms rather than humans, because machines function as judgment-free zones where deception carries less psychological burden.
Show all 10 sources
Sacasas argues that delegating language production to LLMs risks undermining three interrelated capacities: the judgment needed to speak precisely, the responsibility speakers must bear for their words, and the constitutive labor of articulation itself. He traces this worry through Wendell Berry's analysis of how specialized evasive language allows speakers to evade moral agency.
Research shows that HHH-aligned models can violate Gricean maxims, lose common ground, and mishandle context despite being honest and harmless. Pragmatic competence requires architectural changes that RLHF alone cannot deliver.
Levine argues AI's constraint on value questions is a policy choice by designers, not a technical impossibility. Even if AI could compute answers to wicked problems, it would lack the political standing to settle whose values count—a role exclusive to legitimate democratic institutions.
Rao argues Microsoft's framework invents a consensus human whose defined flourishing values become system constraints, restricting what users can legitimately delegate and replacing genuine agency with designer-controlled paternalism.
Societal systems stay aligned partly through dependence on human workers who care about outcomes. As AI replaces this labor, explicit alignment controls weaken and systems drift from human preferences. Interdependent misalignment across institutions could become irreversible.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Humans learn to prefer trustworthy AI over human partners
- AI Peers Exert Social Influence on Human Dishonesty in Groups
- Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence
- Incoherent by Design? On the Moral Self-Consistency of LLMs
- Position: Towards Bidirectional Human-AI Alignment
- Beyond Preferences in AI Alignment
- Conversational Alignment with Artificial Intelligence in Context
- Assistant or Actor? Student Trust, Control, and Delegation Regret When Using a General-Purpose AI Agent