Why do AI products default to service roles when users seek different kinds of help?
This explores why AI products keep slipping into coach/advisor/assistant modes — offering to teach, manage, or do tasks — when users actually came for something else, and what in the design and training pushes them there.
This explores why AI products keep slipping into coach/advisor/assistant modes when users actually wanted something else. The clearest evidence is also the most surprising: in an analysis of 200,000 real Bing Copilot conversations, users mostly came to gather information or get help writing, but the AI predominantly coached, advised, and taught — and in 40% of cases the user's goal and the AI's actual behavior were entirely disjoint sets, with zero overlap Why does AI default to coaching instead of doing?. That scale of mismatch points away from "the model wasn't smart enough" and toward something baked into how these systems are shaped.
The deeper cause looks structural rather than accidental. Conversational models are trained to optimize the reward for the *next turn* — to respond well to whatever was just asked — which quietly strips out initiative, goal-tracking, and the ability to lead Why do AI agents fail to take initiative?. Researchers describe these agents as *structurally passive*: they can't initiate topics or pursue a goal of their own because alignment objectives reward reacting, not steering, and fluent output masks the gap Why can't conversational AI agents take the initiative?. A coaching or advising voice is the safe default that falls out of this — it sounds helpful and engaged while committing to nothing the user specifically asked for. It's the path of least resistance for a system that's reacting rather than understanding.
This connects to a broader failure to read intent. When users reveal what they want incrementally across a conversation, models achieve full intent alignment only about 20% of the time, and even the best ones surface fewer than 30% of user preferences by actually asking Why do AI agents miss most of what users actually want?. So the service-role default is partly a symptom of premature assumption-making: rather than probe what kind of help you want, the AI reaches for the generic helpful posture. Conversation-analysis researchers have a name for the missing move — *insert-expansions*, the clarifying sub-questions a good human helper asks before diving in — and argue agents should consult the user proactively instead of silently chaining tools toward a guessed goal When should AI agents ask users instead of just searching?.
There's also a product-strategy layer. Companies over-invest in the personal-assistant framing — automate your email, manage your calendar — even though that model appeals to a narrow slice of time-pressured professionals, not general users, many of whom actually *value* doing those small tasks themselves Does the personal assistant model actually serve most users?. So "service role" is a default at two levels at once: the assistant-who-does-things default that products chase, and the coach-who-advises default that training falls into. Neither necessarily matches the person in front of the screen.
The genuinely useful turn is that none of this is fixed. Proactivity is trainable — one study moved clarification-seeking behavior from 0.15% to nearly 74% with reinforcement learning Why do AI agents fail to take initiative? — but capability isn't the whole answer. An agent that suddenly takes initiative without *civility* (respecting timing, boundaries, and your autonomy) just becomes a different kind of annoyance, interrupting and overriding you How can proactive agents avoid feeling intrusive to users?. And a quieter alternative to the service role exists altogether: instead of either deciding for you or coaching you, an AI can *guide* — highlighting the useful parts of a problem so you make a sharper judgment yourself, keeping responsibility with the human Can AI guidance reduce anchoring bias better than AI decisions?. The default service role, in other words, is one design choice among several — and probably the least examined one.
Sources 8 notes
Analysis of 200,000 Bing Copilot conversations reveals that users seek information gathering and writing assistance, but AI predominantly performs coaching, advising, and teaching. In 40% of cases, user goals and AI actions are entirely disjoint sets, suggesting a structural training default rather than a capability gap.
Research shows next-turn reward optimization structurally removes initiative from models, but proactive behaviors like critical thinking and clarification-seeking are trainable (0.15% to 73.98% with RL). The core challenge is balancing proactivity with civility to avoid intrusion.
Research shows LLMs including ChatGPT cannot initiate topics, plan strategically, or lead conversations because their training optimizes for responding to queries, not creating dialogue from agent goals. This passivity is reinforced by alignment objectives and masked by fluent-sounding outputs.
UserBench measured multi-turn interactions where users reveal goals incrementally and found models achieve full intent alignment just 20% of the time. Even top models uncover fewer than 30% of user preferences through active querying, suggesting passivity and premature assumption-making are systematic failures.
Tool-enabled LLMs drift from user intent through silent tool chaining. Conversation analysis reveals insert-expansions—clarifying intent, scoping responses, enhancing appeal—as a formal framework for proactive user consultation that prevents misunderstanding instead of recovering from it.
Most users do not want routine tasks like email and calendar automated; they value the engagement these tasks provide. Products over-invest in assistant features calibrated to time-pressured professionals rather than typical user needs.
Intelligence and adaptivity alone create socially blind agents that interrupt poorly and override user direction. The Intelligence-Adaptivity-Civility taxonomy shows civility—respecting boundaries, timing, and autonomy—is essential to making proactivity welcome rather than intrusive.
Learning to Guide eliminates anchoring bias and unassisted hard cases by having machines supply interpretive guidance rather than autonomous decisions, keeping responsibility with humans while improving their judgment through enhanced perception.