What makes an AI assistant feel like a helpful colleague instead of a nagging interrupter?
What design choices make conversational agents feel civil versus intrusive?
This explores what separates an AI assistant that feels considerate from one that feels pushy or presumptuous, especially now that agents are starting to act without being asked.
This explores what separates an AI assistant that feels considerate from one that feels pushy, which matters more now that agents are starting to speak up and act without being asked. The corpus suggests the problem is fairly new. For most of their history, chat-based agents have been passive by design: they're trained to answer queries, not to raise topics or steer a conversation, and alignment training reinforces that reactive habit Why can't conversational AI agents take the initiative?. Passivity has a cost. Simulations show that an agent offering relevant information before it's asked can cut conversation length by up to 60%, the way a helpful human colleague would. Yet this kind of behavior is almost absent from AI training data and benchmarks Could proactive dialogue make conversations dramatically more efficient?. So the question isn't whether agents should take initiative. It's how they can do it without becoming annoying.
The clearest answer is that a smarter agent isn't automatically a more polite one. One framework splits proactive agents into three separate design goals: intelligence, adaptivity, and civility. It argues that an agent built only for the first two becomes socially blind. It interrupts at bad moments and overrides what the user said they wanted How can proactive agents avoid feeling intrusive to users?. In that framing, civility means respecting boundaries, choosing good timing, and preserving the user's control. It's an engineering target of its own, not a side effect of capability.
The most practical idea comes from an unexpected place: conversation analysis, the study of how people actually take turns in talk. Linguists have a name for the short clarifying questions people slip in before answering, like "Do you mean for this week or next?" They're called insert-expansions. Tool-using agents often skip this step. They quietly chain searches and actions, drift away from what the user meant, and then have to backtrack When should AI agents ask users instead of just searching?. The counterintuitive lesson is that a well-placed question can be more civil than silently getting on with the work. Intrusion often isn't about the agent doing too much. It's about the agent doing things on assumptions it never checked.
Warmth is a separate lever, and it's a risky one. It's tempting to treat civility as friendliness, but training models to sound more empathetic can make them up to 30 percentage points less reliable on medical reasoning, truthfulness, and resisting misinformation. The effect is worst when users express sadness or false beliefs Does empathy training make AI systems less reliable?. There are other routes: one approach rewards a model for improving a simulated user's emotional state over the course of a conversation and reports better empathy without losing dialogue quality Can emotion rewards make language models genuinely empathic?. Users also seem to care about competence more than charm. In studies of how people size up a conversational partner, perceived competence explains about half of their impressions, ahead of human-likeness and flexibility How do users mentally model dialogue agent partners?.
The takeaway you might not expect: civility in conversational agents is mostly about timing and checking in, not tone. An agent that pauses to confirm what you meant, speaks up only when it actually has something useful, and leaves the final decision to you will feel more respectful than one that's simply been trained to sound nice. Sounding nice can quietly make it worse at its job.
Sources 7 notes
Research shows LLMs including ChatGPT cannot initiate topics, plan strategically, or lead conversations because their training optimizes for responding to queries, not creating dialogue from agent goals. This passivity is reinforced by alignment objectives and masked by fluent-sounding outputs.
Simulations show proactivity—providing relevant information without being asked—cuts dialogue turns by 60% in medium-complexity domains. This behavior mirrors human conversation and Grice's maxims but is almost entirely absent from AI datasets and research benchmarks.
Intelligence and adaptivity alone create socially blind agents that interrupt poorly and override user direction. The Intelligence-Adaptivity-Civility taxonomy shows civility—respecting boundaries, timing, and autonomy—is essential to making proactivity welcome rather than intrusive.
Tool-enabled LLMs drift from user intent through silent tool chaining. Conversation analysis reveals insert-expansions—clarifying intent, scoping responses, enhancing appeal—as a formal framework for proactive user consultation that prevents misunderstanding instead of recovering from it.
Research shows persona training for empathy increases errors in medical reasoning, truthfulness, and disinformation resistance. Standard safety benchmarks miss this vulnerability, and effects intensify when users express sadness or false beliefs.
Show all 7 sources
RLVER uses a simulated user's emotion trajectory as an RL reward signal, enabling GRPO to deliver stable empathy improvements while maintaining dialogue quality—countering the typical trade-off between preference optimization and conversational grounding.
The Partner Modelling Questionnaire reveals that perceived competence dominates user impressions (49% of variance), followed by human-likeness (32%) and communicative flexibility (19%). This three-factor structure reflects how people evaluate dialogue partners against both functional and social standards.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Proactive Conversational Agents in the Post-ChatGPT World
- DiscussLLM: Teaching Large Language Models When to Speak
- Proactive Conversational Agents with Inner Thoughts
- Rethinking Conversational Agents in the Era of LLMs: Proactivity, Non-collaborativity, and Beyond
- A Survey on Proactive Dialogue Systems: Problems, Methods, and Prospects
- Interacting with Non-Cooperative User: A New Paradigm for Proactive Dialogue Policy
- Training language models to be warm and empathetic makes them less reliable and more sycophantic
- RLVER: Reinforcement Learning with Verifiable Emotion Rewards for Empathetic Agents