Chatbots coach beginners to catch up fast — would letting AI just act on its own erase that head start?
Can AI agents replacing chatbots improve outcomes for less experienced workers?
This explores whether moving from AI that answers questions (chatbots) to AI that takes actions on its own (agents) would widen or shrink the head start that AI currently gives less experienced workers.
This explores whether moving from AI that answers questions (chatbots) to AI that takes actions on its own (agents) would widen or shrink the head start AI currently gives less experienced workers. The corpus has no study that tests this head to head, so the answer has to be pieced together. The pieces point in an unexpected direction. The novice advantage seems to come from AI acting as a coach next to the worker, and today's agents are weakest at the parts of the job where that coaching mattered most.
Start with the best evidence that AI helps beginners. In a study of more than 5,000 customer support agents, AI assistance raised productivity by 15% on average. Most of that gain went to less experienced workers, who got both faster and better, while the most experienced agents got slightly faster but slipped a little on quality Does AI assistance help less experienced workers most?. One reading: the AI was spreading the know-how of top performers to people who hadn't built it yet. That only works while the worker stays in the loop, absorbing the suggestions. An agent that does the task itself changes the deal. The novice is no longer being lifted toward expertise. Their task is being taken over.
The next question is whether agents can actually take it over. Right now, mostly not. In a simulated workplace benchmark, leading agents completed only about 30% of tasks on their own. They failed most on social interaction, navigating professional software, and domain-specific knowledge Why do AI agents fail at workplace social interaction?. Those are the same tacit skills that less experienced workers lack and that the support-desk AI seemed to help them pick up. Part of the reason is structural. Conversational AI is built to respond, not to lead or plan toward a goal Why can't conversational AI agents take the initiative?. Agents that chain tools together quietly also tend to drift from what the user meant, unless they're designed to stop and ask clarifying questions at the right moments When should AI agents ask users instead of just searching?. A novice is the person least able to notice that drift.
There is a constructive path. Some research suggests agents could be more useful by being proactive, offering relevant information before they're asked. In simulations this cut conversation turns by up to 60%, but the behavior is almost absent from current AI training data Could proactive dialogue make conversations dramatically more efficient?. Another line of work gives the AI an explicit list of what it doesn't yet know about the user, which cut harmful advice and sycophancy by 50–75% Do language models know what they don't know about users?. For an inexperienced worker who doesn't know what to ask, an agent that volunteers context and flags gaps could be more valuable than either a passive chatbot or a fully autonomous agent.
The less comfortable part: agents that act raise different ethical problems than assistants that only answer, including questions of trust, manipulation and equity What makes ethics of AI assistants fundamentally different from chatbots?. At the firm level, companies with more AI exposure already replace online marketplace workers with AI faster and more cheaply Do firms substitute labor for AI at different rates?. So whether agents 'improve outcomes for less experienced workers' depends on which outcome you mean. Better task results are plausible if agents are designed to coach and consult. Better career outcomes are much less certain, because the entry-level work where novices once learned is exactly what agents are built to absorb.
Sources 8 notes
A study of 5,172 support agents at a Fortune 500 firm found a 15% average productivity gain from AI assistance, with gains concentrated among less experienced workers who improved both speed and quality. The most experienced agents saw small speed gains but slight quality declines.
TheAgentCompany benchmark shows leading agents achieve 30% task completion in a simulated workplace. Social interaction, professional UI navigation, and domain-specific knowledge are the three primary failure modes, with multi-turn task performance consistently dropping to 35% across enterprise settings.
Research shows LLMs including ChatGPT cannot initiate topics, plan strategically, or lead conversations because their training optimizes for responding to queries, not creating dialogue from agent goals. This passivity is reinforced by alignment objectives and masked by fluent-sounding outputs.
Tool-enabled LLMs drift from user intent through silent tool chaining. Conversation analysis reveals insert-expansions—clarifying intent, scoping responses, enhancing appeal—as a formal framework for proactive user consultation that prevents misunderstanding instead of recovering from it.
Simulations show proactivity—providing relevant information without being asked—cuts dialogue turns by 60% in medium-complexity domains. This behavior mirrors human conversation and Grice's maxims but is almost entirely absent from AI datasets and research benchmarks.
Show all 8 sources
Research shows assistants suffer from sycophancy and hallucination because they have no representation of what remains unknown about users. Adding a schema of labeled unknowns to prompts reduced harmful advice and sycophancy by 50–75% and cut hallucination rates by roughly half.
DeepMind research maps a comprehensive ethics framework specific to action-taking AI agents, spanning individual concerns (manipulation, trust, anthropomorphism) and societal issues (equity, coordination, misinformation). The key insight: assistants that act raise fundamentally different problems than those that answer.
Higher AI-exposed firms replace online labor marketplace workers with AI tools faster and at lower cost than less-exposed firms, suggesting returns to scale in internal AI capability rather than uniform technology diffusion.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Proactive Conversational Agents in the Post-ChatGPT World
- DiscussLLM: Teaching Large Language Models When to Speak
- Proactive Conversational Agents with Inner Thoughts
- Rethinking Conversational Agents in the Era of LLMs: Proactivity, Non-collaborativity, and Beyond
- Intent Mismatch Causes LLMs to Get Lost in Multi-Turn Conversation
- Conversational Alignment with Artificial Intelligence in Context
- Future of Work with AI Agents: Auditing Automation and Augmentation Potential across the U.S. Workforce
- Payrolls to Prompts: Firm-Level Evidence on the Substitution of Labor for AI