Why do AI agents misalign with what users actually want?
UserBench explores how often AI models fully understand user intent across multi-turn interactions. The study reveals that human communication is underspecified, incremental, and indirect — traits that challenge current models to actively clarify goals.
UserBench evaluates agents in multi-turn, preference-driven interactions where simulated users start with underspecified goals and reveal preferences incrementally. The results quantify a gap that existing benchmarks obscure:
- Models provide answers that fully align with ALL user intents only 20% of the time on average
- Even the most advanced models uncover fewer than 30% of all user preferences through active interaction
- Scores drop by over 40% when models must select only one option per dimension (forcing commitment rather than hedging)
The framework identifies three core traits of human communication that make this hard:
- Underspecification — users initiate requests before fully formulating their goals
- Incrementality — intent emerges and evolves across interaction turns
- Indirectness — users obscure or soften their true intent due to social or strategic reasons
These are not edge cases — they are the default condition of human communication. Language is inherently ambiguous (Clark, 1996; Liu et al., 2023), and meaning is co-constructed through interaction.
The disconnect between task completion and user alignment is the critical finding. Standard benchmarks measure whether an agent completes a task — UserBench measures whether the agent completed the right task, from the user's perspective. Current models are task-capable but not user-aligned.
This connects to Why can't users articulate what they want from AI? — the 20% figure quantifies the double gap. And since How do users actually form intent when prompting AI systems?, the incrementality trait confirms that intent-as-binary is a design error, not an edge case.
The finding that models elicit <30% of preferences through active querying connects to Can models learn to ask clarifying questions instead of guessing? — proactive questioning is trainable (0.15% → 73.98%) but is not standard in current deployments.
Source: Design Frameworks
Related concepts in this collection
-
Why can't users articulate what they want from AI?
Explores the cognitive gap between imagining possibilities and expressing them as prompts. Why language interfaces create a harder envisioning task than traditional UI affordances.
the 20% figure quantifies the double gap
-
How do users actually form intent when prompting AI systems?
Users face a 'gulf of envisioning'—they must simultaneously imagine possibilities and express them to language models. This cognitive gap creates breakdowns not from AI incapability but from users struggling to articulate what they truly need.
incrementality confirms intent maturation
-
Can models learn to ask clarifying questions instead of guessing?
Exploring whether large language models can be trained to detect incomplete queries and actively request missing information rather than hallucinating answers or refusing to respond. This matters because conversational agents today remain passive, responding only when prompted.
proactive questioning addresses the preference elicitation gap
-
Why do language models fail in gradually revealed conversations?
Explores why LLMs perform 39% worse when instructions arrive incrementally rather than upfront, and whether they can recover from early mistakes in multi-turn dialogue.
premature assumptions are the mechanism behind the 20%
-
Why can't advanced AI models take initiative in conversation?
Despite extraordinary capability in answering and reasoning, LLMs fundamentally cannot initiate, redirect, or guide exchanges. Understanding this gap—and whether it's fixable—matters for building AI that truly collaborates rather than merely responds.
passivity prevents preference discovery
Click a node to walk · click center to open · click Open full network for a force-directed map
Original note title
agents fully align with all user intents only 20 percent of the time — even best models elicit fewer than 30 percent of preferences through active querying