What makes evaluation easier than envisioning for users?
This explores why people find it easier to react to options than to generate them from scratch — and how AI systems can exploit that asymmetry by turning open-ended 'what do I want?' into a constrained 'is this it?'
This explores why people find it easier to react to options than to generate them from scratch. The corpus has a sharp name for the hard side of that gap: the "gulf of envisioning" — users genuinely can't articulate what they want, and current AI, which responds rather than probes, doesn't help them get there Why can't users articulate what they want from AI?. The key move in that work is exactly your question's premise: structured dialogue that presents model-generated options "shifts the cognitive burden from open-ended envisioning to constrained evaluation." Recognizing something is right is cheap; inventing it is expensive.
Why is recognition cheaper? Partly because intent isn't sitting fully-formed in the user's head waiting to be read out. It matures gradually through "progressive constraint resolution" — each option you react to eliminates possibilities and stabilizes the next choice How do users actually form intent when prompting AI systems?. Evaluation is easier because each act of evaluating is also an act of discovering what you meant. You don't envision the destination; you feel your way toward it one comparison at a time.
That's where the recommendation research connects laterally: people evaluate far better when an item is placed next to others than when judged in isolation, because comparison matches how humans naturally assess things Do comparisons help users evaluate items better than isolated descriptions?. A bare description forces you back into envisioning ("is this what I wanted?"); a comparison hands you a frame ("better or worse than that one?"). The same logic shows up in unexpected places — GUI agents fail when forced to *both* interpret a screen and decide an action at once, but succeed when the screen is pre-parsed so the model only has to evaluate among labeled options Why do vision-only GUI agents struggle with screen interpretation?. Generation-plus-evaluation in one step overloads; evaluation alone flows.
But the corpus also plants a warning flag: easier is not the same as truer. Users routinely *express* satisfaction while remaining internally confused, especially when they don't know what they don't know Does user satisfaction actually measure cognitive understanding?. Smooth, fluent options are seductive — imitation models fool human evaluators with confident style while closing no real capability gap Can imitating ChatGPT fool evaluators into thinking models improved?. So the very thing that makes evaluation easy (you only have to react) is also what makes it gameable: a well-presented wrong answer evaluates beautifully.
The thing worth walking away with: "evaluation is easier than envisioning" isn't just a UX convenience, it's a design principle with a built-in hazard. Good systems lower the burden by offering options to react against; the discipline is making sure those options expand the user's understanding rather than just earning a satisfied click — the same gap between expressed satisfaction and genuine clarity that quietly haunts evaluation everywhere Does user satisfaction actually measure cognitive understanding?.
Sources 6 notes
Intent develops through interaction, not in isolation. Since AI models respond rather than probe, they miss opportunities to help users discover unarticulated requirements. Structured dialogue that presents model-generated options shifts the cognitive burden from open-ended envisioning to constrained evaluation.
Human intent matures through progressive constraint resolution with fluctuating stability, not as a simple present-or-absent condition. The STORM framework and Clarify metric reveal that AI systems fail partly because they cannot access users' internal cognitive states during this evolution.
Relational explanations that compare items carry more decision-relevant information than isolated evaluations because they match how humans naturally assess products. A system extracting aspects from reviews and generating aspect-controlled comparisons produces sentences rated as both accurate and useful for purchase decisions.
OmniParser demonstrates that GPT-4V fails when forced to simultaneously identify icon meanings and predict actions from raw screenshots. Pre-parsing screenshots into structured semantic elements with descriptions lets the model focus solely on action prediction, removing the composite-task bottleneck.
STORM shows users express satisfaction despite internal confusion, especially when unaware of knowledge gaps. Sustained engagement correlates with actual self-understanding, not immediate satisfaction ratings.
Imitation models fool human evaluators by mimicking ChatGPT's confident, fluent style while failing to improve factuality or generalization on novel tasks. The ceiling is set by base model capability, not fine-tuning method—better fundamentals, not shortcuts, drive real improvement.