INQUIRING LINE

If an AI's work is easy to check, people use it — even when the task is hard — but avoid it when they can't verify the answer.

How do workers choose between manual work and AI based on verification ability?

This explores how people decide whether to hand a task to AI or do it themselves, and whether being able to check the AI's output is what drives that choice.


This explores how workers decide when to use AI and when to do the work themselves, and whether being able to check the result is what decides it. The most direct evidence in the collection comes from a small study: in interviews, thirteen junior developers said their main test was whether they could verify the output, not deadlines or how hard the task was Do junior developers choose AI based on their ability to verify results?. They avoided AI for work they couldn't evaluate. They used it most where they already had expertise. That runs against the usual idea that AI helps most where you know least. For these developers, it was useful mainly where they could already have done the work.

The same rule shows up on the machine side too. Wei's "verifier's rule" argues that AI systems get good at tasks in proportion to how easily a solution can be checked. He also argues the gap can be closed ahead of time with answer keys, test suites or measurement tools Does task verifiability determine what AI systems will learn to solve?. Put the two studies together and verifiability is doing the work twice: it shapes what AI learns to do well and what humans trust it with. That may help explain why real delegation has clustered in information-heavy jobs and follows what AI can actually do, not simply where chatbots are popular Where have workers actually delegated tasks to AI?.

The catch is that people are often bad at judging whether they can verify something. In WalkMe's survey, 90% of workers said they feel confident with AI. Only 25% said it works on the first try, and half had spent more time with AI than the task would have taken by hand Why do workers feel confident with AI but get poor results?. One explanation is the "LLM fallacy." When AI output reads smoothly, people blur the line between what they produced and what the tool produced, and start crediting themselves with skills they don't have Do AI-assisted outputs fool users about their own skills?. A worker who thinks they can check the work may only be recognizing that it sounds right.

The less obvious problem is that the strategy can wear itself down over time. The junior developers' rule depends on expertise, and research mapping thousands of workplace AI risk scenarios found that relying on AI agents can gradually erode both skills and the ability to oversee the AI meaningfully Does AI augmentation protect workers from skill erosion?. So the more a worker delegates in the area where they can verify, the more that area may shrink. Social pressure adds to this: people who use AI expect to be seen as less competent and often don't tell colleagues Do people fear judgment when they use AI at work?. That leaves less room to talk openly about where checking actually works.

The collection's direct evidence is thin: one interview study of thirteen people. The broader pattern comes from linking it to related work, not from large studies that measure verification-driven choices directly.


Sources 7 notes

Do junior developers choose AI based on their ability to verify results?

Interviews with thirteen Brazilian junior developers found that the ability to check results—not deadlines or task complexity—drives their decision to use AI. Developers avoid AI for work they cannot evaluate, concentrating its use where they already possess relevant expertise.

Does task verifiability determine what AI systems will learn to solve?

Wei argues that AI solves tasks proportional to how easily solutions can be verified, and that verifiability gaps can be narrowed by pre-investing in answer keys, test suites, or measurement infrastructure. This mechanism explains RL's effectiveness across domains from sudoku to molecular discovery.

Where have workers actually delegated tasks to AI?

Workers have committed AI tasks to structured workflows primarily in information-intensive occupations, following technical capability more than conversational LLM adoption. This gradient differs sharply from routine-task automation predictions and wage patterns reverse at advanced degree levels.

Why do workers feel confident with AI but get poor results?

WalkMe's survey of 2,037 US workers found 90% feel confident using AI, but only 25% report it works on first try and 50% spent more time using AI than doing tasks manually. The gap widened most among younger workers, suggesting overestimation of skill.

Do AI-assisted outputs fool users about their own skills?

Research identifies a systematic cognitive attribution error where individuals integrate AI-generated outputs into their capability identity, believing they possess skills they don't actually have. This occurs when task output is seamless and fluent, obscuring the human-AI boundary.

Show all 7 sources
Does AI augmentation protect workers from skill erosion?

Research mapping 8,356 workplace AI risk scenarios found that augmentation mode does not inherently prevent harm. Overreliance on AI agents can gradually erode worker skills and their capacity to provide meaningful oversight, undermining augmentation's core safety justification.

Do people fear judgment when they use AI at work?

Across four experiments with 4,439 participants, people using AI expected others to judge them as less competent and diligent, and reported lower willingness to disclose AI use to managers and colleagues. The gap suggests a social cost that users foresee and act on.

Papers this line draws on 8

The research behind the notes this line reads — ranked by how closely each paper relates.