Experienced developers in one trial still felt AI had sped them up, though a stopwatch showed they were 19% slower.

This explores whether people's own reports of AI use and AI-driven speedups can be trusted as a measure of real productivity change. The short answer from the collection is: not very much on their own. The clearest evidence is a randomized trial in which experienced open-source developers expected AI tools to make them 24% faster. They were actually 19% slower on real tasks. Even after finishing, they still believed the tools had helped. Outside experts in economics and machine learning made the same overestimate Do AI coding tools actually speed up experienced developers?. So the sense of being faster and the stopwatch can point in opposite directions, even for skilled people.

Why would self-reports drift so far from reality? One reason is that AI changes what a work session feels like. When an assistant produces polished output, people tend to read that smoothness as a sign of their own competence, even though they didn't produce it themselves. Since language models are tuned to sound fluent whether or not the user understands the content, this bias pushes in one consistent direction Does processing ease mislead users about their own competence?. A second reason is that AI often doesn't cut total time at all. Instead it moves time from doing the task to writing prompts and checking what comes back. That interaction can feel like progress, so 'I used AI a lot' and 'I got more done' are easy to confuse Does AI really save time, or just change how we spend it?.

The less obvious problem is that even honest, accurate reports measure the wrong thing if the real question is skill. Productivity gains show up mainly when people apply skills they already have. When workers used AI to learn something new, the gains disappeared and their learning suffered When does AI actually boost worker productivity?. Someone could truthfully say they finished a task faster with AI while getting worse at doing it alone. Automated usage logs don't fix this either. Telemetry records assisted output well, but it can't show whether independent ability is growing or shrinking. In the note's words, it sees 'expertise in use' but not 'expertise forming' Can we measure whether AI erodes independent skill?.

The gap also exists between individuals and the bigger picture. AI-augmented scientists publish about three times as many papers and get far more citations, which looks like a clear productivity win from their side. Across science as a whole, though, the range of topics studied shrinks and collaboration falls Does AI help individual scientists while narrowing scientific focus?. Measures based on what workers have actually handed off to AI also give a different map from measures based on how often people chat with an assistant. Real delegation follows what AI can technically do, not how popular the chat tools are Where have workers actually delegated tasks to AI?. 'How much do you use AI?' and 'what work has AI actually taken over?' are different questions with different answers.

What self-reports are good for is showing expectations, adoption, and how the work feels, and that matters in its own right. The forecasting gap is itself a finding. But to learn what AI does to productivity, the collection points to measured completion times in controlled trials, output tied to specific tasks, and separate tests of what people can still do without AI. If you're reading a headline claim about AI productivity, first check whether it came from a stopwatch or a survey.