Two studies of AI coding tools disagree on whether they save time — could checking the AI's code explain why?

This explores how much of a developer's time goes into checking AI-written code, and whether that checking cancels out the speed AI promises. The corpus has no direct measurement of this. None of these notes gives a number like 'X percent of time spent reviewing.' What it does have is more revealing: two randomized trials that point in opposite directions, and checking is one of the likely reasons they disagree.

In one trial, 16 experienced open-source developers worked on 246 real tasks in codebases they knew well. With AI tools they took 19 percent longer, even though they had expected to be 24 percent faster Do AI coding tools actually speed up experienced developers?. The researchers name low AI reliability as one cause. When suggestions are often wrong, someone who knows the code deeply spends time catching and correcting them. The other trial, with 96 Google engineers on a complex task, found AI features cut time by about 21 percent. Its confidence interval was wide, though, and the result was only statistically significant under some model specifications Do AI coding features actually speed up engineer productivity?. So the honest summary is that the net effect of AI help, after checking costs, is still unsettled. It probably depends on how reliable the tool is and how well the developer knows the codebase.

The less obvious finding is that checking shapes when developers use AI at all, not just how long they spend afterward. Thirteen junior developers in Brazil said the deciding factor was whether they could verify the result, not deadlines or task difficulty Do junior developers choose AI based on their ability to verify results?. They avoided AI for work they couldn't evaluate. That means AI time savings land mostly where the developer already has expertise, which is the opposite of the usual pitch that AI helps you where you know least. The reverse risk is that people stop checking altogether. One note calls this 'cognitive surrender': fluent output builds false confidence, checking is costly, and studies show about 80 percent of AI outputs adopted without challenge When do users stop checking whether AI output is actually backed?. Low measured verification time may therefore mean verification is being skipped, not that it's cheap.

Outside individual coding, the same imbalance shows up at larger scale. Research on AI-assisted science finds generation consistently outpaces verification, which moves the bottleneck from writing to checking Can AI verify research outputs as fast as it generates them?. One position paper argues that if AI speeds up output, review must be automated too, or the pipeline backs up Can human review keep pace with AI-accelerated research generation?. Automated checking is getting closer for code. Structured reasoning templates can judge whether two code patches are equivalent at 93 percent accuracy without running them Can structured reasoning replace code execution for RL rewards?. Verifiers can also run alongside generation with almost no added delay on correct runs Can verifiers monitor reasoning without slowing generation down?.

The takeaway: the useful question may be less 'how long do developers spend verifying?' and more 'who or what does the verifying, and can the developer tell when it was skipped?' The corpus suggests that whether AI-generated code saves time depends more on how cheap and reliable checking is than on how fast the code is generated.