AI made law students much faster at legal analysis but only slightly better, so does that speed hold up in paid client work?

This explores whether the time AI saves on legal tasks in experiments holds up in paid client work. The short answer is that the collection has strong evidence on the first half and almost none on the second, so the gap is the most useful thing to see. The closest direct evidence comes from a randomized trial with law students: GPT-4 made legal analysis much faster at every skill level, but improved quality only slightly and unevenly, with the biggest gains for the weakest students Does GPT-4 actually improve the quality of legal analysis?. That pattern is a warning in itself. Students finished sooner, but the work was not much better, and billable work is judged on the finished product, not on how fast it was drafted.

The practice-side evidence the collection does have concerns errors, not speed. Of 114 US court cases with suspected AI-generated mistakes, 90 percent involved solo practitioners or small firms Do small law firms misuse AI more often than large ones?. Those are the incidents that were caught, not misuse rates by firm size. Still, they show where the speed gain can turn into a liability. Time saved on drafting is lost when a fabricated citation reaches a judge. In paid practice, the verification step is part of the job, and the student trial didn't have to price it in.

Studies from other fields suggest what tends to happen when lab speedups meet real work. Among 5,172 customer support agents, AI raised issues resolved per hour by 15 percent. Most of that gain went to the least experienced agents, while the most experienced saw small speed gains and slight quality declines Does AI assistance help less experienced workers most?. More strikingly, a randomized trial found that early-2025 AI tools made experienced open-source developers 19 percent slower on real tasks in codebases they knew well, even though they had predicted a 24 percent speedup Do AI coding tools actually speed up experienced developers?. A senior lawyer working in their own specialty looks a lot like that developer. Their own expertise is already fast, and checking the AI's output can cost more time than it saves.

Two more findings complicate the picture. Productivity gains appear when workers apply skills they already have, but they disappear when people use AI to learn something new, and learning suffers too When does AI actually boost worker productivity?. For junior associates, whose billable hours are also their training, that tradeoff matters. And even correct AI suggestions can break a person's focus during reasoning, a cost that doesn't show up if you only measure suggestion by suggestion Does AI assistance always help reasoning or does it carry hidden costs?. Legal analysis is exactly that kind of sustained reasoning.

What the collection doesn't cover: no study here measures AI's effect on actual billed hours, law firm economics, or how clients respond. It also doesn't take up the obvious tension that under hourly billing, faster work means fewer billable hours. Here is the lesson you might not have expected to want: when AI meets expert work, the measured speedup tends to shrink, and sometimes it reverses. The people it helps most are the least experienced, who are also the people least able to catch its mistakes.