Lawyering in the Age of Artificial Intelligence

Paper · Source
Domain Specialization in LLMs

Source: Choi, Monahan, Schwarcz, Minnesota Law Review · 2024-11

We conducted the first randomized controlled trial to study the effect of AI assistance on human legal analysis. We randomly assigned law school students to complete realistic legal tasks either with or without the assistance of GPT-4, tracking how long the students took on each task and blind-grading the results.

We found that access to GPT-4 only slightly and inconsistently improved the quality of participants’ legal analysis but induced large and consistent increases in speed. AI assistance improved the quality of output unevenly—where it was useful at all, the lowest-skilled participants saw the largest improvements. On the other hand, AI assistance saved participants roughly the same amount of time regardless of their baseline speed. In follow-up surveys, participants reported increased satisfaction from using AI to complete legal tasks and correctly guessed the tasks for which GPT-4 was most helpful.

These results have important descriptive and normative implications for the future of lawyering. Descriptively, they suggest that AI assistance can significantly improve productivity and satisfaction, and that it can be selectively employed by lawyers in areas where AI is most useful. Because AI tools have an equalizing effect on performance, they may also promote equality in a famously unequal profession. Normatively, our findings suggest that law schools, lawyers, judges, and clients should thoughtfully embrace AI tools and plan for a future in which they will become widespread.

Synthesis notes that discuss concepts related to this paper 4

Notes in this collection most semantically related to the paper, ranked by cosine similarity.

Lines of inquiry this paper opens 24

Research framings built by reading the notes related to this paper — the questions it feeds into.

What are the real-world consequences of AI citation hallucinations? How can AI systems reliably guide voters without introducing political bias? Why do language models hallucinate and how can we prevent it? How do hallucinated citations emerge in AI scholarly output? Does AI deployment reduce or exacerbate workplace inequality and income instability? Does AI-assisted work increase total productivity or just shift time? Why does polished AI output gain credibility despite fundamental verifiability problems? How does AI adoption reshape collaboration patterns in knowledge work? How do educators verify student capability when AI can produce indistinguishable work?