Generative AI in Real-World Workplaces

This report presents the most recent findings of Microsoft’s research initiative on AI and Productivity, which seeks to measure and understand the productivity gains associated with LLM-powered productivity tools like Microsoft Copilot. The report synthesizes research results from over a dozen recent studies conducted by researchers at Microsoft, with a focus on studies of generative AI in actual workplace environments. One of these is, to our knowledge, the largest, randomized controlled trial of the introduction of generative AI into organizations. Overall, the research suggests that generative AI is already aiding workers in becoming more productive in their day-to-day jobs in significant ways. However, the influence of generative AI is subject to variation by role, function, and organization and is contingent upon adoption and utilization. The report explores these variations and underscores the potential for AI to have even greater impact as individuals and organizations recalibrate their work practices to harness AI in the places where it provides the most value.

However, much of this existing literature on AI and productivity is limited in that it consists primarily of lab-based studies.

·      Generative AI is already helping people be measurably more productive in their day-to-day jobs.

·      As expected, the productivity story in real-world workflows is more complex than observed in lab studies.

·      Productivity gains associated with generative AI, including time and accuracy, vary by role, function and organization.

·      Variance in adoption and utilization influences AI’s impact.

·      Early studies suggest generative AI may affect the cognitive effort required for task completion.

largest impact was on novice and low-skilled workers, with very little effect on experienced or highly-skilled workers.

Researchers found that on average, those with Copilot for Microsoft 365 read 11% fewer individual emails and spent 4% less time interacting with them, compared to people without Copilot. Some organizations saw larger effects with relative decreases of up to 20 or 25% in both emails read and time spent interacting with email.

Overall people edited 10% more documents, with heavy users of Word, Excel, and PowerPoint seeing an increase of 13% (on a higher baseline). Some organizations saw increases in the 25-30% range. One hypothesis is this is an early sign of the writing and creation assistance that Copilot provides making it easier to produce and revise output.