Can AI safety nets reduce errors in live clinical practice?

This quality-improvement study, run with Penda Health's primary care clinics in Nairobi, compares 39,849 visits by clinicians with and without access to AI Consult, an LLM tool that serves as a safety net for clinicians. Independent physicians reviewing documentation rated the AI group's visits as having 16% fewer diagnostic errors and 13% fewer treatment errors. A clinician survey reports that every AI-group respondent said the tool improved the care they could deliver, and 75% called the effect "substantial." The authors project that the tool would avert diagnostic errors in 22,000 visits and treatment errors in 29,000 visits a year at Penda alone, but that is their projection, not a measurement. Their summary is that "these results required a clinical workflow-aligned AI Consult implementation and active deployment to encourage clinician uptake."

The paper names two conditions beyond model capability. The first is implementation. The earlier AI Consult v1 required clinicians to stop mid-visit and request feedback, reached about 60% of visits, and left many correct suggestions unheeded. The studied version fires asynchronously at EMR decision points behind a green, yellow or red interface, "explicitly designed to minimize cognitive burden and preserve clinician autonomy." The second is active deployment: the effect was significantly larger in the main period, when deployment strategies ran, than in the induction period. The authors also read a learning effect into a 10–15 point drop in visits that "started red" for treatments in the AI group, while the non-AI group stayed flat. Their conclusion lists "capable models, which are now widely available" as only one of three components.

Against the nearest notes, the paper is a live-care counterpart to offline evaluation. Can language models match therapist empathy in real conversations? rests on isolated responses. This study measures errors in routine visits, which the authors say offline work cannot capture: "real-world patient diversity, designing for and learning from clinician workflows, and deployment towards successful clinician uptake." The workflow emphasis echoes Can clinical experts teach LLMs to annotate complex medical concepts?, which also places clinical-LLM friction in the interface between expert and tool. The direction is reversed, though: that excerpt reports barriers, while this one reports what made a deployed tool work. On the uptake side, Why do patients distrust medical AI systems? concerns patient attitudes. The uptake problem here is clinicians not acting on correct advice.

The excerpt does not establish that the workflow and deployment choices caused the error reduction rather than the model alone. The comparison was access versus no access under a cluster-assigned design, and the studied implementation was never compared with a non-aligned version on error rates. v1 was assessed only for adoption and a 100-case internal safety audit, so "required" is the authors' inference from design rather than a tested contrast. Patient-reported outcomes showed no significant difference, which the authors attribute possibly to measurement sensitivity, a 40% response rate, and a short follow-up. The error ratings are physician judgments of documentation, not observed patient harm, and several authors are affiliated with Penda, as the introduction states. The supportable reading is narrow: in one large Kenyan network, a documentation-based error reduction appeared alongside these implementation and deployment choices. It does not yet show better patient outcomes or transfer to other health systems. The authors call their work "a first step."