Knowing Is Not Enough: Information Retrievability as a Precondition to Effective LLM Oversight

Large language models (LLMs) are increasingly embedded in organizational work, yet their errors often pass human review. Prior research locates such failures in users’ capability to review LLM output or their engagement in doing so. We develop an alternative, retrieval-based account of human oversight and posit that error detection is more effective when oversight-relevant information is accessible to users at the moment of review. Across two randomized lab-in-the- field experiments with 640 customer-facing employees, we show that self-generated explanations improve error detection and strengthen recall of verification-relevant reasoning, while cues that reactivate such reasoning help sustain detection under repeated LLM use. Theoretically, we identify information retrievability as a distinct precondition for effective oversight and specify generative encoding and cue-supported reactivation as mechanisms that build and sustain it. Practically, lightweight onboarding self-explanations and daily retrieval cues can make human oversight more resilient as LLM use becomes routine.

Introduction. Organizations are increasingly adopting large language models (LLMs) to augment knowledge work, such as drafting client communications, answering support requests, and preparing analyses, while retaining human reviewers as a safeguard against the models’ errors [5,24]. This person-in-the-loop design relies on employees to provide oversight and catch consequential mistakes before they reach customers or undermine downstream decisions. However, errors frequently persist through review, even when reviewers are aware of LLM fallibility and prominent system warnings are displayed. For example, lawyers have filed court briefs citing cases that a chatbot fabricated [7]. One common explanation is that fluent, confident, and largely correct LLM output invites acceptance rather than detailed scrutiny. Even a reviewer who was recently trained on a type of LLM error, or has personally encountered it, may still miss the next instance [18,22].

Discussion / Conclusion. Both studies support the same retrieval-based account: oversight can fail when relevant information is not accessible at the moment of review, even in settings where reviewers have received that information and are actively reviewing the output. The results indicate that generating an explanation supported later accessibility of verification-relevant reasoning and that an associated cue helped reactivate that reasoning across repeated use. Building on these findings, we develop three contributions in dialogue with capability- and engagement-based accounts of human oversight. Retrievability as a Distinct and Complementary Precondition for Oversight Our first contribution is to identify retrievability as a distinct precondition for effective oversight. Existing accounts attribute oversight failure either to limited literacy, expertise, or mental models, motivating interventions that build these resources [2,20,39], or to insufficient scrutiny of seemingly reliable systems, motivating interventions that increase attention or evaluative effort [6,28].