Learning To Guide Human Experts Via Personalized Large Language Models

“Consider the problem of diagnosing lung pathologies based on the outcome of an X-ray scan. This task cannot be fully automated, for safety reasons, necessitating human supervision at some step of the process. At the same time, it is difficult for human experts to tackle it alone due to how sensitive the decision is, especially under time pressure. High-stakes tasks like this are natural candidates for hybrid decision making (HDM) approaches that support human decision makers by leveraging AI technology for the purpose of improving decision quality and lowering cognitive effort, without compromising control. Most current approaches to HDM rely on a learning to defer (LTD) setup, in which a machine learning model first assesses whether a decision can be taken in autonomy – i.e., it is either safe or can be answered with confidence – and defers it to a human partner whenever this is not the case [Madras et al., 2018, Mozannar and Sontag, 2020, Keswani et al., 2022, Verma and Nalisnick, 2022, Liu et al., 2022]. Other forms of HDM, like learning to complement [Wilder et al., 2021], prediction under human assistance [De et al., 2020], and algorithmic triage [Raghu et al., 2019, Okati et al., 2021] follow a similar pattern. We argue that this setup is suboptimal and potentially dangerous. The reason is that there is a risk that humans will end up over-trusting the machine’s decisions (a phenomenon known as anchoring bias [Rastogi et al., 2022]). At the same time, whenever the machine opts for deferral, the human is left resolving hard cases completely unassisted. Both situations conflict with the aims of HDM.

As a remedy, we propose learning to guide (LTG), an alternative algorithmic setup that avoids these issues entirely. Instead of proposing potential decisions, in LTG the machine supplies its human partner with interpretable guidance highlighting aspects of the input x that are useful for coming up with a sensible decision. In this setup, responsibilities cannot be shifted: by construction, all decisions are taken (under assistance) by the human in the loop.”