DeLLMa: Decision Making Under Uncertainty with Large Language Models

The potential of large language models (LLMs) as decision support tools is increasingly being explored in fields such as business, engineering, and medicine, which often face challenging tasks of decision-making under uncertainty. In this paper, we show that directly prompting LLMs on these types of decision-making problems can yield poor results, especially as the problem complexity increases. To aid in these tasks, we propose DeLLMa (Decision-making Large Language Model assistant), a framework designed to enhance decision-making accuracy in uncertain environments. DeLLMa involves a multi-step reasoning procedure that integrates recent best practices in scaling inference-time reasoning, drawing upon principles from decision theory and utility theory, to provide an accurate and human-auditable decision-making process.

Inference-time Reasoning in LLMs. Many recent works have leveraged inference time compute to extend the computational power of language models [32, 46]. These can be categorized into three main approaches: (1) instructing the model to generate intermediate reasoning traces [13, 43, 51, 55, 57, 58, 61], (2) decomposing a complex reasoning problem into tangible components [35, 38, 56, 60], and most recently (3) scaling the number of parallel samples to be postprocessed into a final solution [9, 35, 45, 50].

Furthermore, beyond merely making rational decisions, it is crucial to understand why an LLM made a particular decision. This aids in building trust in the decision, assessing its quality, and improving any components that may lead to suboptimal outcomes. The ability to explain decisions and verify decision-making quality—which we refer to as human auditability—is essential for the practical application of LLMs to aid decision making in many real problems

Drawing inspiration from prior work on multi-step reasoning like Chain-of-Thought (CoT) [51] and Tree-of-Thoughts (ToT) [56], in which compute is scaled at inference time, we design a procedure based on classical decision theory, originally designed for rational decision making under uncertainty by humans. Our approach involves three key steps: first, identify and forecast pertinent unknown variables given in-context information; second, elicit a utility function that aligns with the user’s goals; and finally, use this utility function to identify the decision that maximizes expected utility. We call our proposed framework DeLLMa, short for Decision-making Large Language Model assistant.

However, we observe that responses from conventional approaches, such as Self-Consistency and CoT, do not adequately balance available evidence, handle uncertain information, or align with user preferences; we show in Sec. 4 that these methods perform poorly, especially with increasing numbers of actions.