All AI Models are Wrong, but Some are Optimal

Abstract—AI models that predict the future behavior of a system (a.k.a. predictive AI models) are central to intelligent decision-making. However, decision-making using predictive AI models often results in suboptimal performance. This is primarily because AI models are typically constructed to best fit the data, and hence to predict the most likely future rather than to enable high-performance decision-making. The hope that such prediction enables high-performance decisions is neither guaranteed in theory nor established in practice. In fact, there is increasing empirical evidence that predictive models must be tailored to decision-making objectives for performance. In this paper, we establish formal (necessary and sufficient) conditions that a predictive model (AI-based or not) must satisfy for a decision-making policy established using that model to be optimal.We then discuss their implications for building predictive AI models for sequential decision-making.

These predictive models play a central role in intelligent systems by facilitating autonomous decision-making [2]. Despite their potential, predictive AI models are not yet widely adopted for sequential decision-making, in part due to the disappointing performance of the resulting decisions. This performance gap is due to the fact that most real systems are inherently stochastic and, even with abundant data, the predictions of the AI model are always an approximation of the real system’s future behavior [3].

This limitation arises because the construction of the predictive models is generally agnostic to the decision-making objectives, and therefore has no direct relationship to the performance measure of the resulting decision-making scheme.

In sequential decision-making, an agent aims to maximize expected rational utility over time by making decisions based on the latest available information, while accounting for both short- and long-term consequences [17]. For stochastic systems, decisions must also account for future uncertainty and the fact that more information will be available for future decisions.