What People Almost Did: Evaluating LLM Social Simulations Beyond Behavioral Fit

LLM-based social simulations are primarily evaluated for behavioral fit, testing whether agents reproduce the actions or response distributions of the people they are simulating. However, the promise of simulation extends beyond behavioral fit. Simulations can explain human behavior, diagnose barriers, and compare large-scale interventions. These use cases depend on understanding why people acted a certain way, not just what they did. As a result, behavioral fit is insufficient for these types of claims because behavior underdetermines the reasoning process behind it. For instance, the behavior of staying silent may be due to disinterest or suppressed speech, and not answering a call may be due to distrust of the caller or limited phone access. In this paper, we propose representational adequacy as a new evaluation target for LLM-based social simulations. By leveraging LLM reasoning traces, representational adequacy measures whether a simulation’s scenario–reasoning–action triples preserve the reasoning process behind the behavior in a way that is faithful to the population and scenarios being simulated.

Introduction. Researchers simulate people when studying real populations is infeasible. Would teens share more if fewer adults could see their posts? Would more expectant mothers answer an automated health call from a voice they trusted? Running such comparisons on real populations is expensive and sometimes unethical, so simulation allows us to approximate the impact of an intervention before implementing it in real scenarios (Park et al., 2025). In an LLM-based social simulation, a language model simulates a person. It receives a persona, demographic details, memories, or interview transcripts, and acts within a scenario (Park et al., 2024). Where classical agent-based models require the researcher to write out decision rules, an LLM agent generates its own behavior and can be asked what it weighed before acting. The resulting reasoning trace may be generated before the action, stored as a structured state, elicited afterward, or reconstructed from logs. Regardless of how it is obtained, a trace represents a hypothesis about the process behind the agent’s action.

Discussion / Conclusion. Behavioral fit is an important evaluation target for LLM social simulations that predict human behavior. However, it is not sufficient for simulations that are used to explain human behavior, because identical actions can mask underlying reasoning. In response, we propose representational adequacy, a framework for assessing whether simulations represent decision-making processes across the simulated population. The promise of social simulation has always been studying how people would act under conditions we cannot practically or ethically create. Taking that promise seriously means evaluating simulations on more than what people did. It means asking whether simulations preserve what people considered, feared, and almost did.