Should we treat LLM outputs as real empirical data?

A "subtle shift in the meaning of data" is underway: knowledge once derived from empirical observation is now supplemented, or replaced, by information co-produced through human-model interaction. The Foundation Priors paper (2024) provides a formal statistical framework for understanding this shift. LLM-generated outputs are not observations from the world — they are draws from a foundation prior, an intractable, subjectively malleable distribution that reflects both the model's learned patterns and the user's subjective filters.

The provenance of such data is fundamentally uncertain. We have minimal visibility into model architecture and training data, and the prompt design process injects the user's own priors, beliefs, and preferences into the generation mechanism. This makes the generated data epistemically different in kind from empirically collected data, however similar in surface form.

The practical implication is that generative outputs should influence inference only through an explicitly parameterized trust weight (λ) and never by being treated as if drawn from the same process as empirical observations. When framed this way, synthetic data become a source of structured prior information rather than a surrogate for real evidence. The tools the paper develops — integrating across heterogeneous prompts, tempering synthetic data influence through conservative trust, calibrating effect using real observations — formalize what the vault's Tokenization framework describes informally: AI outputs have exchange value (they look and trade like knowledge) but their use value (whether they actually work under their claims) requires independent verification.

Since Does iterative prompt engineering undermine scientific validity?, the Foundation Priors framework provides the formal statistical apparatus for that methodological critique. The self-fulfilling prophecy IS epistemic circularity: prompt iteration reinforcing user priors without empirical anchoring.