SYNTHESIS NOTE
Topics›Evaluations›this note

Does setting temperature to zero actually make LLM outputs reliable?

Explores whether deterministic LLM settings that produce consistent outputs also guarantee reliable judgments, and how to measure true reliability beyond surface consistency.

Synthesis note · 2026-03-28 · sourced from Evaluations

"Can You Trust LLM Judgments?" (2024) introduces a rigorous framework for evaluating LLM-as-a-Judge reliability using McDonald's omega, revealing that the common practice of using fixed seeds and deterministic settings provides false confidence.

The core argument: even with deterministic settings, a single LLM output is one sample from the model's probability distribution. Setting temperature to zero and fixing the seed produces "fixed randomness" — the same output every time, but that output may still be a misleading draw from the distribution. Consistent replication does not guarantee reliability. A perfectly calibrated LLM that says it's 90% confident should be correct 9 out of 10 times — but even a perfectly calibrated LLM can be unreliable if its distribution has high variance.

The framework: prompt the judgment LLM 100 times, varying only the replication while holding all other factors constant. Apply McDonald's omega to assess internal consistency across these replications. This reveals whether the model's judgments are stable properties of the input or artifacts of the sampling process.

The distinction between reliability, confidence, and calibration is critical:

These three are intertwined but distinct. A model can be well-calibrated (confident when right) but unreliable (different answers on different draws). A model can be reliable (always gives the same answer) but poorly calibrated (that consistent answer is wrong).

This connects to Does model confidence predict robustness to prompt changes? — ProSA measures sensitivity to prompt variation, while this measures sensitivity to sampling variation. Both reveal that single evaluations are insufficient. The practical implication: any LLM-as-a-Judge deployment that relies on single-shot evaluation with deterministic settings is providing the illusion of precision without evidence of reliability.

Inquiring lines that read this note 191

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

Why do language models hallucinate and how can we prevent it? Can models develop genuine introspective capability, or only mimic it? Why do standard evaluation practices obscure safety-critical AI failures? What distinguishes genuine communicative competence from surface language performance? What gaps exist between benchmark performance and real deployment outcomes? How does diversity prevent model convergence on superficial patterns? Why do confident AI outputs mislead human trust calibration? Can persona profiles improve LLM prediction accuracy and consistency? How can agents discover and adapt to user preferences during conversation? How can emotionally responsive AI maintain reliability and healthy boundaries? How can we reduce inherent biases in LLM-based evaluation judges? What limits language model accuracy in evaluating ideas? Why does polished AI output gain credibility despite fundamental verifiability problems? How can we detect and account for LLM involvement in academic writing? Can external verification systems adequately replace learned reasoning in AI outputs? Can confidence signals reliably detect flawed reasoning in language models? Can base models hide emergent misalignment through alignment training? Can LLMs distinguish between linguistic form and semantic meaning? How do educators verify student capability when AI can produce indistinguishable work? When should retrieval systems decide to fetch new information? How effectively can test-time voting aggregate diverse reasoning samples? How do reward signal properties affect model reasoning and safety? Why do training associations persist despite contradictory contextual information? What prevents LLMs from applying their reasoning knowledge to improve outputs? Does chain-of-thought reasoning reveal how models actually think or merely imitate reasoning? How do training data quality and composition affect downstream model performance? Can artificial systems establish authority in domains requiring expert judgment? How do users confuse explanation quality with actual system accuracy? What authorization challenges emerge when agents coordinate across system boundaries? How can humans maintain effective oversight as AI systems scale? Do language models reason through disagreement or only accommodate it? Should models ask for clarification when facing ambiguous or under-specified information? How does fine-tuning trade off accuracy against reasoning quality? Can mechanistic interpretability methods reliably reveal what models actually know? How does awareness of evaluation context influence model behavior? How do reward models systematically fail to represent diverse human preferences? How do individually-safe actions create collectively-unsafe outcomes? Which reinforcement learning modifications most improve dialogue quality in language models? What capabilities differentiate diffusion from autoregressive language models? How do models learn from self-generated outputs without cascading failures? How does tokenization reshape what we value in intelligence? How do network effects and self-selection distort aggregated rating accuracy? What human oversight must AI research systems have? How should systems validate code that agents generate? Why does AI verification capability persistently exceed generation capability? How should we measure frontier AI models' cyber exploitation capabilities? What makes process supervision effective for training complex reasoning models? Should governance of agentic AI systems be runtime or design-time? How much of agent capability comes from harness versus the model itself? How should agents coordinate through shared persistent code artifacts? Can smaller specialized models match frontier models on key metrics? Can recurrent computation unlock reasoning capabilities that fixed-depth models cannot? How do curriculum design and feedback approaches affect model learning? How can evaluations be made robust against model reward hacking? Do single-axis benchmarks accurately measure agent capability for real deployment? Why do models reveal hidden associations despite concealment attempts? Can language models reliably simulate personas and predict behavior? Should GUI agents use structured screen representations instead of end-to-end vision? Why do retrieval-augmented generation systems fail in practice despite sound architecture? Do persona-based approaches introduce systematic biases in user simulation? Does reinforcement learning create genuinely new reasoning capabilities or only refine existing ones? How do agents learn to distinguish valuable feedback from noise? Does AI-assisted work increase total productivity or just shift time? How do real-world evaluations reveal AI capabilities that benchmarks hide? Does AI deployment reduce or exacerbate workplace inequality and income instability? How reliably can humans and AI detectors identify machine-generated text? Why do language models fail at sustained therapeutic relationships despite understanding techniques? Can AI systems discover fundamental improvements to their own architectures?

Related concepts in this collection 3

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
12 direct connections · 127 in 2-hop network ·dense cluster Open in graph ↗
Does setting temperature to zero actually make L… Does model confidence predict robustness to prompt… Can LLM judges be fooled by fake credentials and f… Why do preference models favor surface features ov…

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Search by related questions 4

Suggested questions this note speaks to — click to search the collection, or type your own.

How do calibration and reliability differ in LLM judge evaluations?What does McDonald's omega reveal about LLM judgment consistency?How does sampling variation relate to prompt sensitivity as reliability concerns?Can an LLM be well calibrated but still unreliable on single evaluations?
Original note title

deterministic LLM settings create fixed randomness not reliability — a single output remains one draw from the model's probability distribution