Do users worldwide trust confident AI outputs even when wrong?

The cross-linguistic overreliance study shows that the well-documented tendency to over-trust confident LLM outputs is not an English-language or Western-cultural artifact. It is universal.

The LLM side: Models are cross-linguistically overconfident — they generate epistemic markers of certainty at higher rates than their accuracy warrants. But the pattern is linguistically sensitive: models produce the most markers of uncertainty in Japanese and the most markers of certainty in German and Mandarin. The models are tracking real linguistic norms for confidence expression across languages, but they are doing so while systematically overconfident in accuracy.

The user side: Users in all languages rely on confident outputs even when those outputs are wrong. The reliance rate varies cross-linguistically — Japanese users rely significantly more on expressions of uncertainty than English users (consistent with Japanese linguistic norms around face-saving and epistemic humility). But across all languages, confident LLM outputs produce higher user reliance, and overconfident errors are systematically followed.

The mechanism: users are tracking confidence signals, not accuracy signals. Confidence is legible (it comes encoded in language through epistemic markers); accuracy requires independent verification. In the absence of real-time accuracy feedback, users default to confidence as a proxy for reliability. This is a rational heuristic in human-human interaction where confidence often tracks expertise. It is a dangerous heuristic in human-LLM interaction where confidence is a trained linguistic behavior decoupled from epistemic calibration.

This extends Why do language models fail confidently in specialized domains? (which focused on model calibration) to the user behavior level — showing the practical consequence of model overconfidence: systematic user overreliance regardless of linguistic context.

A specific instantiation of overreliance harm comes from AI fact-checking. In a preregistered RCT, AI-generated fact checks did not improve participants' overall ability to discern headline accuracy. Worse, when users opted in to view AI fact checks, they became significantly more likely to share both true and false news — but only more likely to believe false news. Self-selection into AI assistance correlated with increased vulnerability, not decreased. The opt-in users represent a population that actively seeks AI judgment, making them the most susceptible to the confidence-over-accuracy heuristic. See Does AI fact-checking actually help people spot misinformation?.

Fluency activates a folk model of attention. A related but distinct overreliance mechanism: linguistic fluency leads users to read the AI as paying attention to them. In human-human interaction, competent contextual uptake is evidence of attentional presence — a person who responds coherently to what you said has been listening. Users import this inference into AI interaction, treating fluent response as evidence that the system is oriented toward them. Since When should AI systems choose to stay silent? frames when-to-speak design, this fluency/attention conflation is upstream of that question: users do not perceive the AI as a silent partner needing design-imposed speech rules because they already read the fluent AI as attentive. This is distinct from confidence-overreliance — it is not the epistemic-marker signal producing overtrust, but the fluency-signal producing an attribution of attention the AI does not have.

The cross-linguistic finding matters for deployment: LLM overreliance cannot be attributed to English-language user characteristics or Western technology cultures. The risk is embedded in the structure of confident language use, which operates wherever language is used.

Rose-Frame provides a compounding mechanism for overreliance: it identifies three cognitive traps that interact multiplicatively. Overreliance is specifically Trap 2 (mistaking fluency for understanding), which compounds with Trap 1 (treating outputs as ontological facts rather than probabilistic maps) and Trap 3 (confirmation bias from sycophantic outputs that never challenge the user). When all three co-occur, the result is "epistemic drift" — not isolated misjudgments but runaway misinterpretation where each trap reinforces the others. See Why do people trust AI outputs they shouldn't?.

Source: Philosophy Subjectivity