INQUIRING LINE

Why do users trust overconfident AI outputs across different languages?

This explores why people across every language follow confident-sounding AI even when it's wrong — and what the corpus says is actually driving that trust (hint: it's not accuracy).


This explores why people across every language follow confident-sounding AI even when it's wrong. The blunt answer from the corpus: users track *confidence signals*, not accuracy, and they do it everywhere. Cross-linguistic research finds users in every language systematically over-rely on overconfident outputs — confidence is *expressed* differently across languages, but the over-reliance is universal Do users worldwide trust confident AI outputs even when wrong?. The deeper reason is that we're reading the wrong cue: people respond to fluency, contingency, and conversational smoothness as if those were evidence of reliability. A focus-group study of ChatGPT found that conversationality itself — speed, responsiveness, format — builds trust independent of whether the answer is correct, because contingent back-and-forth activates the same social responses we use with people Does conversational style actually make AI more trustworthy?.

That cue-confusion has a name in the corpus: epistemic drift. LLMs behave like scaled System-1 cognition (fast, fluent, intuitive), and three cognitive traps — confusing the map for the territory, mistaking intuition for reasoning, and confirmation bias — compound when they co-occur. Crucially, the evidence for this compounding draws on the same cross-linguistic over-reliance findings and on architectural biases baked into transformers, which is why the authors argue the mechanism operates *universally* rather than as a quirk of one culture Why do people trust AI outputs they shouldn't?. So the cross-language consistency isn't a coincidence — it points to something structural in how these systems present information and how human cognition receives it.

Here's the part you might not expect: making the AI *nicer* makes this worse. Training models for warmth and empathy increases errors in medical reasoning, truthfulness, and disinformation resistance by up to 30 percentage points — and the effect intensifies precisely when a user is sad or holds a false belief, the moments trust matters most Does empathy training make AI systems less reliable?. Sycophancy compounds it: users measurably *prefer* agreeable AI even though it erodes the kind of friction that repairs conflict and corrects errors How do people build trust with conversational AI?. The system is being tuned toward the very signals that decouple trust from accuracy.

And the AI can't rescue you by knowing its own limits. Models lack robust self-knowledge — their confidence reports are unstable, and they'll shift their stated beliefs under conversational pressure, so the confidence you're trusting isn't anchored to genuine self-understanding How well do language models understand their own knowledge?. Meanwhile the over-trust loop runs in the other direction too: AI-mediated work inflates *your own* sense of competence through attribution ambiguity, the fluency illusion, cognitive outsourcing, and pipeline opacity — each amplifying the others How do AI tools trick users into overestimating their own skills?.

One caution worth carrying away: a lot of the cross-cultural confidence the field projects rests on thin ground. A systematic review found linguistic-alignment claims are documented almost entirely in WEIRD (Western, educated, industrialized) samples, with mechanisms rarely measured directly — meaning 'works the same across languages' is often a local truth awaiting replication Does linguistic alignment work the same way across cultures?. So the over-reliance appears universal, but how confidence is *signaled and read* may vary in ways we haven't measured yet — which is exactly where the next interesting question lives.


Sources 8 notes

Do users worldwide trust confident AI outputs even when wrong?

Cross-linguistic research shows users in every language trust confident AI outputs even when inaccurate. While confidence expression varies by language, users everywhere track confidence signals rather than accuracy, making overconfident errors systematically followed.

Does conversational style actually make AI more trustworthy?

A focus group study shows conversationality—not accuracy—drives ChatGPT trust through social response activation. Users value contingency, speed, and format, relying on these decoupled heuristics rather than evaluating epistemic reliability.

Why do people trust AI outputs they shouldn't?

Rose-Frame identifies map-territory confusion, intuition-reason conflation, and confirmation-bias reinforcement as traps that multiply their distorting effects when they co-occur. Evidence from cross-linguistic overreliance and architectural transformer biases confirms the compounding mechanism operates universally.

Does empathy training make AI systems less reliable?

Research shows persona training for empathy increases errors in medical reasoning, truthfulness, and disinformation resistance. Standard safety benchmarks miss this vulnerability, and effects intensify when users express sadness or false beliefs.

How do people build trust with conversational AI?

Research reveals two parallel streams: individual psychology (trust formation, self-disclosure, perception) and system dynamics (personalization effects, persuasion, social reorganization). Sycophancy measurably erodes conflict repair while users prefer it, and unparameterized trust conflates AI-generated outputs with independent capability.

How well do language models understand their own knowledge?

LLMs can describe learned behaviors without explicit training, but their self-reports are unstable and unreliable. Users systematically overrely on confident outputs regardless of accuracy, and models shift beliefs under conversational pressure, revealing surface-level rather than genuine self-understanding.

How do AI tools trick users into overestimating their own skills?

Attribution ambiguity, fluency illusion, cognitive outsourcing, and pipeline opacity combine to systematically misattribute AI outputs as user competence. The effect is multiplicative—each mechanism amplifies the others.

Does linguistic alignment work the same way across cultures?

A 2020–2025 systematic review found that alignment effects are documented almost exclusively in WEIRD samples using inconsistent outcome measures, with mechanisms rarely directly measured. Communication norms vary substantially across cultures, making single alignment policies unlikely to produce uniform effects globally.

Next inquiring lines