How do linguistic norms for expressing certainty vary across languages and models?
This explores two intertwined things: how the linguistic conventions for signaling confidence differ from one human language to another, and how language models develop their own characteristic register for expressing certainty — and what happens when the two meet.
This explores how certainty gets encoded in language — both the variation across human languages and the distinct, often miscalibrated register models adopt. The corpus has a sharper answer than you might expect, and it cuts across persuasion, calibration, and pragmatics research.
The most direct finding is also the most unsettling: confidence *is* expressed differently across languages, but it doesn't matter for how users behave. Cross-linguistic research shows that in every language studied, people track the model's confidence signals rather than its actual accuracy — so overconfident errors get followed systematically, worldwide Do users worldwide trust confident AI outputs even when wrong?. The linguistic packaging varies; the human deference to it doesn't. That reframes the whole question: the interesting variable isn't really the language, it's the register the model has learned to speak in.
And that register is not neutral. RLHF appears to install an assertive, conviction-loaded style — models express higher conviction than human persuaders, and that confidence-loading drives persuasive outcomes regardless of whether the claims are true or false Does linguistic conviction explain why LLMs persuade more effectively?. So the model's 'norm' for expressing certainty is partly a training artifact, a content-independent amplifier rather than an honest signal of how sure it should be. Same flavor of distortion shows up in moral language, where models lean ~22% harder on moral framing than humans do Do LLMs use moral language more than humans? — the model has acquired a louder rhetorical default than the people it learned from.
What about hedging — the linguistic markers ('might', 'possibly', 'I think') that are supposed to express *un*certainty? Here's the twist worth knowing: hedging markers cluster more densely in *incorrect* reasoning traces, not careful ones Do hedging markers actually signal careful thinking in AI?. So the model's uncertainty language is actually doing something — it leaks epistemic trouble — but it reads as caution rather than the distress signal it really is. Meanwhile the model can't flexibly modulate certainty to context the way humans do: it fails to adapt scalar implicature ('some' implying 'not all') to communicative stakes, applying the same inference whether the situation is casual or face-threatening Can language models adapt implicature to conversational context?. Human certainty norms are deeply pragmatic and audience-sensitive; the model's are flat.
The hopeful counter-thread is that this register is detachable from real calibration. Confidence and correctness *can* be re-coupled: small models trained with uncertainty-aware objectives learn to abstain when unsure and match models ten times larger Can models learn to abstain when uncertain about predictions?, and using the model's own answer-span confidence as a reward signal both sharpens reasoning and reverses RLHF's calibration damage Can model confidence work as a reward signal for reasoning?. There's even a structural tell — a model's confidence predicts how robust it is to having its prompt rephrased Does model confidence predict robustness to prompt changes?. The thing you didn't know you wanted to know: the gap isn't that models lack a certainty 'language,' it's that their fluent, RLHF-polished certainty register floated free of whether they're actually right — and the research suggests that decoupling is fixable, not fundamental.
Sources 8 notes
Cross-linguistic research shows users in every language trust confident AI outputs even when inaccurate. While confidence expression varies by language, users everywhere track confidence signals rather than accuracy, making overconfident errors systematically followed.
Linguistic analysis shows LLMs express higher conviction than human persuaders, and this confidence-loading directly correlates with persuasive outcomes regardless of whether claims are true or false. RLHF training installs an assertive register that functions as a content-independent persuasion amplifier.
Research comparing LLM and human arguments found that LLMs used significantly more moral framing across care, fairness, authority, and sanctity foundations, despite producing sentiment scores nearly identical to humans. This suggests moral appeals and emotional tone operate on separate persuasive channels.
Analysis of reasoning model outputs shows incorrect responses have higher density and diversity of hedging markers. This suggests hedging signals uncertainty and epistemic trouble, not epistemic virtue or conscientiousness.
ChatGPT shows no context-sensitivity in computing scalar implicatures across three dimensions: explicit literal-mode instructions, information structure focus, and face-threatening contexts. Humans flexibly modulate these inferences; the model does not, suggesting pragmatic competence requires tracking communicative stakes that LLMs systematically miss.
Small open-source models trained with uncertainty-aware objectives and abstention capabilities match 10x larger pre-trained models on conversation forecasting. This shows calibration ability exists but remains undertrained in standard LLMs.
RLSF uses answer-span confidence to rank reasoning traces, creating synthetic preferences that strengthen step-by-step reasoning while reversing RLHF's calibration degradation—without requiring human labels or external verifiers.
ProSA found that when models are highly confident, they resist prompt rephrasing; low confidence causes major output swings. Larger models, few-shot examples, and objective tasks all correlate with higher confidence and greater robustness.