Why do moderately represented cultures show more flattening than data-poor cultures?
This explores why cultures with a middling amount of training data get assimilated into dominant cultural defaults more than the truly data-scarce ones — a counterintuitive 'just enough data to be confidently wrong' effect.
This explores why moderately-represented cultures get flattened harder than data-poor ones — and the corpus has a sharp mechanistic answer for *how* flattening works, even though the specific moderate-vs-poor comparison has to be assembled laterally rather than read off a single note. The anchor finding is that LLMs don't just produce flattened outputs — they internalize cultural flattening as an architectural pathway: low-resource cultures like Ethiopia and Algeria are *represented through* high-resource cultural proxies inside the model's internal states, not merely in what it says Do LLMs represent low-resource cultures through dominant cultural proxies?. Flattening is routing: a culture gets mapped onto its nearest dominant neighbor.
That routing picture suggests why the *amount* of data matters non-linearly. Work on ideological representation shows that depth of representation scales with feature richness — models can differ by up to 7.3× in how many distinct features they devote to a topic, and richer representations are both harder to steer and more internally consistent Can we measure how deeply models represent political ideology?. Read against the flattening result, a moderately-represented culture is exactly the case with *enough* signal for the model to confidently locate it — but not enough to give it its own dense feature set. So it gets interpolated onto the closest high-resource attractor and held there with confidence. A data-poor culture, by contrast, may be too sparse to confidently assimilate at all; the model has less to over-generalize from, so paradoxically it imposes less of a wrong-but-confident proxy.
The norm-prediction work sharpens the 'confidently wrong' part. Frontier models predict social appropriateness better than any individual human, yet *all of them share identical systematic errors* on unwritten norms Can AI systems learn social norms without embodied experience? Can AI learn social norms better than humans?. That's the signature of statistical pattern-matching that has mastered the dominant distribution and then applies it everywhere — competence at the center, identical blind spots at the margins. A related note makes the deeper point: models achieve top-percentile statistical performance while having no actual cultural participation or meaning-making Why do AI systems fail at social and cultural interpretation?. Flattening isn't a knowledge gap the model knows it has; it's a confident projection from the center outward.
And the cost lands hardest because users don't catch it. Across every language tested, people track an AI's *confidence* signals rather than its accuracy, and systematically over-rely on confident outputs even when wrong Do users worldwide trust confident AI outputs even when wrong?. A moderately-represented culture that gets fluently but wrongly rendered through a dominant proxy produces exactly the kind of confident, plausible output that users won't flag — whereas a data-poor culture more likely triggers visible hedging or refusal. The thing you didn't know you wanted to know: flattening may be worst not where the model knows least, but where it knows *just enough to stop asking*.
Sources 6 notes
Mechanistic interpretability analysis reveals that low-resource cultures like Ethiopia and Algeria are structurally represented through high-resource cultural proxies in internal model states, not just output. This architectural bias persists even when models can produce correct surface-level answers.
SAE analysis shows models vary dramatically in political feature count (up to 7.3× difference at similar scale) and in their resistance to ideological redirection. Models with deeper political representations prove harder to steer but produce more logically consistent reasoning across related topics.
GPT-4.5 predicted appropriateness of 555 social scenarios at the 100th percentile compared to human raters, with Gemini and Claude also exceeding 96% accuracy. However, all models show identical systematic errors, revealing boundaries of pattern-based social understanding that embodied experience may still be necessary to cross.
GPT-4.5 outperformed every individual human at judging social appropriateness across 555 scenarios, challenging the theory that embodied cultural experience is necessary. However, all AI models share identical systematic errors on unwritten norms.
LLMs achieve 100th-percentile performance on norm prediction yet regress on theory-of-mind tasks and cannot generate culturally-resonant interpretations. The pattern shows that statistical competence coexists with absence of actual social understanding and participation.
Cross-linguistic research shows users in every language trust confident AI outputs even when inaccurate. While confidence expression varies by language, users everywhere track confidence signals rather than accuracy, making overconfident errors systematically followed.