Different AI models handle the same political topic very differently — but not mainly in which side they take.
How do different LLMs treat the same political topic differently?
This explores whether different AI models, given the same political subject, actually behave differently. That could mean what they say, how willing they are to engage, and how firmly they hold a view. It also asks where those differences come from.
This explores whether different AI models handle the same political subject differently, and why. The corpus has less than you might expect on models taking different sides. It has more on a less obvious point: the biggest differences between models are in how deep their political understanding goes, how willing they are to engage, and how easily they can be pushed. The left-versus-right slant they show gets far less attention here.
Start with what is going on inside the models. One study looked directly at the internal concepts models use to represent politics. Two models of similar size can differ by as much as 7× in how many distinct political concepts they hold Can we measure how deeply models represent political ideology?. That difference has effects. Models with richer political understanding are harder to steer toward a different ideology, and they reason more consistently across related issues. Models with thinner understanding are easier to push around. So two assistants can give similar-sounding answers about immigration while one has a much more fragile hold on the topic than the other.
The second difference is whether a model engages at all, and here companies' choices matter more than ability. An audit of 7,500 conversations across six assistants found that every one of them readily went along with users on a low-stakes control topic but held back on political ones Do LLM refusals reflect policy choices or capability limits?. They plainly have the ability, so the restraint is a policy decision. Those rules also change depending on the topic and on who the user appears to be. This matches a broader finding that a model's refusals and tone reflect fixed values the company set during training. They are not judgments made about the conversation in front of it Can language models balance competing ethical norms in context?. In practice, much of the difference you notice between assistants on a political question comes from each company's rules about engagement.
The third difference is in effect rather than content. A meta-analysis found that the model family, the conversation format, and the topic area together explained about 82% of the variation in how persuasive AI is across studies. In those studies, GPT-4 consistently persuaded more than Claude 3.x What combination of factors explains differences in LLM persuasiveness?. So the same political argument can move readers by different amounts depending on which model wrote it.
The twist is that a single model may not have one stable position to compare against another's. Models tend to follow whatever argument the user is building rather than defending a view of their own Do LLMs actually hold stable positions or just mirror user arguments?. They also contradict themselves on morally identical scenarios that are simply worded differently, up to 78% of the time Do LLMs apply ethical principles consistently across reframed scenarios?. Across many debate topics, models as a group also cover only about half of the distinct arguments humans make Do language models flatten the range of public arguments?. Models may differ from each other, but together they also narrow political debate in a way no single comparison would show. What the corpus doesn't have is a direct test of several models stating their own positions on the same issue. For that question, these notes give you the structure around it, but not the scoreboard.
Sources 7 notes
SAE analysis shows models vary dramatically in political feature count (up to 7.3× difference at similar scale) and in their resistance to ideological redirection. Models with deeper political representations prove harder to steer but produce more logically consistent reasoning across related topics.
A 7,500-conversation audit across six assistants found all systems readily accommodate users on a low-stakes control topic, while refusing on political topics. This proves the capability exists; what varies is policy-driven engagement rules conditional on topic and user identity.
LLMs cannot perform the situated trade-offs that human pragmatic competence requires. Their ethical principles are structural defaults set at training time, not negotiable moves adapted to context, creating a gap between ethical adherence and communicative appropriateness.
A meta-analysis joint model combining LLM architecture, one-shot versus multi-turn format, and topic domain explained R² = 81.93% of between-study variance. Interactive multi-turn designs and GPT-4 consistently outperformed one-shot formats and Claude 3.x.
Language models generate outputs that match the trajectory implied by each prompt, rather than maintaining stable stances across interactions. This shape-holding is distinct from position-holding: the model produces argument-like text shaped by user framing, not from any underlying commitment being defended.
Show all 7 sources
GPT, Mistral, and Llama produce contradictory responses to morally equivalent scenarios reframed in different ways, with contradiction rates reaching 78% even when the ethical school and underlying situation remain fixed. This suggests stated ethical principles are not stably applied.
Across 23,384 LLM essays on debates, models recover only half of distinct human arguments and reuse hedged sub-arguments—a gap in argumentative structure, not just prose style. Diversity prompting adds noise outside human argument space rather than filling the long tail.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Do LLMs Change Their Minds Like Humans? Diagnosing Human--LLM Divergence in Single-Turn Persuasion Judgments
- Large Language Models Reflect the Ideology of their Creators
- Evaluation Awareness in Language Models Has Limited Effect on Behaviour
- Beyond the Surface: Probing the Ideological Depth of Large Language Models
- Argument Collapse: LLMs Flatten Long-Form Public Debate
- Incoherent by Design? On the Moral Self-Consistency of LLMs
- ChatGPT: towards AI subjectivity
- Conversational Alignment with Artificial Intelligence in Context