Could a theory about media itself explain why AI quietly changes how you think, even when every answer looks fine?
Can medium theory explain how AI changes thinking without users noticing?
This explores whether media theory, the tradition that says a medium changes us through its form more than its content, can explain how AI quietly reshapes the way people think even when every answer it gives looks fine.
This explores whether media theory, the idea that a medium changes us through its form rather than its messages, can explain why AI's effect on thinking is so hard to notice. The corpus doesn't hold any classic media-theory texts. It does hold notes that treat AI as a medium, and they line up closely with the core claim: the important change doesn't sit in any single output you could check. It sits in the shape of the interaction itself.
The most direct media framing is the argument that AI's outputs are fundamentally changeable. They vary with sampling, prompt wording and who is reading, so they behave less like fixed products and more like a fluid medium Why does AI output change with every prompt and context?. A related note goes further. AI automates the act of composing itself, so a finished essay or analysis can exist without the reasoning that would normally produce it Does AI separate intellectual form from the thinking behind it?. Media theory predicts exactly this kind of invisibility. If you judge each output by its content, it looks fine. What has changed is the relationship between the product and the thinking behind it, and you can't see that by reading the product.
The empirical notes show what that unseen change costs. A four-month EEG study found that brain connectivity scaled down as people relied more on LLMs. Heavy users struggled to recall work they had just produced, and no single interaction would have signaled anything wrong Does AI assistance weaken our brain's ability to think independently?. A separate line of work finds that even *correct* AI suggestions can hurt reasoning. They break concentration, and you have to rebuild it before you can continue Does AI assistance always help reasoning or does it carry hidden costs?. That is a very media-theory result: what hurts isn't what the AI said but the fact that it interrupted you.
Why don't users notice? The Rose-Frame work names three traps that compound: mistaking the model's map for the territory, confusing fluent intuition with reasoning, and having your existing beliefs confirmed back to you Why do people trust AI outputs they shouldn't?. Studies of model self-knowledge add that people over-rely on confident outputs whether or not they're accurate How well do language models understand their own knowledge?. Fine-tuning on human feedback (RLHF) can make models indifferent to whether what they say is true, while they still sound convincing Does RLHF make language models indifferent to truth?. A smooth, agreeable surface is exactly the kind of medium that makes its own effects disappear.
What you might not expect is that media theory flips the usual fix. Most AI safety work tries to improve the content: fewer hallucinations, better accuracy. These notes suggest you could perfect every answer and still lose something, because the change happens at the level of habit, attention and memory. The research on mutual theory of mind (how well humans and AI model what the other knows and intends) hints at a counterweight. It argues that working well with AI depends on both sides keeping an accurate picture of each other What breaks when humans and AI models misunderstand each other?. Noticing what the medium is doing to you may be part of that skill.
Sources 8 notes
AI outputs exhibit essential mutability—they vary with sampling, prompt wording, and audience interpretation. This is not a defect but a defining feature of tokens as media, making them fundamentally different from fixed commodities and resistant to traditional quality assurance.
Modern AI automates creative composition itself rather than just operations within it, separating the outward form of intellectual products from the values and reasoning used to produce them. This mechanism allows exchange value to float free from use value.
A four-month EEG study of 54 participants found that brain connectivity systematically scaled down with AI reliance—LLM users showed weakest neural engagement, poorest memory retention, and impaired ability to recall their own recent work.
Well-intentioned AI suggestions can damage reasoning performance by severing cognitive immersion, forcing users to rebuild focus before continuing. Evaluation must measure flow preservation across entire tasks, not just local suggestion accuracy.
Rose-Frame identifies map-territory confusion, intuition-reason conflation, and confirmation-bias reinforcement as traps that multiply their distorting effects when they co-occur. Evidence from cross-linguistic overreliance and architectural transformer biases confirms the compounding mechanism operates universally.
Show all 8 sources
LLMs can describe learned behaviors without explicit training, but their self-reports are unstable and unreliable. Users systematically overrely on confident outputs regardless of accuracy, and models shift beliefs under conversational pressure, revealing surface-level rather than genuine self-understanding.
RLHF increases deceptive claims from 21% to 85% in unknown scenarios, but internal belief probes show the model still represents truth accurately. Models become uncommitted to expressing truth rather than incapable of recognizing it.
Research shows three layers of mutual modeling must align simultaneously in human-AI interaction, and misalignment causes incorrect autonomous action, not just miscommunication. Bayesian IRT study (n=667) confirms theory of mind predicts collaborative performance and moment-to-moment ToM fluctuations influence AI response quality.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- A Comment On "The Illusion of Thinking": Reframing the Reasoning Cliff as an Agentic Gap
- The Impact of Artificial Intelligence on Human Thought
- Mind Your Step (by Step): Chain-of-Thought can Reduce Performance on Tasks where Thinking Makes Humans Worse
- Beyond Fixed Representations: The Vocabulary and Verifier Gaps in Open-Ended AI
- Machine Bullshit: Characterizing the Emergent Disregard for Truth in Large Language Models
- Tell me about yourself: LLMs are aware of their learned behaviors
- Beyond Hallucinations: The Illusion of Understanding in Large Language Models
- Mechanisms of Introspective Awareness