Why do AI outputs lack the stable content of written sentences?
This explores why AI text doesn't hold a fixed meaning the way a written sentence does — why the 'same' output can shift, vary, or fail to commit to a stable claim.
This explores why AI text doesn't hold a fixed meaning the way a written sentence does — why the 'same' output can shift, vary, or fail to commit to a stable claim. The corpus traces this to something deeper than quality: the tokens themselves are mutable by design. AI outputs vary with sampling, prompt wording, and audience — not as a bug but as a defining property of tokens as a medium, which makes them resistant to the kind of fixity we expect from a printed line Why does AI output change with every prompt and context?. A written sentence commits; a generated one is drawn fresh from a distribution each time.
The sharpest demonstration is the regeneration test: ask the same question twice and you get different answers, each internally consistent with the prior context but none representing a fixed commitment. The model holds a superposition of possible characters and samples one at generation time rather than 'being' any of them Do large language models actually commit to a single character?. Stable content requires a speaker who has settled on something. Here there is no settling — only sampling.
There's also a missing-author problem. A written sentence is an utterance: it has an event behind it, someone who meant it. AI produces what one note calls 'event residue' — communicative markers inherited from training data but without the event structure that produces a genuine utterance. The reader supplies the missing orientation, animating the residue into a pseudo-exchange that has structure only on the human side Does AI generate genuine utterances or just text patterns?. Related work frames this as the structural elimination of properties natural text takes for granted — dialogic symmetry, embodied authorship, situatedness — absences rather than surface flaws Does AI-generated text lose core properties of human writing?.
Even where the grammar is flawless, the content stays inert. LLMs master sentence structure but avoid evaluative stance-taking: they lean on descriptively neutral phrasing instead of the status and evidential moves human writers use to commit to a position. The result is organizationally coherent but argumentatively weightless — coherent surface, no settled claim underneath Why does AI writing sound generic despite being grammatically correct?. And the instability runs all the way down to the computation: in some models the 'real' reasoning is computed in early layers, then overwritten in later layers to produce format-compliant filler, so the visible text isn't even a faithful trace of what the model worked out Do transformers hide reasoning before producing filler tokens?.
What you didn't know you wanted to know: this instability mostly doesn't get corrected before it reaches you. Writers edit AI paragraphs only about 23% of the time, and even those edits stay 96% similar to the original — so the unanchored voice propagates almost untouched Do writers actually edit AI-generated text before publishing?. Combine that with the finding that people everywhere track an output's confidence rather than its accuracy Do users worldwide trust confident AI outputs even when wrong?, and the picture sharpens: text that lacks stable content is still received as if it had it.
Sources 8 notes
AI outputs exhibit essential mutability—they vary with sampling, prompt wording, and audience interpretation. This is not a defect but a defining feature of tokens as media, making them fundamentally different from fixed commodities and resistant to traditional quality assurance.
Shanahan's 20-questions test shows LLMs maintain a superposition of consistent objects or characters and sample from that distribution at generation time. Regenerating the same response yields different outputs, each consistent with prior context, proving no fixed commitment exists.
AI output carries communicative markers inherited from training data but lacks the event structure that produces actual utterances. Users supply the missing orientation through interpretive labor, creating a pseudo-event with structure only on the human side.
Research shows artificial text disrupts dialogic symmetry, context continuity, embodied authorship, and political situatedness. These are not surface flaws but structural absences—AI hotel reviews show 80%+ detection accuracy due to inherent falsity about personal experience distinct from human deception.
AI text uses manner nouns and anaphoric references that are descriptively neutral, while human writers use status and evidential nouns that carry evaluative weight. This produces organizationally coherent but argumentatively inert prose.
Logit lens analysis shows models trained with hidden CoT tokens compute correct answers in layers 1-3, then actively suppress these representations in final layers to produce format-compliant filler output. The reasoning is fully recoverable from lower-ranked token predictions.
Writers edited AI-generated paragraphs only 23% of the time, with edits averaging 96% similarity to the original. This means AI's opinionated and distorted voice propagates with minimal human filtering before publication.
Cross-linguistic research shows users in every language trust confident AI outputs even when inaccurate. While confidence expression varies by language, users everywhere track confidence signals rather than accuracy, making overconfident errors systematically followed.