Do writers actually edit AI-generated text before publishing?
This research tests whether the "human-in-the-loop" safeguard against AI text quality issues actually works in practice. It examines how often writers revise AI-generated paragraphs and how substantially they change them.
A common reassurance about AI writing assistance is that humans remain in the loop — they will edit, correct, override. The persona-distortion study tested this assumption directly. Writers were given AI-generated paragraphs and asked to edit them until the text reflected their opinions to their satisfaction. The result: writers edited the AI-generated paragraphs only 23 percent of the time, and most edits were minor — median Levenshtein ratio of 0.96, meaning the edited text was 96 percent identical to the AI's original.
This finding has two implications. First, the standard "human-in-the-loop" defense against AI text quality concerns is empirically wrong at population scale. Editing is rare and shallow when it does occur. The AI's text is reaching its audience in nearly the form the model produced it. Second, this means the persona distortions documented in the same study — opinionated, confident, demographically privileged, emotionally compressed — propagate with minimal human modulation. The distortion is not filtered by the writer's revision; it is embraced or ignored.
This forecloses one common mitigation strategy: relying on the writer to detect and remove distortions before publication. The writer who would have caught and corrected the distortion is the same writer who, the study shows, mostly does not edit and mostly prefers the AI version even after being given the chance to edit it. The distortion arrives at the audience because the writer does not interrupt it. Any intervention that hopes to reduce AI's influence on public discourse cannot rely on the writer-as-gatekeeper assumption — that role, in practice, is not being performed.
Inquiring lines that read this note 186
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
How reliably can humans and AI detectors identify machine-generated text?- Can AI text detectors reliably identify AI-generated websites?
- Why can't algorithms distinguish between human and AI generated content quality?
- Can readers detect when text was written or heavily influenced by AI?
- Is statistical analysis the only reliable way to detect modern AI writing?
- Can token-level watermarks detect synthetic content better than stylometry alone?
- Can AI detectors reliably distinguish human from machine-generated text?
- Can a classifier distinguish machine-written text from poor human writing?
- Does the same rewriting that erases authorship also narrow measurable AI text markers?
- Can detection systems identify AI text rewritten to match human author style?
- Would detectors trained on unaltered AI text catch heavily rewritten versions?
- Can AI-rewritten text still be detected as machine-modified?
- Can readers distinguish machine-generated text from human-written comments?
- Can language models detect AI-generated text in blind evaluation tasks?
- Are AI systems trained to devalue content labeled as machine-generated?
- Can AI detectors confuse distinctive writing style for machine authorship?
- Can AI text detection improve enough to help evaluators make better decisions?
- How accurate are automated AI text detectors compared to human judgment?
- What makes AI-generated punditry different from human expert commentary online?
- Do AI-generated posts crowd out human voices without any coordination or intent?
- Do AI-generated summaries reduce user engagement with original content sites?
- Do AI-generated articles rank worse in Google Search than human-written ones?
- Do AI-generated articles receive less search traffic than human writing?
- Do mixed human-AI posts rank differently than fully generated content?
- Does length of post explain differences in AI rates across formats?
- Can volume-based moderation catch AI-generated top-level posts effectively?
- Do audiences penalize AI-written posts through visible callouts at scale?
- How do subreddit and author identity affect machine-generated text reception?
- Why is machine-generated text concentrated in certain Reddit communities?
- Does algorithmic adjustment of AI content exposure hold as supply grows beyond twelve months?
- How does lower organic traffic affect the economics of human-written web content?
- Can AI involvement in news discussion reduce perceived quality without reducing use?
- Can search performance data distinguish AI-generated content from human-written articles?
- What changes when published text was never written for its readers?
- How does the author-function itself change when AI replaces human authorship?
- Why does production time matter to the meaning of generated text?
- Does AI make writers appear more politically extreme to readers?
- Why does AI writing seem more competent and informative than human writing?
- Does AI writing make authors appear more privileged or educated?
- How does AI assistance affect perceived emotional tone in writing?
- Which reader-rated attributes converge most strongly when writers use AI?
- How does perceived writer confidence shift with AI-assisted composition?
- What structural difference exists between AI posts and human conversational writing?
- Does AI writing erase markers of non-native English speaker identity?
- Can demographic distortion in AI writing affect who appears credible in public discourse?
- When do readers defer to AI text without genuine processing?
- What makes readers treat AI-generated text as authoritative?
- What specific distortions does AI writing assistance introduce into text?
- What textual properties make AI writing feel polished and confident?
- Do AI writing models systematically change the tone or confidence of personal opinions?
- Do writers recognize when AI text misrepresents their actual stance?
- Does AI-assisted writing change how readers perceive the author's demographics or background?
- How do demographic and emotional compression relate to writing quality?
- Why might writers trust AI renderings of their views over their own words?
- What would it take for readers to inspect rather than assume authorship?
- What properties of natural text does artificial text actually eliminate?
- Why does AI-generated content feel flat compared to human commentary?
- Why do AI outputs lack the stable content of written sentences?
- What textual properties cause writers to prefer AI-rewritten versions of their text?
- How do AI rewrites systematically shift how writers appear across demographic dimensions?
- Does AI writing style remain distinct when content is masked or paraphrased?
- Why do human stories land in statistically rarer regions than AI narratives?
- How do changes in human and AI writing distributions shift rarity measures over time?
- Do fluent generated summaries carry false authority over expert judgment?
- Why do AI-inserted text and code suggestions survive at different rates?
- Can readers tell which parts of a document were AI-generated versus human-written?
- How do writers verify and revise AI-generated text before sharing it?
- What prose features actually separate AI text from human writing?
- How much do writers actually edit AI text before publishing it?
- How much do humans edit AI-generated text before publishing?
- Do human readers still recognize authors after heavy AI rewriting?
- Why does topic structure protect authorship signals from AI erasure?
- Does AI assistance distort how readers perceive writer identity and demographics?
- Does AI writing assistance distort a writer's authentic voice and persona?
- Did authors using AI write about different topics than others?
- Can readers actually distinguish AI text from human writing?
- Does engagement with machine text reflect quality or just stylistic acceptance?
- What prose features actually distinguish AI-generated text from human writing?
- Does the semantic weight of AI-written content matter more than sentence count?
- Can writers build AI literacy in readers through interface design choices?
- Why do people rate AI-written text as better than human writing?
- Do measurable differences exist between AI text and human writing?
- Does polished text presentation hide process-level authenticity from readers?
- How do AI tools change the relationship between writing effort and ability signaling?
- How much of AI-assisted comments remain the writer's own words?
- What observable quality dimensions distinguish slop from other forms of poor writing?
- Does AI assistance distort how readers perceive a writer's voice?
- Why does polished prose stop signaling merit once writing becomes easier?
- Can readers reliably distinguish AI-written abstracts from human-written ones?
- Does AI-written text score higher because of presentation alone or judgment shift?
- Do human reviewers detect rhetorical polish as a sign of AI authorship?
- What prose features distinguish automatically generated text from human writing?
- Does polished AI output mislead readers when experts are not directly supervising the writing?
- What evidence exists about writing skill distribution across populations?
- Does AI-generated writing feel polished while remaining harder to understand?
- How do writer preferences for AI output affect their willingness to edit it?
- What interventions beyond writer revision could reduce AI distortion in published content?
- Why do users prefer AI-polished versions of their own writing over originals?
- How do writers decide when to delegate work to AI versus doing it themselves?
- Can task framing influence whether writers experience genuine authorship during co-writing?
- What design changes could reduce unhelpful AI reliance in collaborative writing tools?
- How do authors decide which story components must stay under human control?
- Why does AI ideation benefit individual writers but harm the collective pool?
- Does non-directive framing help writers maintain ownership of ideas from AI suggestions?
- How often should proactive writing assistants interrupt without disrupting cognitive flow?
- What timing strategies work best for delivering AI writing suggestions?
- Do writers prefer configuring AI partners ahead of time or in the moment?
- What makes writers feel self-conscious about their prompts in collaborative spaces?
- Is user preference a reliable target for training AI writing assistants?
- Do writers edit AI assistance enough to fool content filters?
- Can explanation-based AI safeguards work in real-time writing interfaces?
- Does AI writing assistance make different authors sound more alike?
- Does asking AI to preserve voice recover lost authorship signals?
- Can proofreading tools preserve writer voice better than full rewriting features?
- Why do writers hesitate to disclose when they used AI tools?
- Can collaboration with GenAI preserve long-term skill development in writing work?
- How do writers' perceptions of productivity compare to their actual output quality?
- Do writers who own AI output rely more heavily on its suggestions?
- Do writers benefit when they make their AI prompting activity visible to collaborators?
- When AI becomes invisible in writing tools, do writers stop disclosing it?
- Does ownership of AI text lead users to rely more on suggestions?
- Do professional writing services already disadvantage applicants without access to editing help?
- Why do people withhold AI credit even when using personalized text generation?
- What tools or practices help people disclose AI use in their writing?
- How do attribution norms for human ghostwriters compare to AI usage patterns?
- How does reliance on AI change when writers own the final product?
- Does editing AI drafts build skill or replace skill-building practice?
- Does transparency about AI use change how audiences trust the writing?
- How do we discount AI-generated text when we lack cultural literacy for it?
- Does knowing an AI wrote something make people scrutinize it more critically?
- Can disclosure of AI involvement change how evaluators score writing quality?
- How does disclosure of AI involvement change across private versus public writing contexts?
- Does the 'feel of AI' in unedited posts trigger audience backlash and detection?
- Does writer credibility suffer when readers suspect AI involvement?
- Why does the disclosure penalty still hold even when readers have high AI literacy?
- What explains writers' concern that AI disclosure reduces their competence perception?
- What makes readers suspect AI involvement in academic writing they evaluate?
- How much does knowing about AI use actually change how readers judge text?
- Can readers detect AI involvement in writing when not explicitly told?
- Does directly copying AI text into writing change disclosure expectations?
- What reliable traces do generative processes actually leave in finished text?
- Can we verify fabricated text without redesigning the generation process?
- Can marking AI provenance solve the grounding problem for generated text?
- How do citation errors in AI-generated papers differ from human hallucinations?
- Do surface phrases reliably identify unedited machine-generated scholarship?
- How often do AI book summaries fabricate details when spot-checks are random?
- How often do fabricated sources in AI output escape citation checking?
- Can human researchers verify automated research methods before they become uninterpretable?
- Why should AI research prompts be subject to peer review before use?
- How do closed-loop automated venues differ from human-in-the-loop review taxonomies?
- Should rhetorical polish in AI reviews be separated from actual technical accuracy?
- Can human reviewers detect when papers have been rewritten by AI?
- Do AI reviews depend more on writing style than scientific merit?
- How often do researchers violate rules about AI use in review?
- Can polished AI text fool both reviewers and detection methods?
- How often do researchers suspect peer reviews are written by AI?
- Does AI content in reviews correlate with differences in paper quality control?
- Can human reviewers reliably detect AI-written peer review text by sight?
- Can machine review catch flaws in AI-generated work that humans miss?
- How often do AI systems produce papers with undetected factual errors?
- Which feedback loops in AI-mediated review remain unmeasured or rarely observed directly?
- Can humans reliably detect whether research text was written by AI?
- How often do journal editors catch obvious textual problems before publication?
- What percentage of workplace communication now contains AI-generated content?
- How often does LinkedIn wrongly flag legitimate posts as AI-generated?
- Does AI-assisted writing dilute the conversational value of social media?
- Does detecting AI authorship actually improve social media feed quality?
- How much of LinkedIn's feed is genuinely AI-generated versus human-written content?
- Do AI-generated posts get more engagement than human-written ones?
- How much do edited AI responses versus raw outputs affect clinician ratings?
- Why did primary care physicians review only 73% of AI-generated transcripts?
- Do expert physicians also prefer AI-written medical text when it is unlabeled?
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Measuring and Mitigating Persona Distortions from AI Writing Assistance
- Understanding Reader Perception Shifts upon Disclosure of AI Authorship
- Penalizing Transparency? How AI Disclosure and Author Demographics Shape Human and AI Judgments About Writing
- Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews
- What Influences Readers' and Writers' Perceived Necessity of AI Disclosure?
- "It was 80% me, 20% AI": Seeking Authenticity in Co-Writing with Large Language Models
- The AI Ghostwriter Effect: When Users Do Not Perceive Ownership of AI-Generated Text But Self-Declare as Authors
- Is it Cake or is it AI? A Systematic Review of Human Uncertainty in Distinguishing Generative Artificial Intelligence Content
Original note title
The 23 percent edit rate of AI writing assistance establishes that distortions reach audiences in nearly unedited form