When readers think writing is too polished to be human, does that instinct work, or is it closer to chance?

This explores whether human readers and reviewers take polish as a sign that AI wrote something, and whether that instinct holds up. The short answer from the corpus: none of these notes tests 'polish' as a cue directly, but the surrounding evidence suggests that if people use it, it doesn't work. A 30-study review found that human detection of AI content across text, images and voice generally lands around chance, and hasn't improved as the models have Can people reliably spot content made by AI?. When readers do accuse someone of using AI, the accused comments often lack the features that actually mark AI text. That suggests the accusations work more as gatekeeping than as detection, and the people harmed are human writers whose work gets doubted Do unfounded AI accusations harm human writers instead?.

The odd part is that polish really is a signal. Humans just aren't the ones picking it up. Simple, transparent linguistic features identified LLM-written arguments on r/ChangeMyView with 99% accuracy. Part of what gave them away was 'textbook-quality' argument markers and a habit of mirroring the prompt, which real people rarely do Can simple linguistic features detect AI-written arguments?. In fiction, AI stories could be told apart by narrative choices alone, such as how characters act and how time is ordered, even after all the stylistic cues were removed Can AI stories be detected without analyzing writing style?. So the tells exist, but they are spread across many statistical regularities and deep structural choices. Neither is what a reader notices as 'this sounds too smooth.'

In peer review, polish looks less like a red flag and more like a reward. One of three fully AI-generated papers from Sakana AI cleared a double-blind ICLR workshop review before being withdrawn as planned. Its authors later found a citation error, and they judged none of the three good enough for the main conference Can AI-generated papers pass peer review undetected?. On the machine side, simply rewriting a paper's text with an LLM raised AI reviewers' scores by about 0.45 points without changing the science at all Can AI systems safely replace human peer reviewers?. Note that this second result is about AI reviewers, not humans. It does show how easily fluent surface writing can pass for substance.

What seems to drive human judgment is labels, not text. People rated identical passages 13.7 points higher when told a human wrote them, and AI judges showed an even stronger bias Do authorship labels bias how we judge literary quality?. Polish may also be the wrong thing to look for. AI assistance shifts how readers perceive a writer on every one of 29 social traits, making them seem more confident, more extreme and more 'high-quality' Does AI writing assistance change how readers perceive the writer?. Writers edit AI suggestions only 23% of the time, and lightly when they do, so that shifted persona reaches readers unchanged Do writers actually edit AI-generated text before publishing?. Readers may sense something is off without being able to name it. One argument holds that the 'aloofness' people notice in AI posts isn't excess polish at all. It's a missing appeal for the reader's attention, the bid that human writing makes as a basic part of communicating Does AI writing lack the internal appeal to attention that humans use?.

The takeaway you might not have expected: when people go looking for polish, they are looking in the wrong place. The real fingerprints are structural, and only machines read them reliably. What humans actually pick up on is a shift in persona and a missing sense of being addressed. Those are real effects, but people tend to misreport them as 'it's too well written.' A study that directly tests whether reviewers treat fluency as a sign of AI would fill a visible gap in this collection.