When readers call a post 'AI slop,' are they actually spotting machine-written text, or enforcing the community's norms?

This explores whether readers flagging posts as 'AI slop' could catch machine-generated text as well as automated detectors do. The corpus gives a blunt answer, and it changes the question. A matched-control study of 25 million Hacker News and Reddit comments found that the writing features that actually separate AI text from human text do not predict which comments get accused of being slop Do AI slop accusations actually detect AI text?. The flags are not a weak detector. They are measuring something else: whether a comment broke the community's norms about tone, effort or belonging. A slop accusation works as social gatekeeping, not as screening.

This matches a wider finding about human perception. A review of 30 studies found that people's ability to tell AI-made text, images and voices from human-made ones generally sits around chance, and it has not improved as AI output has become more realistic Can people reliably spot content made by AI?. Adding up many flags doesn't fix this. If each person is guessing, a crowd of guessers mostly amplifies shared prejudices, such as distrust of polished prose, certain phrasings or outsiders, rather than revealing who or what actually wrote the text. One limit on the corpus: it has no head-to-head benchmark of user flags against commercial AI detectors, so it can't settle that horse race directly.

What the corpus does suggest is that both flags and detectors may be aimed at the wrong target. 'Slop' usually means low-value content, and authorship is only a rough stand-in for that. Research on evaluating AI systems keeps finding the same pattern: judgments based on surface impressions drift, and judgments based on evidence hold steady. An agent that actively gathers evidence before judging cut evaluation inconsistency roughly a hundredfold compared with a model judging the output directly Can agents evaluate AI outputs more reliably than language models?. Benchmark designers are moving the same way, from a single score toward checkable records of what actually happened Can infrastructure evidence replace terminal scores in benchmark validation?. And splitting a vague quality judgment into specific yes-or-no criteria makes it harder to game with surface tricks Can breaking down instructions into checklists improve AI reward signals?.

Applied to flagging, those findings suggest a different design. A button that asks 'Is this AI?' collects guesses. A flag that asks 'Does this make a claim it doesn't support?' or 'Does this repeat the post above it?' collects observations a reader can actually check. Separate work on monitoring AI agents shows that small, inexpensive monitors trained on carefully filtered reasoning from stronger models can outperform large models that are only prompted Can small models detect scheming by watching actions alone?. Structured human flags with stated reasons could become that kind of training material, while raw 'this is slop' votes would teach a system the community's biases.

The takeaway the question doesn't anticipate is this: user flags are useful information about what a community will tolerate, and that has value of its own. But because they track social belonging rather than how the text was made, treating them as AI detection risks penalizing people who write in an unfamiliar register, while fluent machine text that fits in goes unflagged.