Can you reliably tell AI-written research abstracts from human ones, when ML experts in one study mostly couldn't?

This explores whether tools and people can reliably tell AI-generated text from human writing in research settings. The short answer from this collection: humans mostly can't, a purpose-built classifier did well in one narrow setting, and the corpus doesn't directly test the off-the-shelf detectors that journals and universities actually use. That last gap matters, so treat what follows as partial evidence rather than a verdict.

The clearest research-setting result is about people, not software. When readers with machine-learning expertise were shown research abstracts written by humans, by LLMs, or by humans with LLM editing, they couldn't reliably sort them. They tended to assume a human was involved in all of them Can readers tell LLM abstracts from human ones?. The surprising part comes next: the LLM-edited abstracts got the highest clarity ratings and were preferred 55% of the time even when authorship was disclosed. So the hard case for detection is human-AI blends, which is also the most common kind of AI use in research. It's not clear what a detector should even flag there.

On the software side, the strongest evidence comes from outside academia. Simple, interpretable linguistic features reached 99% accuracy at spotting LLM-written counter-arguments on Reddit's r/ChangeMyView, matching heavier neural detectors Can simple linguistic features detect AI-written arguments?. The tells were telling: LLMs echo the prompt's framing and produce polished, textbook-style argument markers that humans rarely bother with. That suggests detection works best when the genre is narrow, the text is fully machine-written, and the classifier was trained on that exact setting. Research prose breaks all three conditions. Academic writing is already formal and textbook-like, so the stylistic gap that made Reddit arguments easy to catch largely disappears.

A lateral thread is worth following: AI evaluators are also easy to fool with surface signals. LLM judges reliably fall for fake references and rich formatting, and these attacks need no access to the model at all Can LLM judges be fooled by fake credentials and formatting?. Human users show the same pattern, trusting answers with more citations almost as much when the citations are irrelevant Do users trust citations more when there are simply more of them?. Detection is one instance of a broader problem. Both people and models judge text by style and credibility cues, and those cues are exactly what generators are good at producing. Any detector built on style signals should be expected to be evadable.

The question you might not have known to ask is whether detection is the right goal at all. One line of work suggests a different approach: instead of trying to catch AI text, treat LLM output as a draw from the model's learned assumptions rather than as real evidence, and weight it explicitly Should we treat LLM outputs as real empirical data?. In research, the real risk usually isn't who typed the sentences. It's whether the claims, data, and citations are grounded. That is something you can check even when authorship can't be determined.