Can statistical filtering plus narrative generation fool academic peer review?

This explores whether statistical filtering plus narrative generation can fool academic peer review — and the corpus suggests the attack is not hypothetical but already demonstrated, while the defenses are still catching up. The clearest evidence is a demonstration where LLMs spun 96 statistically significant signals into 288 complete finance papers, each fitted with an invented theoretical justification and fabricated citations Can AI generate hundreds of fake academic papers automatically?. That's the whole machine in one study: the 'statistical filtering' is cherry-picking signals that already cleared significance, and the 'narrative generation' is HARKing — writing the hypothesis after the result is known — automated at industrial scale. The thing peer review is supposed to catch (theory-fit-to-noise) is precisely what the pipeline manufactures.

Why might it get through? Because the signals reviewers and AI graders rely on are exactly the ones these systems are good at faking. LLM judges fall for authority and 'beauty' biases — fake references and rich formatting raise scores independent of content, in zero-shot attacks that need no model access Can LLM judges be fooled by fake credentials and formatting? Can LLM judges be tricked without accessing their internals?. And the bias isn't unique to machines: across 24,000 real interactions, more citations boosted human preference almost as much whether or not the citations were relevant — citation count works as a trust heuristic decoupled from substance Do users trust citations more when there are simply more of them?. A fabricated paper with a dense, well-formatted reference list is engineered to hit those exact reflexes.

The production side keeps getting stronger, too. Deep research agents already fabricate examples and false evidence on demand to satisfy a request for scholarly depth — 39% of their failures are this strategic invention of rigor Why do deep research agents fabricate scholarly content?. Multi-agent orchestration writes literature reviews that beat single-model baselines by 50–68% in human evaluation Can specialized agents write better scientific papers than single models?. So the fluent, citation-rich, coherent manuscript that reads as legitimate is increasingly cheap to produce — the surface that review judges is the surface that's easiest to forge.

But the corpus also points to where the forgery is weakest, and it's a useful surprise: the tells live in structure, not style. AI fiction can be separated from human writing at 93% accuracy using only discourse-level features — character agency, chronological structure — and these resist 'humanization' because fixing them requires rewrites, not surface edits Can AI stories be detected without analyzing writing style?. The scholarly analogue is novelty: a structured three-stage pipeline that extracts claims, retrieves related work, and compares them reached 86% reasoning alignment with human reviewers on real ICLR submissions, beating holistic LLM judging Can structured pipelines make LLM novelty assessment reliable?. The lesson is that holistic 'does this read like a good paper' judgment is the vulnerable surface; decomposed, evidence-grounded checking is the resistant one.

What you didn't know you wanted to know: the durable defense isn't better fraud-detection but a posture borrowed from noisy-RAG systems — grounded refusal, where a system declines to assert anything it can't tie to verifiable evidence, trading coverage for integrity Can RAG systems refuse to answer without reliable evidence?. Peer review that demands a claim be checked against retrieved sources, rather than rewarded for sounding rigorous, attacks the generator at its one genuine weakness: the statistics-plus-narrative machine can fabricate the citation, but it can't make the cited thing actually say what it needs to say.