If you rewrite a paper's framing but leave its science alone, do AI reviewers' scores still shift?

This explores whether how a paper is written, rather than what it shows, drives the scores reviewers give it. The corpus has no head-to-head study measuring rhetoric against content for human peer reviewers. It does show that presentation moves scores more than most people would expect. The clearest evidence comes from AI reviewers. When researchers rewrote manuscripts' rhetoric but left the scientific content untouched, LLM reviewer scores shifted measurably How much does rhetorical style shift AI review scores?. The biggest swings came from how evidence was framed and how boldly novelty was claimed. Those are exactly the moves authors already make when they 'sell' a paper.

The weakness also extends past phrasing. LLM judges can be fooled by fake references and rich formatting, without any access to the model Can LLM judges be fooled by fake credentials and formatting?. Human readers share a similar habit: they prefer answers with more citations even when the citations are irrelevant Do users trust citations more when there are simply more of them?. Polish also misleads people. Evaluators have rated AI-generated documents as both human-written and better than real human submissions Does polished writing actually signal better quality work?. AI-written documents in particular make readers see more confidence and quality in the writer Does AI writing assistance change how readers perceive the writer?. In practice, polish was enough for one fully AI-generated paper to score 6.33 at an ICLR workshop, meeting the acceptance threshold. Its own authors later judged none of their three AI-generated papers good enough for the main conference track Can AI-generated papers pass peer review undetected?.

In the counterevidence, content shows up as the bigger force. Across more than 125,000 reviews, it looked as if LLM-assisted reviewers favored LLM-written papers. That pattern disappeared once paper quality was controlled for, because LLM papers were mostly the weaker ones Do LLM reviewers actually favor LLM-written papers?. A study from outside peer review makes a similar point: in debates, the voters' prior beliefs predicted who won better than anything about the language did Does what readers believe matter more than what debaters say?. Many 'style effects' turn out to be effects of who is reading or of quality differences that weren't measured. Polish also doesn't always help. In one admissions program, applicants whose essays were improved by AI were admitted at lower rates Does AI essay use hurt admissions chances despite quality gains?.

If the question is about how reviews themselves are written, the evidence points toward specificity mattering more than eloquence. At ICLR 2025, LLM feedback led 27 percent of reviewers to revise their reviews, and blinded raters judged the revised reviews more informative Can LLM feedback help peer reviewers improve their own reviews?. But when ICML 2026 compared banning LLM use in reviewing with allowing limited use, scores and decisions barely changed Does banning LLM use in peer review change review outcomes?. Together these suggest that changing how reviews are written may have little effect on which papers get in.

So here's what the corpus supports. Rhetoric clearly moves scores, especially for AI reviewers, and the effects are big enough to exploit. But in the large real-world datasets, quality still sorts papers once you control for it. The open risk is the one the AI-review study points to: as more reviewing is automated, evaluation becomes the stage where framing gets the most leverage.