How much AI content appears in peer review at ICLR?

Pangram Labs, a company that sells AI-text detection, analyzed all 19,000 papers and 70,000 reviews from ICLR using the public OpenReview record. Its headline figure is that "21%, or 15,899 reviews, were fully AI-generated," and that "over half of the reviews had some form of AI involvement." On the submission side it reports that 61% of papers were "mostly human-written," that several hundred were fully AI-generated, and that 9% had over 50% AI content. These are the detector's estimates, produced by the company that built the detector. The excerpt gives its false positive rate as 1 in 10,000 on test documents and 1 in 100,000 on held-out arXiv papers, and compares that to a drug test.

The score pattern is the sharpest claim. Pangram finds that "the more AI is present in a review, the higher the score is," and reads this as reviewers outsourcing the judgment itself rather than rephrasing their own view: if AI were only a framing device, average scores on AI and human reviews should match. Sycophancy is offered as one explanation for the positive bias. The company also argues that length no longer signals care. AI reviews run long and carry "filler content" with "low information density," a property it attributes to Shaib et al.'s Measuring AI Slop in Text. It reverses an earlier finding that LLM judges favor their own outputs: here, "the more AI-generated text present in a submission, the worse the reviews are."

Against the nearest notes, this is an observational prevalence scan, not a test of policy. Does banning LLM use in peer review change review outcomes? randomizes the rules and measures what changed. Pangram's data cannot separate AI use from the reviewers who choose it, so it sizes the behavior rather than its effect. The sycophancy account is one candidate mechanism for the score link. How much does rhetorical style shift AI review scores? points to another: scores move with framing even when content is held fixed, so AI-written reviews could score higher on presentation alone. The excerpt tests neither. Its detection concern also rests on a machine check, since Can people reliably spot content made by AI? suggests readers cannot reliably spot this by eye. That fits Pangram's call for conferences to adopt AI detection.

What the excerpt does not establish: it gives no method for how the detector labels a review "fully AI-generated" beyond the error rates, and it does not say how partial AI involvement was scored, so the "over half" figure rests on a loose category. The paper figures also undercount by the company's own caveat, since some fully AI-generated papers were removed before the scan. The score association is a correlation across reviews, and the excerpt does not say whether it controls for paper quality, topic or venue. The 21% is therefore best read as a detector-based estimate with a published error rate, and the score link as an association worth testing rather than a settled cause. The policy conclusion, that fully AI-generated reviews should be sanctioned, is the company's and the excerpt presents it as its own position.