When an AI reviewer's points match a human's, is that proof it's right, or just agreement with one reviewer's partial view?

This asks whether an AI reviewer that 'matches' human reviewers is actually right, or only agreeing. The corpus points clearly toward the second: overlap measures agreement, not correctness, and the two can come apart in surprising ways. The best-known result here is that GPT-4's feedback overlapped with an individual human reviewer's points 30.85% of the time, while two human reviewers overlapped with each other only 28.58% of the time Can GPT-4 feedback match what human reviewers catch?. That sounds like parity with humans. But look at what the baseline shows: human reviewers of the same paper agree on only about 30% of their points. Matching a reviewer means matching one partial, idiosyncratic view of the paper, not some agreed-upon truth.

The less obvious problem is that the most valuable feedback may be the feedback that matches nobody. An agentic reviewer that spends extra compute checking proofs and experiments line by line found critical flaws in papers that had already passed human review at STOC and ICML Can inference scaling help reviewers catch errors humans miss?. Scored by overlap with the human reviews, those catches would count as misses, because no human raised them. A 'match the humans' metric rewards an AI for repeating what reviewers already noticed and penalizes it for finding what they missed.

Agreement can also be produced without any accuracy behind it. AI reviewers show a 'hivemind' effect: they agree with each other more than humans do. And a zero-shot rewrite of a paper's text raised AI review scores by 0.45 points without changing the science at all Can AI systems safely replace human peer reviewers?. Consensus among AI systems therefore says little, and their judgments respond to surface polish. Consumer ratings research shows a similar pattern outside science: later ratings drift toward earlier ones, so a high-agreement aggregate can partly reflect herding rather than independent judgments of quality Do online ratings actually reflect independent customer opinions?. A related agent-safety note makes the general point: a scoring function can compute perfectly and still report something misleading if what it measures has been shaped upstream Can a correct scoring function still mislead about task performance?.

So what would test whether feedback is correct? The corpus offers a few alternatives to overlap. One is to verify specific claims directly, as the proof-checking reviewer does. Another is to have blinded raters judge whether a revised review is more specific and informative. That was the measure used in the ICLR 2025 trial, where 27% of reviewers updated their reviews after LLM feedback Can LLM feedback help peer reviewers improve their own reviews?. A third is to let authors rate how good a review was before they see the accept/reject decision, so their judgment isn't colored by the outcome Can two-stage review and badges fix AI conference peer review?. Each of these asks 'was this right or useful?' instead of 'did it sound like the others?' The takeaway: when a headline says an AI reviewer 'matches humans,' check what humans match each other on first. The bar is lower, and less tied to correctness, than it sounds.