Harvard IQSS working paper by Gary King and Steven Worthington comparing AI-generated reviews with high-quality, paid human reviews of papers submitted to the rapidPeer service; AI reviews won on every criterion measured in blind judgments, with no fabricated sources and about a third the rate of incorrect claims. The paper discusses implications for journal policy.
AI Article Reviews Dominate Human Peer Reviews at Their Best: Evidence and Policy Choices
Gary King and Steven Worthington · Harvard University, Institute for Quantitative Social Science · Working paper, October 22, 2026
Abstract
We show that AI-generated article reviews are now better, by every measure we could construct, than reviews written by well-compensated human experts and judged to be of unusually high quality by journal editors and authors. We then discuss how this result may inform journal policy choices, with concerning downsides as well as potential paths to increase publication speed and quality; reduce adversarial interactions; improve the experience of editors, reviewers, authors, and readers; and advance science.
From the introduction
In this paper, we provide head-to-head experiments, using an unusually high quality set of human reviews, and show that AI now dominates human peer reviews at their best — according to blind judgments by other AI models in every area and by (human) former editors of major scholarly journals. Given prior research on earlier AI models, we had expected AI to win some in some areas, some of the time, but we found that the latest AI wins in them all, by every criterion we measured, by wide margins, with no fabricated sources, and with about a third the rate of incorrect claims of the human reviews. We did not measure the degree to which subsequent revisions improved the paper or the collective consequences of switching to AI reviewing.
The study uses papers submitted to rapidPeer, a pre-submission, double-blind review service developed by Harvard's Institute for Quantitative Social Science, whose paid human reviews are rated as exceptionally high quality by authors and staff.