arXiv stops accepting unrefereed CS review and position papers

The rule

On 31 October the arXiv computer science moderators announced that review articles and position papers will no longer be accepted into the CS category unless the author can show they have already passed peer review at a journal or a conference. The submission has to carry the reference and DOI of the accepted version. Workshop papers do not count, on the grounds that workshop review does not meet the standard of full peer review. Submissions without that documentation will, in the moderators' words, likely be rejected.

The announcement is careful to say that reviews and position papers were never on arXiv's list of accepted content types. In practice they were posted for years when a moderator judged them useful, and many of the surveys people in machine learning actually read went up as preprints first. That informal tolerance is what has now ended, for CS only. The announcement says other categories may follow if they see the same rise in generated submissions.

Why now

The stated reason is volume and quality together. The moderators say CS now receives hundreds of review articles every month, and that most of them are little more than annotated bibliographies with no substantial discussion of open research issues. They say generative models have added to the flood. Since arXiv as a whole was taking about 24,000 submissions a month as of late 2024, a few hundred extra surveys does not sound like much until you remember that each one has to be read by a volunteer.

That is the part of the story that we think outsiders miss. arXiv moderation is human. Every submission passes automated checks and then a moderator in the relevant subject decides whether it is on topic, whether it is in the right category, and whether it is the kind of document arXiv hosts at all. A survey is the most expensive thing to moderate, because judging whether it says anything requires knowing the field it summarises. A language model can produce a plausible forty page survey in an afternoon, and it costs the moderator more time to reject it than it cost the author to make it.

So the rule outsources the judgement. Rather than asking a moderator to decide whether a survey has substance, it asks a journal or a programme committee to have already decided. That is a reasonable move for a service with a fixed budget of volunteer attention, and it is also a retreat from what arXiv was for.

What the preprint model was buying

Paul Ginsparg started arXiv in 1991 so that physicists could read each other's work before the journals got to it. The whole point was to decouple distribution from certification. The endorsement system added in 2004 asks an established author to vouch for a newcomer from an unrecognised institution, but it was designed to check that a submitter is a working scientist, not to check the paper. Content moderation was meant to be light.

Reviews and position papers are exactly the genre where early distribution matters. A survey is most useful in the year the field is moving, and a position paper is by definition an argument someone wants to have before the consensus forms. Routing both through a conference cycle adds months, which in a fast subfield is long enough for the survey to be out of date on arrival and for the position to have been settled without the paper.

There is a fairness cost too. Peer review at a venue that arXiv will accept is not free. Conference registration, travel, and the time to respond to reviews are easier to bear at a well-funded lab than for an independent researcher or a group at a university without a travel budget. The previous regime let those authors put a survey where it would be read and let readers judge it. The new regime asks them to buy a stamp first.

What the rule will not fix

The generated surveys will not stop being written. They will go to the venues arXiv now defers to, which are already complaining about the same thing, or they will go to preprint servers with looser moderation, or they will be reclassified by their authors as something other than a review. A paper with a small experiment bolted onto a long generated literature section is not a review article under any definition the moderators can enforce.

The rule also treats the acceptance as a signal of quality, and that signal is weakening for the same reason arXiv is overloaded. If a fraction of reviewers at large conferences are themselves using language models to write reviews, then acceptance is a noisier certificate than it was, and arXiv is anchoring to it at the moment it is least reliable.

What we would rather see

A cheaper alternative would be to keep accepting unrefereed reviews but to label them, and to let readers filter. arXiv already exposes metadata about versions and cross-lists. A flag reading unrefereed survey, applied by the moderator without reading the paper, moves the judgement to the reader at near zero cost and keeps the distribution channel open. It does nothing about moderator load, which is the actual complaint, unless the automated checks can triage generated text first.

The honest answer is that nobody has a cheap way to tell a useful survey from a generated one without reading it, and that is the same problem every journal and conference has. arXiv has chosen to make it someone else's problem. We understand why, and we still think we lose something when the place that was built to skip the gatekeepers starts asking to see the gatekeeper's receipt.