Can conference organizers keep bad papers out, or can they only catch the problems they can actually check?

This explores the gatekeeping role of conference and journal organizers: where they contribute to bad papers getting accepted, and where their influence works or falls short. The corpus suggests organizers matter a great deal, but not mainly through rules about behavior. What works best for them is choosing which problems they can actually check.

The clearest example is ICLR 2026. Program chairs had LLM-detection tools but did not treat them as automatic filters, because the detectors were too unreliable. Detector flags went to human area chairs as one piece of evidence among several. The chairs did desk-reject papers with confirmed fabricated references, because a fake citation is something you can verify (How can conferences detect and handle LLM misuse in peer review?). Compare ICML 2026, where organizers ran a randomized experiment: some reviewers were banned from using LLMs and others were allowed limited use. Scores and decisions barely moved, and many reviewers broke whichever rule they were given (Does banning LLM use in peer review change review outcomes?). Taken together, the two cases suggest a lesson: organizers do better enforcing checkable facts than policing how people work.

The less comfortable finding is that gatekeepers can themselves be the problem. Research on paper mills documents coordinated networks of brokers and editors across countries who pass submissions among themselves. Some editor groups show retraction rates above 50%. When a journal loses its indexing, these networks move to another one (Does scientific fraud operate through organized networks or individual actors?). Here the problem isn't organizers missing fraud. Part of the organizing layer has been captured by it. Another position paper makes a milder version of the same point for AI conferences: review failures are shared among authors, reviewers and venues. It proposes structural fixes, such as letting authors rate a review before they see the decision and giving reviewers badges for thorough work (Can two-stage review and badges fix AI conference peer review?).

Organizers are also being bypassed or gamed from new directions. Eighteen arXiv manuscripts contained hidden instructions telling AI reviewers to rate them favorably (Are hidden AI prompts in preprints a deceptive research practice?). Simply rewriting a paper's text raised AI review scores without improving the science, and AI reviewers agree with each other more than human reviewers do (Can AI systems safely replace human peer reviewers?). A venue that leans on AI reviewing opens a new way for weak papers to get through. Apparent biases also need care before organizers act on them. LLM-assisted reviewers seemed to favor LLM-written papers, but the effect disappeared once paper quality was controlled for. Those reviewers were simply more lenient toward weaker work in general (Do LLM reviewers actually favor LLM-written papers?). And some papers never pass through organizers at all. MIT's case shows an unreviewed preprint shaping public debate long before anyone questioned how reliable it was (Can unreviewed preprints shape scientific debate before peer review?).

One further detail: organizers can also use their venue as an experiment. At Agents4Science, which accepted AI-assisted research, organizers required authors to disclose how much AI was involved. They found that accepted papers had more human guidance than rejected ones (Do accepted papers need more human guidance than rejected ones?). The corpus has little systematic data on how often organizer decisions directly cause bad acceptances. What it does show is organizers working as system designers. Their leverage lies in choosing which checks are verifiable, which incentives to set, and what to measure.