2% of ICML papers desk rejected because the authors used LLM in their reviews
blog.icml.cc
March 19, 2026
5 min read
🔥🔥🔥🔥🔥
57/100
Summary
AI is becoming integral to researchers' workflows but poses risks to the integrity of peer review if misused. Conferences need to establish rules and policies to address these challenges and enforce disciplinary actions for violations.
Key Takeaways
- ICML 2026 desk-rejected 497 papers due to violations of LLM usage rules by reviewers, representing about 2% of all submissions.
- The conference has implemented two policies regarding LLM use in reviewing: Policy A prohibits LLM use, while Policy B allows it for understanding papers and polishing reviews.
- A total of 795 reviews written by 506 reviewers assigned to Policy A were found to have used LLMs, leading to the removal of these reviews and the rejection of submissions associated with them.
- The method for detecting LLM usage involved watermarking submission PDFs with hidden instructions, which aimed to identify reviews generated by LLMs.
Community Sentiment
Positives
- The detection method for LLM use in reviews demonstrates a robust approach that could enhance the integrity of the peer review process in AI research.
- The fact that only 2% of papers were desk rejected indicates that most authors are adhering to the guidelines, which reflects positively on the community's commitment to ethical standards.
Concerns
- The incident highlights a troubling trend where authors who publicly criticize LLMs may secretly rely on them, raising questions about transparency in the research community.
- The conservative rejection approach suggests that the actual misuse of LLMs in reviews could be significantly higher, indicating a potential crisis of trust in the review process.