N-Day-Bench – Can LLMs find real vulnerabilities in real codebases?
ndaybench.winfunc.com
April 13, 2026
1 min read
49/100
Summary
N-Day-Bench evaluates the ability of frontier language models to identify real-world vulnerabilities disclosed after their knowledge cut-off dates. The benchmark features a standardized testing environment and monthly updates to test cases, focusing on the vulnerability discovery capabilities of large language models.
Key Takeaways
- N-Day-Bench evaluates the ability of language models to discover real-world vulnerabilities disclosed after their knowledge cut-off dates.
- The benchmark is adaptive, with test cases updated monthly and models upgraded to their latest versions.
- OpenAI's GPT-5 achieved the highest average score of 83.93 in the latest benchmark run.
- All benchmark traces are publicly accessible for review.
Coverage & discussions
Hacker News · ndaybench.winfunc.com
Community discussions
Community Sentiment
Positives
- The N-Day-Bench offers an innovative approach to evaluating LLMs in real-world scenarios, which could enhance the security landscape by identifying vulnerabilities effectively.
- Incorporating false-positive rates into the evaluation rubric is crucial, as it addresses a significant concern in model reliability and accuracy.
- The benchmark design and results are seen as interesting and valuable, suggesting a positive reception among those familiar with application security.
Concerns
- The complexity of the evaluation process, including terms like 'Curator' and 'Finder', may alienate those not well-versed in the methodology, highlighting a communication gap.
- There are concerns about the high false-positive rates of the models, which could undermine trust in their effectiveness for real-world applications.
Related Articles
The Hugging Face incident and the road ahead
Aug 26, 2026
Every Model Cheats
Aug 20, 2026
OpenAI’s accidental attack against Hugging Face is science fiction that happened
Jul 23, 2026
Evaluating and mitigating the growing risk of LLM-discovered 0-days
Feb 5, 2026
Artificial intelligence is capitalism's final destination
Oct 16, 2026