Story
N-Day-Bench – Can LLMs find real vulnerabilities in real codebases?
1 report
1 website
1 discussion
Latest report
Overview
N-Day-Bench evaluates the ability of frontier language models to identify real-world vulnerabilities disclosed after their knowledge cut-off dates. The benchmark features a standardized testing environment and monthly updates to test cases, focusing on the vulnerability discovery capabilities of large language models.
Summary from ndaybench.winfunc.com.
Reports
One report so far. Additional coverage will appear here when it is linked to this story.
ndaybench.winfunc.com
N-Day-Bench – Can LLMs find real vulnerabilities in real codebases?
Read report →
Community reactions
Each reaction summarizes sampled comments from the linked discussion.
Hacker News · ndaybench.winfunc.com
Historical count: 5 · 5 saved in sample · Historical sample
Analysis ready
Read this discussion’s reaction
Based on 5 sampled comments from this thread. Analyzed 2026-04-14.
- Praise: The N-Day-Bench offers an innovative approach to evaluating LLMs in real-world scenarios, which could enhance the security landscape by identifying vulnerabilities effectively.
- Praise: Incorporating false-positive rates into the evaluation rubric is crucial, as it addresses a significant concern in model reliability and accuracy.
- Praise: The benchmark design and results are seen as interesting and valuable, suggesting a positive reception among those familiar with application security.
- Concern: The complexity of the evaluation process, including terms like 'Curator' and 'Finder', may alienate those not well-versed in the methodology, highlighting a communication gap.
- Concern: There are concerns about the high false-positive rates of the models, which could undermine trust in their effectiveness for real-world applications.