AI agents are increasingly effective at identifying vulnerabilities in large software systems. Anthropic chose not to release the Mythos model due to concerns over its potential to discover dangerous security flaws.

N-Day-Bench evaluates the ability of frontier language models to identify real-world vulnerabilities disclosed after their knowledge cut-off dates. The benchmark features a standardized testing environment and monthly updates to test cases, focusing on the vulnerability discovery capabilities of large language models.