Story
Are LLMs not getting better?
1 report
1 website
1 discussion
Latest report
Overview
LLMs demonstrate a significant drop in performance when the success criterion shifts from "passes all tests" to "would get approved by the maintainer." The time to reach a 50% success rate decreases from 50 minutes to 8 minutes under the more stringent criterion.
Summary from entropicthoughts.com.
Reports
One report so far. Additional coverage will appear here when it is linked to this story.
entropicthoughts.com
Are LLMs not getting better?
Read report →
Community reactions
Each reaction summarizes sampled comments from the linked discussion.
Hacker News · entropicthoughts.com
Hacker News discussion ↗
Historical count: 5 · 5 saved in sample · Historical sample
Analysis ready
Read this discussion’s reaction
Based on 5 sampled comments from this thread. Analyzed 2026-03-12.
- Praise: There is a clear improvement in AI models from Sonnet 3.7 to Opus 4.0 to Sonnet 4.5, indicating progress in AI capabilities despite some skepticism.
- Praise: The introduction of tools like Claude Code represents a significant advancement in AI applications for coding, driving mainstream adoption.
- Concern: Many users report that AI tools still require significant oversight and often produce inaccurate results, indicating limited improvement in reliability.
- Concern: Perceived improvements in newer models may be largely placebo, as users struggle to differentiate between versions in practical tasks.
- Concern: Overall, there is a consensus that coding abilities of AI models have not noticeably improved over the past year, raising concerns about stagnation in development.