Story
We are changing our developer productivity experiment design
1 report
1 website
1 discussion
Latest report
Overview
METR's previous study indicated a 20% slowdown in task completion among experienced open-source developers using AI tools. A new experiment launched in August 2025 aims to assess AI's impact on developer productivity over time, but participant feedback suggests the data may be unreliable.
Summary from metr.org.
Reports
One report so far. Additional coverage will appear here when it is linked to this story.
Community reactions
Each reaction summarizes sampled comments from the linked discussion.
Hacker News · metr.org
Hacker News discussion ↗
Historical count: 5 · 5 saved in sample · Historical sample
Analysis ready
Read this discussion’s reaction
Based on 5 sampled comments from this thread. Analyzed 2026-02-25.
- Praise: The new study indicates a significant shift in developer productivity, with findings showing a potential speedup of 20%, challenging previous notions of AI's limitations.
- Praise: A notable percentage of developers (30-50%) express a strong preference for using AI, suggesting that AI tools are becoming integral to their workflow and productivity.
- Praise: The observation that experienced developers show greater productivity gains with AI tools highlights the importance of familiarity and skill in leveraging AI effectively.
- Concern: Concerns arise from the potential for rate limits to impact profitability, which could hinder the adoption of AI tools in developer environments.
- Concern: The implication that many developers are avoiding tasks without AI raises questions about the reliability of productivity metrics and the overall impact of AI dependency.