Preliminary data from a longitudinal AI impact study
newsletter.getdx.com
March 11, 2026
2 min read
🔥🔥🔥🔥🔥
45/100
Summary
Preliminary data from a longitudinal AI impact study indicates that AI productivity gains are approximately 10%, rather than the previously suggested 10x. High expectations for AI productivity, influenced by social media and vendor marketing, may not align with these findings.
Key Takeaways
- AI productivity gains in engineering are approximately 10%, significantly lower than the 2-3x increases often suggested by marketing.
- During a study of 40 companies, AI usage increased by an average of 65%, but pull request throughput only rose by 9.97%.
- Developers report that coding is not the primary bottleneck in their workflow, as planning and other human-centric tasks remain largely unchanged.
- The ongoing study will further investigate why some teams achieve greater productivity gains with AI and how leaders can facilitate this.
Community Sentiment
Positives
- A broad 10% productivity improvement across industry is significant, indicating that AI's impact could be transformative, even if it doesn't seem headline-worthy.
- LLMs have effectively reduced the cost of deployment, allowing for quicker iterations and potentially reshaping how coding is approached in the future.
- The rapid iteration capabilities of AI tools are enabling programming teams to work much more efficiently, suggesting a shift in productivity dynamics.
Concerns
- The marginal effect on PR throughput raises concerns about the actual productivity gains from AI, suggesting that the improvements may not be as impactful as anticipated.
- Relying on PR throughput as a metric for AI productivity is flawed, as it does not capture the true potential of AI in transforming development processes.
- Skepticism around the validity of AI productivity enhancements from 2024 data highlights the uncertainty and potential overestimation of AI's impact on developer roles.
