Story
Please Do Not A/B Test My Workflow
1 report
1 website
1 discussion
Latest report
Overview
Anthropic is conducting A/B tests on Claude Code, which has impacted user workflows. The author expresses frustration over the degradation of their workflow due to these tests.
Summary from backnotprop.com.
Reports
One report so far. Additional coverage will appear here when it is linked to this story.
backnotprop.com
Please Do Not A/B Test My Workflow
Read report →
Community reactions
Each reaction summarizes sampled comments from the linked discussion.
Hacker News · backnotprop.com
Hacker News discussion ↗
Historical count: 5 · 5 saved in sample · Historical sample
Analysis ready
Read this discussion’s reaction
Based on 5 sampled comments from this thread. Analyzed 2026-03-14.
- Praise: LLMs can significantly enhance productivity by quickly generating boilerplate content, which can guide users in their thought processes more efficiently than traditional methods.
- Praise: The ability of newer LLM models to perform effectively with less direction indicates advancements in AI capabilities, potentially improving user experience.
- Concern: LLMs currently lack reliability and replicability in results, raising concerns about their suitability for professional workflows, especially when subjected to A/B testing.
- Concern: The risk of LLMs producing misleading or irrelevant information can derail users, emphasizing the need for a solid understanding of the subject matter to utilize them effectively.
- Concern: Open source tools may struggle to compete with proprietary models like Claude Code due to their inability to conduct A/B testing, which limits data-driven design improvements.