MIT's claim that 95% of AI pilots fail - Is this true?

Discussion overview

The thread spent less time debating whether AI creates value than untangling a statistic that commenters believe was mangled in transit. Several readers argued that the underlying research described pilots with no measured return, not projects proven to have failed or produced zero value. In their view, headlines and social-media repetition converted an ambiguity about measurement into a sweeping verdict on AI itself. One commenter pushed back with firsthand enterprise experience, saying focused deployments can save tens of millions of dollars even while many projects go nowhere.

At the same time, nobody portrayed production AI as easy. Commenters described agentic systems as unusually hard to observe: conventional logs can miss partial tool-call failures, so teams need specialized telemetry, robust retries, and evaluation systems that act like AI-specific integration tests. That operational burden makes an unmeasured pilot unsurprising, particularly when businesses lack a credible way to assess model behavior over time. The more skeptical voices therefore treated the 95% headline as depressing and misleading, while conceding that AI projects probably inherit the high failure rates long associated with enterprise IT. The thread’s real conclusion was that weak ROI measurement is not evidence that AI has no ROI, but it is evidence that deploying reliable AI remains difficult.

Points of disagreement

The sharp dispute is whether the 95% figure means AI pilots genuinely fail or merely lack measured returns. Critics say the claim was inflated from a measurement finding into a false declaration that nearly all AI efforts are worthless; the opposing practical view is that, whatever the wording, AI deployments face enough operational complexity that many pilots will indeed stall or fail like other enterprise technology projects.