New AI Is More Than Twice as Good at Diagnosing Diseases as Doctors
gizmodo.com ·
Microsoft’s AI Diagnostic Orchestrator, or MAI-DxO, achieved 83.2% diagnostic accuracy on complex cases from The New England Journal of Medicine, compared with 36.1% for generalist physicians using reference materials. Physicians without those materials scored about 22%. In a separate emergency-department admissions test, MAI-DxO scored 80.3%, versus 46.8% for generalist physicians. Microsoft reported its strongest results with OpenAI’s GPT-5 and additional improvements with GPT-5.5.
MAI-DxO begins with a patient’s chief complaint, then sequentially selects follow-up questions and diagnostic tests before reaching a conclusion. It assigns AI agents roles including proposing diagnoses, choosing tests, challenging assumptions, and weighing costs. Microsoft said simulated-case estimates indicated the system reached diagnoses using fewer tests and procedures at roughly half to one-third of the cost, though those estimates were not actual medical expenses. The findings, first posted as an arXiv preprint last year, have been published in Science.
Microsoft AI chief Mustafa Suleyman said real-world use would require clinical testing. Dr. Dominic King, Microsoft’s AI health lead, said testing was expected to begin at Beth Israel Deaconess Medical Center in Boston within months. The researchers said the performance gap would likely narrow against specialist physicians rather than generalists, and medical experts have said diagnostic benchmark performance does not establish that AI should replace doctors.
What people are saying
The thread treated the headline less as a settled medical breakthrough than as a test of where diagnostic AI genuinely helps and where a flashy benchmark stops short of clinical care.
What readers liked
- Broad medical recall could catch rare and routinely overlooked diagnoses that clinicians miss when they default too quickly to the most familiar…
- AI-backed diagnosis could give rural and generalist providers practical access to specialist-level knowledge where referral networks are thin.
Concerns raised
- A claim of doubled diagnostic accuracy means little without testing the back-and-forth of real consultations, where follow-up questions and physical…
- Hallucinations and false positives could turn encyclopedic recall into harmful overdiagnosis unless systems show calibrated uncertainty and safe…
Read full discussion analysis
Source: Hacker News · Based on 10 sampled comments · Analyzed
Discussion overview
The thread treated the headline less as a settled medical breakthrough than as a test of where diagnostic AI genuinely helps and where a flashy benchmark stops short of clinical care. Many commenters argued that clinicians routinely miss conditions, including ordinary dermatological problems, especially when they rely on common-case heuristics or dismiss patients’ accounts. In that setting, an AI system with broad medical recall looks valuable: it could surface overlooked differentials, give rural generalists specialist-like backup, and let patients disclose history without feeling rushed or judged.
But the strongest pushback was not that machines cannot recognize diseases. It was that diagnosis depends on eliciting a coherent history, performing an exam, noticing what the patient omitted, and handling the consequences of false positives. Several readers doubted a vendor’s claim absent evidence on iterative follow-up, real patients, and hallucination rates; a model that wins a constrained vignette can still fail at the messy conversation in an exam room. Others warned that ubiquitous AI assistance could hollow out the diagnostic judgment trainees acquire through repeated unaided cases. A historical comparison to early expert systems suggested that impressive narrow diagnostic results are not new, while the continued need for doctors shows how difficult deployment and accountability remain. The mood favored AI as a powerful second opinion and access tool, but resisted treating it as an automatic replacement for clinical care.
Points of disagreement
The central dispute is whether superior AI diagnostic performance should justify replacing much of front-line physician judgment. Optimists argue that patients should not receive inferior care merely to preserve doctors’ training opportunities, particularly when AI can catch neglected diseases and extend expertise to underserved settings. Skeptics counter that benchmark accuracy misses interviews, examinations, follow-up questioning, uncertainty, hallucinations, and the human responsibility attached to medical decisions.
What readers liked
- Broad medical recall could catch rare and routinely overlooked diagnoses that clinicians miss when they default too quickly to the most familiar explanation.
- AI-backed diagnosis could give rural and generalist providers practical access to specialist-level knowledge where referral networks are thin.
- A structured AI interview may recover crucial details patients forget or fail to volunteer, improving the raw information behind a diagnosis.
- Patients frustrated by rushed or dismissive appointments see direct AI access as a cheaper, more attentive way to explore their medical history and possible conditions.
Concerns raised
- A claim of doubled diagnostic accuracy means little without testing the back-and-forth of real consultations, where follow-up questions and physical examination often change the answer.
- Hallucinations and false positives could turn encyclopedic recall into harmful overdiagnosis unless systems show calibrated uncertainty and safe escalation behavior.
- Letting AI handle routine cases may deprive trainees of the repeated unaided diagnostic practice needed when the tool is wrong or unavailable.
- Vendor-led performance claims drew distrust because narrow case benchmarks can flatter a model while concealing failures in messy patient-facing care.
Keep reading
Sources and details
gizmodo.com · Published
New AI Is More Than Twice as Good at Diagnosing Diseases as Doctors
Read report →
Community analysis available for 1 of 1 linked discussions.
Additional discussion searches are still incomplete.
Coverage details and dates
Latest available community analysis:
Summaries: 1 current · 0 awaiting update · 0 not verified
Latest summary update:
Hacker News · gizmodo.com