Microsoft’s MAI-DxO Diagnostic AI Gains the Most on Hard Cases and Weaker Models

Microsoft’s MAI-DxO, a diagnostic system that runs several AI agents as a virtual panel of doctors, got 83% of 304 complex New England Journal of Medicine cases right in a study published in Science on October 22, 2026, against 36% for physicians using reference materials. The same study tested the system on top of models from several labs. Those runs show the orchestration layer adds the most on the hardest cases and the weakest models, and almost nothing on routine emergency cases once the model underneath is strong.

MAI-DxO Works a Case Step by Step Like a Panel of Physicians

MAI-DxO, short for Microsoft AI Diagnostic Orchestrator, does not answer a case in one pass. Microsoft AI’s health team describes a set of agents that each propose diagnoses, pick the next question or test, and challenge the leading view across several rounds of debate. The system also weighs what each test is likely to reveal against what it costs. Microsoft built a benchmark to match, called the Sequential Diagnosis Benchmark, or SDBench. Every case begins with limited information, and the AI or physician must decide what to ask for next. SDBench scores the final diagnosis, the evidence requested along the way and the estimated cost of the tests ordered. It draws on the 304 NEJM case records and a set of typical emergency department presentations from the MIMIC-IV-ED dataset.

Microsoft’s Headline Result Pits MAI-DxO Against Generalist Physicians

On the NEJM cases, physicians reached 22% accuracy without outside resources and 36% with textbooks and the internet, against MAI-DxO’s 83%. On the emergency cases, MAI-DxO scored 80% and physicians 47%. Microsoft says one cost-focused configuration reached 81% on the NEJM cases at less than half the cost of the systems it was compared with. TechCrunch’s Rebecca Bellan reported that the physicians were primary care doctors and general internists working alone. The specialists and physician teams who normally handle cases this complex were not part of the comparison.

Model Strength Explains More of the Gain Than Orchestration Does

The study ran MAI-DxO on models from OpenAI, Google, Anthropic, Meta and DeepSeek, and the best results came with OpenAI’s GPT-5 and GPT-5.5. Bellan’s reading of the study data shows how the gains split. GPT-5 alone, with a short basic prompt, scored 78.9% on the complex cases at an estimated $8,400 per case, and MAI-DxO cut about $3,000 from that while adding roughly four points. On GPT-5.5, MAI-DxO lifted complex-case accuracy from 72% to nearly 87% at slightly lower cost. On emergency cases, GPT-5.5 scored about 85% with or without the orchestrator, at similar cost. Google’s Gemini 2.5 Flash rose from 46.4% to nearly 69% with MAI-DxO, yet the gap between weaker and stronger models was still larger than that boost. Microsoft’s researchers told TechCrunch that gains naturally shrink as models approach the ceiling. Microsoft’s own announcement says the gains held with newer models, including GPT-5.5, without changes to the method.

Microsoft Will Test MAI-DxO on Real Patients at Beth Israel Deaconess

Microsoft has partnered with Beth Israel Deaconess Medical Center in Boston on a prospective study. Patients will receive standard care, and those who consent will also interact with MAI-DxO, whose recommendations will be compared with physicians’ independent assessments. Dominic King, vice president of health at Microsoft AI, told TechCrunch the trial will span emergency and urgent care settings. Microsoft is also open-sourcing SDBench for a selection of the NEJM cases, including its scoring and cost-estimation code, so outside researchers can rerun the evaluation. Mustafa Suleyman, who leads Microsoft AI, told TechCrunch the company is training a healthcare foundation model with Mayo Clinic for release next year.

Why This Matters for Engineering Leadership

Multi-agent orchestration is a standing cost in engineering terms. It means more model calls, more latency and more code to maintain, and the MAI-DxO data shows its payoff depends on two things a team can measure. One is the model underneath, and the other is how hard the work is. On GPT-5.5, the same orchestrator was worth about 15 points on the hardest cases and nothing measurable on routine ones. Microsoft’s study design is worth copying for any agent pipeline. It reports the bare model with a basic prompt as a baseline, splits results by difficulty, and records cost per case beside accuracy. Teams running their own orchestration layers should rerun that comparison on every model upgrade and route routine work around the layer when it stops earning its cost. The human baseline deserves the same scrutiny, since a win over generalists working alone says nothing yet about specialists. For more on how these systems are built, see our explainer on multi-agent orchestration architecture.

FAQ