Microsoft unveils high-accuracy medical AI

Microsoft (MSFT) has unveiled MAI-DxO, an AI diagnostic system designed to evaluate patients sequentially, with new peer-reviewed research showing it can significantly outperform generalist physicians on difficult medical cases. In tests based on complex case reports from The New England Journal of Medicine, the system achieved about 83% diagnostic accuracy, compared with roughly 36% for physicians using reference resources and 22% for those working without them. Microsoft describes this as more than twice the accuracy of doctors with references and nearly four times their accuracy without.

The findings were published in Science and are supported by a new evaluation framework, the Sequential Diagnosis Benchmark (SDBench), which compiles challenging scenarios from medical case reports and emergency department datasets. The benchmark is intended to capture the step-by-step nature of real clinical reasoning rather than single-shot question answering.

Unlike systems that rely on one model producing an immediate answer, MAI-DxO coordinates multiple AI agents that act like a panel of physicians. Each agent can propose diagnoses, suggest follow-up questions, choose tests, and challenge emerging conclusions, while the orchestrator combines their input over several rounds. As new information arrives, the system updates its differential diagnosis and weighs each potential test’s value against its estimated cost.

This approach is intended to mirror how clinicians start from a chief complaint, gather history, request investigations, and revise their thinking as evidence accumulates. Researchers argue that modeling this sequential process helps address common diagnostic pitfalls, such as anchoring on an early hypothesis or overlooking alternative explanations.

Across multiple datasets, MAI-DxO delivered large gains over generalist physicians. On complex NEJM cases, it recorded 83% accuracy, while doctors achieved 36% with access to textbooks and the internet and 22% without such resources. Microsoft characterizes this as approximately 2.3 times and 3.7 times higher accuracy, respectively.

On more typical emergency department presentations drawn from the MIMIC-IV-ED dataset and a large admissions database at Beth Israel Deaconess Medical Center, the system reached 80% diagnostic accuracy. Physicians averaged 47% on the same cases, making the AI about 1.7 times more accurate. The Diagnostic Orchestrator also outperformed doctors on these emergency department cases when evaluated using the Beth Israel dataset.

In addition to accuracy, Microsoft evaluated MAI-DxO on estimated diagnostic cost within the simulated cases. The company reports that, on complex NEJM scenarios, the system could reach the correct diagnosis at about half the cost of comparative approaches by using fewer tests and procedures. One configuration achieved 81% accuracy at less than half the cost of alternatives while remaining competitive in performance.

Microsoft states that across both complex and emergency datasets, no human or AI baseline in the study was simultaneously more accurate and less costly than MAI-DxO. The results suggest that the orchestration strategy can help optimize both diagnostic performance and resource use within the benchmarked settings.

The results to date come from simulations using past clinical cases, and Microsoft has begun preparing for clinical evaluation. Through a partnership with Beth Israel Deaconess Medical Center in Boston, the company plans prospective studies of MAI-DxO in real hospital settings. Patients will continue to receive standard care from clinicians, and those who consent will also interact with the AI system in parallel.

Researchers will compare MAI-DxO’s recommendations with physicians’ independent assessments to measure clinical reasoning in practice and understand how the tool might fit into existing workflows. Microsoft positions this as a step toward broader healthcare applications of its AI, alongside ongoing collaborations such as its work with Mayo Clinic on healthcare-focused foundation models.