how automated healthcare fails, how you'd know, and what to do at each tier — every claim sourced, reviewed continuously
Incident
Epic Sepsis Model missed two-thirds of sepsis cases in external validation
June 2021Michigan Medicine, Ann Arbor, MI, USA (model deployed at hundreds of US hospitals)Study findingModels & agentsHuman handoff
An independent validation of Epic's proprietary sepsis score found it far less accurate than the vendor reported: it missed 67% of sepsis cases while alerting on 18% of all hospitalizations.
Sources checked when written 26 September 2026
What happened
Wong and colleagues applied the Epic Sepsis Model retrospectively to 38,455 hospitalizations of 27,697 patients at Michigan Medicine. The hospitalization-level AUC was 0.63, against the developer's reported 0.76 to 0.83. At the alert threshold of 6 the model had 33% sensitivity and 12% positive predictive value; it did not identify 1,709 of the 2,552 patients with sepsis, and it flagged only 183 septic patients (7%) whom clinicians had not already treated with timely antibiotics. Alerts fired on 6,971 hospitalizations. A 2026 multicenter validation of the redesigned version 2 found better discrimination (encounter-level AUROC 0.82 to 0.92) but still low PPV (0.13 to 0.26), high alert burden and wide variation between the four health systems.[1,2]
Documented harm
No individual patient harm documented in the study; the documented effect is missed detection and alert burden at scale.
What it teaches
Vendor-reported accuracy is not local accuracy; validate before go-live.
Counting alerts fired says nothing about the cases the model missed.
A better model version still needs local validation because performance varies by site.
Every alert a model fires is work assigned to a person; count it before deployment.
An alert with low precision teaches clinicians to ignore the alerts that are right.
Information only, not advice. FailSystems is an aggregation and synthesis of published sources. It is not consulting, engineering, legal, regulatory or medical advice, and using it creates no professional relationship. Health systems are complex and no approach fits every organisation: anything you adopt is your own decision, at your own risk, and should be checked against the current official sources and by qualified people who know your setting. Full disclaimer.
Dealing with an incident right now? This site is a reference, not an incident-response service. Activate your organisation's emergency operations plan and incident command, and:
Power loss, disaster or resource needs: go through your local or county emergency management. They escalate to the state, and the state requests FEMA support; hospitals do not call FEMA directly.
A medical device problem: report it to the manufacturer and to FDA MedWatch.
Outside the US: your national emergency number and national cyber agency (in the UK, NCSC).