BACKGROUND AND PURPOSE:

• The Epic Sepsis Model (ESM) has been implemented for sepsis prediction at hundreds of US hospitals, despite a lack of testing of its predictive ability
• Wong et al. (JAMA Intern Med, 2021) externally validated the ESM in the prediction of sepsis and assessed its potential clinical value compared with usual care

METHODS:

• Retrospective cohort study
• Participants
  • Patients ≥18
  • Admitted to a single academic health system
• Exposure
  • ESM score, calculated every 15 minutes | Used an ESM threshold of 6 (range of 5 to 8 suggested by developer)
• Study design
  • Sepsis was defined by a composite of
    • CDC surveillance criteria
    • 2 systemic inflammatory response syndrome criteria and 1 organ dysfunction criterion within 6 hours of one another (International Statistical Classification of Diseases and Related Health Problems, Tenth Revision diagnostic codes)
  • Model discrimination was assessed using the area under the receiver operating characteristic (ROC) curve at
    • Hospitalization-level (when first considered at risk during hospitalization) | 4 hrs | 8 hrs | 12 hrs | 24 hrs
  • Added benefit of ESM was compared to clinical practice | Based on timely administration of antibiotics
  • Alert fatigue was evaluated by comparing the clinical value of different alerting strategies
• Primary outcomes
  • Sepsis

RESULTS:

• 27,697 patients | 38,455 hospitalizations
  • 57% women; median (IQR) age 56 (35 to 69) years
  • Sepsis incidence: 7% (2552 patients)
• ESM’s hospitalization-level ROC: 0.63 (95% CI, 0.62-0.64)
• ESM identified patients with sepsis who did not receive timely administration of antibiotics: 7% (183/2552)
• Sepsis patients missed by the ESM: 1709 (67% of sepsis cases)
• Alert fatigue: ESM generated alerts for 18% of all hospitalized patients

CONCLUSION:

• Among hospitalized patients, the Epic Sepsis Model demonstrated low sensitivity, was a poor predictor of sepsis, and also generated a lot of false positives vs usual clinical care
• The authors state

The increase and growth in deployment of proprietary models has led to an underbelly of confidential, non–peer-reviewed model performance documents that may not accurately reflect real-world model performance

Owing to the ease of integration within the EHR and loose federal regulations, hundreds of US hospitals have begun using these algorithms

Medical professional organizations constructing national guidelines should be cognizant of the broad use of these algorithms and make formal recommendations about their use

Learn More – Primary Sources:

External Validation of a Widely Implemented Proprietary Sepsis Prediction Model in Hospitalized Patients

Related ObG Topics:

Does the Electronic Health Record Match with the Actual Clinical Exam?

To What Extent do Electronic Health Records Systems Contribute to Physician Burnout?

Sepsis and Sepsis-Related Maternal Mortality in the U.S. – Does it Happen More Than We Think?

Do Mandated Sepsis Procedures Reduce Sepsis-Related Deaths?

2019 Cochrane Review Update: Does Corticosteroid Treatment Decrease Death Due to Sepsis?