• About Us
    • Contact Us
    • Login
    • ObGFirst
  • Alerts
  • OB
  • 2T US Atlas
  • The Genome
  • GYN
    • GYN
    • Sexual Health
  • Primary Care
  • Your Practice
  • GrandRounds
  • My Bookshelf
  • COVID-19
About Us Contact Us Login ObGFirst
  • Alerts
  • OB
  • 2T US Atlas
  • The Genome
  • GYN
    • GYN
    • Sexual Health
  • Primary Care
  • Your Practice
  • GrandRounds
  • My Bookshelf
  • COVID-19
Grand Rounds

Validating an EHR Sepsis Prediction Model: How Well Does it Perform?

image_pdfFavoriteLoadingFavorite

BACKGROUND AND PURPOSE:

  • The Epic Sepsis Model (ESM) has been implemented for sepsis prediction at hundreds of US hospitals, despite a lack of testing of its predictive ability
  • Wong et al. (JAMA Intern Med, 2021) externally validated the ESM in the prediction of sepsis and assessed its potential clinical value compared with usual care

METHODS:

  • Retrospective cohort study
  • Participants
    • Patients ≥18
    • Admitted to a single academic health system
  • Exposure
    • ESM score, calculated every 15 minutes | Used an ESM threshold of 6 (range of 5 to 8 suggested by developer)
  • Study design
    • Sepsis was defined by a composite of
      • CDC surveillance criteria
      • 2 systemic inflammatory response syndrome criteria and 1 organ dysfunction criterion within 6 hours of one another (International Statistical Classification of Diseases and Related Health Problems, Tenth Revision diagnostic codes)
    • Model discrimination was assessed using the area under the receiver operating characteristic (ROC) curve at
      • Hospitalization-level (when first considered at risk during hospitalization) | 4 hrs | 8 hrs | 12 hrs | 24 hrs
    • Added benefit of ESM was compared to clinical practice | Based on timely administration of antibiotics
    • Alert fatigue was evaluated by comparing the clinical value of different alerting strategies
  • Primary outcomes
    • Sepsis

RESULTS:

  • 27,697 patients | 38,455 hospitalizations
    • 57% women; median (IQR) age 56 (35 to 69) years
    • Sepsis incidence: 7% (2552 patients)
  • ESM’s hospitalization-level ROC: 0.63 (95% CI, 0.62-0.64)
  • ESM identified patients with sepsis who did not receive timely administration of antibiotics: 7% (183/2552)
  • Sepsis patients missed by the ESM: 1709 (67% of sepsis cases)
  • Alert fatigue: ESM generated alerts for 18% of all hospitalized patients

CONCLUSION:

  • Among hospitalized patients, the Epic Sepsis Model demonstrated low sensitivity, was a poor predictor of sepsis, and also generated a lot of false positives vs usual clinical care
  • The authors state

The increase and growth in deployment of proprietary models has led to an underbelly of confidential, non–peer-reviewed model performance documents that may not accurately reflect real-world model performance

Owing to the ease of integration within the EHR and loose federal regulations, hundreds of US hospitals have begun using these algorithms

Medical professional organizations constructing national guidelines should be cognizant of the broad use of these algorithms and make formal recommendations about their use

Learn More – Primary Sources:

External Validation of a Widely Implemented Proprietary Sepsis Prediction Model in Hospitalized Patients

Want to stay on top of key guidelines and research papers?

ObGFirst® – Try It Free! »

image_pdfFavoriteLoadingFavorite
< Previous
All Grand Rounds Posts
Next >

Related ObG Topics:

Does the Electronic Health Record Match with the Actual Clinical Exam?
To What Extent do Electronic Health Records Systems Contribute to Physician Burnout?
Sepsis and Sepsis-Related Maternal Mortality in the U.S. – Does it Happen More Than We Think?
Do Mandated Sepsis Procedures Reduce Sepsis-Related Deaths?
2019 Cochrane Review Update: Does Corticosteroid Treatment Decrease Death Due to Sepsis?

Sections

  • Alerts
  • OB
  • GYN
    • GYN
    • Sexual Health
  • 2T US Atlas
  • The Genome
  • Primary Care
  • Your Practice
  • Grand Rounds
  • My Bookshelf
  • COVID-19

Are you an
ObG Insider?

Get specially curated clinical summaries delivered to your inbox every week for free

  • Site Map/
  • © ObG Project/
  • Terms and Conditions/
  • Privacy/
  • Contact Us/
© ObG Project
SSL Certificate


  • Already an ObGFirst Member?
    Welcome back

    Log In

    Want to sign up?
    Get guideline notifications
    CME Included

    Sign Up

Sign In

Lost your password?

Sign Up for ObGFirst and Stay Ahead

  • - Professional guideline notifications
  • - Daily summary of a clinically relevant
    research paper
  • - Includes 1 hour of CME every month

ObGFirst Free Trial

Already a Member of ObGFirst®?

Please log in to ObGFirst to access the 2T US Atlas

Password Trouble?

Not an ObGFirst® Member Yet?

  • - Access 2T US Atlas
  • - Guideline notifications
  • - Daily research paper summaries
  • - And lots more!
ObGFirst Free Trial

Media - Internet

Computer System Requirements

OBG Project CME requires a modern web browser (Internet Explorer 10+, Mozilla Firefox, Apple Safari, Google Chrome, Microsoft Edge). Certain educational activities may require additional software to view multimedia, presentation, or printable versions of their content. These activities will be marked as such and will provide links to the required software. That software may be: Adobe Flash, Apple QuickTime, Adobe Acrobat, Microsoft PowerPoint, Windows Media Player, or Real Networks Real One Player.

Disclosure of Unlabeled Use

This educational activity may contain discussion of published and/or investigational uses of agents that are not indicated by the FDA. The planners of this activity do not recommend the use of any agent outside of the labeled indications.

The opinions expressed in the educational activity are those of the faculty and do not necessarily represent the views of the planners. Please refer to the official prescribing information for each product for discussion of approved indications, contraindications, and warnings.

Disclaimer

Participants have an implied responsibility to use the newly acquired information to enhance patient outcomes and their own professional development. The information
presented in this activity is not meant to serve as a guideline for patient management. Any procedures, medications, or other courses of diagnosis or treatment discussed or suggested in this activity should not be used by clinicians without evaluation of their patient’s conditions and possible contraindications and/or dangers in use, review of any applicable manufacturer’s product information, and comparison with recommendations of other authorities.

Jointly provided by

NOT ENOUGH CME HOURS

It appears you don't have enough CME Hours to take this Post-Test. Feel free to buy additional CME hours or upgrade your current CME subscription plan

Subscribe

JOIN OBGFIRST AND GET CME/CE CREDITS

One of the benefits of an ObGFirst subscription is the ability to earn CME/CE credits from the ObG entries you read. Tap the button to learn more about ObGFirst

Learn More
Leaving ObG Website

You are now leaving the ObG website and on your way to PRIORITY at UCSF, an independent website. Therefore, we are not responsible for the content or availability of this site