improvement from may 2016 front office to front line ...decision to evaluate or treat an infant...

www.jcrinc.com

Improvement fromFront Office to Front Line

May 2016Volume 42 Number 5

Making Room for Improvement in Reporting of QI StudiesFeaturesPerformance Improvement■ Reporting Quality Improvement Interventions: A Call to Action■ How Well Is Quality Improvement Described in the Perioperative Care

Literature? A Systematic Review

Performance Measures ■ Will Hospitals Finally “Do the Right Thing”? Providing Evidence-

Based Tobacco Dependence Treatments to Hospitalized Patients Who Smoke

■ Two Years in the Life of a University Hospital Tobacco CessationService: Recommendations for Improving the Quality of Referrals

■ Improving Quality of Care for Hospitalized Smokers with HIV:Tobacco Dependence Treatment Referral and Utilization

Methods, Tools, and Strategies■ Anatomy of Inpatient Falls: Examining Fall Events Captured by

Depth-Sensor Technology

Departments Tool Tutorial■ Development and Implementation of an Early-Onset Sepsis

Calculator to Guide Antibiotic Management in LatePreterm and Term Neonates

“The standard of reporting of quality interventions and QI techniques in surgery is often suboptimal, making it

difficult to determine whether an intervention can be replicated and used to

deliver a positive effect in another setting.”

—How Well Is Quality Improvement Described in the Perioperative Care

Literature? A Systematic Review (p. 196)

The Joint Commission Journal on Quality and Patient Safety

Volume 42 Number 5May 2016232

One of the most common management decisions pedi-atricians face involves early-onset sepsis (EOS). Which

infants should be evaluated? Which infants should receive empiric antibiotics? If an infant is clinically ill, the decision is straightforward, as most pediatricians will treat empirically while they await the results of blood cultures or other diagnos-tic tests. However, in most cases the newborn is well appearing or only has minor physiologic disturbances (tachypnea, tem-perature instability). In general, the presence of risk factors such as group B Streptococcus (GBS), inadequate intrapartum anti-biotic prophyl axis (IAP), maternal chorioamnionitis, and pro-longed rupture of membranes (≥ 18 hours) has been used to decide which infants to evaluate with a blood culture and/or treat empirically with antibiotics. However, making a rational decision to evaluate or treat an infant should depend on the probability of infection as well as the cost-benefit relationship between unnecessarily treating uninfected infants and delaying antibiotic treatment in infected infants.

Widespread use of IAP has led to a sharp decline in EOS from GBS.1–4 The Centers for Disease Control and Prevention (CDC) reports that the rate of EOS secondary to GBS has de-creased from 1.7 cases per 1,000 births (1993) to 0.28 cases per 1,000 births.5 During this time, the incidence of non-GBS EOS has declined as well.6,7 Although the population risk has declined, how do we determine an infant’s individual risk? The CDC 2002 (and revised 2010) guidelines for the prevention of neonatal GBS disease provide algorithms for the evaluation of infants at risk for GBS EOS.6,7 Presumably, infants with a higher probability of EOS should have an evaluation or empir-ic treatment recommended, but the algorithm neither gives an explicit risk nor specifies risk levels for evaluation or treatment. Although simple to use, the CDC algorithm has several draw-backs. Information is lost as continuous variables are dichoto-mized (that is, rupture of membranes ≥ 18 hours and gestational

age < 37 weeks). Using chorioamnionitis, a clinical and thus less reliable predictor, is also problematic. In the current era, when internet access and smartphones are ubiquitous, more sophis-ticated, multivariate risk prediction models can be employed.

Under current CDC guidelines, the percentage of infants be-ing treated with antibiotics is ~200-fold higher than the inci-dence of EOS. In 2008 and 2009, clinicians at Brigham and Women’s Hospital (Boston) treated 8% of well-appearing in-fants born at ≥ 34 weeks’ gestation with antibiotics, while the incidence of EOS among those infants was only 0.4 cases per 1,000 live births.8 Similarly, an internal review of births in Kai-ser Permanente Northern California between 2010 and 2013 demonstrated that 15% of infants born ≥ 34 weeks’ gestation had blood cultures, and 5.4% received antibiotics, despite an incidence of EOS of only 0.3 cases per 1,000 live births.

Unnecessary evaluations and antibiotic treatment are not risk free. In the short term, obtaining blood cultures may result in infant discomfort and parental anxiety. Given the low incidence of EOS, false positive cultures will outnumber the true posi-tive and inevitably lead to more laboratory tests and/or unneed-ed antibiotic exposure. Infants receiving antibiotics are often admitted to a neonatal ICU for treatment; this may interrupt parental bonding and breastfeeding. Finally, an emerging liter-ature suggests that early antibiotic exposure may be associated with diseases later in childhood. Studies have shown an associa-tion between early antibiotic exposure and asthma,9–16 allergic/autoimmune disease,17–21 and obesity.22–25 Additional studies are needed to delineate the relationship between early antibiotic ex-posure on the neonatal microbiome and later disease. Prudence should dictate a more conservative stance toward exposing large numbers of newborns to broad-spectrum antibiotics.

Our goal was to implement an evidence-based approach for the evaluation and management of newborns suspected of EOS. To achieve this goal, we adapted two predictive models

Tool Tutorial

Development and Implementation of an Early-Onset Sepsis Calculator to Guide Antibiotic Management in Late Preterm and Term NeonatesReaders may submit Tool Tutorial inquiries and submissions to Steven Berman, [email protected].

Michael W. Kuzniewicz, MD, MPH; Eileen M. Walsh, RN, MPH; Sherian Li, MS; Allen Fischer, MD; Gabriel J. Escobar, MD

Copyright 2016 The Joint Commission


Volume 42 Number 5May 2016 233

developed by our research team for use in an integrated health care delivery system. In this article, we describe the strategy and the tools that we employed. The central component of our ef-forts was to take advantage of new technologies not used by the CDC guideline—Web-based analytic tools, smart phones, and an administrative infrastructure for establishing consensus across a network of 14 delivery hospitals.

Tool Development and Description Early-OnsEt sEpsis risk prEdictiOn MOdEl

In two recent reports,26,27 our team described the development and validation of two linked predictive models for EOS. These models employ a Bayesian approach. The first model establish-es a newborn’s baseline probability of EOS entirely on the basis of the maternal risk factors (or EOS risk at birth). EOS risk at birth is calculated using gestational age and maternal variables that are available at the moment of birth (highest antepartum temperature, GBS carriage status, duration of rupture of mem-branes, and the nature and timing of IAP).

To maximize the reliability of the variables, we favored ob-jective variables such as “highest maternal antepartum tempera-ture” over more subjective variables such as “physician diagnosis of chorioamnionitis.” We used nonparametric smoothing meth-ods to incorporate continuous variables. Thus, instead of dichot-omizing continuous variables, as is done in the CDC algorithm, the relationship of each variable to the outcome was analyzed separately and then combined into a multivariate model.

Early-OnsEt sEpsis calculatOr

One difficulty with a multivariate risk prediction model is that it is not as simple to use as the CDC flow diagram. Ex-pecting clinicians to manually calculate risk using regression model coefficients is unrealistic. To facilitate clinical adoption, we built Web-based calculators for desktop Web browsers and smartphones that allow clinicians to quickly enter patient-spe-cific data and calculate individualized risk of EOS.28 Physi-cians deter mine sepsis risk using the calculator when admitting a patient and writing his or her history and physical (H&P). As had been done in the past under CDC guidelines, nursing staff alert pediatric providers of risk factors (GBS–positive sta-tus, maternal fever, obstetricians’ chorioamnionitis diagnoses), or clinical symptoms (tachypnea, increased work of breathing, fever), particularly in those instances in which a physician was not completing the H&P immediately. In those instances, pe-diatric providers calculate the initial sepsis risk and determine appropriate management.

dissEMinatiOn Of Early-OnsEt sEpsis calculatOr

KPNC serves a population of 3.9 million members, which constitutes nearly 40% of the insured population in Northern California. Fourteen hospitals have obstetrical and neonatal ser-vices, which care for approximately 35,000 newborns each year. Newborn care is provided by a combination of hospital-based pediatricians, neonatal nurse practitioners, and neonatologists. Two of the facilities also have pediatric residents involved in the care of newborns. The EOS calculator and an explanation of the risk prediction model were presented to the neonatology chiefs with a “soft launch” (no guidance was given to staff with respect to intervention thresholds) in December 2012. This strategy permitted staff to familiarize themselves with the calculator and probability of EOS at birth.

Critical to the introduction of the EOS calculator was an ad-ministrative structure that has developed at KPNC during the past decade. Three key venues for the introduction of initiatives are the Perinatal Research Unit (PRU), the Neonatology Chiefs Group, and the electronic health record (EHR) (Kaiser Perma-nente HealthConnect [KPHC]) governance group. Often, an idea will be brought forward at the Chiefs group. A request goes out to the regionwide journal club for a review of the literature and to the PRU for analysis of internal data. After a consensus is built around a specific clinical practice, those ideas go back to the journal club for further vetting and then back to the Chiefs for endorsement. Whenever possible, tools are built into the EHR to support and reinforce that new practice.

incOrpOratiOn Of clinical prEsEntatiOn

The second predictive model for EOS that we developed quantifies how the baseline risk is modified by the infant’s clin-ical examination.27 Using a variety of statistical and visualiza-tion techniques, our team analyzed newborn data to define a hierarchical classification scheme for a newborn’s evolving clin-ical examination in the first 12 hours of life. We defined three clinical-presentation categories (clinical illness, equivocal, and well appearing) and calculated likelihood ratios (LRs) for each. Through a combination of recursive partitioning (classification and regression trees) and logistic regression, we stratified infants on the basis of both their clinical presentation and the three lev-els of sepsis risk at birth (< 0.65 cases/1,000 live births, 0.65–1.54 cases /1,000 live births, ≥ 1.54 cases/1,000 live births). On the basis of the posterior probability of EOS in these strata and estimated numbers needed to treat (NNT), management choic-es were suggested (continued observation, observe and evaluate, and treat empirically). In August 2013, at a regional KPNC journal club for neonatal staff, we formally introduced the EOS




calculator, as well as thresholds for treatment based on this strat-ification scheme.

prOblEMs with Our Original risk stratificatiOn Our original stratification strategy suffered from several lim-

itations. Collapsing EOS risk at birth into three categories re-sults in significant information loss, which is most pronounced in the highest-risk group. Although the low (< 0.65 cases/1,000 live births) and medium (0.65–1.54/1,000 live births) EOS risk at birth groups had relatively narrow ranges of risk, a much broader range of risk is present in the high-risk (≥ 1.54 cases/1,000 live births) group. Therefore, information loss was particularly problematic among well-appearing infants in the highest-risk group. The mean posterior probability in this group was 6.74 cases/1,000 infants (NNT, 148). However, infants whose risk was close to the lower bound of the high-risk range would ac-tually have a posterior probability of < 1 case /1,000 live births (NNT ≥ 1,000) when the LR for a normal exam is taken into account. In addition, the strata means were determined from the nested case-control study, not a population-based sample. Because the incidence of EOS is relatively rare, the nested-case control study with three controls for case, by design, results in an EOS–enriched cohort.

Consequently, we examined more recent data (KPNC 2010–2012 births) and calculated EOS risk at birth for all infants. We found a left-skewed risk distribution (Figure 1, right); that is, a large percentage of infants in our original high-risk stratum had a risk near the lower bound of the distribution. Thus, applying our initial stratification schema would result in the treatment of a large number of infants in the high-risk group who, if well appearing, would have a low individual risk.

dEtErMining individual risk Of EOs (incOrpOrating thE clinical prEsEntatiOn)

To remedy this issue, we utilized Bayes’s theorem (prior odds multiplied by the LR equals the posterior odds) to calculate in-dividual posterior probabilities for EOS risk for each infant. The LR equals the probability of a particular finding in individ-uals with disease divided by the probability of a particular find-ing in individuals without disease. We converted the probability of EOS risk at birth to odds to determine the prior odds (Prob-ability = odds / [odds+1]). The finding of interest is the infant’s evolving clinical presentation after birth. We used LRs from our prior work,27 as follows:

■■ Clinical illness, 14.5 (95% confidence interval [CI] 10.2–21.2)

■■ Equivocal presentation, 3.75 (95% CI 2.83–5.00)

■■ Well appearing, 0.36 (95% CI 0.31–0.41) We wanted to give clinicians a single value for EOS risk rath-

er than a range. However, because the LR estimate has an uncer-tainty bound, we reasoned that using the point estimate of the LR could be dangerous (and lead to underestimation of an in-fant’s risk). To be conservative and estimate the maximum EOS risk, we used the upper limit of the 95% confidence interval of the LR instead of the point estimate of the LR itself (21.20 for clinical illness, 5.00 for equivocal presentation, and 0.41 for well-appearing presentation). Figure 2 (page 235) illustrates the effect of clinical presentation on the posterior probability of EOS for a given EOS risk at birth (prior probability). A well- appearing presentation will decrease the posterior probability, an equivocal presentation increases the posterior probability, and clinical illness greatly increases the posterior probability (and has the most dramatic effect).

sEtting trEatMEnt thrEshOlds

Now that we could determine an individual’s risk for EOS, we needed to define management thresholds. We decided to establish two: one for continued observation/blood culture and one for empiric antibiotics/blood culture. Unfortunately, there are no randomized controlled trials to inform the probability of infection at which the benefits of starting antibiotics imme-diately exceed the risk and costs of delaying treatment. It is important to remember that the treatment choice is not that of

Distribution of Early-Onset Sepsis (EOS) Risk at Birth in High-Risk Group, Kaiser

Permanente Northern California, 2010–2012

Figure 1. The calculated EOS risk at birth of infants in the original high-risk stratum had a risk near the lower bound of the distribution, indicating a need to revise the original model.

EOS Risk at Birth (Cases per 1,000 Live Births)

Per

cent

age




antibiotics versus no antibiotics but, rather, between (a) starting antibiotics empirically, (b) drawing a blood culture and wait-ing to start antibiotics if the infant becomes clinically ill or the blood culture becomes positive, or (c) observing the infant and waiting to start antibiotics if the infant becomes clinically ill. On September 4, 2012, we convened a one-day symposium of local experts in the San Francisco Bay Area and queried the KPNC chiefs of neonatology to help determine a risk threshold for in-tervention. We reached consensus for (a) posterior probability of EOS of ≥ 1/1,000 live births (NNT29 1,000) to obtain a blood culture, monitor vital signs every 4 hours in the mother-baby unit, and remain in the hospital until the culture was incubated for 24 hours, and (b) a posterior probability of EOS of ≥ 3/1,000 (NNT 333) to obtain a blood culture and start empiric antibi-otics. We reasoned that these NNT were relatively conservative. For a comparison in perinatal medicine, the risk of uterine rup-ture complicating vaginal birth after cesarean section (VBAC) is ~1%, leading to death or severe neurologic injury in the infant in approximately 10% of those births.30–33 Thus, the risk of death or severe neurologic injury after VBAC is about 1/1,000 VBACs or a number needed to harm of 1,000.

rEvisiOns tO thE Early-OnsEt sEpsis calculatOr tO iMprOvE clinical accEptancE

In July 2014 we added several enhancements to the calcu-lator. We modified some of the specifications in our original papers to improve face validity and clinician acceptance. These modifications centered on the need to facilitate dissemination across 14 hospitals. To broaden the ability to apply the calcula-

tor to different populations, we added the ability to select the baseline prevalence of EOS in the population being managed. Because the predictive model was developed using data from the nested-case control study, its intercept term needs to be ad-justed to reflect the prevalence of EOS in the population to which it is being applied.26,34

The original EOS calculator was adjusted for an EOS prev-alence of 0.6 cases per 1,000 live births—the prevalence of the population from which the nested cases and controls were drawn.8 However, in KPNC the current (2010–2012) preva-lence of EOS was 0.3 cases per 1,000 live births. The revised cal-culator allows the user to select a baseline prevalence of 0.3–0.6/ 1,000 live births, in increments of 0.1.

The original risk prediction model used broad-spectrum an-tibiotics < 4 hours prior to delivery and GBS–specific antibi-otics any time prior to delivery as predictors that reduced the probability of EOS. However, even with the large study pop-ulation of 608,014, there were not enough cases for antibiot-ic administration to be included in the model as a continuous variable. Further, antibiotic timing as displayed in the calcula-tor lacked face validity because clinical efficacy of antibiotics administered shortly before delivery was questioned. Conse-quently, we reviewed the IAP timing literature with respect to efficacy of preventing EOS. Evidence exists that IAP had some effect even if administered as little as 30 minutes prior to de-livery, but the effectiveness of IAP in preventing GBS if given ≥ 2 hours was ≥ 89%.35–38 Therefore, we elected to be more conservative than the risk-prediction model by stipulating that antibiotic administration must occur at least 2 hours before de-livery, which increased face validity and clinician acceptance. In deciding on adequate IAP for GBS, we instructed clinicians to include erythromycin and clindamycin as adequate prophylaxis only if GBS cultures had shown sensitivity to these antibiot-ics. We also modified the clinical presentation descriptions and timing specifications. Because infants often have transient respi-ratory distress transitioning to ex-utero life, we explicitly state that the need for respiratory support needed to occur beyond the delivery room. We added high-flow nasal cannula (which has become more widely used) to the list of respiratory support. We expanded the study criterion of seizure to the more inclusive criterion of encephalopathy. For the supplemental oxygen re-quirement, we added a caveat that supplemental oxygen needed to maintain pulse oximetry of > 90% for > 2 hours to avoid mis-classification. For the equivocal presentation, we clarified time periods regarding vital signs abnormalities.

We expanded the time frame to identify clinical illness or equivocal presentation from 12 hours to 24 hours, which is

.0001 .0002 .0005 .001 .002 .003 .005 .007 .008 .009 .010

Sepsis Risk @ Birth (Prior Probability) = 0.5/1,000

PROBABILITY of SEPSIS

LR 0.41

LR 5.0

LR 21.2

Well Appearing

Equivocal

Clinical Illness

Effect of Clinical Presentation on Early-Onset Sepsis (EOS) Risk

Figure 2. The effect of likelihood ratios (LRs) for clinical presentation on the posterior probability of EOS is shown.




consistent with a more conservative stance. Table 1 (above) summarizes our modified criteria. The most notable enhance-ment was adding posterior probabilities for each of the three clinical exam categories. This provides the user with (a) the in-dividual probability for EOS, incorporating both the EOS risk at birth and clinical presentation, and (b) a clinical recommen-dation based on it. The clinical recommendations play an im-portant purpose, as we found that in the comment phase absent recommendations, physicians often set their own thresholds for intervention (rather than the consensus thresholds endorsed by the neonatology chiefs). One of the goals of the EOS calcula-tor was to reduce practice variability so that the efficacy, as well as the safety, of the treatment thresholds can be evaluated. Al-though physicians remain free to do what they wish, it is much harder to stray from the recommendations if the calculator—whose output is visible to nurses and other physicians—gives a specific recommendation rather than just a number. Prior to de-veloping the calculator, we saw tremendous variability between centers with regard to antibiotic usage. When a clinician en-ters in the maternal and infant data, the calculator produces the risk of EOS and clinical recommendations for all three clinical presentations. If an infant developed clinical symptoms several hours after birth, the physician would apply the clinical recom-mendation based on the new clinical presentation status. There is no need to reenter data into the calculator because it provides recommendations on all three clinical presentations.

Also, we introduced two additional safeguards into the cal-

culator. Although the LR for clinical illness was high (21), it was theoretically possibly to have an EOS risk at birth so low such that, when multiplied by 21, a posterior probability of < 3 cases/ 1,000 live births (clinical recommendation would be not to initi-ate antibiotics) would have resulted. We did not want physicians to withhold antibiotics in an infant with clinical illness, even if his or her posterior probability was below the consensus threshold. Therefore, the EOS calculator clinical recommendation for an infant with clinical illness and a posterior probability of < 3 cases/ 1,000 live births reads, “Strongly consider antibiotics.”

Finally, we recognized that a septic infant might appear well at birth and then develop symptoms later. Because a ma-jor component of the EOS calculator is clinical presentation and its effect on the posterior probability, we wanted to ensure that infants with a higher EOS risk at birth (≥ 1 case/1,000 live births) would not merely have their posterior probability re-duced by a reassuring exam shortly after birth. The infant might then be placed in the newborn nursery under routine care and have the identification of clinical symptoms missed or delayed. As a result, we added an additional clinical recommendation on the basis of the EOS risk at birth: If the EOS risk at birth was ≥ 1 case/1,000 live births, the clinical recommendation is for vital signs every 4 hours for 24 hours to encourage more timely iden-tification of any clinical symptoms that might arise.

Use of the Early-Onset Sepsis CalculatorIn December 2012 we made the EOS calculator available in both a computer-formatted website (http://www.kp.org\EOScalc [available as of May 1, 2016]) and a smartphone-formatted dis-play (http://www.Newborn sepsiscalculator.org). The computer- formatted website is shown in Figure 3 (page 237, also available in color in online article). Key issues surrounding implementa-tion of the EOS calculator and their solutions are summarized in Table 2 (page 238).

The calculators are being used widely at KPNC. The calcula-tor has been accessed by users in all 50 states, with 52% of page visits by users in California, and by users in 124 other countries. Figure 4 (page 238) shows the monthly use of the desktop cal-culator. The mobile application currently receives 1,500 page views per month.

Monitoring the Impact and Safety of the Early-Onset Sepsis CalculatorWe are actively monitoring the impact of the EOS calculator by following monthly rates of sepsis evaluations in the first 24 hours (defined by obtaining a blood culture), as well as antibi-otics administered in the first 24 hours (ascertained from the

Table 1. Classification of Clinical Presentation

Clinical illness 1. Persistent need for NCPAP/ HFNC/mechanical ventilation (outside of the delivery room)

2. Hemodynamic instability requiring vasoactive drugs

3. Neonatal encephalopathy/Perinatal depression• Seizure• Apgar Score @ 5 minutes < 5

4. Need for supplemental O2 > 2 hours to maintain oxygen saturations > 90% (outside of the delivery room)

Equivocal presentation

1. Persistent physiologic abnormality > 4 hours• Tachycardia (HR > 160)• Tachypnea (RR > 60)• Temperature instability (> 100.4˚F or < 97.5˚F)• Respiratory distress (grunting, flaring, or

retracting) not requiring supplemental O2

2. Two or more physiologic abnormalities lasting for > 2 hours

Well appearing 1. No persistent physiologic abnormalitiesNCPAP, nasal continuous positive airway pressure; HFNC, high-flow nasal cannula; HR, heart rate; RR, respiratory rate.




electronic medication administration record). Rates are deter-mined for KPNC, as well as individual medical centers. To en-sure that we are not missing infants with EOS with our lower rate of sepsis evaluations, we are tracking any readmissions in the first week of life with a positive blood or cerebrospinal fluid (CSF) culture. As an integrated system, we can track all subse-quent laboratory evaluations and readmissions anywhere in the KPNC system. In addition, we review every EOS case to de-termine if antibiotics would have been started earlier under the CDC guidelines and if any adverse outcomes occurred, such as meningitis, need for mechanical ventilation, need for inotropic agents, and death. We plan to publish data on utilization and safety after 18 months of implementation of the EOS calculator.

Results and Lessons LearnedWe have developed a Web-based EOS calculator that provides perinatal practitioners with an explicit risk estimate. The cal-culator’s Bayesian structure reflects the actual flow of clinical experience while simultaneously providing the capability of up-dating a risk estimate. By calculating explicit risk and using a guideline with specific intervention thresholds, it is possible to better target the use of antibiotics in high-risk situations, while forgoing their use when the risk is low. Use of the calculator fits well with standard hospital work flow, which has contributed to its broad adoption within KPNC. The potential benefits of re-duced antibiotic exposure in low-risk situations is significant, as emerging evidence has shown associations between early antibi-otic therapy and subsequent diseases such as asthma, inflamma-tory bowel disease, and childhood obesity.22,39–43

Reduction in sepsis evaluations and antibiotic usage will have to be weighed against other important clinical outcomes. Hospital readmission for an infant with sepsis despite an as-sessment that the infant was “low-risk” is the obvious concern. However, this scenario is unlikely, given that nearly all cases of EOS present with clinical illness before 24 hours, and most be-fore 12 hours of age.44–46 We have an active surveillance system. EOS cases (if any) are reviewed on a monthly basis. In addition, because KPNC is an integrated system in which newborns are covered under their mother’s insurance for at least the first 30 days, we are tracking any readmissions with a positive blood or CSF culture in the first seven days of life.

An important element of the EOS calculator is that it pro-vides individualized risk estimates in conjunction with man-agement recommendations (as opposed to the CDC’s isolated management recommendations). First, it provides physicians (and potentially parents) with an actual numeric risk value. This is beneficial as one weights the benefit and risks of treat-

ment with nontreatment. Treatment thresholds become more explicit and transparent, enabling the physician to use the ac-tual risk value to explain to parents why treatment is or is not needed. The EOS calculator also allows for a further Bayesian approach in select cases with use of the LR for the complete blood count.46,47

To implement the calculator, modifications from the ini-tial research studies were necessary to increase face validity, add additional safeguards, and promote clinical adoption. Our ad-ministrative structure supported the physician education and consensus building necessary to change clinical practice. Links to the calculator and documentation templates in the EHR fur-ther support and standardize the new quantitative tool as it is adopted in an entire hospital network. Anytime one strays from the specifications in a research study, there is the possibility of not obtaining the identical results produced by the study. How-ever, a protocol that does not garner clinical acceptance will

Early-Onset Sepsis (EOS) Online Calculator

Figure 3. A computer-formatted screenshot of the EOS calculator website, as shown at http://www.kp.org\EOScalc (available as of May 1, 2016).




never be put into practice. Because we were departing from the established CDC algorithm, it was particularly important that clinicians understood the reasoning behind the EOS calculator, that the approach had clinical face validity, and that additional safeguards were built into the approach. The modifications we made were always more conservative than the original studies.

We have demonstrated the translation of a relatively complex, statistically sophisticated health services research tool into daily clinical practice. The bedrock of any such effort is the quality of the research design and the subsequent data analysis. Transla-tion to clinical practice in a large hospital network requires a ro-bust administrative infrastructure that facilitates organizational learning and consensus building. Change in clinical practice is enforced by effective implementation of the EHR. J

References1. Bizzarro MJ, et al. Seventy-five years of neonatal sepsis at Yale: 1928-2003. Pediatrics. 2005;116:595–602.2. Moore MR, Schrag SJ, Schuchat A. Effects of intrapartum antimicrobial prophylaxis for prevention of group-B-streptococcal disease on the incidence and ecology of early-onset neonatal sepsis. Lancet Infect Dis. 2003;3:201–213.3. Phares CR, et al. Epidemiology of invasive group B streptococcal disease in the United States, 1999-2005. JAMA. 2008 May 7;299:2056–2065.4. Puopolo KM, Eichenwald EC. No change in the incidence of ampicillin- resistant, neonatal, early-onset sepsis over 18 years. Pediatrics. 2010;125:e1031–1038.5. Centers for Disease Control and Prevention. Group B Strep (GBS): Clinical Overview. (Updated: Jun 1, 2014.) Accessed Mar 28, 2016. http://www.cdc.gov/groupbstrep/clinicians/clinical-overview.html. Published 2015.6. Schrag S, et al. Prevention of perinatal group B streptococcal disease. Revised guidelines from CDC. MMWR Recomm Rep. 2002 Aug 16;51(RR-11):1–22.7. Verani JR, McGee L, Schrag SJ; Division of Bacterial Diseases, National Center for Immunization and Respiratory Diseases, Centers for Disease Con-trol and Prevention. Prevention of perinatal group B streptococcal disease: Revised guidelines from CDC, 2010. MMWR Recomm Rep. 2010 Nov 19; 59(RR-10):1–36.8. Mukhopadhyay S, Eichenwald EC, Puopolo KM. Neonatal early-onset sep-sis evaluations among well-appearing infants: Projected impact of changes in CDC GBS guidelines. J Perinatol. 2013;33:198–205.9. Droste JH, et al. Does the use of antibiotics in early childhood increase the risk of asthma and allergic disease? Clin Exp Allergy. 2000;30:1547–1553.10. Muc M, Padez C, Pinto AM. Exposure to paracetamol and antibiotics in early life and elevated risk of asthma in childhood. Adv Exp Med Biol. 2013; 788:393–400.11. Murk W, Risnes KR, Bracken MB. Prenatal or early-life exposure to anti-

Michael W. Kuzniewicz, MD, MPH, is Director, Perinatal Research Unit, Kaiser Permanente Division of Research, Oakland, California. Eileen M. Walsh, RN, MPH, is Senior Research Project Manager and Sherian Li, MS, is Senior Data Analyst, Kaiser Permanente Di-vision of Research. Allen Fischer, MD, is Regional Director of Neo-natology, and Gabriel J. Escobar, MD, is Regional Director, Hospital Operations Research, Kaiser Permanente Northern California, Oak-land. Please address correspondence to Michael W. Kuzniewicz, [email protected].

Early-Onset Sepsis (EOS) Online Calculator: Page Visits per Month

Figure 4. Page visits per month for the EOS Online Calculator are shown.

Table 2. Key Issues in Incorporating Early-Onset Sepsis Calculator into Clinical Practice

Issue SolutionClinical acceptance of calculator

Avoided “black box” calculator

Extensive education on development of risk prediction model and components

Information loss due to recom mendations based on EOS risk in large infant subsets

Use of actual likelihood ratios to generate individual risk estimates

Limitations of clinical exam categories in original report

Modification made to original clinical categories

Safety concerns around uncertainty of risk estimates

Use of likelihood ratios’ upper 95% confidence intervals for calculations to maximize risk estimates

Manual calculation of model coefficients impractical

Development of Web-based calculator

How to translate probabilities into discrete clinical actions

Recommendations embedded in calculator

Departing from CDC guideline

Educational component highlighting evidenced-based approach with analyses of more recent KPNC data

Prospective data collection strategy implemented that includes balancing measures (such as readmissions for EOS and complications due to delayed antibiotic treatment), tracking possible adverse outcomes

EOS, early-onset sepsis; CDC, Centers for Disease Control and Prevention; KPNC, Kaiser Permanente Northern California.




biotics and risk of childhood asthma: A systematic review. Pediatrics. 2011; 127: 1125–1138.12. Ong MS, Umetsu DT, Mandl KD. Consequences of antibiotics and in-fections in infancy: Bugs, drugs, and wheezing. Ann Allergy Asthma Immunol. 2014;112:441–445.e1.13. Risnes KR, et al. Antibiotic exposure by 6 months and asthma and allergy at 6 years: Findings in a cohort of 1,401 US children. Am J Epidemiol. 2011 Feb 1;173:310–318.14. Sobko T, et al. Neonatal sepsis, antibiotic therapy and later risk of asthma and allergy. Paediatric Perinat Epidemiol. 2010;24:88–92.15. Stensballe LG, et al. Use of antibiotics during pregnancy increases the risk of asthma in early childhood. J Pediatr. 2013;162:832–838.e3.16. Sun W, et al. Early-life antibiotic use is associated with wheezing among children with high atopic risk: A prospective European study. J Asthma. 2015; 52:647–652.17. Tsakok T, et al. Does early life exposure to antibiotics increase the risk of eczema? A systematic review. Br J Dermatol. 2013;169:983–991.18. Schokker D, et al. Early-life environmental variation affects intestinal microbiota and immune development in new-born piglets. PLoS one. 2014 Jun 18;9:e100040.19. Mårild K, et al. Antibiotic exposure and the development of coeliac disease: A nationwide case-control study. BMC Gastroenterol. 2013 Jul 8;13:109.20. Li M, Wang M, Donovan SM. Early development of the gut microbiome and immune-mediated childhood disorders. Semin Reprod Med. 2014;32:74–86.21. Arvonen M, et al. Repeated exposure to antibiotics in infancy: A predis-posing factor for juvenile idiopathic arthritis or a sign of this group’s greater susceptibility to infections? J Rheumatol. 2015;42:521–526.22. Trasande L, et al. Infant antibiotic exposures and early-life body mass. Int J Obes (London). 2013;37:16–23.23. Murphy R, et al. Antibiotic treatment during infancy and increased body mass index in boys: an international cross-sectional study. Int J Obes (London). 2014;38:1115–1119.24. Bailey LC, et al. Association of antibiotics in infancy with early childhood obesity. JAMA Pediatr. 2014;168:1063–1069.25. Azad MB, et al. Infant antibiotic exposure and the development of childhood overweight and central adiposity. Int J Obes (London). 2014;38:1290–1298.26. Puopolo KM, et al. Estimating the probability of neonatal early-onset in-fection on the basis of maternal risk factors. Pediatrics. 2011;128:e1155–1163.27. Escobar GJ, et al. Stratification of risk of early-onset sepsis in newborns ≥ 34 weeks’ gestation. Pediatrics. 2014;133:30–36.28. Kaiser Permanente. Probability of Neonatal Early-Onset Infection Based on Maternal Risk Factors. 2015. Accessed Mar 28, 2016. https://www.dor.kaiser .org/external/DORExternal/research/InfectionProbabilityCalculatorOLD .aspx.29. Walka MM, et al. Complete blood counts from umbilical cords of healthy term newborns by two automated cytometers. Acta Haematol. 1998;100: 167–173.

30. Barger MK, et al. Maternal and newborn outcomes following uterine rup-ture among women without versus those with a prior cesarean. J Matern Fetal Neonatal Med. 2013;26:183–187.31. Barger MK, et al. Severe maternal and perinatal outcomes from uterine rupture among women at term with a trial of labor. J Perinatol. 2012;32:837–843.32. Ouzounian JG, et al. Vaginal birth after cesarean section: Risk of uterine rupture with labor induction. Am J Perinatol. 2011;28:593–596.33. Signore C. VBAC: What does the evidence show? Clin Obstet Gynecol. 2012;55:961–968.34. Kelley CT. Solving Nonlinear Equations with Newton’s Method. Philadel-phia: Society for Industrial and Applied Mathematics, 2003.35. Colombo DF, et al. Optimal timing of ampicillin administration to preg-nant women for establishing bactericidal levels in the prophylaxis of group B Streptococcus. Am J Obstet Gynecol. 2006;194:466–470.36. de Cueto M, et al. Timing of intrapartum ampicillin and prevention of ver-tical transmission of group B Streptococcus. Obstet Gynecol. 1998;91:112–114.37. Lin FY, et al. The effectiveness of risk-based intrapartum chemoprophylaxis for the prevention of early-onset neonatal group B streptococcal disease. Am J Obstet Gynecol. 2001;184:1204–1210.38. Illuzzi JL, Bracken MB. Duration of intrapartum prophylaxis for neo-natal group B streptococcal disease: A systematic review. Obstet Gynecol. 2006;108:1254–1265.39. Russell SL, et al. Early life antibiotic-driven changes in microbiota enhance susceptibility to allergic asthma. EMBO Rep. 2012 May 1;13:440–447.40. Kozyrskyj AL, Ernst P, Becker AB. Increased risk of childhood asthma from antibiotic use in early life. Chest. 2007;131:1753–1759.41. Kim BS, Jeon YS, Chun J. Current status and future promise of the human microbiome. Pediatr Gastroenterol Hepatol Nutr. 2013;16:71–79.42. Hviid A, Svanström H, Frisch M. Antibiotic use and inflammatory bowel diseases in childhood. Gut. 2011;60:49–54.43. Ajslev TA, et al. Childhood overweight after establishment of the gut mi-crobiota: The role of delivery mode, pre-pregnancy weight and early adminis-tration of antibiotics. Int J Obes (London). 2011;35:522–529.44. Escobar GJ, et al. Neonatal sepsis workups in infants ≥ 2000 grams at birth: A population-based study. Pediatrics. 2000;106(2 Pt 1):256–263.45. Ottolini MC, et al. Utility of complete blood count and blood culture screening to diagnose neonatal sepsis in the asymptomatic at risk newborn. Pediatr Infect Dis J. 2003;22:430–434.46. Bromberger P, et al. The influence of intrapartum antibiotics on the clinical spectrum of early-onset group B streptococcal infection in term infants. Pediat-rics. 2000;106(2 Pt 1):244–250.46. Newman TB, et al. Combining immature and total neutrophil counts to predict early onset sepsis in term and late preterm newborns: Use of the I/T2. Pediatr Infect Dis J. 2014;33:798–802.47. Newman TB, et al. Interpreting complete blood counts soon after birth in newborns at risk for sepsis. Pediatrics. 2010;126:903–909.


improvement from may 2016 front office to front line ...decision to evaluate or treat an infant...

Documents