Outcomes of Antimicrobial Stewardship –

What Metrics are Worth Measuring?

Scott Bergman, PharmD, FCCP, FIDSA, BCPSPharmacy Coordinator, Nebraska Medicine

Antimicrobial Stewardship Program

@bergmanscott #NebSteward2019

Co-coordinator Nebraska ASAP

Clinical Associate Professor

UNMC College of Pharmacy

Objectives1. Describe metrics that can be used to measure outcomes of antimicrobial stewardship programs

2. Design a plan for monitoring the impact of your antimicrobial stewardship intervention on antibiotic use and quality

2

Post-Summit Activity: Using the information from previous sessions, choose one high-priority problem that you would like your antimicrobial stewardship program to address within the next 6 to 12 months.

• Identify an antimicrobial stewardship intervention that best fits a real problem within your practice model

• Identify additional resources or partners that are needed to make this strategy successful

• Design a plan for monitoring the impact of your antimicrobial stewardship intervention on antibiotic use and quality

The Problem with Metrics

What is GOOD about this measurement?What is BAD about this measurement?

Goals of Stewardship Programs

McGowan JE. Infect Control Hosp Epidemiol. 2012; 33(4): 331-337., Wagner B, et al. Infect Control HospEpidemiol. 2014; 35(10): 1209-1228. Dodds Ashley ES, et al. Clin Infect Dis. 2014; 59(S3): S112-S121.

Improve Patient

Outcomes

Improve Patient Safety

Reduce Antimicrobial

Resistance

Reduce Antimicrobial Expenditures

Is There an Ideal Antibiotic Quality Measure?

• Easy to assess compliance

• Will improve practice, change behavior

• Has no unintended consequences

• Practical for any healthcare setting (acute care, long term care, adult/pediatrics)

• Useful to many audiences– Stewardship program staff

– Clinicians

– Accreditation/regulatory agencies

So many metrics!

• Antimicrobial consumption

• Appropriateness of therapy

• Time to appropriate therapy

• Documented indication for

antimicrobial therapy

Dodds Ashley ES, et al. Clin Infect Dis. 2014; 59(S3): S112-S121. Tamma PD. Infect Dis Clin N Am. 2011; 25: 245-260. Toth NR, et al. Am J Health-Syst Pharm. 2010; 67:746-749.

• Infection-related mortality• Length of stay• Readmission rates• Clostridium difficile rates• Antimicrobial resistance rates• Clinical success / cure

Outcomes MeasuresProcess Measures

Structure Measures

• Components of the stewardship

program, e.g. Core Elements

Measures of Antibiotic Use

•Defined Daily Dose (DDD)– A standardized metric for drug exposure endorsed by the World Health

Organization

– Limitation: the WHO standard dose is not always the dose that is used in most patients

•Days of Therapy (DOT)– A single day of drug administration regardless of number of doses or

strength

– Considered by many as a more realistic estimate of use

– Limited by data available from electronic systems

– THIS IS THE PREFERRED REPORTING MEASURE FOR NHSN!!

Polk RE. Clin Infect Dis 2007;44:664-670

NHSN AUR Option

The Centers for Disease Control and Prevention (CDC) National Healthcare Safety Network (NHSN) Antibiotic Use and Resistance (AUR) Option

National Objectives

– Monitor and improve antimicrobial prescribing

– Identify, understand, and respond to antimicrobial resistance patterns or trends

Requires barcode medication administration data or electronic medication administration records to be extracted

– Cost and time commitment involved

SAAR: The CDC’s suggested antibiotic use measure

• Standardized Antibiotic Administration Ratio

• Based on Days of Therapy (administered) per 1000 patient days present

• Risk adjusted by facility and location characteristics

– Benefits: the standard for national benchmarking, endorsed by CDC and National Quality Forum

– Limitations: Accuracy of benchmarking dependent on increasing participation in NHSN Antibiotic Use Reporting

SAAR Overview

• Available to facilities reporting data to NHSN’s AU Option from specified patient care locations

• Observed-to-Predicted Ratio (similar to SIR)

• First developed in 2015 using data reported prior

• As more facilities are enroll, additional patient care locations are validated for comparison

• Update: 2017 baseline, released in December 2018

– Greatly enhanced the risk adjustments

• New antimicrobial categories also added

Adult SAAR Agent Groups2014 Baseline 2017 Baseline Includes (Examples)

Broad-spectrum agents used for hospital-onset/multi-drug resistant infections

Broad-spectrum agents used predominately for hospital-onset infections

Piperacillin-tazobactam, cefepime, ceftazidime, meropenem, aminoglycosides

Broad-spectrum agents used predominately for community-acquiredinfections

same Flouroquinolones, ceftriaxone, cefuroxime, oral 2nd-3rd generation cephalosporins, ertapenem

Anti-MRSA agents Agents used for resistant Gm-positive infections (e.g. MRSA)

Vancomycin IV, linezolid, daptomycin, ceftaroline

Agents predominately for surgical site infection prophylaxis (removed)

(New) Narrow spectrum Beta-lactam agents

Cefazolin, cefoxitin, cephalexin, amox ± clav, ampicillin ± sulbactam, naf/oxacillin

All antibacterial agents same Everything reported

New Agents used predominately for extensively resistant bacteria (Rates only)

Tigecycline, Ceftolozane/tazobactam,ceftazidime/avibactam, IV polymyxin B & colistin

New Agents posing the highest risk for Clostridium difficile infection

3rd & 4th gen cephalosporins,flouroquinolones, clindamycin

New Antifungal agents used for invasive candidiasis

Fluconazole, echinocandins (micafungin)

Pediatric SAAR Agent GroupsAll New 2017 Baseline Includes (Examples)

Broad-spectrum agents used for hospital-onset/multi-drug resistant infections

Piperacillin-tazobactam, cefepime, ceftazidime,meropenem, ertapenem, aminoglycosides, Flouroquinolones

Broad-spectrum agents used predominately for community-acquired infections

Ceftriaxone, oral 2nd-3rd gen cephs, amoxicillin-clavulanate, ampicillin-sulbactam,

Narrow agents used predominately for community-acquired infections

Amoxicillin, naf/oxacillin,Cefazolin, cefoxitin, cephalexin

Agents used for resistant Gm-positive agents (e.g. MRSA)

Vancomycin IV, linezolid, daptomycin, ceftaroline, clindamycin

Azithromycin Azithromycin

Agents used predominately for extensively resistant bacteria (Rates only)

Ceftolozane-tazobactam, ceftazidime-avibactam, polymyxin & colistin IV, tigecycline

Agents posing the highest risk for Clostridium difficile infection

3rd & 4th gen cephalosporins,flouroquinolones, clindamycin

All antibacterial agents Everything reported, including above

Antifungal agents used for invasive candidiasis Fluconazole, echinocandins (micafungin)

NICU SAAR Agent Groups, coming Dec 2019

Ceftazidime

2017 baseline SAAR models

Adults SAARs available for:

– Medical, med-surg, surgical wards

– Medical, med-surg, surgical ICUs

– Step-down units (new)

– General hematology-oncology wards (new)

Pediatric SAARs (new) available for:

– Medical, med-surg, ICUs

– Medical, med-surg, surgical wards

AU Example

SAAR Example

2014 Baseline Example

AU Example, 2017 Baseline

SAAR Example, 2017 Baseline

0.5

0.6

0.7

0.8

0.9

1.0

1.1

1.2

1.3

1.4

1.5

20

17

M1

1

20

17

M1

2

20

18

M0

1

20

18

M0

2

20

18

M0

3

20

18

M0

4

20

18

M0

5

20

18

M0

6

20

18

M0

7

20

18

M0

8

20

18

M0

9

20

18

M1

0

20

18

M1

1

20

18

M1

2

20

19

M0

1

All Antimicrobial Agents Used in Adult ICUs, Wards, Step Down Units and Oncology Units

SAAR: Standard Antimicrobial Administration Ratio

Linear (SAAR: Standard Antimicrobial Administration Ratio)

SAAR not available

• Oncology step-down, ICU or BMT

• Pediatric oncology

• Cardiac locations

– Wards, step-down or ICUs

• Neuro locations

• Rates are available though

Long-term Care Facilities

• Antibiotic Starts/1,000 Resident-Days

• Days of Therapy/1,000 Resident-Days

• Take into account short-term post-acute stays

Antibiotic Use Metric*Percent Short-Stay Resident-Days

p-ValueLow Medium High

Starts/1000 Resident-Days (SD)

7.5(5.3)

9.8(3.0)

13.1(4.9)

<0.05

Days of Therapy/1000Resident-Days (SD)

128.1 (68.5)

128.3 (51.9)

179.3 (58.0)

0.12

Long-term Care Metrics

• Assess starts for appropriateness by McGeer or Loeb criteria

• Determine if adequate therapy based on susceptibilities

Ambulatory Clinic Stewardship

0

10

20

30

40

50

60

70

80

E-mail only Tools Delivered to MedicalDirector

Tools to Med Director +Patient-Empowerment

Antibiotic Prescription for Acute Bronchitis

Pre Post-Intervention

P<0.05

Activity: Show me the data

Discuss: What data do you have available for any stewardship metrics? How well do you

think it reflects your stewardship goals?

What influences what we measure?

• Expert opinion

• Local preferences, ASP goals

• Ease of measurement

• Regulatory requirements

• Financial pressures!

Example: C. difficile testing intervention

Example: C. difficile testing trends

Example: C. difficile event tracking

Jan

uar

y

Mar

ch

May

July

Sep

tem

ber

No

vem

ber

Jan

uar

y

Mar

ch

May

July

Sep

tem

ber

No

vem

ber

Jan

uar

y

Mar

ch

May

July

Sep

tem

ber

No

vem

ber

Jan

uar

y

Mar

ch

May

July

Sep

tem

ber

No

vem

ber

Jan

uar

y

Mar

ch

Rat

e p

er

1,0

00

Pt

Day

s

Month

Clostridium Difficile LabID Events2015- 2019

HO Rate

Types of Interventions should guide metrics

Adapted from Moerhing RW Anderson DJ, Curr Infect Dis Rep 2012; 14(6):592-600

RestrictionsOrder Sets

Audit and real-time feedbackIV to PO programsDose Optimization

Audit reportsEducational programs

Guidelines

Technology-based alerts:Bug-drug mismatch

de-escalationDuration of therapy

ProviderAntibiotic

prescriptionPatient

disposition

Front EndBefore Rx

Back EndAfter Rx

Act

ive

Pass

ive

Discussion: What metrics would help

monitor the success/failure of

formulary restrictions?

Example: Formulary Restriction with Preauthorization

ID Restricted + Criteria

ID Restricted

Criteria-Monitored

Unrestricted

Representative Example Goal Supported by RestrictionRestriction Tier

Daptomycin

Linezolid

Polymyxins

Meropenem

Aztreonam

Quinolones

Nafcillin

Cefazolin

↓ resistance, ↓ cost

↓ resistance, ↓ cost, ↑ safety

↓ resistance, ↑ safety

↓ resistance

↓ resistance, ↓ cost, ↑ outcome

↓ resistance, ↑ outcome

↑ outcome

↑ outcome Toxi

city

, res

ista

nce

, su

per

infe

ctio

n, c

ost

,

n

ee

d t

o p

rese

rve

eff

icac

y

Athans V et al. Am J Health-Syst Pharm, 2015

What about the other ASP strategies

Education

Guidelines & clinical pathways

Prospective audit with intervention and feedback

Antimicrobial order forms

Combination therapy

Dose optimization

Streamlining/de-escalation

IV to PO

Activity: For your chosen ASP problem, what metrics represent the relevant goals?

Stewardship goals Possible Metric Your ideas?

Improve patient outcome MortalityReadmissionClinical cure

Improve patient safety C. difficileAcute kidney injuryAdverse drug events

Reduce resistance Antibiogram trends

Reduce cost Drug expendituresDrug utilization LOS

Importance of ASP Outcomes*

Bumpass JB et al. Clinical Infectious Diseases 2014;59(S3):S108–11

Outcome Actually used

Most important*

Admins Pharmacy Director

P&T Committee

ID Physician

Abx USE 73% 15% 2% 22% 32% 2%

Abx COST 73% 10% 41% 56% 15% 0%

Appropriate 51% 56% 5% 5% 15% 27%

Mortality 7% 34% 2% 5% 2% 37%

LOS 12% 22% 5% 0% 2% 7%

*survey of 94 physicians and pharmacists in acute care hospitals

Perceived importance

Problems with Cost Measures

•Evaluating drug related cost misses the bigger picture of

overall length of stay

•Drug prices are not always comparable over time and

between institutions

•Economic endpoints should be used to complement other

stewardship goals

•Like drug utilization, cost should be adjusted by census

Problems with Antibiotic Consumption

• Never focus on just one drug– “Squeezing the balloon”

• Consider measuring and aggregating drugs with a common target or feature– e.g. MRSA agents, all quinolones

• Some antibiotic use is necessary! – “Zero” is not an appropriate goal!

Nicasio AM, et al. J Crit Care 2010;25:69-77.

Problems with Resistance

• Difficult to Measure

• Changes can take a long time

• Confounded by multiple factors (e.g. infection

control, community endemnicity)

• Breakpoint and testing changes over time

can bias results

Philips I. Clin Infect Dis 2004;33(Suppl 3):S130-S132. Cook PP, et al. J Antimicrob

Chemother 2004;53:853-9. MacDougall C, et al. Clin Microbiol Rev 2005;18:638-56.

Most of these same problems are also problems with measuring clinical outcomes and safety

Putting it togethercomprehensive ASP metrics via Delphi method

Measure Strong agreement (%)

Domain 1: ConsumptionDOTsDDDs

8050

Domain 2: Resistance# patients with drug-resistant organismsDe-escalation/optimized therapy

7870

Domain 3: OutcomesAntimicrobial related organism mortalityAll cause mortalityConservable days of therapy among certain patientsUnplanned 30d readmission

787080

100

Adapted from Morris AM et al. Infect Control Hosp Epidemiol 2012;33(5):500-506

Tons of great examples

• Automated reports for ASP outcomes and cost metrics

– Nowak MA et al. Am J Health-Syst Pharm. 2012; 69:1500-8

• Comprehensive review of ASP outcomes

– Griffith M et al. Expert Rev Anti Infect Ther 2012: 10(1):63-73

– Moehring RW et al. Clin Infect Dis. 2017; 64(3):377-83

• Discussion of process and outcome metrics in ASPs

– Khadem TM et al. Pharmacotherapy. 2012 Aug;32(8):688-706

Using the data

• Feedback to your target audience(s)

– Stewardship team, front line clinicians, administrators, patients?

• Have a routine communication method

– Newsletter, presentation, bulletin board, etc

• What are your highlights?

– These will vary by audience

Education

40%

50%

60%

70%

80%

90%

100%

2011 (207) 2012 (283) 2013 (297) 2014 (277) 2015 (214) 2016 (232) 2017 (310) 2018 (343)

Pe

rce

nt

Susc

ep

tib

lePseudomonas aeruginosa

Aztreonam

Cefepime

Levofloxacin

Meropenem

Piperacillin/tazobactam

Tobramycin

Allergy Assessment & Graded Challenge

https://www.nebraskamed.com/for-providers/asp/plans

Education Graded Challenge Order

set

Penicillin Allergy Guidance document

Example of targeted antibiotic

Using the data

• There is currently no regulatory or other standard goal for antibiotic use

– You can set the goal. The goal cannot be zero

• Benchmarking allows comparison of use (not appropriateness) across similar institutions

– High performers may represent best practices

– Low performers may represent inappropriate prescribing

– Need to adjust for confounding factors before making assumptions

Summary: Measure Something!

•Process

– Antimicrobial use

– Compliance with policy

•Outcomes

– Resistance

– Cost

– Safety

– Mortality

– LOS

•Comparative

– Over time

– Benchmark with like institutions

•Compelling

– Demonstrate value

– Reduce variation

– Know your audience

Outcomes of Antimicrobial Stewardship –

What Metrics are Worth Measuring?

Scott Bergman, PharmD, FCCP, FIDSA, BCPS

Antimicrobial Stewardship Coordinator

Nebraska Medicine/UNMC, NE ASAP

@bergmanscott #NebSteward2019

scbergman@nebraskamed.com

Assessment #1Which of these statements describes the measure “Days of Therapy”?A. A single day of drug administration regardless of strength or

number of doses

B. Considered a more realistic estimate of use than Defined Daily Doses

C. Used as part of reporting antibiotic use to the National Healthcare Safety Network

D. All of these

50

Assessment #2Which of the following metrics would NOT be considered a “process measure”?

A. Compliance with institutional guideline

B. Documentation of indication of therapy

C.C. difficile infection rate

D.Time to appropriate therapy

51