Development and Use of Ambulatory Adverse Event Trigger Tools

Jonathan R. Nebeker

AHRQ CONFERENCE SEPT. 29, 2010

BOSTON UNIVERSITY SCHOOL OF PUBLIC HEALTH, BOSTON, MA, VA CENTER FOR ORGANIZATION, LEADERSHIP AND MANAGEMENT RESEARCH (COLMR), BOSTON MAAKROSEN@BU.EDU

Acknowledgements

PI Amy Rosen, PhD Co-PI Jonathan Nebeker,

MD, MS Co-Investigators:

Stephan Gaehde, MD Haytham Kaafarani, MD,

MPH

Brenna Long, MA Hillary Mull, MPP Brian Nordberg, BS Steve Pickard, MS Peter Rivard, PhD Lucy Savitz, PhD, MBA Chris Shanahan, MD, MPH Stephanie Shimada, PhD

Sponsored by AHRQ Contract No. HHSA290200600012, Task Order Officer: Amy Helwig, MD

What’s new?

ADE Surveillance rules Complex to maximize PPV and Sensitivity Action-Oriented to make a difference for

patient Use input from system-based focus

groups and experts with practical experience

Study Flow

Epidemiological Design

Mull H & Nebeker JR, Informatics Tools for the Development of Action-Oriented Adverse Drug Event Triggers, 2008. AMIA Annual Symposium Proceedings. Nov 6:505-9. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2655939/

Example of Complex Trigger

[Dose change in GFR reducer or renal toxin and subsequent creatinine > 50% from average baseline creatinine value or > 33% from last post dose change creatinine value] AND (No new trimethoprim within 1 day prior to 7 days after creatinine lab value firing) AND (GFR Reducer or renal toxin used within 90 days prior to lab value)

Suppress If: Renal toxin or GFR reducer dose reduced 0-7 days after lab) OR ( new creatinine lab result within 1-6days of firing lab value)

Results: Focus Group/Interviews

Delirium

WBC

Platelets

Hypokalemia

Hyperkalemia

Creatinine

Warfarin

0% 20% 40% 60% 80% 100%

LikelyUnclearUnlikely

Delphi Highlights

50% national experts, 50% local clinical leaders

Got perspectives that represented emphasis of research or clinical experience. ED doc liked delirium trigger

Various, contradictory input on thresholds

Helpful modifications Interesting insights on what would be

useful

Analysis

Sample Requested sets of 2,000 pts based on

exposure to target meds Got 1,059 to 9,339 -> IRB amendment Restricted to patients with 2 notes in 1 year

Analysis 1° PPV = number of patients with

AE/number of cases triggered 2° Sensitivity, if prevalence >5% (for 25%

CI width) (highest was 4%)

Ignore the CI behind the curtain

Nebeker JR, Stoddard GJ & Rosen AK, Considering Sensitivity and Positive Predictive Value in Comparing the Performance of Triggers Systems for Iatrogenic Adverse Events, Proceedings of the Trigger and TIDS Expert Meeting, Agency for Healthcare Research and Quality, June 2008. Bethesda, MD. http://www.ahrq.gov/QUAL/triggers/triggers2.htm

End Points

Harm PSO Harm scale WHO Causality criteria (certain or probable) Explicit criteria Lab values without symptoms are not harm

Change in care (useful) E.g., prevented additional lab draw

Trigger Performance

Trigger Reviewed/ Flagged

Harm PPV (CI)

Change PPV (CI)

Either PPV (CI)

Creatinine 40/57 0 60%(43-75%)

60%(43-75%)

WBC 0 - - -

Platelets 17/17 6%(0-29%)

53%(28-77%)

59%(33-82%)

Warfarin 96/677 13%(7-21%)

14%(7-22%)

19%(12-28%)

Hypokalemia

85/90 17%(9-26%)

42%(31-54%)

58%(46-68%)

Hyperkalemia

50/64 8%(2-19%)

28%(16-43%)

36%(23-51%)

Delirium 68/359 10%(4-20%)

10%(4-20%)

19%(11-31%)

Summary

Harm is in the eye of the beholder: less or more? Doomed to benchmarking apples with

oranges? Explicit criteria

Potential to affect practice Are complex trigger better?