Ascertainment of cardiovascular (CV) outcomes via physician review of medical records using a standardized case definition aims to maximize data quality, but is resource intensive.

Adjudication of CV outcomes is limited to a quarter of Women’s Health Initiative (WHI) participants starting in 2010.

To assess whether Medicare data can be used to reliably identify CV outcomes in WHI.

Four projects, focusing on coronary heart disease (CHD), stroke, venous thromboembolism (VTE), and peripheral artery disease (PAD), contributed to this comparison.

Study population

WHI participants with fee-for-service Medicare coverage at baseline (1993-1998)

Some analyses also included participants who aged in

Excluded participants who had WHI outcome of interest prior to entering Medicare

Medicare data through 2007 (CHD, stroke, PAD) or 2010 (VTE) were used

Project specific cohorts selected based on availability of WHI adjudicated outcomes and other factors

Divided into training (data shown) and test sets

Coding algorithms

Evaluated and modified as needed in training set

Used hospital claims (MedPAR file) and ICD-9-CM discharge diagnoses (in any position) and procedure codes

Some analyses also used CPT/HCPCS codes from Carrier/Outpatient claims files

Performed person-based analysis

Assessed agreement

Kappa statistic

Sensitivity (SN), specificity (SP), positive predictive value (PPV), negative predictive value (NPV)

WHI diagnosis defined as “gold standard”

Assessed reasons for disagreement

Additional analyses excluded records that were not informative with regard to performance of Medicare data

E.g., WHI medical record not received

Dale R. Burwen1, Garnet Anderson2, Dominic Cirillo3, Mark Hlatky4, Kamakshi Lakshminarayan5, Joseph Larson2, Marian Limacher6, Karen Margolis7, Matthew Mell4, Mary Pettinger2, Ross Prentice2, Roberta Ray2, Beth Virnig5, Chunyuan Wu2, Jacques E. Rossouw1

1National Heart, Lung, and Blood Institute, Bethesda, MD; 2Fred Hutchinson Cancer Research Center, Seattle, WA; 3University of Iowa, Iowa City, IA; 4Stanford University, Stanford, CA; 5University of Minnesota, Minneapolis, MN; 6University of Florida, Gainesville, FL; 7HealthPartners, Minneapolis, MN

Comparison of Medicare Claims vs. Physician Adjudication for Identifying Cardiovascular Outcomes in the Women’s Health Initiative

OBJECTIVE

METHODS

RESULTS

SUMMARY AND CONCLUSION

BACKGROUND

Table 2. Sensitivity of Medicare Data for Identifying WHI Outcomes.

Table 3. Kappa Statistic Comparing Agreement Between Medicare and WHI Outcomes.

Table 1. Positive Predictive Value of Medicare Data for Identifying WHI Outcomes.

Condition* MI CABG PCI Stroke VTE Carotid AAA LE PAD

Total participants with Medicare events 1501 891 1586 745 261 317 41 255

PPV (%), All 71 89 86 68 62 81 63 58

WHI hospitalization reported† 81 94 94 80 72 92 67 73

WHI hospitalization adjudicated‡ 86 95 95 89 91 94 74§ 82§

Among participants with an outcome in both WHI and Medicare data, dates were within +/-30 days for most events

Examples: 96% for Stroke; 94% for MI; 85% for lower extremity (LE) PAD

Details for Stroke (n=505):

83% on same date; 88% within +/- 1 day; 91% within +/-3 days; 95% within +/- 7 days; 96% within +/- 30 days

Range (WHI date – Medicare date) -4269 to 682 days

Specificity and NPV were >98% for all conditions; PPV, sensitivity, and kappa are provided in Tables 1-3

* MI, myocardial infarction; CABG, coronary artery bypass graft surgery; PCI, percutaneous coronary intervention; VTE, venous thromboembolism (including deep venous thrombosis and pulmonary embolism); Carotid, carotid artery treatment procedure; AAA, abdominal aortic aneurysm treatment procedure; LE PAD, lower extremity peripheral artery disease treatment procedure † Participants with events in Medicare data only were excluded if no WHI hospitalization recorded within +/-30 days ‡ Participants with events in Medicare data only were excluded if no WHI hospitalization adjudicated within +/-30 days§Further evaluation by reviewing Medicare physician claims and selected medical records yielded PPV estimate of 91%- 94% for AAA and 92%-95% for LE PAD

ACKNOWLEDGMENTS

Key reason for a Medicare event without WHI match was lack of WHI report of hospitalization

E.g., Inadequate recall; disability/death and lack of proxy report

Even if hospitalization reported to WHI, not all were adjudicated

E.g., No medical records received; reported reason for hospitalization did not meet criteria for adjudication

Limiting analysis to Medicare events that could be evaluated with WHI adjudicated records increased PPV

Some AAA and LE PAD procedures found in Medicare hospital claims but not WHI data were corroborated by Medicare physician claims or review of selected medical records (data not shown)

Estimated PPV was high

Test set results (data not shown) were similar to training set results

CHD, Stroke, PAD (VTE pending)

Although we defined WHI outcomes as the gold standard, ascertainment of events depends on reporting of hospitalization by participants

This reporting is similar across trial arms, but can be incomplete

The adjudication process ensures that virtually all WHI identified events are true

Medicare data miss some events in WHI, yet record others not identified in WHI

The associated PPV is imperfect, however

PPV is high for CABG, PCI and carotid procedures (when analysis was limited to participants with an adjudicated medical record)

We estimate it may be similar or almost as high for AAA and LE PAD

PPV was slightly lower but reasonably high for MI and stroke

For VTE, a relatively larger proportion of hospitalizations reported were not adjudicated

Therefore more caution is needed in generalizing from the subset adjudicated, and the best estimate of the PPV may be lower

Additional VTE coding algorithms are being tested

We thank the other members of the working groups for the 4 contributing projects (CHD, stroke, VTE, PAD). We gratefully acknowledge the dedication of the WHI participants, and investigators and staff who have contributed to the WHI.

The WHI program is funded by the National Heart, Lung, and Blood Institute, National Institutes of Health, U.S. Department of Health and Human Services through contracts HHSN268201100046C, HHSN268201100001C, HHSN268201100002C, HHSN268201100003C, HHSN268201100004C, and HHSN271201100004C.

Condition* MI CABG PCI Stroke VTE Carotid AAA LE PAD

Total participants included in analysis 37397 37397 37397 24428 8001 48656 48966 50467

Kappa, All 0.74 0.91 0.88 0.75 0.71 0.86 0.74 0.65

WHI hospitalization reported† 0.79 0.94 0.92 0.83 0.77 0.92 0.76 0.73

WHI hospitalization adjudicated‡ 0.82 0.94 0.92 0.88 0.87 0.93 0.81 0.77

Condition* MI CABG PCI Stroke VTE Carotid AAA LE PAD

Total participants with WHI events 1345 848 1516 582 192 281 29 204

Sensitivity (%), All 79 94 90 87 84 92 90 73