Comparison of Physician Rankings on Performance Quality Composites in the Care of Hypertensive Patients
AcademyHealth, June 9, 2008Washington, DC
Weifeng Weng, Gerald K. Arnold, Eric S. Holmboe, Rebecca S. Lipner
ABIM and Maintenance of Certification (MOC)
ABIM certifies physicians in internal medicine and its subspecialties
Certification is time-limited: 10 year duration Renew certificates through MOC program MOC requires demonstration of
– Professionalism– Lifelong Learning– Cognitive Expertise– Practice performance
Practice Improvement Module (PIMTM)
Performance Report
Performance Report
Improvement
Chart audit Patient survey
Impact
plan
do
study
act
Practice survey
Based on Picker patient and CAHPS
surveys
Based on Wagner’s Chronic Care Model & IHI’s Idealized
Office Design
Evidence-based guidelines
Research Question In P4P programs
– Clinical measures dominate – Patient survey and Practice system survey
measures used less frequently– Typically, rewards awarded by relative ranking– Typically not all three data streams used
Do physician performance rankings (and rewards) vary considerably when different combinations of the three data streams are used?
Methods
Physician database with patient-level data Standardized composite scores for the three
data streams(1) Chart audit
(2) Patient survey
(3) Practice systems
Super-composite scores: Combine composites Examine changes in physician rankings
Physician and Patient Samples
659 Physicians– Mean Age: 44 (SD = 6.4), 26% female– 61% general internists, 39% subspecialists (largely
nephrologists and cardiologists)– 29% in solo practice
Patients– Chart audit: 13,096 patients, age 18-75, 51% male – Patient survey: 14,913 patients, age 18-75, 53% male
Chart Audit Individual Measures
Measures
Physicianlevel ICC
N forRm
=0.85Mean (SD) Rho 95% CIBlood pressure control 0.52 (0.19) 0.12 0.10 to 0.14 42LDL control 0.55 (0.20) 0.16 0.14 to 0.18 29Complete lipid profile 0.82 (0.22) 0.43 0.39 to 0.46 8Urine protein test 0.81 (0.26) 0.64 0.60 to 0.68 3Annual serum creatinine test 0.88 (0.16) 0.36 0.32 to 0.40 10DM co-morbidity documentation or screen test 0.93 (0.15) 0.59 0.54 to 0.64 4Aspirin therapy for eligible patients 0.74 (0.25) 0.40 0.36 to 0.44 9Smoking status and cessation advice and treatment 0.96 (0.08) 0.36 0.30 to 0.42 10Counseling for diet and physical activity 0.89 (0.18) 0.51 0.47 to 0.56 5
Outcome variables are risk adjusted for co-morbidity conditions:BP control <130/80 for pts with dm or stroke co-morbidities, <140/90 for rest.LDL control <100 for pts with major risks, <130 for pts with other risks, <160 for the rest.
Patient Survey Individual Measures
Measures
Physicianlevel ICC
N for Rm
=0.85Mean (SD) Rho 95% CIOverall hypertension care 0.88 (0.12) 0.20 0.17 to 0.23 22Encouraging/answering questions 0.83 (0.14) 0.19 0.16 to 0.22 24Providing information on Medication side effects 0.73 (0.17) 0.17 0.14 to 0.19 28Providing information on foods to eat & avoid 0.60 (0.17) 0.12 0.10 to 0.14 41Providing information on taking medication properly 0.86 (0.13) 0.19 0.16 to 0.22 24
Practice System Individual Measures
Measures (# of measures) Mean SD Alpha
Information Management (27) 0.54 0.24 0.89
Patient activation & communication (28) 0.56 0.23 0.88
Access & communication with patients (7) 0.63 0.27 0.73
Safety & efficiency (11) 0.72 0.24 0.82
Practice team (8) 0.72 0.32 0.82
Consultation & referral (4) 0.62 0.31 0.65
Practice system survey of 89 questions
Distribution of Physician Performance Composite Scores
C+P+SC+SC+PSystem (S)
Patient (P)
Chart (C)
-8
-4
-6
0
-2
4
2
6
Correlations among Composites
Chart Patient System C+P C+S C+P+S
Chart (C) 1.00 0.22 0.19 0.76 0.72 0.66
Patient (P) 1.00 0.15 0.77 0.22 0.63
System (S) 1.00 0.20 0.78 0.66
C+P 1.00 0.59 0.84
C+S 1.00 0.87
C+P+S 1.00
15% 18%
10% 10% 12%
15%15%
9%12%
14%
0%
10%
20%
30%
40%
Patient System C+P C+S C+P+S
Percent who change rankings by more than one quartile*
Baseline: Chart
* One quartile counts for 164 rank positions
Ranks better than chart Ranks worse than chart
8% 8%
2% 3% 2%
7% 8%
2%0.3%0.3%
0%
5%
10%
15%
20%
Patient System C+P C+S C+P+S
Percent who change rankings by more than two quartiles*
* Two quartile counts for 329 rank positions
Baseline: Chart
Ranks better than chart Ranks worse than chart
Examples of Extreme Discordance of Performance – more than three quartiles
Ranks and (z scores)
Physician Chart (C) Patient (P) System (S) C+P C+S C+P+S
1 24 592 83 356 15 187
(1.3) (-1.2) (1.1) (0.08) (2.4) (1.2)
2652 287 4.5 644 586 538
(-3.6) (0.23) (1.7) (-3.4) (-1.9) (-1.7)
Rank 1 = Best; Rank 659 = Worst
Conclusions
Measuring multiple dimensions in the quality of patient care is complex– Very moderate correlations among three data
streams
– Rankings change considerably depending on combinations
A profile that incorporates more than one aspect of patient care tells a different story than any one of them alone
Limitations and Future Research
Self-report data for chart and system data Participants are volunteers Need more robust risk adjusters Investigate other analytic approaches for
combining individual measures into composites
Investigate stability of pass/fail decisions