creating composite measures using factor analysis: the total illness burden index sherrie h. kaplan,...
TRANSCRIPT
“Creating Composite Measures Using Factor Analysis:
The Total Illness Burden Index
Sherrie H. Kaplan, PhD, MPH
Professor of Medicine
UC Irvine School of Medicine
Academy Health ARM
June 8-10, 2008
Some Background…
Role of Purpose of Measurement
• Changes content of aggregate measure
• Changes tolerance of error
• Changes psychometric requirements of aggregate
• Changes ‘level of confidence’, dissemination strategy
How to create composites:Lessons from psychometrics…
• Choose measures that Choose measures that broadly broadly represent represent underlying (latent) construct (sampling underlying (latent) construct (sampling from domain of observables); each item from domain of observables); each item adds unique informationadds unique information
• Hypothesize structure of items in Hypothesize structure of items in composites before analysis (what composites before analysis (what measures what?)measures what?)
How to create composites:Lessons from psychometrics…
• Conduct Conduct confirmatory confirmatory cluster, latent cluster, latent variable analyses (construct validity)variable analyses (construct validity)
• Decide on scoring methods (simple Decide on scoring methods (simple algebraic sum, weighting, conjunctive or algebraic sum, weighting, conjunctive or compensatory); compensatory); testtest scoring methods scoring methods
• Test reliability, predictive validity of Test reliability, predictive validity of derived compositederived composite
Models for Composite Scoring
• Conjunctive scoring (‘ands’): highest, lowest levels achieved define score– Rheumatoid arthritis trials: patient responded if:
• at least a 20% improvement in tender joint count and • 20% improvement in swollen joint count and • at least 20% improvement in 3 out of 5 of the
following: pain assessment, global assessment, physician assessment, etc.
• Compensatory scoring (‘ors’): high scores on one component make up for low scores on another
Models for weighting• Expert defined
– Conditioned by ‘expert’ representation
• Regression-based– Conditioned by database (provider,
patient sample, sample size)
• Factor analysis-based– Conditioned by variables included in
factor analysis
• Reliability-based– Conditioned by database (sample size)
Classic Measurement Theory: Using Factor Analysis to Create Composites
• Each factor represents ‘latent’ construct
• Correlations of items with factors (factor ‘loadings’) represent statistical structure of set of variables
• Factor analysis does not require items have difficulty structure
,
Cronbach’s alpha
• Measure of internal consistency reliability
• Given by formula:
– Where:• N = number of tests
• σYi2 = variance of item i
• σx2 = total test variance
,
Cronbach’s alpha
• Alpha is unbiased reliability estimator if items have equal covariances (means and item variances may differ); i.e. have common factor in factor analysis
Total Illness Burden: The Latent Construct
• Patient-reported composite measure of severity of multiple diseases
• Taken together represent increasing risk for substantial declines in health and increased risk for mortality (1-5 years post initial observation)
Purposes of Measurement
• Post hoc case-mix adjustment
• A priori risk stratification of clinical trials
• Improve clinical decision making for ‘tailoring’ treatment
Subdimensions…
• Pulmonary disease• Heart disease• Stroke and neurologic disease• Gastrointestinal conditions• Other cancers (excluding prostate)• Arthritis• Foot and leg conditions
Subdimensions (cont’)
• Eye and vision conditions• Hearing problems• Hypertension • Diabetes
Sample Questions: COPD
1. During the past 6 months, how often did you have wheezing?
a. Never
b. Once or twice
c. About once a week
d. Several times a week
e. Several times a day
Sample Questions: COPD
4. During the past 6 months, did you use extra pillows in order to sleep at night because of problems with your breathing?
a. No
b. Yes, 1 pillow
c. Yes, 2 pillows
d. Yes, 3 or more pillows
Steps in Constructing Subdimensions
• Transformed variables to uniform metric by clinical definition of severity
• Tested reliability of clinically defined scale (Cronbach’s alpha > .70)
• Created composite of each subdimension using simple algebraic sum, mean
• Items in each subdimension varied
• Validated each subdimension as scale using SF-36, etc.
Steps in Constructing Composite
• Conducted principal components analysis, higher order factor analysis using scales as entries
• First factor explained 67% of variance
• Other factors had Eigen values, scree indicating single factor solution
• Factor loadings ranged from .40 - .70
• Used factor loadings to create composite
• Validated derived composite
Understanding and Reducing Disparities in Diabetes Care:Coached Care for Diabetes
Sherrie H. Kaplan, PhD, MPH
Sheldon Greenfield, MD
NovoNordisk
Lund, Sweden
May 28-20, 2008
Characteristics of Patient SampleCharacteristics Registry
(n=3,894)Survey Sample
(n=1001)
Mean age 58.9 60.1
% Male 43.2 39.3
% White 33.1 25.8
% Hispanic 50.5 48.9
% Asian 16.4 25.3
% Medicare 21.4 20.9
% Medicaid 50.7 54.1
% Commercial 19.2 16.2
% Uninsured 8.7 8.7
Principal Components Analysis: First Factor
TIBI Scale Sample 1 Sample 2GI disease .604 .628
Atherosclerotic heart dis .650 .621
Neurologic problems .433 .360
Hearing problems .452 .388
Hypertension .328 .448
Cardiopulmonary .712 .704
Feet problems .613 .587
Arthritis .449 .618
Vision problems .475 .367
Cronbach’s alpha (.799)
TIBI Scale Scale Mean α if item deleted
GI disease .58 .767Atherosclerotic heart dis .79 .783Neurologic problems .19 .799Hearing problems .36 .798Hypertension .74 .799Cardiopulmonary .81 .755Feet problems .67 .787Arthritis .57 .790Vision problems .51 .798
Correlation of TIBI with Patient-Reported Health Status Measures by Ethnic Group
Health Status Measures
Whites Mexican-American
Vietnamese
SF-36 PFI10 -.63 -.38 -.55CESD .50 .50 .55Diabetes Burden .37 .33 .37
Other TIBI Validation Studies…
Preventing Cardiovascular Disease: Identification of Co-Morbidity Subgroups who may not Benefit from Aggressive Diabetes Management
Greenfield S, Nicolucci A, Pellegrini F, Kaplan SH
QuED Study
• Prospective cohort study of consecutively enrolled patients with diabetes who completed TIBI at enrollment in Italian Quality of Care and Outcomes in Type 2 Diabetes Study Group (n=2,613)
QuED Study
Patient Characteristics
Mean age 62.7 [10.3]
% Female 45
% < 5 yrs education 52
% BMI > 30 28
HbA1c value 7.2
QuED: Total 5-yr CV Events by TIBI
TIBI Group HR 95% HR CI P-value
0-3 1
3-6 0.95 0.61-1.47 .81
6-9 1.11 0.74-1.67 .61
9-12 1.46 1.02-2.1 .04
>12 1.57 1.16-2.12 .003
QuED: % 5-yr Survival by TIBI
TIBI Group % HR (95% CI) P-value
0-3 85.9 1
3-6 85.0 1.11(.75-1.65) .59
6-9 80.2 1.42(1.00-2.02) .05
9-12 80.0 1.41(0.96-2.08) .08
>12 75.6 1.63(1.25-2.13) <.000
“Complex diabetes patients, those with the greatest burden from competing co-morbidities (highest TIBI scores) may benefit less from aggressive glycemic control due to their increased risk of mortality from other causes before those benefits could be realized.”
Conclusions
• Using factor analysis, possible to derive latent construct that reflects patients’ “total illness burden”
• Potentially useful in case-mix adjustment, clinical trials design, clinical decision making
• Future research aimed at improving sensitivity, specificity, particularly at ‘intermediate ranges’