1

HPM 214 Course Review

March 9, 2015 (9:00-11:50 am)

HPM 214 http://hpm214.med.ucla.edu/

911 Broxton Avenue

Los Angeles, CA 90024

Where we are now in HPM214

1. Introduction

2. Profile Measures (SF-36 due)

3. Preference-Based Measures

4. Designing Measures

5. Evaluating Measures

6. PROMIS/IRT/Internet Panels

7. Reviews of Manuscripts

8. Course Review (Cognitive interviews due)

9. Final Exam (3/16/15)2

HPM 214 Assignments

• Class participation (25%)

• Two class assignments (25%)– Complete the SF-36 v2 survey at

http://www.sf-36.org/demos/SF-36v2.html – Conduct and summarize 5 cognitive interviews with

a self-administered HRQOL survey.  

• Extra credit (2-page critique of published HRQOL article).

3

U.S. Health Care Issues

• Access to care – ~ 50 million people without health insurance

• Costs of care– Expenditures ~ $ 2.7 Trillion

• Effectiveness (quality) of care4

How Do We Know If Care Is Effective?

• Effective care maximizes probability of

desired health outcomes

– Health outcome measures indicate whether

care is effective

Cost ↓

Effectiveness ↑

5

What Are Health Outcomes?

• Traditional clinical endpoints– Death, disease occurrence, other adverse events

– Clinical measures/biological indicators• Blood pressure• Blood hemoglobin level• Symptoms (e.g. fever)

• Health-Related Quality of Life

7

HRQOL is Multi-Dimensional

HRQOL

Physical Mental Social

8

Health-Related Quality of Life (HRQOL)

How the person FEELs (well-being)• Emotional well-being• Pain• Energy

What the person can DO (functioning)• Self-care • Role • Social

9

HRQOL is Not

Quality of environment

Type of housing

Level of income

Social Support

Patient-Reported Outcomes (PROs)

“Any report coming from patients about a health condition and its treatment”(U.S. FDA, 2006)

Including• Health-related quality of life (HRQOL)• Satisfaction with treatment• Patient reports about care• Needs assessment• Adherence to treatment

Patient-Reported Outcomes (PROs)

“Any report coming from patients about a health condition and its treatment”(U.S. FDA, 2006)

Including• Health-related quality of life (HRQOL)• Satisfaction with treatment• Patient reports about care• Needs assessment• Adherence to treatment

Patient-Reported Measures (PRMs)

• Background characteristics– Age, education, income

• Health care experiences– Reports about care (e.g., communication)

• Behavior– Adherence to physician recommendations

• Outcomes– Satisfaction with care– HRQOL

Uses of HRQOL Measures

• Monitoring population (and subgroups)

• Observational studiesObservational studies

• Clinical trials Clinical trials

• Clinical practiceClinical practice

HS 214, Winter 01·11·10

Behavioral Risk Factor Surveillance System (BRFSS)

• Telephone interview (random digit dialing) of nationwide survey of U.S. adults

• % reporting poor or fair health about 16%

Greater % of fair or poor health reported Greater % of fair or poor health reported by older adults (33% for 75+ vs. 9% for 18-24)by older adults (33% for 75+ vs. 9% for 18-24)

Greater % of fair or poor health reported Greater % of fair or poor health reported by older adults (33% for 75+ vs. 9% for 18-24)by older adults (33% for 75+ vs. 9% for 18-24)

Greater % of fair or poor health Greater % of fair or poor health reported reported

by females (17%) than males (15%) by females (17%) than males (15%)

Greater % of fair or poor health Greater % of fair or poor health reported reported

by females (17%) than males (15%) by females (17%) than males (15%)

Uses of HRQOL Measures

• Monitoring population (and subgroups)Monitoring population (and subgroups)

• Observational studies

• Clinical trials Clinical trials

• Clinical practiceClinical practice

Observational Study

•Observation of groups •(non-random assignment)

Outcomes

•Survival•Clinical•HRQOL

Casemix adjustment needed• + Conditions/comorbidity • + Severity• + Demographics•

Uses of HRQOL Measures

• Monitoring population (and subgroups)Monitoring population (and subgroups)

• Observational studiesObservational studies

• Clinical trialsClinical trials

• Clinical practice Clinical practice

Randomized Trial Design• Outcomes

– Survival

– Clinical

– HRQOL

• Control for case-mix may not be required

StudyPopulation

RandomizeRandomize

Intervention Group

Control Group

Uses of HRQOL Measures

• Monitoring population (and subgroups)Monitoring population (and subgroups)

• Observational studiesObservational studies

• Clinical trials Clinical trials

• Clinical practice

HRQOL Assessment by Providers May

• Facilitate patient-physician communication• Improve clinician understanding of patients’ problems

(particularly those of a psychosocial nature)

Detmar SB, Aaronson NK. Quality of life assessment in daily clinical oncology practice: a feasibility study. Eur J Cancer. 1998;34(8):1181-6.

Detmar SB, Muller MJ, Schornagel JH, Wever LD, Aaronson NK. Health-related quality-of-life assessments and patient-physician communication: a randomized controlled trial. J Am Med Assoc. 2002;288(23):3027-34.

Hess R, Tindle H, Conroy MB, et al. A randomized controlled pilot trial of the Functional Assessment Screening Tablet to engage patients at the point of care. JGIM. 2014; 29(12):1641-1649.

Velikova G, Brown JM, Smith AB, Selby PJ. Computer-based quality of life questionnaires may contribute to doctor-patient interactions in oncology. Br J Cancer. 2002;86(1):51-9.

Velikova G, Booth L, Smith AB, et al. Measuring quality of life in routine oncologypractice improves communication and patient well-being: a randomized controlled trial. JClin Oncol. 2004;22(4):714-24.

24

In general, how would you rate your health?

Excellent

Very Good

Good

Fair

Poor

25

Does your health now limit you inwalking more than a mile?

(If so, how much?)

Yes, limited a lotYes, limited a littleNo, not limited at all

26

How much of the time during the past 4 weeks have you been happy?

None of the time

A little of the time

Some of the time

Most of the time

All of the time

27

- Profile: Targeted vs. Generic

- Preference

Types of HRQOL Measures

28

Targeted HRQOL Measures

• Designed to be relevant to particular group.• Sensitive to small, but clinically-important

changes.• More familiar and actionable for clinicians.• Enhance respondent cooperation.

29

Kidney-Disease Targeted Items

During the last 30 days, to what extent were you bothered by cramps during dialysis?

Not at all bothered

Somewhat bothered

Moderately bothered

Very much bothered

Extremely bothered

30

SF-36 Generic Profile Measure • Physical functioning (10 items)

• Role limitations/physical (4 items)

• Role limitations/emotional (3 items)

• Social functioning (2 items)

• Emotional well-being (5 items)

• Energy/fatigue (4 items)

• Pain (2 items)

• General health perceptions (5 items)

31

Scoring HRQOL Profile Scales• Average or sum all items in the same scale.

• Transform average or sum to• 0 (worse) to 100 (best) possible range• z-score (mean = 0, SD = 1)• T-score (mean = 50, SD = 10)

32

HRQOL in HIV Compared to otherChronic Illnesses and General Population

0 10 20 30 40 50 60

Asymptomatic

Symptomatic

AIDS

General Pop

Epilepsy

GERD

Prostate disease

Depression

Diabetes

ESRD

MSEmot.Phy func

Hays et al. (2000), American Journal of MedicineT-score metric

33

HRQOL in HIV Compared to otherChronic Illnesses and General Population

0 10 20 30 40 50 60

Asymptomatic

Symptomatic

AIDS

General Pop

Epilepsy

GERD

Prostate disease

Depression

Diabetes

ESRD

MSEmot.Phy func

Hays et al. (2000), American Journal of MedicineT-score metric

34

HRQOL in HIV Compared to otherChronic Illnesses and General Population

0 10 20 30 40 50 60

Asymptomatic

Symptomatic

AIDS

General Pop

Epilepsy

GERD

Prostate disease

Depression

Diabetes

ESRD

MSEmot.Phy func

Hays et al. (2000), American Journal of MedicineT-score metric

35

Hypertension

Diabetes

Current Depression

Stewart, A.L., Hays, R.D., Wells, K.B., Rogers, W.H., Spritzer, K.L., & Greenfield, S. (1994). Long-termfunctioning and well-being outcomes associated with physical activity and exercise in patients withchronic conditions in the Medical Outcomes Study. Journal of Clinical Epidemiology, 47, 719-730.

Physical Functioning in Relation to Time

Spent Exercising 2-years Before

Low High

Total Time Spent Exercising

84

82

80

78

76

74

72

70

68

66

64

62

0-100 range

36

SF-36 PCS and MCSPCS_z = (PF_Z * 0.42) + (RP_Z * 0.35) +

(BP_Z * 0.32) + (GH_Z * 0.25) + (EF_Z * 0.03) + (SF_Z * -.01) + (RE_Z * -.19) + (EW_Z * -.22)

MCS_z = (PF_Z * -.23) + (RP_Z * -.12) + (BP_Z * -.10) + (GH_Z * -.02) + (EF_Z * 0.24) + (SF_Z * 0.27) + (RE_Z * 0.43) + (EW_Z * 0.49)

PCS = (PCS_z*10) + 50MCS = (MCS_z*10) + 50

SF-12

• Items by Scale– General health (1)– Physical functioning (3b, 3d)– Role-Physical (4b, 4c)– Role-Emotional (5b, 5c)– Bodily pain (8)– Emotional well-being (9d, 9f)– Energy/fatigue (9e)– Social functioning (10)

37

38

Debate About Summary Scores

•Taft, C., Karlsson, J., & Sullivan, M. (2001). Do SF-36 component score accurately summarize subscale scores? Quality of Life Research, 10, 395-404.•Ware, J. E., & Kosinski, M. (2001). Interpreting SF-36 summary health measures: A response. Quality of Life Research, 10, 405-413.•Taft, C., Karlsson, J., & Sullivan, M. (2001). Reply to Drs Ware and Kosinski. Quality of Life Research, 10, 415-420.

39

Farivar et al. alternative weights

PCS_z = (PF_z * .20) + (RP_z * .31) + (BP_z * .23) +

(GH_z * .20) + (EF_z * .13) + (SF_z * .11) +

(RE_z * .03) + (EW_z * -.03)

MCS_z = (PF_z * -.02) + (RP_z * .03) + (BP_z * .04) + (GH_z * .10) + (EF_z * .29) + (SF_z * .14) +

(RE_z * .20) + (EW_z * .35)

•Farivar, S. S., Cunningham, W. E., & Hays, R. D. (2007). Correlated physical and mental health summary scores •for the SF-36 and SF-12 health survey, V. 1. Health and Quality of Life Outcomes, 5: 54. [PMCID: PMC2065865]

40

Is Complementary and Alternative Medicine (CAM) Better than Standard Care (SC)?

0

10

20

30

40

50

60

70

80

90

100

CAMCAM

SCSCCAMCAM

SCSC

PhysicalHealth

CAM > SC

Mental Health

SC > CAM

41

Does Taking Medicine for HIV Lead to Worse HRQOL?

1 No deaddead2 No deaddead

3 No 50 4 No 75 5 No 100 6 Yes 0 7 Yes 25 8 Yes 50 9 Yes 75 10 Yes 100

Subject Antiretrovirals HRQOL (0-100)

No Antiretroviral 3 75Yes Antiretoviral 5 50

Group n HRQOL

42

Cost-Effectiveness of Health Care

Cost ↓

Effectiveness (“Utility”) ↑

http://www.ukmi.nhs.uk/Research/pharma_res.asp

Preference Elicitation• Standard gamble (SG)• Time trade-off (TTO)• Rating scale (RS)

– http://araw.mede.uic.edu/cgi-bin/utility.cgi

SG > TTO > RS SG = TTOa

SG = RSb (Where a and b are less than 1)

• Also discrete choice experiments44

45

46

SF-6D

Brazier et al. (1998, 2002)— 6-dimensional classification

(collapsed role scales, dropped general health)

— Uses 12 SF-36 items (PF: 3a, b, j; R: 4c, 5b; SF: 10; BP: 7, 8; MH: 9b, f; EN: 9e)

--- About 18,000 possible states-— 249 states rated by sample of 836

from UK general populationhttp://www.shef.ac.uk/scharr/sections/heds/mvh/sf-6d

SF-6D Example

• Mean = 0.73 (SD = 0.14)

• Adjusted R-squared of 39% for 43 dfs

• Only 2 of 23 conditions had non-significant associations (melanoma, endometrial cancer)

48

HRQOL in SEER-Medicare Health Outcomes Study (n = 126,366)

0.73

0.74

0.75

0.76

0.77

0.78

0.79

0.8

0.81

0.82

No Condition Hypertension Arthritis-Hand Stroke COPD Arthritis-Hip

SF-6D (range = 0.30-1.00; SD = 0.14) by Condition

Controlling for age, gender, race/ethnicity, education, income, and marital status.

Summary of SF-6D Example

• Unique associations of multiple chronic conditions on health-related quality of life are generally similar and additive, not interactive

• The largest unique associations of chronic conditions with health-related quality of life among Medicare managed care beneficiaries was observed for four chronic medical conditions– Stroke, COPD/asthma, sciatica, arthritis of the hip

• Advanced stage of cancer is associated with noteworthy decrement in health-related quality of life for four “big” cancers (breast, prostate, colorectal, lung)

50

End goal is measure that is “Psychometrically Sound”

• Same people get same scores

• Different people get different scores and differ in the way you expect

• Measure works the same way for different groups (age, gender, race/ethnicity)

• Measure is practical

51

First law of survey development:

Only do it when necessary

52

Second law: Know thy respondent

53

Third law: Practice before you play

“Cut and try, see how it looks and sounds, see how people react to it, and then cut again, and try again” Converse & Presser (1986, p. 78)

Identify problems with

– Comprehension of items (stem/response options)– Retrieval of information– Skip patterns– Response burden

54

Fourth law: Keep it simple and short

55

Fifth law: Believe the survey respondent, but only so much

Four Levels of Measurement

• Nominal (categorical)

• Ordinal (rank)

• Interval (numerical)

• Ratio (numerical)

Levels of Measurement and Their Properties

PropertyProperty

Level Level Magnitude Magnitude Equal Equal

IntervalInterval Absolute 0Absolute 0

Nominal Nominal NoNo NoNo NoNo

Ordinal Ordinal YesYes NoNo NoNo

Interval Interval YesYes YesYes No No

Ratio Ratio YesYes YesYes YesYes

Measurement Range for HRQOL Measures

NominalNominal OrdinalOrdinal IntervalInterval RatioRatio

Variability• Responses fall in each response category

• Distribution approximates bell-shaped “normal” curve (68.2%, 95.4%, and 99.6%)

Reliability

Reliability is the degree to which the same score is obtained for thing being measured (person, plant or whatever) when that thing hasn’t changed.

– Ratio of signal to noise

Kappa Coefficient of Agreement(Corrects for Chance)

kappa =(observed - chance)

(1 - chance)

“Quality Index”

Reliability Minimum Standards

• 0.70 or above (for group comparisons)

• 0.90 or higher (for individual assessment)

SEM = SD (1- reliability)1/2 95% CI = true score +/- 1.96 x SEM

if z-score = 0, then CI: -.62 to +.62 when reliability = 0.90Width of CI is 1.24 z-score units

63

Range of reliability estimates

0.80-0.90 for blood pressure

0.70-0.90 for multi-item self-report scales

Hahn, E. A., Cella, D., et al. (2007). Precision of health-related

quality-of-life data compared with other clinical measures.

Mayo Clin Proceedings, 82 (10), 1244-1254.

Multitrait Scaling Analysis

• Internal consistency reliability– Item convergence

• Item discrimination

65

Item-scale correlation matrix

Depress Anxiety Anger Item #1 0.50* 0.50 0.50 Item #2 0.50* 0.50 0.50 Item #3 0.50* 0.50 0.50 Item #4 0.50 0.50* 0.50 Item #5 0.50 0.50* 0.50 Item #6 0.50 0.50* 0.50 Item #7 0.50 0.50 0.50* Item #8 0.50 0.50 0.50* Item #9 0.50 0.50 0.50* *Item-scale correlation, corrected for overlap.

Validity

• Does instrument measure what it is supposed to measure?

• Content Validity– Includes face validity

• Construct Validity– Many synonyms

67

6

2

17

5

0

2

4

6

8

10

12

14

16

18

<35 35-44 45-54 >55

%

Dead

(n=676) (n=754) (n=1181) (n=609)

SF-36 Physical Health Component Score (PCS)—T scoreSF-36 Physical Health Component Score (PCS)—T score

Ware et al. (1994). SF-36 Physical and Mental Health Summary Scales: A User’s Manual.

Self-Reports of Physical Health Predict Five-Year Mortality

Evaluating Construct ValidityScale Age Obesity ESRD Nursing

Home Resident

Physical Functioning

Medium (-). Small (-) Large (-) Large (-)

Depressive Symptoms

? Small (+) ? Medium (+)

Cohen effect size rules of thumb (d = 0.2, 0.5, and 0.8):Small correlation = 0.100Medium correlation = 0.243Large correlation = 0.371r = d / [(d2 + 4).5] = 0.8 / [(0.82 + 4).5] = 0.8 / [(0.64 + 4).5] = 0.8 / [( 4.64).5] = 0.8 / 2.154 = 0.371 (Beware r’s of 0.10, 0.30 and 0.50 are often cited as small, medium, and large.)

Responsiveness to Change

• HRQOL measures should be responsive to interventions that changes HRQOL

• Need external indicators of change (Anchors)

Minimally Important Difference (MID)

• External anchors– Self-report– Provider report– Clinical measure – Intervention

• Anchor correlated with change on target measure at 0.371 or higher

• Anchor indicates “minimal” change

Listed below are a few statements about your relationships with others. How much is each statement TRUE or FALSE for you?

1. I am always courteous even to people who are disagreeable.2. There have been occasions when I took advantage of someone.3. I sometimes try to get even rather than forgive and forget.4. I sometimes feel resentful when I don’t get my way.5. No matter who I’m talking to, I’m always a good listener.

Definitely true; Mostly true; Don’t know; Mostly false; Definitely false

72

0 10 20 30 40 50 60

General PopEpilepsy

GERDProstate disease

DepressionDiabetes

ESRDMS

AsymptomaticSymptomatic

AIDS

East-West

Physical Functioning and Emotional Well-Being at Baseline

for 54 Patients at UCLA-Center for East West Medicine

EWBPhysical

MS = multiple sclerosis; ESRD = end-stage renal disease; GERD = gastroesophageal reflux disease.

72

73

Significant Improvement in all but 1 of SF-36 Scales (Change is in T-score metric)

Change t-test prob.

PF-10 1.7 2.38 .0208

RP-4 4.1 3.81 .0004

BP-2 3.6 2.59 .0125

GH-5 2.4 2.86 .0061

EN-4 5.1 4.33 .0001

SF-2 4.7 3.51 .0009

RE-3 1.5 0.96 .3400

EWB-5 4.3 3.20 .0023

PCS 2.8 3.23 .0021

MCS 3.9 2.82 .0067

Effect Size

(Follow-up – Baseline)/ SDbaseline

Cohen’s Rule of Thumb:

ES = 0.20 Small

ES = 0.50 Medium

ES = 0.80 Large

74

75

Effect Sizes for Changes in SF-36 Scores

0

5

10

15

20

25

30

35

40

45

50

PFI Role-P Pain Gen H Energy Social Role-E EWB PCS MCS

Baseline

Followup

0.13 0.35 0.35 0.21 0.53 0.36 0.11 0.41 0.24 0.30

Effect Size

PFI = Physical Functioning; Role-P = Role-Physical; Pain = Bodily Pain; Gen H=General Health; Energy = Energy/Fatigue; Social = Social Functioning; Role-E = Role-Emotional; EWB = Emotional Well-being; PCS = Physical Component Summary; MCS =Mental Component Summary.0.11 0.13 0.21 0.24 0.30 0.35 0.35 0.36 0.41 0.53

76

Defining a Responder: Reliable Change Index

(RCI)

)( )2(12

SEM

XX

xxbl rSDSEM 1

Note: SDbl = standard deviation at baseline rxx = reliability

77

7-31% Improve Significantly

% Improving % Declining Difference

PF-10 13% 2% + 11%

RP-4 31% 2% + 29%

BP-2 22% 7% + 15%

GH-5 7% 0% + 7%

EN-4 9% 2% + 7%

SF-2 17% 4% + 13%

RE-3 15% 15% 0%

EWB-5 19% 4% + 15%

PCS 24% 7% + 17%

MCS 22% 11% + 11%

Item Responses and Trait Levels

Item 1 Item 2 Item 3

Person 1 Person 2Person 3

TraitContinuum

78

Reliability Target for Use of Measures with Individuals

Reliability ranges from 0-1 0.90 or above is goal

SE = SD (1- reliability)1/2

Reliability = 1 – (SE/10)2

Reliability = 0.90 when SE = 3.2 95% CI = true score +/- 1.96 x SE

79

Convenience Internet Panels• PROs

– Relatively inexpensive and faster– Able to get to low incidence subgroups

• CONs– Data integrity

• False answers• Answering too fast• Same answer repeatedly• Duplicate surveys from same person

– Respondents may differ from intended target on measured (more educated) and on unmeasured characteristics

Probability Panels

• Selection probabilities known. – Need sampling frame (denominator)

• Get internet access for those without it.

81

Example Questions

1) What is the difference between a profile and preference-based measure?

2) Name a profile measure.

3) Name a preference-based measure.

4) What is a quality-adjusted life year?

5) What does “ACE” stand for?

82

Example Questions (2)1) What is the difference between a PRM and a

PRO?

2) What are the 3 underlying dimensions of HRQOL?

3) Is social support an indicator of HRQOL?

4) What is known about using HRQOL in clinical practice?

5) How much of the time during the last 4 weeks have you been happy?” is an item that measures what?

83

Questions