measuring health-related quality of life in neurology clinical

Measuring Health-Related Quality of Life in Neurology Clinical Research:

Neuro-QOL Item Banks and Disease Targeted Scales

Cindy Nowinski, Deborah Miller, Amy Peterman, Richard Gershon,

David Victorson, Seung Choi, Jin-Shei Lai, Rita Bode, Claudia Scala-Moy, and David

Cella

http://www.neuroqol.org/default.aspx

Objectives of Neuro-QOLRFP-NIH-NINDS-03-04

Develop a core set of QOL questions that cut across chronic adult and pediatric neurologic disorders – “Item Banks”

Develop supplemental questions that address concerns specific to targeted diseases or subgroups of patients – “Scales”

Create a publicly available, adaptable and sustainable system allowing clinical researchers access to common item repositories (English and Spanish Language) and computerized adaptive testing (“CAT”)

Project Overview

• Awarded end of 2004• Phase I – Bank and scale development

- completed• Phase II - Field testing

- in progress• Public release – Fall 2010

“Item Bank”• A large collection of items measuring a single

domain • Items in a bank are linked on a common

metric• Items are selected to maximize precision and

retain clinical relevance• Basis for tailored/adaptive testing

– Short forms and CATs

Phase I: Bank and Scale Development

Key Goals• Identify target neurological conditions • Identify domains• Select/create items in English and Spanish

Goal 1 – Identify Target Conditions

• Adult– Epilepsy– Multiple Sclerosis– Parkinson’s disease– Stroke– ALS

• Pediatric – Epilepsy– Muscular dystrophies

Data Sources: Literature, Expert Surveys (N=152) and Consensus Panel

PHYSICALFunction/Health Symptoms

Fatigue

Sleep

Mobility/Ambulation

Sexual

Bowel/Bladder

ADL’s/UE’s

MENTALEmotional Health Cognitive Health

Perceived

Applied

Depression

Stigma

Positive Psych Fn

Anxiety

Personality Change

End of Life Concerns

SOCIALRole Participation Role Satisfaction

B=Bank T=Targeted Scale TD=Targeted scale developed but not tested

Goal 2 -Adult Domains Drawn from Expert Survey, Patient Focus Groups and Individual Interviews

B-includes communication problems

BB

TD

TD

T

T

B

B

B

B

B

B B

T

TD

PHYSICALFunction/Health Symptoms

Pain

Fatigue

Mobility/Ambulation

ADL’s/UE’s

MENTALEmotional Health Cognitive Health

Depression

Stigma

Anxiety

SOCIAL

B=BankT=Targeted Scale

Goal 2 - Pediatric Domain Framework for Item Banks and Scales

B

B T

T

B

B

TT

B

Perceived Cognitive Function

Goal 3 – Select/Create Items

Psycho-metricTesting

Item Bank(IRT-calibrated items)

0.0

0.5

1.0

1.5

2.0

2.5

-3 -2 -1 0 1 2 3

Theta

Info

rma

tio

n

0.0

0.2

0.4

0.6

0.8

1.0

-3 -2 -1 0 1 2 3

Theta

Pro

ba

bil

ity o

f R

esp

on

se

Short FormInstruments

CAT

Literature Review

Item Pool

Patient Focus Groups

Expert Input

Existing Items

Questionnaireadministered to largerepresentative sample

SecondaryData Analysis

CognitiveTesting

TranslationExpertReview

Newly Written

Items

Overlap between Neuro-QOL, PROMIS and AM-PACDomain Neuro-QOL PROMIS AM-PAC

Social Role Performance X X --

Social Role Satisfaction X X --

Mobility/Ambulation X X X

Self Care/Upper Extremity X X X

Depression X X --

Anxiety/Fear X X --

Positive Psychological Function X X --

Fatigue/Weakness X X --

Sleep Disturbance X X --

Pain X X --

Stigma X -- --

Perceived Cognitive Deficits X -- --

Applied Cognition X -- X

Personality and Behavioral Change X -- --

Phase II: Testing and Refinement of Final HRQL Instrument

• Primary Goals– Confirm factor structure of primary domains

• Unidimensionality of sub-domains– Conduct provisional differential item functioning (DIF) analyses– Produce item calibrations for all items in each sub-domain– Create profile scores for each disease population– Develop Short Forms– Test-retest validation for short forms– Assess longitudinal use of the forms

Neuro-QOL Testing Highlights• WAVE Ia: Online clinical testing in adults (n=511)

and children (n=59) - Complete

• WAVE Ib: Online English and Spanish general population calibration testing for adults (n= 3000) and children (n=1500) – Complete

• WAVE II: Clinical validation testing of IRT-calibrated short forms (n=800 baseline; n=400 7 day; n=400 180 day) – In progress

Analysis for Wave Ia & Ib• N>500

– PROMIS analysis plan • Reeve et al, 2007; or

– Evaluation of unidimensionality• Descriptive statistics

• Factor analysis

– Estimation of Item Parameter using 2-PL IRT

• N < 500 (Wave 1a Pediatric)– Descriptive statistics and Rasch analysis

Results – Wave Ia Adults (N=511)

Fatigue Stigma Personality # of items being tested 20 26 20 # of items retained 19 24 18 # of factors 1 1 1

-3 -2 -1 0 1 2 30

10

20

30

40

50

60

70

Scale Score

Info

rma

tio

n

Test Information and Measurement Error

0

0.18

0.36

0.53

0.71

0.89

Stan

da

rd E

rror

Fatigue

-3 -2 -1 0 1 2 30

10

20

30

40

50

60

70

80

90

100

Scale Score

Info

rmat

ion


0

0.16

0.32

0.47

0.63

0.79

Standard E

rror

-3 -2 -1 0 1 2 30

10

20

30

40

Scale Score

Info

rmat

ion


0

0.11

0.21

0.32

0.43

0.53

Standard E

rror

Stigma Personality

Results – Wave Ia Pediatric (N=59)

Fatigue Perceived Cognitive Function

Pain Stigma

# of Items tested

13 20 10 20

# of items retained

All 13 items are retained

19 items are retained All 10 items are retained

18 items are retained

Results – Wave Ib Pediatric Emotion

Depression Anger Anxiety # of items being tested 19 8 19 # of items retained 18 8 19 # of factors 1 1 1

-3 -2 -1 0 1 2 30

10

20

30

40

50

60

70

Scale Score

Info

rmat

ion


0

0.18

0.36

0.53

0.71

0.89

Standard E

rror

Depression

-3 -2 -1 0 1 2 30

10

20

30

40

50

Scale Score

Info

rmat

ion


0

0.21

0.42

0.63

0.84

1.04

Standard E

rror

Anger Anxiety

-3 -2 -1 0 1 2 30

10

20

30

40

50

60

70

Scale Score

Info

rmat

ion


0

0.20

0.41

0.61

0.81

1.02

Standard E

rror

Results – Wave Ib Pediatric Social Function

• Started with– “sociability” (item n=30), and – “social role” (item n=8)

• Ended with – “interaction with peers” (item n=17), and – “interaction with adults” (item n=8)

Results – Wave Ib Pediatric Social Function (cont.)

-3 -2 -1 0 1 2 30

5

10

15

20

25

Scale Score

Info

rmat

ion


0

0.21

0.42

0.62

0.83

1.04

Standard E

rror

Interaction with Adults

-3 -2 -1 0 1 2 30

10

20

30

40

50

Scale Score

Info

rmat

ion


0

0.20

0.39

0.59

0.78

0.98

Standard Error

Interaction with Peers

However…….

• Sleep– All 20 items are tested in Wave II due to ambiguity of

factor structure

• Physical Function (both adults and pediatrics) – Skewed distribution– 20 items were selected based on clinician input

Reliability and Item Total Correlation Examples

Domain Alpha i-total r

Positive Psychological Function .984 .60 - .91

Depression .983 .64 - .90

Anxiety .970 .56 - .87

Perceived Cognitive Dysfunction .982 .57 - .85

Applied Cognitive Function .973 .54 - .78

CFA Fit Statistic Examples

CFI TLI RMSEA #LID

>=.15

PPF .933 .991 .205 2

DEP .946 .993 .134 1

ANX .832 .962 .232 58

PCD .904 .992 .086 4

ACF .820 .973 .124 30

Adult Item Bank Information FunctionsPositive Psychological Function

T-score

rel.=.90

rel.=.95

0

20

40

60

80

100

10 20 30 40 50 60 70 80 90

Depression

T-score

rel.=.90

rel.=.95

0

20

40

60

80

100

10 20 30 40 50 60 70 80 90

Anxiety

T-score

rel.=.90

rel.=.95

0

10

20

30

40

50

60

10 20 30 40 50 60 70 80 90

Perceived Cognitive Deficiency

T-score

rel.=.90rel.=.95

0

50

100

150

10 20 30 40 50 60 70 80 90

Applied Cognitive Function

T-score

rel.=.90

rel.=.95

0

20

40

60

80

100

120

140

10 20 30 40 50 60 70 80 90

Stigma

T-score

rel.=.90

rel.=.95

0

20

40

60

80

100

10 20 30 40 50 60 70 80 90

Mobility Ambulation

T-score

rel.=.90rel.=.95

0

50

100

150

10 20 30 40 50 60 70 80 90

Upper Extremity ADL

T-score

rel.=.90rel.=.950

100

200

300

400

500

600

10 20 30 40 50 60 70 80 90

Social Role Ability

T-score

rel.=.90rel.=.95

0

50

100

150

200

250

10 20 30 40 50 60 70 80 90

Social Role Satisfaction

T-score

rel.=.90rel.=.95

0

50

100

150

200

250

300

10 20 30 40 50 60 70 80 90

The shaded areas represent the effective measurement ranges where a reliability of roughly .95 is attainable.

Wave II Clinical Validation Testing is Ongoing

Assessment I (Baseline)

Assessment II (7 days)

Assessment III(180days)

Stroke 100 X X

Multiple Sclerosis 100 X X

Parkinson’s Disease 100 X X

Adult Epilepsy 100 X X

ALS 100 X X

Proxies matched to Stroke 100 X X

Pediatric Epilepsy 50 X X

Proxies matched to above Epilepsy

50 X X

Pediatric Muscular Dystrophy 50 X X

Proxies matched to above MD 50 X X

Maximum Subtotal 800

Thank you from the Neuro-QOL Investigators

David Cella, PhDCindy Nowinski, MD, PhDFrancois Bethoux, MDRichard Gershon, PhDAllen Heinemann, PhDAlan Jette, PhDDeborah Miller, PhDAmy Peterman, PhDAnthony Reder, MDDavid Victorson, PhDClaudia Scala-Moy, PhD

Northwestern UniversityNorthShore University HealthSystemCleveland Clinic FoundationNorthwestern UniversityRehabilitation Institute of ChicagoBoston UniversityCleveland Clinic FoundationU of North Carolina- CharlotteUniversity of ChicagoNorthwestern University

NINDS Project Officer

www.NeuroQOL.org

Connections to other studies


Location

50

52

54

56

I needed medical instructions repeated because I could not keep them straightI forgot to do things like turn off the stove or turn on my alarm clockI made mistakes when writing down phone numbersMy speech was difficult for others to understandAfter I made a mistake, I got stuck and couldn’t figure out a new way to goWhen I was reading I needed to use a ruler or my finger to keep track of which line I was onI was not as good a listener as usualI was told that I start talking before the other person finishesI was told that I repeat myselfI often didn’t recognize my mistakes right awayI had trouble remembering the name of a familiar personI had trouble spelling words correctly when writingI used the wrong word when I referred to an objectI reacted slowly to things that were said or doneI had trouble adding or subtracting numbers in my headI had to repeat myself so others could understand meI had trouble planning out steps of a taskI had trouble remembering new information, like phone numbers or simple instructionsMy thinking was confusedI had trouble recalling the name of an objectI communicated by gestures, for example, moving my head, pointing or sign languageI had trouble saying what I mean in conversations with othersI had trouble keeping track of the day or dateMy thinking was slowI had to work really hard to pay attention or I would make a mistakeI had difficulty paying attention for a long period of timeI had trouble getting started on very simple tasksI had trouble making decisionsI had to read something several times to understand itI made simple mistakes more easilyI had trouble remembering whether I did things I was supposed to doWhen I had something to do that takes a long time, I had trouble deciding where to startI had trouble thinking clearlyI had difficulty doing more than one thing at a timeI felt like my mind went blankI had trouble keeping track of what I was doing if I was interruptedI had trouble concentratingI had trouble finding the right word(s) to express myselfI walked into a room and forgot what I meant to get or do there

Applied Cognitive Function

Location

35

36

37

38

39

40

41 remembering a list of 4 or 5 errands without writing it down?keeping important personal papers such as bills, insurance documents and tax forms organized?reading a long book (over 100 pages) over a number of days?taking care of complicated tasks like managing a checking account or getting appliances fixed?handling an unfamiliar problem?organizing what you want to say?checking the accuracy of financial documents?doing calculations in your head while shopping?planning for and completing regularly scheduled weekly tasks?explaining how to do something involving several steps to another person?managing your time to do most of your daily activities?counting the correct amount of money when making purchases?have planning what to do in the day?learning new tasks or instructions?carrying on a conversation with a familiar person in a noisy environment?planning an activity several days in advance?remembering where things were placed or put away?using a local street map to locate a new store or doctor's office?putting words together to form grammatically correct sentences?remembering to take medications at the appropriate time?planning for and keeping appointments that are not part of your weekly routine?carrying on a conversation with a small group of familiar people?dialing familiar numbers such as a family member or doctor?reading and following complex instructions?composing a brief note or e-mail to someone?understanding pictures that explain how to assemble something?looking up a phone number or address in the phone book?making yourself understood to other people during ordinary conversations?understanding familiar people during ordinary conversations?

2 4 6 8 10 12

0.0

0.2

0.4

0.6

0.8

1.0


Test Length

SE

T-score=30T-score=40T-score=50T-score=60T-score=70

measuring health-related quality of life in neurology clinical

Documents