1

Class 2Measurement Terminology and

Locating Measures

September 28, 2006

Anita L. StewartInstitute for Health & Aging

University of California, San Francisco

2

Overview of Class 2

Measurement terminology– Confusion, variation, and overlap

Types of response scales– Contribution to concept definition

Measurement models– Depicting structure of measures

Locating measures

3

Measurement

Assignment of numbers to aspects of objects or events according to a rule

Assignment of numbers to an unobservable characteristic of individuals (construct)

4

Construct, Concept, Variable, Latent Trait, Latent Variable

Terms defined differently by different investigators/authors

Meaning is very similar– considerable overlap among them,

Often used interchangeably

5

Construct/Concept

A variable that is relatively abstract as opposed to concrete and is defined or operationalized in terms of observed indicators

An idea developed or constructed through informed scientific theory. Concepts that are directly or indirectly observable.

An abstraction based on observations of certain behaviors or characteristics (e.g. pain, stress)

6

Latent Trait, Factor

Latent - present but not visible, unobservable Latent trait - unobservable set of characteristics

that can be empirically inferred and estimated through answers to a set of questions

(Medical Care glossary) Factor - a latent variable or theoretical construct

operationalized in terms of associations among indicators in a factor analysis

(Stewart and Ware, 1992 glossary)

7

Latent Variable, Manifest Variable

Latent variable - unobservable latent trait given a label

Manifest variable is the “observed” information, i.e., the answers to items used to infer the latent trait

8

Domain, Dimension, Sub-domain, Component

Domain: a state, attitude, perception, behavior, or other sphere of action or thought related to health …. All entities in a single domain have some property in common ….

(Patrick & Erickson, 1993) Dimension: a distinct component of a

multidimensional construct that can be theoretically or empirically specified (e.g., physical and mental health)

(Stewart & Ware, 1992, glossary)

9

Example: Dimension, Component, Sub-domain

Mental health (dimension)– Psychological distress (component)

» Depression (sub-domain)» Anxiety

– Psychological well-being» Positive affect» Sense of belonging

10

Components of Domains of MOS Long-form Measures

Role Role limitations due to:Functioning -- physical health

-- emotional problems

Social Social limitations due to healthFunctioning Sexual problems

Family functioningMarital functioning

Pain Pain severityEffects of painDays with pain

11

Multidimensional and Unidimensional

A multidimensional “measure” or “instrument” provides several scores– A “profile” of scores

Unidimensional measure - all items measure only one concept– Only one scale score is obtained

Dimensionality must be empirically tested– e.g., factor analysis identifies one factor,

or more than one factor

12

Example of Multidimensional “Instrument” -

Patient Satisfaction Questionnaire (PSQ) 55 items, 18 subscales, e.g.

– Access to care– Technical quality– Interpersonal manner– Explanations– Continuity of care

Marshall, GN Psychological Assessment, 5:477-483, 1993

13

Example of a Multidimensional “Instrument” – SF-36Measures 8 health concepts (36 items): Physical functioning Role limitations due to physical health Role limitations due to emotional problems Social functioning Pain Vitality (energy/fatigue) Mental health Health perceptions

14

Example of Unidimensional “Measure”

Perceived Stress Scale (PSS) 14 items, subjective experiences of stress

– felt confident could handle life’s problems

– able to control irritations in your life

– difficulties piling up so high, could not overcome them

Single score from all itemsCohen, S, J Health Soc Behav 24:385-396, 1983

15

Instrument, Profile, Battery, Measure, Questionnaire, Survey, Tool, Inventory Instrument - typically refers to a published,

named measure or set of measures Profile - same, but all scores on the same

metric and can be compared (0-100 scales) Battery (of independent measures)

– collection of measures from diverse sources Measure - single- or multi-item scale or

index (umbrella term)

16

Generic vs. Disease-Specific Approaches

Generic– Applicable across many conditions and

diseases (e.g., sleep, health perceptions, fatigue, pain)

Disease-specific– Specific to particular disease or condition

(e.g., arthritis, cancer, depression)» Generic approach (e.g., general dexterity scale for arthritis

study)» Specific (nausea due to chemotherapy, back pain)

17

Generic or Disease Specific bySingle Domain, Profile, or Index

Single domain

Profile Summary index

Generic Pain

Fatigue

SF-36

SIP

PCS

MCS

QWB

Disease specific

Nausea

Back pain

Diabetes impact

18

Preference Based or Utility Measures

Utilities are numeric measurements that reflect the desirability people associate with a health state or condition– Value of a particular health state

– Preference for a health state compared to a different one

19

Questionnaire, Survey, Tool, Inventory

Questionnaire - any of the above formatted into a “questionnaire” or set of measures

Survey - same as questionnaire Tool = Questionnaire = Survey? Inventory?

20

Examples of Names of “Instruments”

SF-36 Short Form Health Survey Beck Depression Inventory Health Assessment Questionnaire Sickness Impact Profile Center for Epidemiological Studies

Depression Scale

21

Example of a Battery

A survey for a particular study containing the following measures/instruments:– SF-36 (8 domains)– Perceived stress scale (1 domain)– Social support survey (4 domains)– Pain measure

Typical in clinical trials, epidemiological studies– Each measure serves particular purpose (outcome,

predictor, mediator, covariate)

22

Measure, Scale, Index, Item

Measure - single- or multi-item scale or index (umbrella term)

Scale - aggregation of items from one concept or domain, scored according to an accepted scaling method

Index - aggregation of 2 or more scales into a summary score

Item - a single question or statement including its response scale

23

Overview of Class 2

Measurement terminology– Confusion, variation, and overlap

Types of response scales– Contribution to concept definition

Measurement models– Depicting structure of measures

Locating measures

24

Item

Consists of an item stem and a response scale or response choices

Item stem – – The question or statement

Response scale or choices –– Answers presented to a respondent with which to

respond to a (closed-ended) question

25

Example of a Response Scale

How often have you felt tired inthe past week? 1 Never 2 Once or twice 3 A few times 4 Fairly often 5 Very often

Responsescale

Itemstem

26

Response Choice Formats

Verbal descriptors of all levels Verbal descriptors of endpoints only Verbal descriptors of endpoints plus

midpoint Visual analogue scales:

10cm line with endpoints labeled

27

Format: Verbal Descriptors of All Levels

1 - All of the time2 - Most of the time3 - Some of the time4 - A little of the time5 - None of the time

28

Vague, Imprecise Quantifiers

How often?– Very often, pretty often, not too often

– Sometimes, often, never How much?

– Too little, about right, too much

– Below average, average, above average

Bradburn NM, Vague quantifiers. Public Opinion Quart 1979, 92-101.

29

Intensity Scales – “How distressed have you been?”

1 – Not at all

2 - Slightly

3 - Moderately

4 - Quite a bit

5 - Extremely

1 – Not at all

2 - A little

3 - A fair amount

4 - Much

5 - Very much

30

Intensity Scales – “How distressed have you been?” (even more scales)

1 – Not at all

2 - A little

3 - Fairly

4 - Quite

5 - Extremely

1 – Not at all

2 - A little

3 - Somewhat

4 - Very much

31

“How Often” Response Scales

1 - Never

2 - Once or twice

3 - A few times

4 - Often

1 – Never

2 - Hardly ever

3 - Some days

4 - Most days

5 - Almost every day

6 - Always, every day

32

“How Often” Response Scales - more

1 - Never

2 - Almost never

3 - Sometimes

4 - Fairly often

5 - Very often

6 - Always

1 - Once or twice

2 - A few times

3 - Fairly often

4 - Very often

5 - Almost every day

6 - Every day

33

Format: Verbal Descriptors of Endpoints Only

0 1 2 3 4 5 6 7 8 9 10 No Pain pain as bad

as you can imagine

34

Format: Verbal Descriptors of Endpoints with Midpoint Labeled

Very Very

much 1 2 3 4 5 6 7 muchworse No better

change

35

Format: Verbal Descriptors of Endpoints with Midpoint Labeled

-3 -2 -1 0 1 2 3Very No Very much change muchworse better

36

Format: Visual Analogue Scale, 10 cm Line

No Pain pain as bad

as you can imagine

37

Optimal Number of Response Choices

Are 100 levels better than 5 or 10? 5-points is probably optimal for respondents

– Especially if items will be combined into multi-item scales

For single-item measures– 7 points may be preferred– Allows distribution over 5 points since people

seldom use endpoints Hard to come up with labels for 7-point scales

38

Format: Verbal Descriptors of Endpoints Only

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

No Pain pain as bad

as you can imagine

39

Magic Number 7 + 2

George Miller determined that 7 “chunks” of information is the most that short-term memory can retain– Plus or minus two (5-9 possible)

Seven (+ 2) is also the most points that people can discriminate along a continuum

Principle has held up well in measurement

Miller, GM Psychological Bulletin, 63:81-97, 1956

40

Can You Make a 7-point Intensity Scale? (How physically active were you this week?)

1 - Not at all active

2 - A little bit active

3 - Moderately active

4 - Quite active

5 - Extremely active

1 - Not at all active

2 - A little active

3 - Fairly active

4 - Quite active

5 - Very active

6 - Extremely active

41

Overview of Class 2

Measurement terminology– Confusion, variation, and overlap

Types of response scales– Contribution to concept definition

Measurement models– Depicting structure of measures

Locating measures

42

Measurement Models

The dimensional structure of a measure in terms of how the items related to the constructs (latent traits, factors, concepts)

Can be depicted as a list or visually Standard visual format for structural

equation modeling (SEM)

43

Measurement Models

Physical Functioning (4 items) Psychological Distress (7 items)

44

Measurement Model (List format)

Physical Functioning defined in terms of:– Walking

– Climbing stairs

– Bending

– Reaching

45

Measurement Model (Visual format)

Physical Functioning

WalkingClimbing

StairsBending Reaching

46

Measurement Model (List format)

Psychological distress– Depression

» Sad» Lost interest» Can’t get going

– Anxiety» Restless» Nervous

47

Measurement Model (Visual format)

Psychological Distress

Depression Anxiety

SadLost

interestCan’t

get goingRestless Nervous

48

Measurement Model: Family Factors (List format)

Family environment– Conflict– Cohesion

Parenting behaviors– Nurturing– Discipline– Expectations

49

Measurement Model: Family Factors (Visual format)

Family Factors

Family Environment

Parenting Behaviors

Conflict Cohesion

Fight a lot

Throw things

Help and support

Together-ness

50

Measurement Model: Family Factors (Visual format)

Family Factors

Parenting Behaviors

Nurturing Discipline Expectations

PlayTake to

park

Spank Once

per week

Yell at for

whining

Wash &dry hands

Put Awaytoys

Family Environment

51

What’s in a Label?

Don’t be fooled

52

Content Areas: Four Depression Measures

CES-D HAM-D Beck GDS

Sleep problems X X X X

Concentration X X

Fatigue, energy X X

Appetite loss X X

Depressed mood X X X

Anxiety, restlessness X X

Irritability X X

Sense of failure X X

Social withdrawal X X

Guilt X

Self-criticism X X

53

Content Areas: Four Standard Measures of Physical Functioning

AIMS MOS HAQ SIP

Walking X X X X

Climbing stairs or steps X X X X

Bending, kneeling X X X X

Lifting, carrying X

Getting out of bed X

Bathing X X X

Running errands X

Opening jars X

Vigorous activities X X X

54

MOS Physical Functioning: SF-36 versus SF-12

SF-36 SF-12 Vigorous activities X Moderate activities X X Lift/carry X Climb several flights stairs X X Climb one flight stairs X Bend/kneel/stoop X Walk > 1 mile X Walk several blocks X Walk 1 block X Bathe/dress X

55

MOS Mental Health Items:SF-36 versus SF-12

SF-36 SF-12 Nervous X Down in the dumps X Calm and peaceful X X Downhearted and blue X X Happy person X

56

Overview of Class 2

Measurement terminology– Confusion, variation, and overlap

Types of response scales– Contribution to concept definition

Measurement models– Depicting structure of measures

Locating measures

57

Locating Specific Measures

Compendia Web Organizations, foundations National and state surveys Large research studies Many other sources

58

Locating Specific Measures: Compendia

Compendia of measures– Many books review and critique various

measures Web based review of such books:

www.measurementexperts.org/instrument/book_compendium.asp

59

Locating Specific Measures: Compendia by Web

Health and Psychosocial Instruments (HaPI)– Over 120,000 instruments

– http:/gateway.ovid.com/

– Need ovid account – UCSF has one if use computer at work (or vpn) – no need to enter password

60

Locating Specific Measures: Compendia by Web

Measurement Excellence and Training Resource Information Center (METRIC)

– http://www.measurementexperts.org

– Click on “find an instrument”

61

Locating Specific Measures: Organizations and Foundations

RAND publishes measures, scoring manuals, and lists citations– Measures of quality of care, patient satisfaction– Measures of health-related quality of life

» Generic and disease specific

– All Medical Outcomes Study measures http://www.rand.org/health/

– Go to “surveys and tools”

62

Inter-University Consortium for Political and Social Research

http://www.icpsr.umich.edu/

Maintains archive of social science data Membership-based organization – over 500

member colleges/universities– UCSF is a member

Can search website using keywords to locate studies, data, and questionnaires

63

Commonwealth Fund

The Fund tracks trends in health coverage, access and quality, and general policy/practice issues in the U.S. and internationally. All information is available free of charge. Raw data are available to researchers one year post release.

Surveys and data: – http://www.cmwf.org/surveys/surveys.htm

64

Commonwealth Fund: Example

2001 Health Care Quality Survey This survey assessed racial and ethnic

differences in patient experiences with the health care system, focusing on such quality measures as patient–physician communication and access to care. 

http://www.cmwf.org/surveys/surveys_show.htm?doc_id=228171

65

Locating Specific Measures: National Surveys

National Center for Health Statistics– Surveys and data collection systems

Can download any survey or portion of survey

Summary of Surveys (see handout)– http://

www.cdc.gov/nchs/data/NCHS_Survey_Matrix.pdf#search=%22Summary%20of%20Surveys%20CDC%22

66

National Center for Health Statistics: Examples of Surveys

National Health Interview Survey National Health and Nutrition Examination Survey National Health Care Survey

– Ambulatory health care data (NAMCS)– National Home and Hospice Care Survey

National Survey of Family Growth National Maternal and Infant Health Survey Longitudinal Studies of Aging (LSOA)

www.cdc.gov/nchs/express.htm

67

Locating Measures: National Agencies

Agency for Healthcare Research and Quality (AHRQ)– National Quality Measures Clearinghouse

– Consumer Assessment of Health Plans Survey (CAHPS)

– National Healthcare Quality Report

– National Healthcare Disparities Report

68

Locating Measures: National Agencies

Centers for Disease Control Behavioral Risk Factor Surveillance

System– Surveys of health behaviors

http://www.cdc.gov/brfss/questionnaires/index.htm

69

Locating Specific Measures: NCI

National Institutes of Health: special initiatives on measures– National Cancer Institute

Health Information National Trends Survey (HINTS)– Working group compiled cancer screening

questions, identified best ones, conducted extensive pretesting, cognitive interviewing

– Measures are on the NCI web site

70

Locating Measures: State Surveys

http://www.chis.ucla.edu/ California Health Interview Survey

(CHIS) “Methodology – review questionnaires”

71

Locating Measures: Projects and Centers

Projects/Centers– Toolkit to measure end of life care (TIME)– Stanford Patient Education Research Center– Michigan Diabetes Research and Training Center– Resource Centers for Minority Aging Research

Proprietary measures– SF-36, SF-12 (have to pay for instruments, scoring

manuals)

72

Tool Kit of Instruments to Measure End of Life Care (TIME)

www.chcr.brown.edu/pcoc/toolkit.htm Instruments to measure “patient-focused,

family centered” care Annotated bibliographies of instruments

that measure outcomes of care at the end of life

73

TIME Annotated Bibliographies Available:

Quality of life Pain and other

symptoms Emotional and

cognitive symptoms Advance care

planning

Spirituality Continuity of care Caregiver well-being Grief and

bereavement and more

74

Michigan Diabetes Research and Training Center: Survey Instruments

www.med.umich.edu/mdrtc/survey/

•Diabetes Care Profile (DCP) •Diabetes History (DMH) •Diabetes Knowledge Test (DKT) •Diabetes Attitude Scale (DAS-3) •Diabetes Empowerment Scale (DES) •Michigan Neuropathy Screening Instrument (MNSI)

75

Locating Specific Measures

Large research studies on topic similar to yours– Health ABC, CARDIA

– Women’s Health Initiative

– EPESE – Established Populations for Epidemiological Studies of the Elderly

76

Locating Measures: Large Studies

Study of Women’s Health Across the Nation (SWAN)– physical, biological, psychological, and

social changes of women during their middle years

www.edc.gsph.pitt.edu/swan/public Questionnaire can be downloaded

77

UCSF Data Coordinating Center

https://rds.epi-ucsf.org/ go to eForms Warehouse link (lower left)

– Specific study? ALL

– Annotated? NO

– Specific content search?» pulldown menu with types of measures

78

Proprietary and Non-Proprietary Measures: An Example

SF 36– www.sf-36.org

RAND SF-36: non proprietary– www.rand.org/health/survesy/sf36item/

79

Locating Specific Measures: Bibliographic Searches

Published measurement articles– Medline Searches

– MESH headings or keywords:» health status indicators

» outcome assessment (health care)

» psychometrics (methods)

» questionnaire

» measurement

80

Locating Specific Measures: Finding Authors of Measures

Published research using measure you are interested in– Unpublished measures often described in

methods– Authors may provide measures

81

Summary

Don’t let the terminology get you down– Be aware of alternative uses of words/labels

Measurement models – SEM depiction is useful

Locating measures– Maintain a file of measures of concepts you

are interested in

82

Next Week: Class 3

Process of selecting measures Classical methods of scale construction Rationale and criteria for multi-item scales

Might want to do readings for class 3 before class