T10 OUTCOME ASSESSMENTWhy, what and how?
Dr. Frederike van Wijck & John Dennis
Learning OutcomePlan a safe, effective and appropriate
intervention, i.e.: • Design and adapt appropriate progressive
physical activity programme(s) after stroke using findings from the physical/ exercise assessments, etc…
Demonstrate competency in relevant assessment procedures:
• Monitor clients’ progress against agreed goals
Content1. Measuring outcomes: why?
2. Measurement: general principles
3. Measuring outcomes: what?– General framework: the ICF– Specific suggestions for the exercise-after-
stroke setting
4. Using outcome measures in an exercise after stroke setting: how?
Outcome measure – a definition:
“ a test or scale administered by therapists that has been shown to measure accurately a particular attribute of interest to patients and therapists and is expected to be influenced by the intervention”
(Mayo, 1995)
1. Measuring outcomes: why?
1. Screening: testing eligibility for exercise
2. Baseline assessment: establishing starting point for exercise programme
3. Follow-up assessment: charting change following exercise
4. Monitoring: to chart adherence and identify adverse effects
This session: baseline and follow-up assessment using outcome measures
1. Measuring outcomes: why not?
Common reasons for not using outcome measures:• It takes time away from the actual exercise• It is a burden for participants• It’s complex and a hassle for the instructors• You need training – we don’t have time for
that• What do these measures tell you anyway – I
know if something works!
Do you??!
SCIENCE VS. COMMON SENSE
Science:
“knowledge, ascertained by observation and experiment, critically tested, systematised and brought under general principles”
Cambridge English Dictionary
Common sense:
“normal understanding, good practical sense in every day affairs, general feeling (of mankind or community)”
Oxford English Dictionary
• Common sense is not good enough for exercise instructors/ health care professionals;
• Exercise/ rehabilitation/ health care needs to be based on science!
Content1. Measuring outcomes: why?
2. Measurement: general principles
3. Measuring outcomes: what?– General framework: the ICF– Specific suggestions for the exercise-after-
stroke setting
4. Using outcome measures in an exercise after stroke setting: how?
2. Measurement: general principles
Characteristics of good outcome measures:
1. Relevant
2. Valid
3. Reliable
4. Sensitive to change
5. Practicable
6. Results can be easily communicated(Wade, 1992)
Characteristics of good outcome measures
Relevance:
the pertinence of the information
Consider:Is this information useful – what does it tell
me?What am I going to do with the information?
Characteristics of good outcome measures
Validity:
the measure does what it is purported to do
Consider:• Which idea/ construct does this measure
address?
Characteristics of good outcome measures
Reliability:
the measure gives the same result each time the same quantity is measured.
Consider:– Intra-rater variation– Inter-rater variation-> Importance of protocols! (tutorial)
Characteristics of good outcome measures
Sensitivity to change:
the measure can detect changes that are relevant
Consider:• On what scale is/ are the item(s) scored?
E.g.:– 0/ 1 or Yes/ No– 0-10 (Visual Analogue Scale)
• Floor and ceiling effects
Characteristics of good outcome measures
Practicability:
the measure is quick and easy to use
Consider:• Amount of information required• Duration of the process• Complexity of the process• Burden on client (and you!)
Characteristics of good outcome measures
Communicability:
The results can easily be reported and understood
Consider:• Amount• Format• Standardisation of the information
Summary - general principles of measurement:
Characteristics of good outcome measures:
1. Relevant
2. Valid
3. Reliable
4. Sensitive to change
5. Practicable
6. Results can be easily communicated(Wade, 1992)
Content1. Measuring outcomes: why?
2. Measurement: general principles
3. Measuring outcomes: what?– General framework: the ICF– Specific suggestions for the exercise-after-
stroke setting
4. Using outcome measures in an exercise after stroke setting: how?
3. Measuring outcomes: what?
A general framework for outcome measurement in clinical practice: the ICF
International Classification of Functioning, Disability and Health
“Aim of the ICF classification is to provide a standard language and framework for the description of health and health-related
states.”
http://www.who.int/classification/icf/intros/ICF-Eng-Intro.pdf
ICF definitions
Impairments are problems in body function or structure such as a significant deviation or loss.
Activity limitations are difficulties an individual may have in executing activities.
Participation restrictions are problems an individual may experience in involvement in life situations.
ICF model
http://www.who.int/classification/icf/intros/ICF-Eng-Intro.pdf
The ICF: A general framework for outcome measurement in rehabilitation
Activity Limitations
Impairments Participation Restrictions
Health Condition
Outcome measures and the ICF
Which outcome measures do you use in your work ?
Where do they fit within the ICF?
Can you think of one outcome measure in each of the ICF domains for a person who has had a stroke?
Outcome measures for exercise after stroke
Activity LimitationsExample?
ImpairmentsExample?
Participation Restrictions
Example?
Person with stroke
3. Measuring outcomes: what/ how?
Suggested outcome measures for exercise after stroke specifically :
• 6 min. walk/ 10 m. walk
• Timed up and Go
• Visual Analogue Scale (VAS)
• Stroke Impact Scale
+ Register: for monitoring adherence
Onto: Measuring Outcomes: how?
• 6 min. walk/ 10 m walk
• VAS
• Timed up and Go
• Stroke Impact Scale
http://figuredrawings.com/Animation.html
6-minute/ 10 m. walk test
Construct: ?
Evaluation?
6-minute walk testConstruct: maximum walking distance in 6 minutes• Relevance: functional test for exercise endurance,
O2 uptake• Validity: good• Reliability: high• Sensitivity: ?• Practicability: good• Reporting: easy (distance (m))• Normative data for healthy people aged 60-89 yrs:
345-623 m(Steffen et al., 2002)
Timed Up and Go
Construct: ?
Evaluation?
Timed Up and Go
Construct: time to stand up from arm chair, walk 3 m, turn, walk back, sit down
• Relevance: functional test for basic mobility for frail elderly in community
• Validity: acceptable• Reliability: moderate - high• Sensitivity: ?• Practicability: good• Reporting: easy (time (s))• Normative data for healthy people aged 60-89 yrs:
7-12 s(Steffen et al., 2002)
VAS
Construct: ?
Evaluation?
VAS
Construct: person’s perception of a particular construct
• Relevance: depending on what is assessed. Can be used to assess individual goal attainment
• Validity: generally good• Reliability: generally high• Sensitivity: high • Practicability: caution with stroke, esp. higher
cortical problems and neglect (Price et al., 1999)• Reporting: easy
Correct use of VAS in stroke
Price et al. (1999), p. 1359.
Correct use of VAS in stroke
Price et al. (1999), p. 1360.
Stroke Impact Scale
Construct: ?
Evaluation?
Stroke Impact ScaleConstruct: the person’s perceived impact of stroke across
range of domains (incl. movement, ADL, cognition, communication, emotion, participation)
• Relevance: high• Validity: good (devised with target population)• Reliability: moderate – very high• Sensitivity: each item on 5-point scale + one VAS item• Practicability: mixed• Reporting: time-consuming but can be done by mail• Normative data: not applicable• Interpretation: change between 10-15 points clinically
meaningful(Duncan et al., 2003)
Pitfalls of measurement – and how to fix them
Problem
• Error:– Systematic– Random
• Wrong signals:– False +– False -
Solution?
Pitfalls of measurement – and how to fix them
Problem
• Error:– Systematic– Random
• Wrong signals:– False +– False -
Solutions:
• Errors:– Calibrate your instrument– Use standardised protocol
• Wrong signals:– Check sensitivity– Verify with other information
Pitfalls of measurement – and how to fix them
Problem
• Error:– Systematic– Random
• Wrong signals:– False +– False -
Solutions:
• Errors:– Calibrate your instrument– Use standardised protocol
• Wrong signals:– Check sensitivity– Verify with other information
Errors are inherent in any form of measurement!
Always be aware and try to reduce.
Summary
Outcome measures are necessary to:
• Establish baseline for exercise
• Evaluate change following exercise
-> science underpinning your work
EVIDENCE BASED PRACTICE
(meten is weten!)
Choosing your Outcome Measure how to go about it
Safe?
NO
YES
Relevant?
NO
YES
Science Robust?
NO
YES
Practicable?
NO
YES
GO
T
H
I
N
K
A
G
A
I
N
Safe?
NO
YES
Relevant?
NO
YES
Science Robust?
NO
YES
Practicable?
NO
YES
GO
T
References
• DUNCAN, P. W., LAI, S. M., BODE, R. K., PERERA, S. & DEROSA, J. (2003b) Stroke Impact Scale-16: A brief assessment of physical function. Neurology, 60, 291-6.
• DUNCAN, P. W. Stroke Impact Scale (SIS). Rehabilitation Outcomes Research Centre, US Department of Veteran Affairs. Available from: http://www1.va.gov/rorc/stroke_impact.cfm (last accessed 05/12/05).
• WADE, D. T. (1992) Measurement in Neurological Rehabilitation, Oxford, Oxford University Press.
• WORLD HEALTH ORGANISATION (2001). International Classification of Functioning, Disability and Health. Available from http://www.who.int/classifications/icf/en/