reliability and validity introduction to study skills & research methods (hl10040) dr james...
TRANSCRIPT
Reliability and Validity
Introduction to Study Skills & Research Methods (HL10040)
Dr James Betts
Lecture Outline:•Definition of Terms
•Types of Validity
•Threats to Validity
•Types of Reliability
•Threats to Reliability
•Introduction to Measurement Error.
Some definitions…• Validity
• Reliability
• Objectivity
Types of Experimental Validity• Internal
• External
Logical Statistical
Validity
Logical Validity
Statistical Validity
Logical/Statistical Validity
Interesting Example: Breast Cancer• Incidence: ~1 % (0.8 %)
(i.e. a positive result should be detected for approximately 1 in every 100 women tested)
• Sensitivity: ~90 % (87 %)(the mammogram is sensitive enough that approximately 90 in every 100 breast cancer patients will receive a positive result)
• Specificity: ~90 % (93 %)(the mammogram is specific enough that approximately 90 in every 100 healthy patients will receive a negative result).
Data from Kerlikowske et al. (1996)
Quick Test
• What is the probability that a patient receiving a positive result actually has breast
cancer?
Threats to Internal Validity
e.g.
O1 T O2
PO3
RO4
Threats to Internal/External Validity
Threats to Internal/External Validity
Threats to External Validity
Reliability
Subject 1 60 ml.kg-1.min-1 60 ml.kg-1.min-1 60 ml.kg-1.min-1
Subject 2 55 ml.kg-1.min-1 55 ml.kg-1.min-1 55 ml.kg-1.min-1
Subject 3 70 ml.kg-1.min-1 70 ml.kg-1.min-1 70 ml.kg-1.min-1
Reliability
Subject 1 60 ml.kg-1.min-1 65 ml.kg-1.min-1 65 ml.kg-1.min-1
Subject 2 55 ml.kg-1.min-1 60 ml.kg-1.min-1 60 ml.kg-1.min-1
Subject 3 70 ml.kg-1.min-1 75 ml.kg-1.min-1 75 ml.kg-1.min-1
Reliability
Subject 1 60 ml.kg-1.min-1 72 ml.kg-1.min-1 57 ml.kg-1.min-1
Subject 2 55 ml.kg-1.min-1 61 ml.kg-1.min-1 52 ml.kg-1.min-1
Subject 3 70 ml.kg-1.min-1 40 ml.kg-1.min-1 84 ml.kg-1.min-1
Types of Reliability
Subject 1 60 ml.kg-1.min-1 63 ml.kg-1.min-1 57 ml.kg-1.min-1
Subject 2 55 ml.kg-1.min-1 56 ml.kg-1.min-1 48 ml.kg-1.min-1
Subject 3 70 ml.kg-1.min-1 65 ml.kg-1.min-1 66 ml.kg-1.min-1
Rater Reliability (Objectivity)• Intrarater reliability
• Interrater reliability
Threats to Reliability
Measurement Errors• Ultimately, reliability is dependent on the
degree of measurement error in a given study
• The overall error in any measurement is comprised of both systematic and random error
• We will address measurement error further next week…
Literature Search Assignment
• The handout lists 8 questions which can be answered through retrieving the corresponding source articles
• Answer as many as possible and hand them in at next week’s lecture
• Does not contribute to final assessment but will be marked
• DO NOT contact author or order articles.
Selected Reading• Atkinson, G. and A. M. Nevill. Statistical methods for
assessing measurement error (Reliability) in variables relevant to sports medicine. Sports Medicine. 26:217-238, 1998.
• Holmes, T. H. Ten categories of statistical errors: a guide for research in endocrinology and metabolism. American Journal of Physiology. 286: E495-501.
• Thomas J. R. & Nelson J. K. (2001) Research Methods in Physical Activity, 4th edition. Champaign, Illinois: Human Kinetics
