what do you think about a doctor who uses the wrong treatment, either wilfully or through ignorance,...
TRANSCRIPT
What do you think about a doctor who uses the wrong treatment, either wilfully or through ignorance, or who uses the right treatment wrongly (such as by giving the wrong dose of a drug)?
Most people would agree that such behaviour is unprofessional, arguably unethical, and certainly unacceptable.
Derived from: Altman DG. The Scandal of Poor Medical Research. BMJ, 1994; 308:283
What do you think about researchers who use the wrong techniques (either wilfully or in ignorance), use the right techniques wrongly, misinterpret their results, report their results selectively or draw unjustified conclusions? We should be appalled… but numerous studies of the medical literature have shown that all of the above phenomena are common.
Derived from: Altman DG. The Scandal of Poor Medical Research. BMJ, 1994; 308:283
Understanding your results Research Talk
2015
Dr Emily [email protected]
Office for Research, Western Centre for Health Research & EducationCentre for Epidemiology and Biostatistics, Melbourne School of Population
and Global Health, University of Melbourne
Overview
• Defining your research question – PICOS• Describing data• Understanding the results
– Estimates reported in the literature– Interpreting 95% confidence intervals and p-
values ~ Statistical Inference
Research question
Participants / population• neonates
Intervention / exposure• 14 day administration of antenatal corticosteroids
Comparison• 7 day administration of antenatal corticosteroids
Outcome • Neonatal mortality and neonatal morbidity
Study design• RCT
Murphy et al. The Lancet, 2008; 372:2143-2151.
Research question
Research question
Participants / population• Neonates
Intervention / exposure• 14 day administration of antenatal corticosteroids
Comparison• 7 day administration of antenatal corticosteroids
Outcome • Neonatal mortality and neonatal morbidity
Study design• RCT
Research question
Participants / population• Women at high risk of preterm birth
Intervention / exposure• 14 day administration of antenatal corticosteroids
Comparison• 7 day administration of antenatal corticosteroids
Outcome • Neonatal mortality and neonatal morbidity
Study design• RCT
Study designs
The general idea…
– Evaluate whether a risk factor (or
preventative factor) increases (decreases)
the risk of an outcome (e.g. disease, death,
etc)
exposure
outcome
time
Overview
• Defining your research question – PICOS• Describing data• Understanding the results
– Estimates reported in the literature– Interpreting 95% confidence intervals and p-
values ~ Statistical Inference
Study designs
The general idea…
– Evaluate whether a risk factor (or
preventative factor) increases (decreases)
the risk of an outcome (e.g. disease, death,
etc)
exposure
outcome
time
Murphy et al. The Lancet, 2008; 372:2143-2151.
Summarising the data
Summarising the data
Dreyfus et al. Journal of Pediatrics, 2015 online.
Summarising the data
Summarising the data
Numerical
Categorical
Continuous (age, weight, height)
Discrete (length of stay, # of hospital visits)
Nominal (sex, blood group)
Ordinal (tumour stage, quintile of SES)
Discrete
Nominal
Ordinal
• Which variables are categorical? – Sex (Male/Female)– Country of birth (Australia/Elsewhere)
• Which variables are continuous? – Age (years)– Length of stay (days)
Summarising the data
Summarising the data
Summarising the data
Summarising the data
Summarising the data
0.0
5.1
.15
.2D
ens
ity
44 46 48 50 52 54Age (years)
Stata command: histogram Age
Summarising the data
Standard deviation
Mean = 49.8 years
= 2.1 years
Note, 95% of observations lie within approximately ±2×SD of the mean. In this example, 95% of observations lie within 45.6 and 54.0 years.
Summarising the data
Summarising the data
Summarising the data
0.0
5.1
.15
.2D
ens
ity
0 5 10 15 20 25 30 35Length of stay (days)
Stata command: hist LOS
Summarising the data
Stata command: hist LOS, normal
Summarising the data
Mean = 5 days
Summarising the data
Mean = 5 days
Median = 50th percentile= 4 days
Summarising the data
Mean = 5 days
Standard deviation
Median = 4 days
Mean is not a good measure of central tendency and standard deviation is not a good measures of spread for a skewed distribution
Note, 95% of observations lie within approximately ±2SD of the mean. In this example, 95% of observations lie within -4.8 and 14.8 daysBUT they don’t because LOS can’t be negative!
Summarising the data
Inter-quartile range (IQR) = lower quartile – upper quartile= 25th percentile – 75th percentile = 2 to 6 days
Median = 50th percentile= 4 days
Summarising the data
Summarising the data
Spread
Spread
Central tendency
Central tendency
Summarising the data
Summarising the data
Positive skew
Negative skew
Data variable - numerical
Plot histogram
Normally distributed
NOT normally distributed
Unimodal Multimodal
MeanStandard deviationMinimum-maximum
MedianInter-quartile rangeMinimum-maximum
Categorise variable
Summarising the data
Simpson et al. J Fam Plan and Rep Health Care, 2001; 27:234-236.
Summarising the data
Absolutely critical to choosing the appropriate form of statistical analysis
Normally distributed
SkewedNumerical
Categorical
Continuous (age, weight, height)
Discrete (length of stay, # of hospital visits)
Nominal (sex, blood group)
Ordinal (tumour stage, quintile of SES)
Overview
• Defining your research question – PICOS• Describing data• Understanding the results
– Estimates reported in the literature– Interpreting 95% confidence intervals and p-
values ~ Statistical Inference
Study designs
The general idea…
– Evaluate whether a risk factor (or
preventative factor) increases (decreases)
the risk of an outcome (e.g. disease, death,
etc)
exposure
outcome
time
Estimates reported in the literature
– Risk differences– Odds ratios / risk ratio – logistic regression– Beta-coefficients – linear regression
Summarising the data
Normally distributed
SkewedNumerical
Categorical
Continuous (age, weight, height)
Discrete (length of stay, # of hospital visits)
Nominal (sex, blood group)
Ordinal (tumour stage, quintile of SES)
Measures of association – binary outcome
Binary variables – two categories only
(also termed – dichotomous variable)
Examples: • Outcome – diseased or healthy; alive or dead• Exposure – male or female; smoker or non-smoker;
treatment or control group
Comparing two proportions
With outcome (diseased)
Without outcome (disease free)
Total
Exposed (group 1)
d1 h1 n1
Unexposed (group 0)
d0 h0 n0
Total d h n
• Proportion of all subjects experiencing outcome, p = d/n
• Proportion of exposed group, p1 = d1/n1
• Proportion of unexposed group, p0 = d0/n0
Comparing two proportions - TBM Trial
Adults with tuberculous meningitis randomly allocated into 2 treatment groups:
1. Dexamethasone
2. Placebo
Outcome measure: Death during 9 months following start of treatment.
Research question:
Can treatment with dexamethasone reduce the risk of death among adults with tuberculous meningitis?
Thwaites et al 2004
Comparing two proportions
Death during 9 months post start of treatment
Treatment group Yes No Total
Dexamethasone(group 1)
87 187 274
Placebo (group 0)
112 159 271
Total 199 346 545
Thwaites et al 2004
Comparing two proportions - TBM Trial
Measure of effect Formula
Risk difference p1-p0
Risk Ratio (RR) p1/p0
Odds Ratio (OR) (d1/h1)/(d0/h0)
When there is no association between exposure and outcome:
– Risk difference = 0 – Risk ratio (RR) = 1– Odds Ratio (OR) = 1
Comparing two proportions
Death during 9 months post start of treatment
Treatment group Yes No Total
Dexamethasone(group 1)
87 (d1) 187 (h1) 274 (n1)
Placebo (group 0)
112 (d0) 159 (h0) 271 (n0)
Total 199 346 545
Risk difference = p1-p0 = (87/274)-(112/271) = -0.095
Risk ratio = p1/p0 = (87/274)/(112/271) = 0.77
Odds ratio = (d1/h1)/(d0/h0) = (87/187)/(112/159) = 0.66
Thwaites et al 2004
Comparing two proportions - TBM Trial
Estimates reported in the literature
– Risk differences– Odds ratios / risk ratio – logistic regression– Beta-coefficients – linear regression
Summarising the data
Normally distributed
SkewedNumerical
Categorical
Continuous (age, weight, height)
Discrete (length of stay, # of hospital visits)
Nominal (sex, blood group)
Ordinal (tumour stage, quintile of SES)
Linear regression
Dreyfus et al. Journal of Pediatrics, 2015 online.
There are four assumptions underlying our linear regression model:
Linearity (outcome and exposure)
Normality (residual variation)
Independence (of observations)
Homoscedasticity (constant variance)
Linear regression
Overview
• Defining your research question – PICOS• Describing data• Understanding the results
– Estimates reported in the literature– Interpreting 95% confidence intervals and p-
values ~ Statistical Inference
Statistical Inference
We follow a standard four-step process
1) Sample size
2) Estimate of the effect size
3) Calculate a confidence interval
4) Derive a p-value to test the hypothesis of no association
Statistical Inference
Statistical Inference
P-value
How likely is it we would see a
difference this big
IF
There was NO realdifference betweenthe populations?
What is theprobability (P-value)
of finding theobserved difference
IF
The null hypothesis is true?
Statistical Inference
0.0001
0.001
0.01
0.1
1P-v
alu
e
Increasing evidence againstthe null hypothesis with
decreasing P-value
Weak evidence againstthe null hypothesis
Interpretation of p-values
Strong evidence againstthe null hypothesis
Statistical Inference
Statistical Inference
Overweight and obese adults living in the UK
300 adults participating in a RCT comparing 2 dietary interventions
Mean weight loss after 4 weeksAtkins group – 4.40 kgWeight Watchers group – 2.86 kg
Source: Truby H et al. BMJ 2007
Example: Randomised controlled trial of weight loss programmes in the UK
Group nSample mean
Weight loss after 4 weeks (kg)
Sample standard deviation
Sample standard error
Atkins 57 4.40 2.45 0.32
Weight Watchers
58 2.86 2.23 0.29
1) Estimate of difference in population mean weight loss after 4 weeks between Atkins & Weight Watchers groups = 4.40 – 2.86 = 1.54 kg2) 95% CI: 0.67 kg to 2.41 kg
Source: Truby H et al. BMJ 2007
Statistical Inference
Interpretation
1) We found a difference of 1.54 kg in mean weight loss
after 4 weeks between the Atkins & Weight Watchers
diet groups.
2) From the 95% confidence interval, the true difference
could be as much as 2.41 kg (much greater weight
loss for Atkins diet) or 0.67 kg (marginally greater
weight loss for the Atkins diet compared with Weight
Watchers).
Statistical Inference
P-value: comparing two groups
How likely is it we would see a
difference this big
IF
There was NO realdifference betweenthe populations?
What is theprobability (P-value)
of finding theobserved difference
IF
The null hypothesis is true?
Statistical Inference
Null hypothesis – There is no difference in the population mean weight loss after 4 weeks between the Atkins and Weight Watchers groups
2-sided p-value <0.001
Thus the probability of observing a difference of at least 1.54 kg in the sample means of the two groups, assuming the null hypothesis is true,
is <0.001 or <0.1%.
Statistical Inference
Presenting the results
1) Sample size300 adults participating in a RCT comparing 2 dietary
interventions
2) Estimate of the effect sizeMean weight loss after 4 weeks for Atkins group compared to Weight watchers: 1.54 kg
3) Calculate a confidence interval 95% CI for difference in population means: 0.67 kg to 2.41 kg
4) Derive a p-value to test the hypothesis of no association
P-value < 0.001
Statistical Inference
Overview
• Defining your research question – PICOS• Describing data• Understanding the results
– Estimates reported in the literature– Interpreting 95% confidence intervals and p-
values ~ Statistical Inference