Understanding Study Design & Statistics

Dr Malachy O. Columb FRCA, FFICM

University Hospital of South ManchesterNWRAG Workshop, Bolton, May 2015

COIs: Interesting Confllicts!

Editorial Board Roles: European Journal of Anaesthesiology British Journal of Anaesthesia International Journal of Obstetric

Anesthesia

Manuscript Types

(7) Meta-analysis & systematic reviews (6) Original research – PDBRCT (5) Original research – other RCT (4) Original research – observational (3) Original research – retrospective (2) Narrative reviews – including editorials (1) Case reports, abstracts & letters

Manuscript Types

(7) Original research – PDBRCT (6) Original research – other RCT (5) Original research – observational (4) Original research – retrospective (3) Meta-analysis & systematic reviews (2) Narrative reviews – including

editorials (1) Case reports, abstracts & letters

Statistics: Definition

…the discipline concerned with the treatment of numerical data derived from groups of individuals…

Data

…are always plural…

‘Datum’ is the singular…

Types of Data

Numerical – continuous & discrete Categorical – binary, nominal, ordinal

Hypotheses Null hypothesis (HO) Alternative hypothesis (HA) P value and 95% confidence interval Two-sided by convention One-sided are rarely appropriate Equivalence, Non-inferiority, Superiority

(Margins) Inequality is the usual HA

Potencies and probabilities: One-sided P values suggest a one-sided story! Columb MO, Polley LS. Anesthesia & Analgesia 2001;92:278-9

Controlling Bias - Design

Prospective > Retrospective Double Blind > Single Blind > Unblinded Randomised Controlled Trial >

Unrandomised PDBRCT > Propensity Score Matching! PROBE (Single Blind)

Sample Size

Power analysis and sample size calculations. Columb MO, Stevens A. Current Anaesthesia & Critical Care 2008; 19: 12-4.

Sample size

Minimum difference that is (clinically) important

Defines primary outcome! Multiple comparisons!

Power analysis and sample size calculations. Columb MO, Stevens A. Current Anaesthesia & Critical Care 2008; 19: 12-4.

Estimate of SD

Published research Pilot data Empirical approach 1/5th – ‘one fifth’ of the range

Power analysis and sample size calculations. Columb MO, Stevens A. Current Anaesthesia & Critical Care 2008; 19: 12-4.

One-Fifth Range

4 SD = 95.4% of values 6 SD = 99.7% of values Take 1/5th range to approximate SD 20% of the range

Power analysis and sample size calculations. Columb MO, Stevens A. Current Anaesthesia & Critical Care 2008; 19: 12-4.

Standardised Difference

Difference / SD

Power analysis and sample size calculations. Columb MO, Stevens A. Current Anaesthesia & Critical Care 2008; 19: 12-4.

Standardised Difference = 1.0Power and sample size estimation Armitage & Berry pp 195-202

Sample size estimate for comparison of 2 equal groups1 minimum difference to be significant1 SD

0.05 type 1 error rate, P value0.8 power

1.959964 Z 2 alpha 2 sided0.841621 Z 2 beta 1 sided

18 = minimum n for each group 20 = nonparametric

Nonparametric Adjustment

Add 16% more subjects per group!

Power analysis and sample size calculations. Columb MO, Stevens A. Current Anaesthesia & Critical Care 2008; 19: 12-4.

Sample Size - Proportions

Power analysis and sample size calculations. Columb MO, Stevens A. Current Anaesthesia & Critical Care 2008; 19: 12-4.

Standardised Difference = 1.0Sample size estimate for comparison of 2 proportions

0.75 proportion 10.25 proportion 20.05 type 1 error rate, P value0.8 power

1.959964 Z 2 alpha 2 sided0.841621 Z 2 beta 1 sided

0.5 pooled proportion

14 = minimum n per group for uncorrected Chi square test18 = Fleiss continuity correction, minimum n per group

Descriptive Statistics

Sample Mean (SD) – 68% of data Median [interquartiles, range] Count/frequency

Inferential Statistics - Precision Population estimates; precision Differences in means, medians,

proportions Mean or mean difference Sampling theory!

Population(variable X)

Distribution of sample means(variable )

Population of means(variable )

µ

µ

Sample 1 Sample jSample 3Sample 2x1,x2.....xn x1,x2.....xn x1,x2.....xn x1,x2.....xn

1 2 3 j

......... .....

Randomization

x

100 random samples of size 4

100 random samples of size 250

100 random samples of size 50

100 random samples of size 20

Inferential Statistics - Precision SD of sampled means is the SE of mean SE mean = SD / n SEM = 68%CI, (precision) SEM x 1.96 = 95%CI (precision) Test statistic = difference / SE difference P value

Significance P value – ‘probability of the observed

difference or greater assuming the null hypothesis’

Type I or alpha error <0.05; false +ve Type II or beta error <0.20; false -ve Multiple comparisons - Bonferroni correction Corrections to 95% CI of difference

Group TestsGroups Parametric Parametric

repeated Nonparametric Nonparametric

repeated Proportions

1 One sample t

One sample t

Wilcoxon signed rank

Wilcoxon signed rank

Binomial

2 Unpaired t Paired t Mann-Whitney U

Wilcoxon matched pairs

Chi square or Fisher’s exact

>2 ANOVA Repeated measures ANOVA

Kruskal-Wallis

Friedman’s Chi square or expanded Fisher

Statistical Analyses

Correlation – Pearson, Spearman, intraclass Regression – linear, logistic, probit, survival Diagnostics – sensitivity, specificity, ROC

curves Reference intervals – normal range Agreement – kappa, Bland-Altman plots

Transformations

Table. Some useful transformations

Types of data Transformations

Positive skew, increasing variance Logarithmic = ln x, log10 x

Negative skew, decreasing variance Square power = x2

Count; Poisson distribution Square root =x

Time; survival data Inverse = 1/x

Proportion; p, Binomial distribution Logit = ln (p/1-p) Arcsin = Arcsinp

Probit = Cumulative Normal Deviate

Time-to-Event: Log Transformation

Analyses for RCT Per-Protocol (PP)

Received allocated treatment and completed protocol Largest estimate of effect size Selection bias for post-treatment withdrawals

Treatment-Received (TR) Received allocated treatment May not have completed the protocol Selection bias for pre-treatment withdrawals

Intention-to-Treat (ITT) All randomised subjects – NO WITHDRAWALS May or may not have received the intervention Underestimates true effect size of treatment Most robust analysis

MOCPASS – columbmo@msn.com

Power and sample size estimation Armitage & Berry pp 195-202

Sample size estimate for comparison of 2 equal groups

20 minimum difference to be significant20 SD

0.05 type 1 error rate, P value0.8 power

1.959964 Z 2 alpha 2 sided0.841621 Z 2 beta 1 sided

18 = minimum n for each group 20 = nonparametric

Power estimation for comparison of 2 equal groups

20 minimum difference to be significant20 SD

0.05 type 1 error rate, P value20 sample size n in each group

1.959964 Z 2 alpha 2 sided1.202314 Z 2 beta 1 sided

0.89 = power of study

Sample size estimate for comparison of 2 proportions

0.25 proportion 10.75 proportion 20.05 type 1 error rate, P value

0.8 power

1.959964 Z 2 alpha 2 sided0.841621 Z 2 beta 1 sided

0.5 pooled proportion

14 = minimum n per group for uncorrected Chi square test18 = Fleiss continuity correction, minimum n per group

Power estimation for comparison of 2 equal groups

0.25 proportion 10.75 proportion 20.05 type 1 error rate, P value

20 n per group

1.959964 Z 2 alpha 2 sided1.388312 Z 2 beta 1 sided1.002815 Z 2 beta, corrected

0.5 pooled proportion

0.917479 = power for uncorrected Chi square test0.84 = power with Fleiss continuity correction