malmo 30 03-2012

44
The unity of all science consists alone in the method, not its material. Pearson K. The grammar of science. London, Black, 1892.

Upload: jonas-ranstam

Post on 29-May-2015

179 views

Category:

Documents


2 download

TRANSCRIPT

Page 1: Malmo 30 03-2012

The unity of all science consists alone in the method, not its material.

Pearson K. The grammar of science. London, Black, 1892.

Page 2: Malmo 30 03-2012

Statistics is the study of uncertainty.

Savage LJ. The foundations of statistics. New York, Wiley, 1954.

Page 3: Malmo 30 03-2012

The aim of statistics reviewing

Accurate and transparent description of the uncertainty in presented findings.

“Statisticians are experts in handling uncertainty”.

Lindley DV. The philosophy of statistics. The Statistician 2000;49:293-337.

Page 4: Malmo 30 03-2012

Experi-mental

Study Study design design

Obser-vational

Few ethics concernsBias adjustmentsExternal validityLong follow up

Few sample size restrictions

Ethics concernsRandomization

Controlled conditionsInternal validityShort follow up

Sample size restrictions

Medical research methodology

Page 5: Malmo 30 03-2012

Experi-mental

Study Study design design

Obser-vational

Internal validity by design (blocking of known risk factors and randomization of

other)

Potential for confounding: none

Internal validity by statistical analysis (confounding adjustment for known and

measured risk factors)

Potential for confounding: massive

Statistical aspects – internal validity

Page 6: Malmo 30 03-2012

Confounder (or case-mix) adjustmentHow much of the variation in endpoints can be explained by known factors, and how much has unknown causes?

Variation with unknown origin

95%-99% Arthroplasty revision

85%-95% EQ-5D, SF36

70%-80% Coronary heart disease

Page 7: Malmo 30 03-2012

Risk factors, confounding, and the illusion of statistical control'...it is essential to remember that “statistical control” is nothing more than a highly fallible process filled with judgment calls that often go unnoticed in practice.'

Christenfeld NJS, Sloan RP, Carrol D, Greenland S.Psychosomatic Medicine 2004;66:868–875

Page 8: Malmo 30 03-2012

Simple modelMultiple model

(or multivariable, but not multivariate)

Linear regression analysis

Page 9: Malmo 30 03-2012

Stepwise regression

Page 10: Malmo 30 03-2012
Page 11: Malmo 30 03-2012
Page 12: Malmo 30 03-2012

Statistics

We calculated odds ratios by logistic regression analysis, to estimate the relationship between failure of the osteotomy and possible preoperative risk factors. We performed multivariate, stepwise (backward) logistic regression and entered variables with a p-value of ≤ 0.05 into the model.

Page 13: Malmo 30 03-2012

Unified theory of bias Bias can be reduced to or explained by 3 structures

1. Reverse causation

Outcome precedes exposure measurement or outcome can have effect on exposure. Measurement error or Information bias.

2. Common cause

Confounding by association, confounding by indication.

3. Conditioning on common effects

Collider, selection bias, time varying confounding.

Page 14: Malmo 30 03-2012

Covariate selectionAdequate Background Knowledge

Confounder identification must be based on understanding of the causal structure linking the variables being studied (treatment and disease).

Condition on the minimal set of variables necessary to remove confounding.

Inadequate Background Knowledge

Remove known instrumental variables, colliders, intermediates (variables with post treatment measurement.

Page 15: Malmo 30 03-2012

ConfoundingUnder-adjustment

occurs when a confounder is not adjusted for.

Over-adjustment

can occur from adjusting instrumental variables, intermediate variables, colliders, variables caused by outcome.

Page 16: Malmo 30 03-2012

Confounder

Common cause, i.e., confounder

Confounder L distort the effect of treatment A on disease Y

Always adjust for confounders, unless small data set and confounder has strong association with treatment and week association with outcome

Page 17: Malmo 30 03-2012

Confounder example

A = treatment1: statin alone0: niacin alone

L = Baseline Cholesterol1: LDL ≥ 160 mg/dL0: LDL < 160 mg/dL

Y = Myocardial infarction1: Yes0: No

Page 18: Malmo 30 03-2012

Intermediate variableAdjusting for intermediate variable I in a fixed covariate model will remove the effect of treatment A on disease/outcome Y

In a fixed covariate model we do not want to include variables influenced by A or Y

Page 19: Malmo 30 03-2012

Intermediate exampleA = treatment1: statin alone0: niacin alone

I = Post-treatment Cholesterol1: LDL ≥ 160 mg/dL0: LDL < 160 mg/dL

Y = Myocardial infarction1: Yes0: No

Page 20: Malmo 30 03-2012

ColliderAdjusting for a collider can produce bias

Conditioning on common effect F without adjustment of U1 or U2 will induce an association between U1 and U2, which will confound the association between A and Y

Page 21: Malmo 30 03-2012

Collider example

Page 22: Malmo 30 03-2012

Variables associated with treatment or disease onlyInclusion of variables associated with treatment only can cause bias and imprecision

Variables associated with disease but not treatment (risk factors) can be included in models. They are expected to decrease variance of treatment effect without increasing bias

Including variables associated with disease reduces the chance of missing important confounders

Page 23: Malmo 30 03-2012

Reality is complicated

Page 24: Malmo 30 03-2012

http://www.dagitty.net

Page 25: Malmo 30 03-2012

http://www.dagitty.net

Page 26: Malmo 30 03-2012
Page 27: Malmo 30 03-2012

Any claim coming from an observational study is most likely to be wrong

12 randomised trials have tested 52 observational claims (about the effects of vitamine B6, B12, C, D, E, beta carotene, hormone replace- ment therapy, folic acid and selenium).

“They all confirmed no claims in the direction of the observational claim. We repeat that figure: 0 out of 52. To put it in another way, 100% of the observational claims failed to replicate. In fact, five claims (9.6%) are statistically significant in the opposite direction to the observational claim.”

Young S, Karr A. Deming, data and observational studies. Significance, September 2011.

Page 28: Malmo 30 03-2012
Page 29: Malmo 30 03-2012

Aetiology Study scope Study scope Treatment

Pre-specified hypotheses

Confirmation

Legislation, regulatory guidelines

Uncertainty intolerance

Hypothesis generation

Exploration

Academic analysis freedom

Uncertainty tolerance

Medical research methodology

Page 30: Malmo 30 03-2012

Aetiology Study scope Study scope Treatment

Randomized clinical trials

Patient registerstudies

Epidemiologicalstudies

Laboratory experiments

Medical research methodology

Experi-mental

Study Study design design

Obser-vational

Page 31: Malmo 30 03-2012

Aetiology Study scope Study scope Treatment

Protected type-1 error rate for specified endpoints

Sample size based on the type-2 error rate

Specified type-1 and -2 error uncertainty

(confidence intervals)

No multiplicity consideration for safety endpoints

Multiplicity issuesnot addressed

Sample size not based on type-2 error rate

Bonferroni correction within endpoints

Few type-2 error considerations

Statistical aspects - precision

Experi-mental

Study Study design design

Obser-vational

Page 32: Malmo 30 03-2012

Aetiology Study scope Study scope Treatment

Experi-mental

Study Study design design

Obser-vational

Drug development

Phase 1Discovery(Phase 0)

Phase 2Phase 3

Phase 4

PMS(Phase 5)

Page 33: Malmo 30 03-2012

Aetiology Study orientation Study orientation Treatment

Device development

Randomizedperformance

trials

Safetyfollow-up

in registries

Biomechanicsin vitro, etc.

Experi-mental

Study Study design design

Obser-vational

Page 34: Malmo 30 03-2012

It is impossible to do clinical research so badly that it cannot be published

“There seems to be no study too fragmented, no hypothesis too trivial, no literature citation too biased or too egotistical, no design too warped, no methodology too bungled, no presentation of results too inaccurate, no argument too circular, no conclusions too trifling or too unjustified, and no grammar and syntax too offensive for a paper to end up in print.”

Drummond Rennie 1986 (editor of NEJM and JAMA)

Page 35: Malmo 30 03-2012
Page 36: Malmo 30 03-2012
Page 37: Malmo 30 03-2012
Page 38: Malmo 30 03-2012
Page 39: Malmo 30 03-2012
Page 40: Malmo 30 03-2012

Arthroplasty registry analyses

Crucial issues

- Fulfillment of methodological assumptions (Gaussian distr, homogeneity of variance, proportionality, linearity, etc.)- Confounding adjustment (risk factors, causality, linearity, etc.) - Clinical significance and estimation uncertainty (95%CI).

Should be avoided

- P-value culture- Bonferroni correction- Post-hoc power- Predictions

Page 41: Malmo 30 03-2012
Page 42: Malmo 30 03-2012

Thank you for your attention!

Page 43: Malmo 30 03-2012
Page 44: Malmo 30 03-2012

Indicators for statistical review- Randomized trials

- Patient registry (safety) studies

- Analyses of knees, hips, elbows... (bilateral observations)

- Pseudo-replicates (esp. in laboratory experiments)

- “No difference” manuscripts

- Stepwise regression

- ???