Science = generalizable knowledge

Predictive and reliable information

Collected in some subjects and generalized to others

Sampling problems often exists, but not always

Qualitative research methods

Are only meaningful without sampling problems(constant outcome, deterministic events)

or when sampling problems are irrelevant(one observation is sufficient to reject the hypothesis)

Quantitative methods (statistical)

Are used to quantify sampling uncertainty

Sampling uncertainty is caused by variability

Statistics is primarily about variability

The confusing EQ-5D index

A study of variability

9/24/11

Swedish Knee Arthoplasty Register

The distribution is important when discussing improvement

9/24/11

The Poisson distribution

Siméon Poisson (1781–1840)

Defined by: λ

9/24/11

The Gaussian distribution

Abraham de Moivre (1667–1754)

Defined by: µ and σ (the latter usually assumed constant)

9/24/11

Xie F Li S-C, Luo N, LO N-N,Yeo S-J,Yang KY, Fong KY, Thumboo J. Comparison of the EuroQol and Short Form 6D in Singapore Multiethnic Asian Knee Osteoarthritis Patients Scheduled for Total Knee Replacement. Arthritis & Rheumatism (Arthritis Care & Research) 2007;57:1043–1049

Empirical EQ-5D distribution

9/24/11

A Gaussian mixture distribution?

Defined by: µ1,µ

2,σ

1,σ

2 and w

9/24/11

Mean = 0.58SD = 0.21

Mean = 0.58SD = 0.21

70%13%

87%

30%

!

Hospital A Hospital B

9/24/11

Studying change in EQ-5D

It has been suggested that pairwise differences between pre- and postoperative EQ-5D values are normally distributed and can be meaningfully interpreted.

9/24/11

Studying change in EQ-5D

It has been suggested that pairwise differences between pre- and postoperative EQ-5D values are normally distributed and can be meaningfully interpreted.

It can easily be shown that this is not correct

The sum of two bimodal distribution has a distribution with three modes, the difference four.

9/24/11

Preop EQ-5D Postop EQ-5D

Delta EQ-5D

Empirical EQ-5D data from knee patients in Trelleborg 2007-2008

Delta EQ-5D

9/24/11

9/24/11

9/24/11

9/24/11

9/24/11

Additional problem with analyses of change

Change is confounded by association with baseline

X = pre-operative (baseline) value Y = postoperative (follow up) value

Y-X correlates with X

Solution

When analyzing change, adjust for imbalance at baseline

(This is an almost perfect case-mix adjustment!)

9/24/11

2. Gävleborg2. Kronoberg

1. Stockholm

3. Östergötland

9/24/11

1. Stockholm

2. Kronoberg

2. Gävleborg

3. Östergötland

9/24/11

9/24/11

EQ-5D ProblemsConventional analyses

Mean values not interpretable

Confidence intervals not reliable (Calculated assuming Gaussian distribution)

P-values not reliable (Student's t-test, ANOVA, etc. requires Gaussian distribution and homogeneous variance)

9/24/11

EQ-5D Problems cont'd

Non-parametric analysis

Median value may not exist.

Confidence intervals not reliable (calculated assuming Gaussian or binomial distribution).

P-values not reliable (Wilcoxon's MPSR-test requires a symmetrical distribution, Mann-Whitney U-test requires distributions with identical shape.

9/24/11

EQ-5D Problems cont'd

Adjusting for baseline

How meaningful is the outcome of an ANCOVA with variables having non-Gaussian, multimodal distributions (with different number of modes)? What do these residuals look like?

9/24/11

EQ-5D Problems cont'd

Alternative analyses methods?

- Mixture distribution analysis (mixdist library for R)

- Multi-state Markov analysis (msm library for R)

9/24/11

9/24/11

9/24/11

9/24/11

9/24/11

“This is about clinical improvement, not science”

9/24/11

Swedish law defines clinical improvement work (CIW) as “not research”

Some CIW projects include experiments on patients

- No ethics approval is required (or can be applied for)- No informed consent- No scientific planning or evaluation of the experiments- No formal publication of studies and results

9/24/11

Regression analysis

- Adjusting for baseline

- Models only including statistically significant factors

- Stepwise regression methods

9/24/11

What factors should be included in a linear model (ANCOVA)?

Y = b0 + b

1X

1 + b

2X

2 + … + b

nX

n + e

This is a multiple or multivariable analysis but not multivariate.

Xi is a variable (factor or covariate)

bi is the effect on Y of one unit change in X

i

Assume that Y is blood pressure and X1 an indicator of anti-

hypertensive treatment. bi will then estimate the treatment effect in

terms of blood pressure reduction.

9/24/11

Linear models

Answer

It depends on a) the purpose of the study and b) the study design used.

1. Purpose: (black-box) prediction

Any variable can be included as long as it increases the sensitivity and specificity of the prediction, and as long as results (b

i) are not

interpreted in terms of causal effects.

2. Purpose: effect estimation

The variables needed to produce valid (bi and their s.e.) should be

included.

9/24/11

Linear models

1. Common for all designs

Include baseline when analyzing change in a continuous variable.

2. Randomized trial

Include randomization stratification factors (for valid standard errors).

3. Observational study

Include potential confounding factors (for valid regression coefficients).

9/24/11

Linear models

How should confounding factors be included?

1. By the investigator's reasoning.

2. By reviewing other publications on the same endpoint.

3. By performing sensitivity analyses.

4. But not by using hypothesis testing or stepwise regression analysis.

9/24/11

Parsons et al. A systematic survey of the quality of research reporting in general orthopaedic journals. J Bone Joint Surg Br 2011;93-B,1154-9

9/24/11

9/24/11