Statistics don’t lie – do people?

Janez StareFaculty of Medicine, Ljubljana

USA Today has come out with a new survey– apparently, three out of four people make up 75% of the population.

David Letterman

On the other hand

It's amazing how authoritative you can soundjust by quoting some statistics ...

And certainly

Without data it is anyone’s opinion ...(In God we trust; all others must bring data.)

2

So – statisticians don’t lie?

3

A researcher viewed 107 publishedstudies comparing a new drug and a traditional therapy and found "studiesof new drugs sponsored by drug companies were more likely to favor those drugs than studies supported by noncommercial entities". In not a single case was a drug or treatment manufactured by the sponsoring company found inferior to another company's product.

4

Cigarette manufacturer Lorillard claimed that "TRIUMPH BEATS MERIT" because "an amazing 60 percent said Triumph tastes as good or better than Merit.“

Actually, 36 percent preferred Triumph, 24 percent said they were equal, and 40 percent preferred Merit.

A typical lie

5

Phases of research

• Planning• Collecting data• Data Analysis (together with description)• Interpretation of results

We can ‘lie’ in every phase!

6

Planning of research and data collection

Example:100 measurements on one sheet of paper100 measurements on another sheet

But – measurements are paired!And the guy doesn’t know how!

When we plan our research, we must know what methods of analysis will be used!

7

Missing data!

Example: duration of labourTwo phasesMeasured variables:Duration of the first phase x1

Dur. of the second phase x2

Total duration x3

We got: !!! 31 xx

8

Some lying graphs

New York Times

‘Figures don’t lie, but liars can figure’

9

Washington Post

10

11

What a fall!!

12

Lower rang is better!!

13

And some desperately bad graphs

?

14

hospital n dead % dead

1 64 3 4,7 2 49 6 12,2 3 67 1 1,5 4 68 1 1,5 5 70 5 7,1 6 45 1 2,2 7 73 7 9,6 8 97 3 3,1 9 125 10 8,0 10 80 2 2,5 11 46 4 8,7

Analysis

Does hospital 2 stand out?

And what if hospitals are compared to some standard (say 5%)?

15

PID-PAB ANALIZA

COOP WONCA vprašalnik:

SKUPNO: sešteti točke iz posameznih vprašanj (minimalno številoje 6, maksimalno pa 30). Primerjati skupini s PAB in brez PAB gledena skupno število točk.

ANALIZA

Analizirati, kako posamezne spremenljivke vplivajo na kvalitetoživljenja (COOP WONCA vprašalnik), tako na posamezne vidikekvalitete življenja kot na skupno oceno (seštevek točk).Analizirati ločeno za bolnike s PAB in ločeno za paciente brezPAB, ter za celo skupino pacientov skupaj. Analizirati vsaj:starost, spol, BMI, pas, sistolični in diastolični tlak,hemoglobin, s-glukoza, s-K, urea, kreatinin, CRP, celokupniholesterol, HDL, LDL, trigliceridi, u-proteini, u-glukoza, SCORE,minimalni GI, znižan GI min, aterosklerotična bolezen, anginapectoris, akutni koronarni sindrom, zožitev karotidne arterije,

16

ishemični napad, možganska kap, intermitentna klavdikacija,klavdikacijska razdalja, ishemija uda, bolezni v družini, sladkornabolezen, hipirlipidemije, arterijska hipertenzija, kajenje,razdražljivost, spanje, alkohol, sadje, zelenjava, zmerno gibanje,intenzivno gibanje, individualno svetovanje, skupinsko svetovanje,antiagregacijska terapija-skupaj, lipolitiki-skupaj, ACE insartani-skupaj, antihipertenzivi, diuretiki, številozdravil-skupaj (to naj bo nova spremenljivka)

The guy wanted 1114 tables with corresponding tests!

17

2 4 6 8 10 12 14

24

68

1012

14

Something here

And

som

ethi

ng h

ere

Do the assumptions hold?

18

When we need to know a bit more

Example: Somebody was ‘explaining’ GDP for eleven years with seven variables in a regression equation. He got R2 = 0,95.

Wow! Bravo!But:The expected value of R2 = 0,7 (under the null R2 = 0)!!

19

The famous 5 percent (or 1%)

• Examples of reviews: – Please state that side effects were NOT different (p = 0.058).

– Either something IS significantly different or IT IS NOT.

– Please delete discussion of non-statistically significant results from the text.

• Fisher• How much is 5%?• What is the difference between 5,1% and 4,9%?

20

1990 1995 2000 2005

020

040

060

080

0

year

dead

New law

Interpretation of results

21

Years

Sur

viva

l Pro

babi

lity

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

0.0

0.2

0.4

0.6

0.8

1.0

men

women

Survival after AMI by sex

22

Years

Sur

viva

l Pro

babi

lity

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

0.0

0.2

0.4

0.6

0.8

1.0

men

women

Adjusted to: age=61

Predicted survival by sex after controlling for age

23

Relative survival of men and women

24

There are no routine statistical questions, there are only questionable statistical routines

D.R. Cox

25

Statistics !can’t lie – people candon’t lie – people do.

26