2/9/8

Getting the most out of a biostatistics consultation

John Pearson

University of Otago, Christchurch

2

Themes

• Who we are

• What we do

• How we work

3

Full time consultantsAssoc Prof Elisabeth WellsDepartment of Public Health and General PracticeEmail elisabeth.wells@otago.ac.nz

Research InterestsPsychiatric epidemiology Survey methods

Dr John PearsonDepartment of PathologyEmail john.pearson@otago.ac.nz

Research InterestsMicroarrays Genetics Bioinformatics Survey Methods Statistical Computing

4

How to find us

• http://www.chmeds.ac.nz/departments/pubhealth/biostat.htm

• Stephen Sharp (admin, ground floor) can make bookings – stephen.sharp@otago.ac.nz – Tel:  378 6026

• Supervisors should attend the first meeting

5

Two-way communication issues in consultingClient:

Be patient with a biostatistician who doesn’t know your research area and is struggling to understand enough to advise well

Biostatistician:

Be respectful and try to communicate with non-statisticians, taking account of the amount of statistics they know and trying to translate, if required

6

Biostatistical consulting

UOC biostatistics webpagehttp://www.uoc.otago.ac.nz/departments/pubhealth/biostat.htm

says:

“Biostatistical consultancy involves advice on study design, methodology, computer software, data analysis and preparation, and revision of articles and reports. The biostatisticians can also act as collaborators during research analysis or on long term studies, and provide training in statistical methods.”

7

Biostatistics webpage (ctd)

“There is no charge for consultation services provided by biostatistical consultancy staff for students and members of staff of the School and the Canterbury District Health Board. However biostatistical work should be costed as part of research grant applications.”

WE ARE FREE BUT IN SHORT SUPPLY

8

Process

• Think first (preferably think lots)

• Read the literature

• Consult a biostatistician, well in advance of an ethics or grant application

In this sequence each step may result in your going back to an earlier step and revising

9

Reasons for revising

Examples:

• Literature may show your project has been done or that you need to measure other variables

• The biostatistician’s questions may require you to re-read articles or search again

• There may be ways of analysing the data which you had not thought of so you may revise your aims and objectives

10

Looping back and revising

• Looping back and revising is part of the usual process of setting up a research project

• The more you can think through things in advance the better

• Nonetheless revision is often required as you think further, and in more detail, and is nothing to be ashamed of

11

What to bring to a biostatistician

• Aims/objectives/hypotheses• Suggested study design• Expected size of effect based on the

literature/clinical experience or

the desired precision of some estimate• Any key papers • Draft data collection instrument (if

available)

12

Study design• Observational studies:

- cross-sectional studies- cohort studies- case-control studies

• Intervention/experimental studies:- for human trials usually only one or two interventions are used- for laboratory studies there will usually be a dose variable with several levels and often other experimental conditions as well

13

Study design

• Is a case-control study unmatched or matched (individually or frequency matched)

• In a treatment trial are there parallel groups or a cross-over design

• In follow-up of patients is it better to study more patients or fewer more often?

Within the main study designs there will be options:

14

What sample size do I need

• This is sometimes asked with no context

• That is like asking a travel agent ‘What does it cost to travel?’

• The answer depends on what you want to do.

15

Effect size or single estimate

• Estimating the size of an effect (eg the difference between Tx A and Tx B) and testing the hypothesis that there is an effect

• Estimating a single quantity (eg the proportion of the population living below the poverty line) – no hypothesis here

When planning a study you need to decide if you are:

16

Effect sizes expected

• Use the literature as a guide to the size of effect you might find (similar studies, similar treatment for different problem, size of effect that would change clinical practice or policy)

• Consider differences in proportions, odds ratios, or differences in means (try to find out the standard deviations if possible)

17

Effect size and sample size

• For comparisons, expected effect sizes have to be thought through before a statistician can come up with a sample size

• Various scenarios can be looked at before deciding one is best

• If sample size is fixed, then it is possible to see what size of effect could be detected

18

Power and sample size

• Ethics applications mention the power of the study. This is the probability of obtaining a statistically significant result in your study if the world is as you suppose it (ie there is an effect of the size you propose).

• Power is often set at 80% (odds of 4:1) but sometimes at 90% or 95% for definitive studies.

19

Power (ctd)

• If the power is less than 80% then there is not much point in doing the study

• Biostatisticians don’t like multiple small studies which are too small to detect the likely effects of different treatments

20

Hypothesis testing vs confidence intervals• In the mid 1980s biostatisticians tried to

move health research away from significance testing (results reported as significant or not) to confidence intervals around an estimate

• The estimate shows the most likely effect and the confidence interval shows the interval within which the true value is likely to fall

21

Precision of single estimates• Sometimes there is no ‘size of effect’ but just the

precision of a single estimate.• This is often how sample sizes for national

surveys are set but it also can apply to small surveys

• You may want to know if only a small, moderate or high proportion of patients had a 12 month follow-up outpatient visit

• However if the survey result is 40% you want to know the precision (“margin of error”); 10%-70% is a different estimate from 35%-45%

22

Planning for precision

• If 4 out of 20 children have asthma then the prevalence (P) is 20%, with a 95% confidence interval (CI) of (8%, 42%)

• For 20/100, P=20%, 95%CI=13%, 29%• For 40/200, P=20%, 95%CI=15%, 26%

Planning for precision is based on the size of the confidence interval expected:

Agresti, A., and Coull, B. Approximate is better than 'exact' for interval estimation of binomial proportions. The American Statistician 52: 119-126, 1998

23

Planning for precision

Planning for precision is based on the size of the confidence interval expected:

CI for P=0.2

0.0

0.2

0.4

0.6

0 50 100 150 200

Sample size

Pro

port

ion

24

Planning for precision

• N=100 P=50% 95%CI=40%, 60%

• N=100 P=5% 95%CI=2%, 11%

Precision depends on P so it is necessary to look at plausible values for P

What precision do you want?What precision can you afford?

Precision vs proportion

50

100

200

0

0.05

0.1

0.15

0.2

0 0.2 0.4 0.6

Proportion

CI w

idth

25

Diminishing returns

• Precision and power depend on the square root of N, not N

• To 1/2 a confidence interval you need to 4x the sample size, (not 2x it)

• Costs increase linearly with sample size

• There are diminishing returns from larger and larger samples

26

Data collection

• This should be planned well in advance of starting collection

• A database or EXCEL spreadsheet should be set up

• Procedures for checking data are required

Biostatisticians can advise on data collection:

27

Questionnaires

• Most biostatisticians have experience in questionnaire design

• Biostatisticians will think about data entry issues, coding and analysis of the data, not just the questions themselves

• Questionnaires may look easy but there are many traps

28

Computing

• Almost no research is done without computers now

• Biostatisticians can advise on what software packages to use (but we use only some of them so if you choose another then the support will be limited.)

29

Analysis

• How much you know

• Whether or not you are a student (students have to learn to do their own analyses)

• How complicated the analyses are

Whether you carry out your own analyses or whether a biostatistician does them depends on:

30

Writing up the results

Biostatisticians can help with this:

• For papers or reports biostatisticians often write up the statistical methods and may write some or much of the results

• Students have to write their own results but will receive guidance from their supervisor(s) and may check with a biostatistician

31

Responding to referee criticism

• Rebutting criticism or making design changes suggested by grant reviewers

• Rebutting criticism or carrying out additional analyses from journal or report reviewers

The point of referees is to find weaknesses in studies (as well as to praise good points). Depending on the issues raised biostatisticians may be involved in:

32

Summary

• Biostatisticians can be involved with all stages of a project from design to publication

• Sometimes they are required only for technical advice at some part (but fixing problems which occurred because advice was not sought earlier is not liked)

• Sometimes they are part of the team

33

Take home messages

• Check statistics early

• Statisticians are mere mortals