teaching registrars research methods study design landon myer phd senior lecturer, infectious...
TRANSCRIPT
Teaching Registrars Research Methods
Study design
Landon Myer PhD
Senior Lecturer, Infectious Diseases Epidemiology Unit, School of Public Health, UCT
Orientation to today’s session• So far:
– Introduction– Study protocol– Reviewing the Literature
• Today: → Study design ←
• To come:– Sampling– Measurement– Data analysis– Ethics
All of one interrelated
process
Overview• How to start to select a study design
• Framework for considering different study designs– Key similarities & differences
• Introduction to each major category of study design– Focus on major functional features– Strengths & limitations– Examples
• ExercisesAsk questions throughout!
Note on terminology
• “Outcomes”– Health outcome of interest in the study– Disease, death, side effect, complication
• (stats: “dependent variables”)
• “Exposures”– Measures that may be associated with the
outcome – Possible “risk factors”, “causes”,
“determinants” • (stats: “independent variables”)
I. Framework for thinking about study designs
What are study designs?
• Structured approaches to address specific research questions
• Provide general guidelines for thinking about specific aspects of study conduct:– sampling populations – systematically collecting measurements– analysing data
• Strengths & limitations of specific designs are well-established
How to select a study design
• Start with a good study question – Relevant
• Addresses topic of significance to health of local population / health care services
– Novel • Makes meaningful contribution to existing
knowledge ~ new insights– Feasible
• Not overly ambitious Creativity
Different types of study questions lead to different types of study designs
• Descriptive– What is the prevalence of condition Z in a
specific population?
• Analytic – What are the factors associated condition Z?
Is condition X a risk factor for condition Z?
• Diagnostic– How good is test Q in detecting condition Z?
Selecting the right study design option
– Relevant • Design allows you to answer your research
question
– Novel • Design allows meaningful contribution to existing
knowledge ~ new insights
– Feasible• Design allows study to be done within available
time and funds
– Simple• ALWAYS avoid ALL unnecessary complexities
Types of study designs
• Many types– Most are some variation on general themes
presented here
• All designs based on same basic principles – key differences in how study design samples
participants with respect to• “exposures” (risk factors, patient characteristics) • “outcomes” (diseases, conditions)
Choice of study design closely related to other aspects of protocol
1. Study design choices inform how you will sample a specific study population
• in a way that its understood how the participants in the study relate to the population in general
2. Study design choices inform the most appropriate measurements to collect on participants in a standardised manner (create data)
3. Study designs will point to the most appropriate analysis of data to answer study question:
– Descriptive• Calculate the proportion of the study population with
condition Z (incident or prevalent)
– Analytic• Compare the frequency of condition Z among groups
of the population
– Diagnostic• Calculate the validity (sensitivity/specificity) or
reliability of test Q in detecting condition Z
Broad categories of options in study design
• Cross-sectional
• Case report / case series
• Case-control
• Cohort
• Randomised Controlled Trial (RCT)
Broad categories of options in study design
• Cross-sectional
• Case series
• Case-control
• Cohort
• Randomised Controlled Trial
Descriptive
Analytic
Diagnostic?
Broad categories of options in study design
• Cross-sectional
• Case report/series
• Case-control
• Cohort
• RCT
Observational designs:investigator is only observing distribution of variables (risk factors, diseases, etc) ‘in nature’
Experimental designs:investigator assigns study conditions ~ usually testing an intervention (many variations here)
Key differences between study designs
1. How participants are sampled– Are participants sampled according to exposure
status, disease status, neither, both?
2. When measurements are taken– Are some variables measured before others, or
are measurements taken all at once?
3. How outcome variables are measured– Incident or prevalent outcomes (morb/mort)?
4. Are there comparison groups involved?
5. Is design observational or experimental?
Time marches on
• Onset of conditions takes place over time in populations
• Different study designs deal with the onset of conditions through time in different ways
• Critical to understand how your choice of study design handles the timing of – Identification of participants– Measurement of variables (exposure,
disease)
1
10
9
8
7
6
5
3
X= onset ofcondition of interest
11
2
4
X
X
X
X
X
X
Died
Died
Died
Died
Time
Died
II. Details on categories of study designs
a. Case report & case seriesb. Case-controlc. Cross-sectionald. Cohorte. RCT & other experimental
designs
Case-report & case-series
• Cases– people with health outcome– depends on what is of interest
• Case report / series– Describes
• characteristics of disease / condition• characteristics of individual that may be associated
with the condition
Issues in case-only designs
• Useful for descriptive purposes only
• Implicit comparisions to what is ‘expected’ or ‘normal’
• Why might this be problematic?
vs
Case-control studies
• Set of cases (usually from health service)
• Comparable set of controls (various
sources)
• Both groups evaluated on characteristics /
‘exposures’ of interest
– Compare distribution of ‘exposure’ in cases
and controls
Cases
Controls
Exposed
Unexposed
Exposed
Unexposed
Cases
Controls
Exposed
Unexposed
Exposed
Unexposed
Time
• 50 new cases of severe asthma identified at RXH in 12-month period, all <5 yrs of age
• These cases are compared to 90 children <5 years attending RXH for orthopedic surgery (who do not have asthma)
• Cases and controls are compared on maternal hairspray use since child’s birth
Example: Does children’s inhalation of hairspray facilitates development of asthma?
Children with asthma
Children without asthma
Hairspray
No hairspray
Hairspray
No hairspray
Odds ratio in 2x2 table
Odds ratio = (A/C) / (B/D)
A
E- DC
BE+
Cases Controls
(DB)
(CA)
Strengths & limitations of case-control study
• Relatively simple & quick approach to address analytic questions
• Ideal to study rare diseases (vs cohort)
• Cases & potential controls are accessible in health care setting
• Choice of the wrong control group ~ selection bias
• Cases may over-report past exposures ~ information bias
Cross-sectional studies
• Most common form of research ~ “surveys”
• Measure all variables on participants at
same point in time (approximately)
• Measure prevalent disease (not incidence)
Time
X Exposure
X Disease
Sampling
Collect data on outcome (disease) and exposure (risk factors)
Exposed
Diseased
Exposed
Not diseased
Not exposed
Diseased
Not exposed
Not Diseased
Defined population
Example: How severe is disease among
rheumatoid arthritis patients attending GSH?
• Study population: patients attending rheumatology clinic at GSH during one month period
• Measures: degree of disease severity (outcome); demographics, disease history, treatment history (exposures)
• Analysis: prevalence of severe disease in clinic population; association between severity of disease and different exposures
Benefits of cross-sectional study
• Feasibility → easy to do– In health care setting, can work from existing
records (consent issues)
– Low cost, rapid • Not waiting for incident outcomes to develop
• Can calculate prevalence– Often most relevant measure for burden of
disease, informing health care strategies– Measure of association: calculate Odds Ratio
for prevalent disease
Issues in cross-sectional studies
• Measuring prevalent disease only– Prevalence incorporates incidence of disease
AND duration of disease– Risk factors for prevalent disease often
different from risk factors for incident disease
• Issues of timing (temporality) are a problem– Exactly when did disease develop? – Did exposures come before or after onset of
disease?
Cohort studies
• Start with group of individuals without the
outcome of interest: “at risk”
• Follow forward in time to observe
incidence of disease (a rate)
• Can be descriptive or analytic
– If analytic question, then measure exposures
on cohort at the beginning of the study
Do not develop outcome of interest
Develop outcome of interest
At risk participants(without outcome)
Time
Cohort studies can be purely descriptive
Eg: What is the rate of remission among men treated for prostate cancer at GSH?
Do not develop outcome of interest
Develop the outcome of interest
No exposure
Exposure
Study population without the outcome of interest
Do not develop the outcome of interest
Develop the outcome of interest
Time
Analytic cohort studyEg: Do β-blockers increase risk of renal transplant rejection?
Types of cohort studies
• Prospective – Following cohort forward through time from
present– Most common approach
• Retrospective– Assemble cohort from medical records,– “follow” based on records– Follow-up is in the past (can extend into
present)
A B
DC
Exposed
Unexposed
New cases of outcome
Participants who do not develop outcome
Total number of exposed = A + B
Total number of unexposed = C + D
Total number of participants = A + B +C + D
Measure of association in a cohort study: relative risk (aka risk ratio, rate ratio)
RR = [A/(A+B)] / [C/(C+D)]
Strengths + problems in cohort studies
• Strengths– Can calcluate rates of new events~ valuable– Timing of exposure before disease assured– Good for studying health effects of rare
exposures (can select an exposed cohort)
• Weaknesses– Participants ‘self-select’ their exposure status~
leads to confounding, bias– Take time, resources (if prospective)– Many subjects needed for rare outcomes
Randomised controlled trials
• Principal experimental design in medical research
• Like a cohort study, except exposure status is assigned by investigator (randomly)– not just observed
• Complex, take time → costly
• RCTs are usually best design for testing the impact of a specific intervention in improving a specific health outcome
Do not develop outcome of interest
Have outcome of interest
No exposure
Exposure
Randomisation
Study population without the outcome of interest
Do not develop outcome of interest
Have outcome of interest
Time
Key features of RCT• Randomisation
• Removes selection bias or confounding• Alternation or other assignment schemes are bad
idea
• Use of concurrent control groups• Vs Before/After studies
• Blinding whenever possible • Blind investigators: prevents information biases• Blind participants: prevents selective behaviour
change during the trial• Not just in trials
• RCTs are important tools
• But can encounter major problems that hinder interpretation of results
– Generalizability: Trial participants are highly selected individuals
• often not representative of general population at risk
– Complexity: Trials procedures can be complex (and costly)
• when key design features breakdown, the experiment is compromised
Other experimental designs• For an experiment, need to compare 2
states: with intervention vs without
• Before/after studies
Median survival of cryptococcal meningitis in HIV+ before
Median survival of cryptococcal meningitis in HIV+ after
Introduction of Pfizerfluconazole donation programme at GFJ
• Controlled before/after studies
Rate of postoperative sepsis
Rate of postoperative sepsis
Rate of postoperative sepsis
Rate of postoperative sepsis
MMHintervention
NSHcontrol
vs
New training in sterile procedures
No new training
BEFORE AFTER
0
5 0 0 0
1 0 0 0 0
1 5 0 0 0
2 0 0 0 0
2 5 0 0 0
1 9 8 0 1 9 8 4 1 9 9 0 1 9 9 5 2 0 0 00
5 0
1 0 0
1 5 0
2 0 0
2 5 0
3 0 0
• Time-series studies
# of pap smears performed in Western Cape
Rate of advanced cervical cancer cases per 100,000
III. Conclusion
The “hierarchy” of study designs
• Frequently see framework for comparing evidence based on the study design used
RCT / experiments
Cohort
Case-control
Cross-sectional
Case series/report
‘better evidence’~ more valid
‘worse evidence’~ less valid
Not (nearly) so simple
• The study design alone does not make the evidence from a study better or worse
• The details of how a study is conducted is what matters– Rigour in design, sampling, measurements,
analysis
• This is why the Methods section is the most important part of scientific papers
Wrap-up
• Framework for thinking about study designs when developing research ideas for MMed – Start with a good research question– Understand different study design options– Select the most feasible study design based on
the study question • Balance time, funding, available data sources
– Understand the strengths and limitations of your approach
– Be able to justify your choice of study designs
Resources to learn more
• Consultations re: study design, conduct, analysis – Ask for help before you start collecting data!!– Email Dr Jim teWaterNaude (to ID appropriate
support within School of Public Health)
• Self-learning– Hulley SB, Cummings SR. Designing Clinical Research– Gordis L. Epidemiology– Szklo M, Nieto J. Epidemiology: Beyond the Basics– Friedman LM, Fundamentals of Clinical Trials
IV. Examples
• An investigator is interested in studying the association between schizophrenia and measles vaccinations.– Hypothesis: childhood vaccinations
predispose individuals to develop schizophrenia in later life
– What study designs are possible?– Which design would you recommend and
why?
#1
• A study among outpatients attending the GSH diabetes clinic during 2004 collects data on 1432 patients.
• Each patient is included once only in the dataset (ie, info from their first visit during 2004)
• Data are collected on patient & family history, past treatment, knowledge of disease & its management, disease severity (GTT)
• Medicine Registrar decides to examine whether patients with more severe disease have better knowledge of disease & management
• What kind of study of this? What measures can be calculated?
#2
• You have collected records from your weekly clinic with information on 152 patients– you have data on:
– patient demographic characteristics– detailed clinical information on morbidity– medical history– risk behaviours (smoking, drinking) – medications
– You need to write an MMed. Quickly.– Identify a research question, a study design,
and describe these briefly. • What are the strengths & limitations of the study
design you have selected in answering your specific question?
#3
• A study of neural tube defects and antenatal folate supplementation, lasting 10 years, follows 10,000 pregnancies in which women used folate supplements, and 10,000 pregnancies in which no supplements were used.
– Among women taking folate supplements, 50 cases of neural tube defects were observed
– Among women not taking supplements, 150 cases of neural tube defects were observed.
#4
– What type of study is this?
– What is the appropriate measure of
association?
– Draw up a 2x2 table, calculate the measure
and interpret in one sentence
#4
#5• Another study of folate supplementation and
neural tube defects uses a hospital referral system to identify all cases of neural tube defects in the local population.
– 200 cases are identified over 10 years (50 among women using supplements, 150 among women not using supplements).
– For comparison, investigators select 800 control pregnancies (where no neural tube defects were observed) at random from the same population (of whom 498 use folate supplements and 492 are unexposed).
• What type of study is this?
• What is the appropriate measure of association?
• Draw up a 2x2 table, calculate the measure and interpret in one sentence.
• Comparing #4 and #5, what is the principle advantage of the case-control design to cohort design?
#5