designing clinical research global health clinical scholars program, 2006 george w. rutherford, m.d....

Designing Clinical Research

Global Health Clinical Scholars Program, 2006

George W. Rutherford, M.D.Institute for Global Health

Lecture topics

Wednesday 6 September 20061. The research question2. Study populations and sampling3. Variables and measurements4. Study design I: cross-sectional studies,

case-control studies5. Study design II: cohort studies and

experiments

Lecture topics

Wednesday 13 September 20066. Statistical inference

7. Sample size calculations

8. Ethical conduct of research

9. Questionnaire design, pre-testing and quality control

10. Data management and cleaning

The research question

Lecture 1

Designing Clinical ResearchGlobal Health Clinical Scholars Program

Sections of the study protocol

• Research question• Significance (background)• Study design• Study population and sampling• Variables and measurements• Statistical issues• Ethical issues• Quality control and data management

The research question

• All studies should start with a research question that addresses what the investigator would like to know

• Goal is to find an important research question that can be developed into a feasible and valid study plan

The research question

• Usual format (analytic studies):

“In a population of [study population], is [predictor variable] associated with [outcome variable]?”

Examples of research questions

• In a population of injection drug users with HIV infection, is HAART use associated with slower progression of chronic hepatitis B?

• In a population of children with falciparum malaria, is HIV associated with higher mortality?

• In a population of Croatian merchant seaman, is unprotected sex with sex workers in African ports associated with HIV infection?

The research question

• Format for descriptive study “In a population of [study population], what

is the prevalence (or mean, median, etc.) of [outcome variable]?”

• Most studies will have more than one research question

• The research protocol flows from the research question

What is the research question in this abstract?

Acute respiratory tract infections caused by Streptococcus pneumoniae are a leading cause of morbidity and mortality in young children. We evaluated the efficacy of a 9-valent pneumococcal conjugate vaccine in a randomized, double-blind study in Soweto, South Africa. At 6, 10 and 14 weeks of age, 19,922 children received the 9-valent pneumococcal polysaccharide vaccine conjugated to a noncatalytic cross-reacting mutant of diphtheria toxin (CRM197), and 19,914 received placebo…

Klugman KP, Madhi SA, Huebner RE, et al. A trial of 9-valent pneumococcal conjugate vaccine in children with and without HIV infection.

N Engl J Med 2003; 349:1341-8.

In a population of South African infants is immunization with a 9-valent pneumococcal conjugate vaccine associated with invasive pneumococcal disease?

In a population of [study population] is [predictor variable] associated with [outcome variable]?

Origins of a research question

• Mastering the literature Scholarship -- be a scholar and know the

literature Get a mentor

• Be alert to new ideas and techniques Importance of teaching

• Be imaginative

FINER characteristics

• Feasible

• Interesting to the investigator

• Novel

• Ethical

• Relevant

FINER characteristicsCriteria for feasibility

• Adequate number of subjects

• Adequate technical expertise

• Affordable in time and money

• Manageable in scope

FINER characteristicsCriteria for novelty

• Confirms or refutes previous findings

• Extends previous findings

• Provides new findings

FINER characteristicsCriteria for relevance

• To scientific knowledge

• To clinic, public health or health policy

• To future research directions

Study populations and sampling

Lecture 2

Designing Clinical ResearchGlobal Health Clinical Scholars Program

Sections of the study protocol

• Research question• Significance (background)• Study design• Study population and sampling• Variables and measurements• Statistical issues• Ethical issues• Quality control and data management

Basic terms and concepts

• A population is a complete set of people or objects (e.g., clinics, neighborhoods) with a specified set of characteristics Clinical Demographic Geographic Period of time

• A sample is a subset of the population

Target populations and study samples

• The target population is the group of all people in the world to which results will apply (e.g., inpatients with malaria)

• The accessible population is the subset of the target population that is potentially available to the investigator for study (e.g., pediatric inpatients with malaria at Mulago Hospital)

• The study population or study sample is the subset of the accessible population that is asked to participate in the study (e.g., a random 20% sample of all pediatric inpatients with malaria at Mulago Hospital)

Target populations and study samples

• In the research question, the population refers to the target population Defined by clinical, demographic and

geographic characteristics

• The research plan specifies the accessible population and how to sample it to get the study population

Target populations and study samples

• Findings in the study sample are inferred to be present in the accessible population and, by extension, in the target population

• Study findings are generalized from the study sample to the accessible population to the target population

Selecting a study sample from the accessible population

• Rather than selecting participants from the entire accessible population, we often define a subset of participants for the study using inclusion and exclusion criteria and then choose participants

Selecting a study sample from the accessible population

• Inclusion criteria are the main characteristics of the target population - demographic, clinical, geographic and temporal

• Exclusion criteria are the subset of individuals in the accessible population that should not be included (other causes for outcomes, poor data quality) The fewer the exclusion criteria, the more

generalizable the results

Inclusion criteria specify characteristics relevant to the research question and efficiency

• Demographic characteristics

• Clinical characteristics

• Geographic characteristics

• Temporal characteristics

• Male injection drug users 20-40 years old

• HIV infection

• Attending investigator’s clinic

• Seen between January 1 and December 31, 2004

Exclusion criteria specify subsets not studied ( limits generalizability)

• A high likelihood of being lost to follow up

• An inability to provide good data

• Being at high risk for side effects

• Characteristics that make it unethical to withhold study treatment

• Not a permanent resident of Zagreb

• History of encephalopathy or meningitis

• ≥2 failed HAART regimens• Applies primarily to

experiments

Relationship between target population study population and actual subjects recruited

Study sample or study population, further defined by inclusion and exclusion criteria

Accessible population

Optimizing subject selection: a delicate balancing act

Feasibility

Accessibility

Cost

Time/Efficiency

Generalizability

Accuracy

Diversity

Adequate Size

At the end, will I believe the findings and will others believe me?

Validity of study population

• External validity? How different are those invited to participate from

those eligible to participate in the accessible population but not invited, that is, not selected for the sample (demographics, illness status, geography, risk profile)?

• Internal validity? How different are those who choose to participate

in comparison to those who are invited but decline?

What are the target, accessible and study populations in this abstract?Acute respiratory tract infections caused by Streptococcus pneumoniae are a leading cause of morbidity and mortality in young children. We evaluated the efficacy of a 9-valent pneumococcal conjugate vaccine in a randomized, double-blind study in Soweto, South Africa. At 6, 10 and 14 weeks of age, 19,922 children received the 9-valent pneumococcal polysaccharide vaccine conjugated to a noncatalytic cross-reacting mutant of diphtheria toxin (CRM197), and 19,914 received placebo…

Klugman KP, Madhi SA, Huebner RE, et al. A trial of 9-valent pneumococcal conjugate vaccine in children with and without HIV

infection. N Engl J Med 2003; 349:1341-8.

“The 9-valent pneumococcal conjugate vaccine is currently under development for licensure in both developed and developing countries but has not yet been licensed for use. Our study provides evidence to support the wider development and use of this vaccine to prevent invasive pneumococcal disease, reduce antibiotic resistance among pneumococcal strains, and diminish the incidence of pneumonia in children.”

Subject Recruitment:How to Get the “People?”

• Successful recruitment generally means… response, generalizable sample, adequate size

• For database only studies (Not usually a big problem)

• For hands-on studies (e.g., surveys, cohorts, trials)

Expect that it will be harder than you think! Use reasonable inclusion/exclusion criteria Acceptable burden to subjects with potential

benefits Try to minimize subject non-response (non-

response bias)

Purpose of sampling

• Advantage of sampling (as opposed to including everyone in the target population) Efficiency Allows investigator to draw inferences about a

large population by examining a sample at relatively small cost in time and effort

• Disadvantages of sampling Introduces error If sample is insufficiently representative of target

population, findings may not generalize to target population

Sampling techniques

• Non-probability sampling• Quasi-probability sampling• Probability sampling

Types of non-probability sampling

• Convenience sample Population accessible to investigator No need to randomly select participants Representativeness? Selection bias?

• Consecutive sample All members of target population accessible to investigator

recruited as they present for care, etc. Minimizes selection bias

• Chain-referral or snowball sample• Quasi-probability sampling

Respondent-driven sampling Time-location sampling

Types of probability sampling

• Probability sampling Each unit of population has equal chance

of being selected to participate

• Simple random sample

• Stratified random sample

• Cluster sample

• Systematic sample

Demonstration of sampling

• Simple random sample

• Stratified random sample

• Cluster sample

• Systematic sample

Sampling and recruitment example

• RQ: Among patients on ART is there an association between consuming a Mediterranean diet and HIV-related lipohypertrophy?

• What is the target population? All patients worldwide on ART

• What is the accessible population? HIV-infected patients at UHID on ART for >1 year

Inclusion criteria

• Sampling frame: current attendees at UHID or any attendance ever (including dead patients)?

• Inclusion criteria: all vs. subset? Demography: men only? Geography/administrative: Zagreb or

Rijeka or both Time period: one year, one month

Exclusion criteria

• Previously diagnosed lipohypertrophy or hyperlipidemia

• Altered mental status (inaccurate dietary history)

Remember: exclusion criteria are applied to a subset of individuals in the accessible population that should not be included (other causes for outcomes, poor data quality), and the fewer the exclusion criteria, the more generalizable the results to the accessible and target populations.

Sampling strategies

• You decide you need to include 240 patients and approximately 500 patients are currently under care at UHID that meet the major inclusion criteria (≥1 year of ART, male)

• Name three sampling strategies and their advantages and disadvantages Consecutive sample Random sample Stratified sample (by region of origin)

Variables and measurements

Lecture 3

Designing Clinical ResearchGlobal Health Clinical Scholars Program

Sections of the study protocol

• Research question• Significance (background)• Study design• Study population and sampling (subjects)• Variables and measurements• Statistical issues• Ethical issues• Quality control and data management

“The most elegant design of a clinical study will not overcome the damage caused by unreliable or imprecise measurement.”

J.L. Fleiss (1986)

Fleiss JL. The design and analysis of clinical experiments. New York: John Wiley and Sons, 1986:1-5.

Predictor* Outcome

Confounding variables*

Effect Modifiers*

Types of variables

*Generally categorized as exposures

Classification of variables

Continuous Quantitative intervals

with typical ranking Examples:

• Number of sexual partners/yr

• Number of drinks/day• CD4+ cells• Hours

Categorical Dichotomous (yes/no)

• Death, HIV, condom use at last intercourse

Nominal (no order) • Ethnicity, occupation

Ordinal (ordered rank)• Educational level

Continuous variables are or are not normally distributed

Measurement scales

Type of measurement

Characteristics of variable

Example Descriptive statistics

Information content

Categorical Nominal

Unordered categories

Sex, blood type,

alive/dead

Counts, proportions

Lower

Ordinal Ordered categories with non-quantifiable

intervals

Degree of pain,

ethnicity

Also medians

Intermediate

Continuous or ordered discreet

Ranked spectrum with quantifiable

intervals

Weight, number of

sexual partners

Also means, standard

deviations

Higher

Additional “exposure” considerations

• Dose issues Cumulative exposure (i.e., total dose) Exposure rate (e.g., per unit time)

• Time issues When exposure first started When it ended Exposure distribution (e.g., constant vs.

intermittent)

Typical data sources

• Surveys and questionnaires

• Interviews • Diaries• Direct observation• Environmental

measurements• Laboratory tests

• Databases and registries

• Medical records• Physiologic

measurements• Biomarkers (e.g., DNA)• Imaging tests• Pathology

Measurements

• Describe items of interest in study that can be analyzed statistically

• Often measured using scales

• Goals are: Precision (free of random error) Accuracy (free of systematic error or bias)

General measurement goals

• Precision You get the same result when measured

repeatedly measure a continuous variable but it is much more precise (that is, lower standard deviation) -- within and between subjects and over time

• Accuracy It also represents what the value is really

supposed to be (that is, more accurate) and with higher sensitivity and specificity

Precision

• Reproducibility of measurement• Precision increases power to detect effects• Sources of imprecision

Observer Instrument Subject (biological variability)

• Assessing precision Within instrument or observer Between instrument or observer

Accuracy

• How much does the measured variable reflect what it’s supposed to represent

• Accuracy increases validity• Assessed by comparison with a reference

standard (“gold standard”); validity for abstract variables

• Sources of inaccuracy (systematic bias) Observer Instrument Subject

Precision and accuracy

• •• •

• •• • • •

• •

• •

• •

Precise but not accurate Neither precise nor accurate

Accurate but not precise Both precise and accurate

Qualities of individual measures

• Sensitive

• Specific

• Appropriate

• Objective

• Range of values

Improving precision and accuracy of variables Reducing measurement bias

• Standardize methods• Pretest• Refine/automate

instrument• Train & evaluate staff• Timely editing, coding

& correcting of forms• Multiple

measurements

• Validate against “gold standard”

• Use less obtrusive measures

• Blind staff doing measurements to exposure status

• Institute quality control measures during data collection, processing, and analysis

Study measurements

Example of measurements

• As part of study of the association between a Mediterranean diet and lipohypertrophy, you want to measure triceps skin fold thickness

• Name three ways to improve precision and accuracy

• Standardize method• Train staff• Use multiple measures• Blind staff as to

Mediterranean diet status• Check results each day

for patterns of unusual or missed data

Study design I:Cross-sectional and case-control

studies

Lecture 4

Designing Clinical ResearchGlobal Health Clinical Scholars Program

Sections of the study protocol

• Research question• Significance (background)• Study design• Study population and sampling• Variables and measurements• Statistical issues• Ethical issues• Quality control and data management

The research question

• In analytic studies:

“In a population of [study population], is [predictor variable] associated with [outcome variable]?”

• In descriptive studies:

“In a population of [study population], what is the prevalence of [outcome variable]?”

Predictor variable(independent)

Outcome variable(dependent)

Types of clinical studies

• Studies with no variables Case studies, case series, editorials,

opinions, reviews

• Studies with single variables Descriptive studies and surveys

• Studies with ≥2 variables Experiments Observational studies Meta-analyses and systematic reviews

Hierarchy of clinical study types

Descriptive studies

Experimental studies

Cohort Case-control Cross-sectional

Observational studies

Analytic studies

Descriptive studies

• Report distribution of single variable• Variable can be categorical or

continuous• Categorical = percentage with and

without condition, risk factor, etc• Continuous = central tendencies

(means, medians, modes, standard deviations, standard errors)

Descriptive studies:examples

• Proportion of trauma patients at Mulago Hospital who present in coma

• Proportion of adult outpatients in a general medical clinic in Cape Town with depression

• Median time from road traffic accident to Accident & Casualty Department at Muhimbili Hospital

Analytical studies

• Describe relationship between predictor variable(s) and outcome variable(s)

• Experiments Investigator manipulates predictor variable

• Observational studies Investigator observes effect of predictor

variable on outcome variable

Observational studies

• Cross-sectional studies

• Case-control studies

• Cohort studies

Inferring causality from observational studies

• Predictor variable precedes outcome variable• Strength of association• Relationship observed in studies with different

designs• Strength of association increases as

exposure to predictor variable increases (dose response)

• Biological plausibilityBradford-Hill, 1965

Cross-sectional studies

• Describe the relationship between predictor variable and outcome variable at a single point in time

• Has advantage of being inexpensive and relatively easy and participants are not lost to follow up

• Cannot differentiate cause-effect and effect-cause relationships

• Not suited for rare outcomes (e.g., risk factors for cancers) unless sample size is huge

Cross-sectional studies

Predictor variable

at time of study

Outcome variable

at time of study

Total

Present Absent

Present a b a+b

Absent c d c+d

Total a+c b+d NRisk ratio = a/(a+c)or prevalence ratio b/(b+d)

Case-control studies

• Type of observational study

• Used to study rare outcomes

• Because case-control can establish temporal relationship between predictor and outcome variables, avoid problem of effect-cause relationships

Case-control studies

• Retrospective• Compare subjects identified with the

outcome (cases) at the beginning of the study and subjects know to not have the outcome (controls)

• Determines exposure to the predictor variable in the past

• Examines relationship between predictor (past) and outcome (present) in both groups

THE PAST THE PRESENT

Cases

Controls

Absent

Absent

Present

Present

Subjects with outcome

Subjects without outcome

Predictor variable(risk factor, exposure)

Case-control studies

Limitations

• Cannot determine the incidence or prevalence or the outcome in the population

• Prone to both sampling bias, recall bias and confounding

• Relative risk is approximated by odds ratio

Odds ratio

Predictor variable

Outcome variable

Present Absent

Present a b

Absent c d

OR = a/(a+b)/c/(c+d) = a/b/c/d = ad/bc if a+b ~ b and c+d ~ c (i.e., outcomes rare)

Bias in case-control studies

• Sampling bias Cases and controls are sampled

separately

• Measurement bias Retrospective - recall bias

Controlling confounding by matching

• Matching is a strategy for controlling confounding for specified variables

• Creates pairs, triplets, quadruplets of cases and matched controls

• Typically demographic variables (e.g., sex, age, risk group) are matched

• Beware of overmatching!• Need to use special statistical tests for

analysis (e.g., McNemar’s matched pair analysis)

Case-control studyExample

• Cryptococcal meningitis is an uncommon but serious complication of HIV infection in Thailand (<5% of AIDS patients)

• Azole prophylaxis has been done intermittently

• Investigator wishes to know the effect of azole prophylaxis on the incidence of cryptococcal meningitis

Case-control studiesExample

• Research question:

In a population of Thai AIDS patients is azole (fluconazole or itraconazole) prophylaxis associated with decreased incidence of cryptococcal meningitis?

Case-control study:Example

• Cases are all Thai AIDS patients alive at any time since 1996 (HAART era) who have developed laboratory-diagnosed cryptococcal meningitis

• How are you going to choose controls?• Do you need to match? On what

criteria would you match?

THE PAST THE PRESENT

Cases

Controls

Yes

Yes

No

No

Cryptococcal meningitis

No crytpococcal meningits

Received azole prophylaxis?

Exposure to azoles among cases and controls

02468

10121416

Cases Controls

Azoles

No azoles

Exposure to azoles among cases and controls

Azole prophylaxis

Cryptococcal meningitis

TotalYes (case) No (control)

Yes 5 15 20

No 15 5 20

Total 20 20 40

Interpretation

• Odds ratio = 5*5/15*15 = 25/225 = 0.11

• Patients who received azoles had nine-fold lower odds of developing cryptococcal meningitis

Study design II:Cohort studies and experiments

Lecture 5

Designing Clinical ResearchGlobal Health Clinical Scholars Program

Cohort studies

• A cohort (follow-up, longitudinal) study is a comparative, “observational” study in which subjects are grouped by their exposure status, i.e., whether or not the subject was exposed to a suspected risk factor

• Has both descriptive (incidence of certain outcome) and analytic (analyze associations between predictor variables and outcome variables) components

Cohort studies

• The subjects, exposed and unexposed to the predictor variable (risk factor), are followed forward in time to determine if one or more new outcomes (diseases) occur• Subjects should not have outcome variable

on entry• No new subjects allowed in after initial

recruitment• The rates of disease incidence among the

exposed and unexposed groups are determined and compared.

Elements of a cohort study

• Selection of sample from population

• Measures predictor variables in sample

• Follow population for period of time

• Measure outcome variable

Types of cohort studies

• Prospective Predictor variables are measured before

outcomes occur (e.g., older age and risk of death among HIV-infected patients)

• Retrospective Define sample and collect predictor

variables after outcomes have occurred, using pre-existing data

Strengths of cohort studies

• Know that predictor variable was present before outcome variable occurred

• Directly measure incidence of a disease outcome

• Can study multiple outcomes of a single exposure

Weaknesses of cohort studies

• Expensive and inefficient for studying rare outcomes

• Often need long follow-up period or a very large population

• Loss to follow-up can affect validity of findings

Analysis of cohort studies

Outcome

Predictor Present Absent Total

Present (exposed)

a b a+b

Absent

(unexposed)

c d c+d

Total a+c b+d N

Analysis of cohort studies

• Denominators are standardized to person-years of exposure (usually per 100 person-years)

• Statistics: Incidence = a+c/N (incidence density) Incidence in exposed = a/a+b Incidence in unexposed = c/c+d Relative risk = Incidenceexp = a/(a+b)

Incidenceunexp c/(c+d)

Relative risk

• Estimates the magnitude of an association and is equivalent to the probability that the outcome will occur given the exposure in comparison to non-exposed persons

Incidence in exposed group (Ie)RR=

Incidence in non-exposed group (Io)

RR=1 No associationRR>1 Risk of outcome increasedRR<1 Risk of outcome decreased (protective factor)

Analysis of cohort studies

• Because of varying follow-up periods and drop out, often need to use time-to-event analysis (Kaplan-Meier or survival analysis)

• When using survival analysis, associations between predictors and outcomes are expressed in Cox proportional hazards models as hazards

Kaplan-Meier analysis

Experiments

• Investigator controls the predictor variable (intervention or treatment)

• Major advantage over observational studies is ability to demonstrate causality

• Randomization controls unmeasured confounding

• Only for mature research questions

Types of experiments

• Randomized controlled trials• Other randomized designs

Controlled clinical trials (non-randomized between-group design)

Factorial design Matched pairs randomization Group or cluster randomization

• Within-group designs Times-series design (pre-post) Cross-over design

What types of interventions can be studied in experiments?

• Drugs

• Medical devices (e.g., female condom)

• Behavioral interventions

• Social and policy interventions

• Surgical procedures

• Organization of health-care systems

• Combinations of the above

Randomized controlled trial

The Present The Future

Treatment Sick Well

Sample R

Placebo Sick Well

Analysis of randomized controlled trial

• Analyzed like cohort study with RR• Intention to treat analysis

Most conservative interpretation Include all persons assigned to intervention

group (including those who did not get treatment or dropped out)

• Subgroup analysis Groups identified pre-randomization

Monitoring trials

• Elements to monitor Outcomes Adverse events

• Stopping rules Independent data and safety monitoring

board