Current Issues in the Design of Cluster

Randomization Trials

Allan Donner, PhD, FRSC

Department of Epidemiology and Biostatistics

The University of Western Ontario

London, Canada

Robarts Clinical Trials,

Robarts Research Institute

London, Canada

What Are Cluster Randomization Trials

Cluster randomization trials are experiments in

which intact social units or clusters of individuals

rather than independent individuals are randomly

allocated to intervention groups.

Examples:

Medical practices selected as the randomization

unit in trials evaluating the efficacy of disease

screening programs

Communities selected as the randomization unit

in trials evaluating the effectiveness of new

vaccines in developing countries

Hospitals selected as the randomization unit in

trials evaluating educational guidelines directed at

physicians and/or administrators

Reasons for Adopting Cluster Randomization

Administrative convenience

To obtain cooperation of investigators

Ethical considerations

To enhance subject compliance

To avoid treatment group contamination

Intervention naturally applied at the cluster level

Role in vaccine trials

Randomization of geographic areas can be used to

capture indirect (herd ) effects of vaccination.

Incidence of disease among non-vaccinated individuals

in the intervention group is compared to incidence of

disease in the control group.

Unit of Randomization vs. Unit of Analysis

A key property of cluster randomization trials is

that inferences are frequently intended to apply

at the individual level while randomization is at

the cluster or group level. Thus the unit of

randomization may be different from the unit of

analysis.

In this case, the lack of independence among

individuals in the same cluster, i.e., intracluster

correlation, creates special methodological

challenges in both design and analysis.

Quantifying the Effect of Clustering

Consider a trial in which k clusters of size m are

randomly assigned to each of an experimental and

control group denoted by i =1 and 2 respectively. Let

denote the sample mean of the response variable in

the group

Then assuming is normally distributed with common

variance

where is the coefficient of intracluster correlation.

Equivalently,

If clusters of size are randomized to

each of two treatment groups,

then the effective sample size per group is

given by

Application of standard sample size approaches

leads to an underpowered study (Type II error)

Application of standard statistical methods

generally tends to bias p-values downwards, i.e.,

could lead to spurious statistical significance

(Type I error)

Examples:

COMPLETELY RANDOMIZED DESIGN

Study Purpose:

To evaluate the effectiveness of vitamin A

supplements on childhood mortality.

450 villages in Indonesia were randomly assigned

to ether participate in a vitamin A supplementation

scheme, or serve as a control. One year mortality

rates were compared in the two groups.

Sommer et al. (1986)

MATCHED PAIR DESIGN

Study Purpose:

The COMMIT community intervention trial(1995) was

designed to promote smoking cessation using a

variety of community resources. The primary

outcome measure was the 5-year smoking cessation

rate.

Unit of Randomization: Community

Number of Matched Pairs: 11

Matching factors: size, pop. density

STRATIFIED DESIGN

Study Purpose:

The purpose of the WHO antenatal trial was to

compare the impact of two programmes of antenatal

care on the health of mothers and newborns.

Unit of Randomization: Antenatal care clinic.

Stratification Variables: Primary stratification was

by country: Thailand,

Cuba, Argentina, Saudi

Arabia.

Number of

Clusters per Stratum: Ranged from 12 to 17.

Villar et al. (2001)

CLUSTER-CROSSOVER DESIGN

All participating clusters receive both intervention and

control in a sequence determined at random

Reduces total number of clusters required with a

considerable increase in study duration

Turner(2007)

STEPPED WEDGE DESIGN

All clusters eventually cross over but only from the

control to intervention at a time point determined at

random

Allows clusters to be enrolled gradually over time

Hussey and Hughes(2007)

Key question:

Are the trial inferences aimed at

(i) the individual subject? or

(ii) at a naturally defined cluster of individuals?

This decision guides both the choice of design and

the approach to the analysis

Selecting the Unit of Inference

Example 1:

The intervention consists of a blood pressure

screening program intended to lower the risk of

cardiovascular mortality

Bass et al (1986)

Example 2:

The intervention consists of educational

guidelines for the management of hyperlipidaemia

intended to increase the proportion of eligible patients

prescribed lipid-lowering drugs

Diwan et al (1995)

Consider a trial randomizing hospitals designed

to assess the effect of obtaining a second clinical

opinion on the decision to proceed with a

caesarian section operation (Altabe et al 2008).

The target of the intervention was the hospital rate

of caesarian section.

In this case, the hospital was the natural unit of

inference and standard methods of sample size

estimation and analysis applied at the cluster

level.

Key point:

If the unit of inference is at the cluster level then an

analysis at the cluster level is appropriate, and no

consideration need be given to the intracluster

correlation coefficient.

From the perspective of sample size estimation and

analysis the challenges are no different from those

that arise in individually randomized trials.

Simplified data collection and lower cost

Can be applied to any outcome variable

Exact statistical inferences can always be constructed

Can be adapted to adjust for baseline imbalances

Informed consent issues may be eased

Cluster level Analyses:Advantages

A Common Misconception

Investigators have occasionally claimed that cluster

level analyses will only provide valid statistical

inferences when the intracluster correlation

coefficient

However this is too stringent a claim.

For balanced designs, cluster level analyses

provide inferences which are equivalent to those

obtained using a mixed linear regression model

for any value of

Threats to trial validity?

Selection bias in the recruitment of patients over time

Experimental contamination

i)Risk of selection bias

Consider a trial using a simple randomization scheme

practices in which physicians are asked to identify as

well as to treat selected patients (e.g. Kinmouth et al

[1998]).

Can cluster randomization introduce bias through the

way patients are differentially recruited across

treatment groups?

Source of bias:

If the physician’s practice has already been randomized, recruitment for patient participation cannot be done blindly with respect to intervention group.

If physicians in the experimental group are more diligent in seeking out patients than in the control group, or tend to identify patients who are less ill, bias may result.

Unbiased estimates of the effect of intervention can be assured only if analyses are

(i) based on data from all cluster members

or

(ii) based on a random sub-sample of cluster

members or

Possible Solutions:

Identify eligible patients in each practice prior to randomization.

If eligible patients are identified after randomization, recruitment should be done by an individual independent of the trial.

Torgerson (2001) Farrin et al (2005)

ii) Is fear of contamination a legitimate

reason adopting cluster randomization?

Suppose under individual randomization a

proportion of R of the control group patients experience

the same success rate P1 as seen among experimental

patients.

Then the difference that can be detected is

reduced to

and the required sample size must be inflated by the

factor IF = 1/ (1-R)2

e.g. If R =0.30, then IF =2.04

But under unrestricted cluster randomization, the

inflation factor 1+(m-1) might be more

Farrin et al (2005)

However…..

Uncertainty concerning true level of contamination

Presence of contamination under individual

randomization will lead to an underestimate of the

effect of treatment

Logistical difficulties a factor-giving different

treatments at random to different patients in the same

office.

Can create pairs of clusters between which travel is

difficult.

Issues Involving Informed Consent

Two distinct levels of informed consent must be

distinguished in cluster randomization trials:

(i) informed consent for randomization (usually

provided by a ‘decision-maker’)

(ii) informed consent for participants given that

randomization has occurred.

By analogy to current ethical requirements for

clinical trials, it would be unethical not to obtain

informed consent from every cluster member prior to

random assignment.

Is such a strict analogy required for

community randomized trials?

“Ethical advice indicated that, since we were only

providing information to clinicians, there was no

reason to seek patient consent”

Wyatt et al (1998)

“Eliciting consent from the many thousands of patients

involved would create severe budgetary and logistic

obstacles.

The central human rights committee at the Hines

Coordinating Center and those at the participating

sites have deemed this type of quality improvement

research exempt from direct patients consent”.

US Dept. of Veterans Affairs

The Effect of Level of Intervention

Edwards et al. (1999) distinguished cluster

randomization trials by the level of intervention:

for “individual-cluster” trials intervention is provided

directly to individual study subjects (e.g., STD

treatment for prevention of HIV).

for “cluster-cluster” trials intervention is provided at

the cluster level (e.g., effect of local medical opinion

leaders on patient treatment).

The need for consent by individual study subjects is

deemed of particular concern for “individual cluster”

trials.

The need for consent by individual study subjects is

deemed of particular concern for “individual cluster”

trials.

However these distinctions may only be relevant

when randomizing larger clusters (e.g., work sites,

classrooms, communities).

Is Informed Consent Required for Cluster-

Cluster Trials?

“Informed consent to randomization may not be

required if it is not possible to approach

subjects at the time of randomization.

However potential subjects approached after

randomization must be provided with a detailed

description of the interventions in the trial arm to

which their clusters have been randomized”

McRae et al(2009)

Cluster Randomization Trials and the Zelen Randomized Consent Design

The requirements of informed consent are often perceived to be a barrier to patient accrual. Randomized consent designs were introduced by Zelen (1990) to lower this barrier, thus hastening trial completion.

In the single consent design patients are randomly assigned to one of two groups , but with only patients in the intervention group asked to provide informed consent. If not, they may be offered the control therapy.

Analyses are conducted based on the intervention to

which patients were initially assigned, i.e., analysis by

intention to treat.

Randomized consent designs have proved quite

controversial because of concern for the ethical

implications of randomizing subjects prior to obtaining

their consent.

But this strategy is typical of many cluster

randomized trials!

Factors Influencing Loss of Precision

Cluster Randomization Trials

1. Interventions often applied on a group basis

2. Some studies permit the immigration of new

subjects after baseline.

3. Entire clusters, rather than just individuals, may be

lost to follow-up.

4. Difficulties associated with prevention trials

Why are so many CRT’s unpowered?

“Generally the size of effects has been meager in

relation to the effort expended”

“We often do not have the resources to detect medium

effects”

Susser (1995)

Strategies for Increasing Precision

Restrict inclusion criteria

e.g. recruit practices of same size, with physicians having similar years of experience in a controlled geographic area.

Conduct a baseline survey of the prevalence of the trial outcome

- serves as powerful risk factor for outcome

- provides estimate of

- allows trial personnel to gain experience

- can help to identify potential stratification factors

Increasing cluster size provides diminishing

returns in statistical power if

e.g. If increasing beyond 100 provides little

statistical gain.

Donner and Klar (2004)

Consider sample size re-estimation as part of an

interim analysis.

Lake et al (2001)

Recognize likely size of in primary care

research (0.01 to 0.05 for outcome variables; 0.05

to 0.15 for process variables).

Campbell et al (2001)

Gulliford et al (2004)

IMPACT ON POWER OF INCREASING THE NUMBER

OF CLUSTERS VS. INCREASING CLUSTER SIZE:

Let

As

As

Statistical Challenges in the

Meta-Analysis of Cluster

Randomization Trials

Meta-analyses combining cluster randomization trials raise methodologic issues beyond those raised by meta-analyses which include only individually randomized trials:

- Increased Trial Heterogeneity

- Difficulties in Estimating Design Effects from Individual Trials

- Special Issues Involving Publication Bias

- Choice of Statistical Method

- Assessment of Trial Quality

Study Heterogeneity

Sources of heterogeneity in meta-analyses involving cluster randomization trials:

(i) Heterogeneity of study design e.g., completely randomized, matched-pair, stratified

(ii) Heterogeneity in unit of randomization e.g., worksites

schools

households

individuals

(iii) Variation in eligibility criteria at both the cluster and individual level

(iv) Wide variation in methodological quality

Example of Design Heterogeneity in a

Meta-Analysis:

Brunner et al. (1997) reported on a meta-analysis of dietary interventions aimed at reducing cardiovascular risk factors.

17 randomized trials with a total of 6893 participants were included in the meta-analysis.

16 of the trials were individually randomized, one was cluster randomized.

The one cluster randomized trial (allocating worksites) included approximately 42% of the 6893 participants.

In this case, the effective sample size for the meta-analysis is clearly much less than 6893.

Difficulties in Estimating Design Effects

from Individual Trials

For trials randomizing clusters of size m to two

intervention groups, ‘design effect’ is given by

Estimation of the design effect is crucial for optimal

weighting of the trials to be combined.

However many trial reports fail to provide estimates of the design effect (Laopaiboon [2003], Thomas et al. [2003]). Therefore meta-analysts have resorted to various ad-hoc strategies to obtain the required information:

(i) increasing the variances of estimated intervention effects by an arbitrary amount (e.g., Fawzi et al. [1993]).

(ii) using external data to obtain imputed values of the design effect (e.g., Rooney and Murray [1996]).

(iii) using regression methods to estimate design effects from information on design effects reported in other trials (e.g., Beaton et al. [1993]).

(iv) excluding cluster randomization trials from a meta-analysis (e.g., Xin et al. [2003]).

Assessing Quality of Individual Trials

Issues here particularly relevant to cluster randomization trials include:

1. Justification for randomizing clusters rather than individuals.

2. Unambiguous definition of the randomization unit.

3. Eligibility criteria defined at both the individual and cluster level.

4. Accounting for clustering in both the estimation of trial size and in the analysis.

5. Reporting of observed intracluster correlation and/or design effect.

Special Issues Involving Publication Bias

Complication in cluster randomization trials is that many

such studies ignore the clustering in the analysis. These

studies may then be reported as statistically significant

when in fact a correctly performed analysis would not show

significance.

Conversely, if the analysis is correctly performed but power

considerations were ignored in the design, a finding of non-

significance may be misleading.

Combining Trials in Which One Cluster Only

Has Been Assigned to Each Group

Community intervention trials have been reported in which

exactly one cluster has been assigned to the intervention

group and one to the control group.

These invariably result in interpretational difficulties due to

the confounding of two sources of variation:

(i) variation in response due to the effect of intervention.

(ii) natural variation that exists between two clusters even in the

absence of an intervention effect.

However such trials could potentially be included in a meta-

analysis provided the randomization unit and subject

population for such studies were comparable to those of the

other trials included. Under these conditions, the

intracluster correlation could also be assumed to be

comparable to, and hence estimable from, the other trials.

SOME FUTURE CHALLENGES:

What are the best methods of quantifying trial

heterogeneity?

Development of efficient meta-regression techniques

Methods for combining individually randomized trials

with cluster randomized trials

OTHER ISSUES:

1. Is it fair to combine cluster randomization trials that have used very different units of randomization? Or very different designs?

2. Since estimates of intracluster correlation and design effects are often unreported, what are the best strategies for dealing with this problem?

3. What are the best methods to adjust for clustering effects in meta-analyses for different choices of endpoints and study designs?

3. Are fixed effects or random effects models for cluster trials most appropriate?

4. Should trials which include only one cluster per intervention group be included in a meta-analysis?

5. Can checklists be developed that assign quality scores to individual trials? If so, how many points should be deducted when a study ignores clustering in the design or analysis?

Why adopt a non-randomized design ?

Intervention has already been implemented on a

major scale, raising ethical issues.

Often less expensive, easier logistics

Easier to explain to public officials ,gain public

acceptance.

The availability of a small number of clusters is not a

reason to avoid a randomized design!

Can match or stratify in the design

Possibility of imbalance remains a problem

Estimation of the ICC: unresolved issues

i)Confidence interval construction for community

intervention trials having a large number of small

clusters and a binary outcome.

ii)Definition of the ICC for time-to-event outcomes with

censored data.

- Includes a checklist of items that should be included in a trial report

- Extends CONSORT statement for reporting of individually randomized trials (Begg et al, JAMA, 1996)

- Key Points • Rationale for adopting cluster design

• Incorporation of clustering into sample size estimation and analysis

• Chart showing flow of clusters through the trial, from assignment to analysis

EXTENSION OF CONSORT STATEMENT TO

CLUSTER RANDOMIZATION TRIALS

(CAMPBELL ET AL, BMJ, 2004)