Blackwell Science, LtdOxford, UKJEPJournal of Evaluation in Clinical Practice1356-1294Blackwell Publishing Ltd 2004115479483Original ArticleCluster randomized controlled trialsS.
Puffer
et al
Journal of Evaluation in Clinical Practice, 11, 5, 479–483 doi:10.1111/j.1365-2753.2005.00568.x
Cluster randomized controlled trials
Suezann Puffer BSc,1 David J. Torgerson PhD2 and Judith Watson PhD3
1
Research Assistant, 2Director, 3Research Fellow, York Trials Unit, Department of Health Sciences, University of York,
York, UK
Correspondence Abstract
David J. Torgerson Cluster randomized controlled trial (RCT), in which groups or clusters of
York Trials Unit
individuals rather than individuals themselves are randomized, are increas-
Department of Health Sciences
Area 4 ingly common. Indeed, for the evaluation of certain types of intervention
Seebohm Rowntree Buiding (such as those used in health promotion and educational interventions) a
University of York cluster randomized trial is virtually the only valid approach. However, clus-
York YO10 5DD
ter trials are generally more difficult to design and execute than individually
UK
E-mail: djt6@[Link] randomized studies, and some design features of a cluster trial may make it
particularly vulnerable to a range of threats that can introduce bias. In this
Keywords: cluster, controlled trial, paper we discuss the issues that can lead to bias in cluster randomized trials
methodology, randomized
and conclude with some suggestions for avoiding these problems.
Accepted for publication:
16 September 2004
promotion trials among children are often under-
Introduction
taken within a school setting by randomizing whole
In most clinical trials participants are randomized as schools to a novel health promotion curriculum, for
individuals to different treatments. However, some- example.
times individual allocation is not possible or desir- There are various reasons why a cluster design may
able and groups of individuals are randomized be appropriate. For some types of intervention clus-
instead. This is known as cluster or group randomiza- ter randomization may be the only feasible approach.
tion. In a cluster trial groups of participants (such as For example, if we wished to test the effectiveness of
general practices or hospital wards) are the unit of a set of clinical guidelines on patient outcome, we
randomization. Periods of time can also form a clus- would randomize doctors to receive the guidelines or
ter. Thus, weeks of a year might be randomized so to act as controls. The effectiveness of the guidelines
that an intervention might be implemented during would be tested on the patients who are registered
1 week and withdrawn during the control week. The to those doctors. This approach is probably the only
use of cluster randomization, and appropriate statis- feasible and practicable way of evaluating such an
tical analysis, in the context of education was dis- intervention. An alternative approach would be to
cussed as early as 1940 by Lindquist (Lindquist 1940). randomize individual patients to doctors who had
This is because many educational evaluations are received the guidelines and to those who had not.
within a naturally occurring clusters (e.g. classroom Whilst this approach is, in theory, feasible, in practice
or school) and therefore the only feasible approach it would be difficult or even impossible to implement.
of undertaking a randomized controlled trial (RCT) Evaluating a vaccine or an intervention to prevent a
is to use cluster allocation. Cluster trials are increas- communicable disease may also require cluster ran-
ing in popularity among health service researchers domization to avoid the ‘herd’ effect. In a trial of
(Donner & Klar 2000; Bland 2004). Indeed, health communicable diseases the probability of an individ-
© 2005 Blackwell Publishing Ltd 479
S. Puffer et al.
ual contracting the disease will be dependent upon substantial drawbacks to the use of this design.
the proportion of the surrounding population who Firstly, compared with an individually randomized
are already immune to the disease. Individual ran- trial testing the same hypothesis, cluster randomiza-
domization will underestimate the effectiveness of tion requires a significantly larger sample size. This is
the vaccine because, in real life situations, a propor- because standard sample size calculations assume
tion of the population who remain unvaccinated will that outcomes between individuals are uncorrelated.
receive some protection from the bulk of the popu- However, when participants are randomized by
lation who are vaccinated. Thus, the only sensible cluster this assumption no longer holds. The greater
method of evaluating such interventions is through the correlation between individuals within a cluster
allocating people in clusters. [known as the intracluster correlation coefficient
Cluster randomization is often used to avoid ‘con- (ICC)] the greater the number of participants
tamination’ between those receiving the intervention required. The ICC is the proportion of the total vari-
and those who are not. For example, a randomized ance of the outcome that can be explained by the
trial of an educational intervention to improve the variation between clusters. Typically, cluster trials
diet and lifestyle of people at high risk of heart dis- need 50–100% more participants to achieve the
ease might use a cluster design to avoid the inter- same statistical power as an individually randomized
vention group telling the control group about the trial: this is known as the design effect. For example,
treatment. If this did occur, then there is a danger of in a review of 36 cluster trials by Puffer and col-
dilution bias resulting in a Type II error. To avoid leagues, only five papers reported the design effect,
dilution bias participants may be randomized by and these ranged from 21% to 200% with an aver-
general practice on the basis that, compared with age (median) of 120% (Puffer et al. 2003).
individual randomization, participants in the inter- In general, cluster trials with fewer than five clus-
vention practices are less likely to contaminate par- ters per arm are inadvisable (Medical Research
ticipants in the control practices. It is this need to Council 2002). This is because there will be an inad-
avoid the possibility of contamination that has led to equate number to balance out cluster level confound-
an increased use of cluster randomization. ing. Randomization given sufficient numbers will
Although RCTs are the most robust evaluative result in factors that could affect participant outcome
method, poorly conducted trials are susceptible to a to be balanced across the groups. In a cluster trial
number of factors that can bias their results. Meth- there will be factors at the cluster level as well as the
odological reviews of individually randomized trials participant level that need to be balanced.
have shown that rigorously conducted trials produce Secondly, the analysis of cluster trials is not
different effect estimates from those which are straightforward. If the clustering is not taken into
poorly conducted (Shulz et al. 1994; Kjaergaard account and the data are treated as independent, the
et al. 2001). However, whilst cluster randomized tri- result may be P-values that are too small and confi-
als are more complex to design and execute than dence intervals that are too narrow (Bland 2004).
individually randomized trials (Donner & Klar Thus, leading to conclusions that may be false. This
2000), they have received relatively little attention. problem has been known for more than 60 years
Thus, the implications of cluster randomization have (Lindquist 1940). Despite this many cluster random-
not been well understood. Within this paper we ized trials still use statistical approaches that are
attempt to highlight the particular difficulties with appropriate only for individually randomized stud-
conducting cluster randomized trials and identify ies. Statistical methods that account for the cluster-
potential solutions to avoiding or minimizing these ing effect are necessary. These may take the form of
problems. relatively simple methods such as a two sample t-
test, which instead of using individual patient values,
compares cluster means (Kerry & Bland 1998).
Statistical issues
Alternatively, more complex approaches such as
Whilst cluster randomization may be advantageous multi-level modelling may be required. The most
for the evaluation of specific interventions, there are appropriate analytical technique will depend on the
480 © 2005 Blackwell Publishing Ltd, Journal of Evaluation in Clinical Practice, 11, 5, 479–483
Cluster randomized controlled trials
study design, the number of clusters and the number ment is often a possibility. Furthermore, selection
of individuals per cluster (Medical Research Council bias can be introduced if participants withheld their
2002). consent for either treatment or data collection, par-
ticularly if this occurs differentially across the treat-
ment groups. This is a well documented disadvantage
Selection bias
of acquiring consent after randomization in individ-
The statistical drawbacks of using cluster randomiza- ually randomized trials [known as Zelen’s Method
tion are relatively well documented within the liter- (Zelen 1990)] because some refusal of treatment or
ature (Murray 1998; Donner & Klar 2000), and are data collection usually occurs (Altman et al. 1995).
now therefore given at least some consideration in This is less problematic in non-Zelen designs, as
the majority of cluster trials published in major jour- participants are told in advance about the treatment
nals (Puffer et al. 2003; Bland 2004; Eldridge et al. options and if they decline to be exposed to one of
2004). However, cluster trials are prone to a series of the options they are not randomized.
other problems that can bias their results, and until Whatever the reasons for differences in recruit-
fairly recently these issues have received insufficient ment, the consequences are potentially the same:
attention (Puffer et al. 2003). These issues are dis- selection bias is introduced and the trial results are
cussed below. unreliable. In a recent review Puffer and colleagues
In cluster trials potential bias can be introduced at identified a sample of 36 cluster randomized trials
two levels, the first of which is the cluster level. The published in three major medical journals, between
randomization of clusters needs to be undertaken 1997 and 2002 (Puffer et al. 2003). They found that 15
carefully and preferably independently. Otherwise it of these trials could have experienced bias in their
is possible for biased allocation to occur. That is, clus- recruitment of participants. Seven of these 15 trials
ters are not allocated randomly and are instead allo- showed some evidence of consenting differential
cated to a particular arm on the basis of reasons that numbers of participants or excluding participants in
might affect outcome. As has happened in individu- a selective fashion. Furthermore, whilst having no
ally randomized trials (Shulz 1995), it is theoretically evidence of bias in the original published paper, one
possible for the allocation of clusters to be subverted. of the eight remaining trials was later subsequently
Furthermore, once clusters have been randomized it found to have experienced recruitment bias (Jordhoy
is important to retain the cluster in its allocated et al. 2002). Thus, it was found that 25% of cluster
group to avoid the risk of attrition bias. trials published in major clinical journals suffered
The second level at which bias can occur is after potential selection bias.
clusters have been allocated and when individual A more recent review of 152 cluster randomized
participants are recruited into the study. If the person trials undertaken in primary care found that only
recruiting participants has both knowledge of the eight (5%) had evidence for differential recruitment
clinical characteristics of the participants and of the (Eldridge et al. 2004). However, unlike the work by
allocation schedule, biased recruitment can occur. Puffer and colleagues, in this review each trial was
Subversion within individually randomized trials can not carefully scrutinized to ascertain whether or not
occur when participants with poor prognostic char- there was a problem of biased recruitment (Eldridge,
acteristics are randomized so that they are more personal communication).
likely to enter the ‘unfavoured’ group (Shulz 1995; Given that cluster trials are more difficult to
Kennedy & Grant 1997). Furthermore, evidence for undertake successfully than individually randomized
the biasing effects of allocation foreknowledge has studies we make the following suggestions that trial-
been shown on treatment effect sizes (Shulz et al. ists might consider when designing such trials.
1994; Kjaergaard et al. 2001). Consequently, those
recruiting participants in individually randomized
Individual allocation
trials ought to be blind to the allocation schedule.
In cluster trials, however, it is not always possible As discussed above, one of the main reasons for
to conceal treatment allocation. Thus, biased recruit- using cluster randomization is to overcome the per-
© 2005 Blackwell Publishing Ltd, Journal of Evaluation in Clinical Practice, 11, 5, 479–483 481
S. Puffer et al.
ceived threat of contamination. However, whilst this patient identification, by identifying either more
threat may be real in some situations, in others there cases or patients with different clinical characteristics
may be very little contamination. Indeed, in sample compared with GPs who did not have training. This
size terms it has been demonstrated that even when process would lead to samples of participants in the
there are relatively high contamination rates (e.g. control and treatment groups who were fundamen-
20%) it may still be more efficient to randomize tally different at baseline leading to bias. Alterna-
more patients in an individual trial and accept a tively, or in addition, some GPs may consciously or
diluted effect size (Slymen & Hovell 1997; Torgerson unconsciously ‘subvert’ the trial by recruiting partic-
2001). Thus, it is best to avoid using cluster trial ipants with characteristics such that the study will be
methods if at all possible. However, this advice can- shown to be successful. To reduce the possibility of
not always be adhered to, and for the robust evalua- either of these problems King and colleagues used
tion of particular interventions cluster methods are practice receptionists to identify and recruit partici-
required. pants. Receptionists from both groups of practices
would have had identical training and therefore
should recruit similar trial participants, which should
Prior identification of participants
avoid recruitment bias.
In an attempt to avoid recruitment bias, in some
instances it may be possible to identify participants
Reporting of cluster trials
before cluster allocation. For example, if we consider
the evaluation of a school-based health promotion Cluster trials must not only be designed and analysed
campaign, it would be possible to identify children appropriately but also reported in such a way that
within schools or individual classes before cluster allows readers to assess trial quality and understand
allocation. The children and their parents are pre- how the conclusions were reached. The reporting of
sented with the possible alternatives and are asked trials in general is of poor quality (Huwiler-Muntener
for their consent to participate. Once consent is et al. 2002). However, since the introduction of the
obtained, schools or classes can be randomized to the Consolidated Standards of Reporting Trials (CON-
different groups. Failure to identify children before SORT) statement in 1996 (Begg et al. 1996), and the
cluster allocation has the potential to introduce bias. revised version in 2001 (Moher et al. 2001), this issue
Again, however, prior identification is not always has received much attention and improvements have
possible. been made. The CONSORT statement includes a
checklist of items that should be included when
reporting a trial, which are evidence-based whenever
Independent recruitment
possible and are regularly reviewed (Altman et al.
Where prior identification is not feasible, an ‘inde- 2001). However, the original statement focused on
pendent’ person needs to identify and recruit partic- the reporting of individually randomized trials and
ipants. Furthermore, this person should ideally be did not allow for the special methodological circum-
blind to group allocation. Allowing a person with stances of cluster trials. Indeed a recent systematic
foreknowledge of group allocation to recruit partici- review of cluster trials in primary care highlighted
pants could introduce bias and should be avoided. the problem of poor reporting and found no
An example of independent recruitment is provided improvement in reporting of cluster trials between
from a trial by King and colleagues (King et al. 2002). 1997 and 2000 (Eldridge et al. 2004). Recently,
In this trial general practitioners (GPs), in the inter- Campbell and colleagues have developed an
vention group, were trained to diagnose and treat extended version of the CONSORT statement to
depression. Clearly GPs in both groups were aware include cluster randomized trials (Campbell et al.
of their allocation and if they identified trial partici- 2004). The development of this statement will hope-
pants bias could result in one of two ways. First, fully increase awareness of the implications of cluster
because of the nature of the training, if it were effec- randomization, improve the reporting of these trials
tive, we might expect the GPs to actually improve and aid critical appraisals of trial quality.
482 © 2005 Blackwell Publishing Ltd, Journal of Evaluation in Clinical Practice, 11, 5, 479–483
Cluster randomized controlled trials
in cluster randomized trials of palliative care. Palliative
Conclusions Medicine 16, 43–49.
Because of the difficulties of conducting cluster Kennedy A.M. & Grant A. (1997) Subversion of allocation
RCTs, they should be avoided unless individually in a randomised controlled trial. Controlled Clinical
Trials 18 (Suppl. 3), 77S–78S.
randomized trials are inferior or practically impossi-
Kerry S.M. & Bland M. (1998) Analysis of a trial ran-
ble. Researchers should justify why an individually
domised in clusters. British Medical Journal 316, 54.
randomized trial is not appropriate and explain
King M., Davidson O., Taylor F., Haines A., Sharp D. &
clearly why a cluster trial is preferable. The particular Turner R. (2002) Effectiveness of teaching general prac-
difficulties of using this design should be given suffi- titioners skills in brief cognitive behaviour therapy to
cient attention from the initial planning stages of the treat patients with depression: randomised controlled
trial. Furthermore, particular attention should be trial. British Medical Journal 324, 947–952.
given to the reporting of the trial to ensure readers Kjaergaard L.L., Villumsen J. & Cloud C. (2001) Reported
are able to assess the methodological quality. methodologic quality and discrepancies between large
and small randomized trials in meta-analyses. Annals of
Internal Medicine 135, 982–989.
References
Lindquist E.F. (1940) Statistical Analysis in Educational
Altman D.G., Schulz K.F., Moher D., Egger M., Davidoff Research. Houghton Mifflin, Boston, USA.
F., Elbourne D., Gøtzsche P. & Lang T. (2001) The Medical Research Council (2002) Cluster Randomised
revised CONSORT statement for reporting randomised Trials: Methodological and Ethical Considerations.
trials: explanation and elaboration. Annals of Internal MRC, London. Available at:
Medicine 134, 663–694. [Link]
Altman D.G., Whitehead J., Parmar M.K.B., Stenning S.P., Moher D., Schulz K.F., Altman D.G. & CONSORT
Fayers P.M. & Machin D. (1995) Randomised consent Group. (2001) The CONSORT statement: revised rec-
designs in cancer clinical trials. European Journal of ommendations for improving the quality of reports of
Cancer 31A, 1934–1944. parallel-group randomised trials. Lancet 357, 1191–
Begg C., Cho M., Eastwood S. et al. (1996) Improving the 1194.
quality of reporting of randomised controlled trials. The Murray D.M. (1998) Design and Analysis of Group-
CONSORT statement. Journal of the American Medical Randomized Trials. Oxford University Press, New York.
Association 276, 637–639. Puffer S., Torgerson D.J. & Watson J. (2003) Evidence for
Bland J.M. (2004) Cluster randomised trials in the medical risk of bias in cluster randomised trials: review of recent
literature: two bibliometric surveys. BMC Medical trials published in three general medical journals. British
Research Methodology 4, 21. Medical Journal 327, 785–789.
Campbell M.K., Elbourne D.R. & Altman D.G. for the Shulz K.F. (1995) Subverting randomisation in controlled
CONSORT Group (2004) CONSORT statement: exten- trials. Journal of the American Medical Association 274,
sion to cluster randomised trials. British Medical Journal 1456–1458.
328, 702–708. Shulz K.F., Chalmers I., Grimes D.A. & Altman D.G.
Donner A. & Klar N. (2000) Design and Analysis of (1994) Assessing the quality of randomization from
Cluster Randomised Trials in Health Research. Arnold, reports of controlled trials in obstetrics and gynaecology
London. journals. Journal of the American Medical Association
Eldridge S.M., Ashby D., Feder G.S., Rudnicka A.R. & 272, 125–128.
Ukoumunne O.C. (2004) Lessons for cluster randomised Slymen D.J. & Hovell M.F. (1997) Cluster versus individ-
trials in the twenty-first century: a systematic review of ual randomization in adolescent tobacco and alcohol
trials in primary care. Clinical Trials 1, 80–90. studies: illustrations for design decisions. International
Huwiler-Muntener K., Juni P., Junker C. & Egger M. Journal of Epidemiology 26, 765–771.
(2002) Quality of reporting of randomized trials as a Torgerson D.J. (2001) Contamination in trials: is cluster
measure of methodologic quality. Journal of the Ameri- randomisation the answer? British Medical Journal 322,
can Medical Association 287, 2801–2804. 355–357.
Jordhoy M.S., Fayers P.M., Ahlner-Elmqvist M. & Kaasa S. Zelen M. (1990) Randomized consent designs for clinical
(2002) Lack of concealment may lead to selection bias trials: an update. Statistics in Medicine 9, 645–656.
© 2005 Blackwell Publishing Ltd, Journal of Evaluation in Clinical Practice, 11, 5, 479–483 483