the systematic review of systematic reviews has arrived!
TRANSCRIPT
ARTICLE IN PRESS
1356-689X/$ -
doi:10.1016/j.m
Manual Therapy 11 (2006) 91–92
www.elsevier.com/locate/math
Editorial
The systematic review of systematic reviews has arrived!
April saw the publication of Ernst and Canter’s (2006)paper, ‘‘A systematic review of systematic reviews of
spinal manipulation’’. The summary statement concludedthat ‘‘considering the possibility of adverse effects thisreview does not suggest that spinal manipulation is arecommendable treatment’’.
The media hype in the United Kingdom rose to adeafening crescendo with most of the media focus onosteopathy and chiropractic, even though the review hasincluded a number of studies incorporating manipula-tive physiotherapy.
Systematic reviews are usually regarded as the peak ofevidence to judge the efficacy of an intervention. Thus itis disappointing to be able to detect fundamentalproblems in this review, considering its impact in thepopular press. The paper Ernst and Canter (2006)includes systematic reviews focusing on a heterogeneous
Evidence based authority –future ? !
Opinions:-respected authorities
based on clinical evidence
Well designednon-experimental studies
from more than one centre
Well designed non-randomised trials
e.g. cohort, case-controlled
At least one RCT of appropriate size
Systematic reviews
Systematic reviewsof meta-analyses
Systematic reviewsof systematic reviews
Meta-analyses
collection of problems (low back pain, neck pain, neckproblems, chronic headache, non-spinal pain syndrome,dysmenorrhoea, infantile colic, asthma, cervicogenic
see front matter r 2006 Published by Elsevier Ltd.
ath.2006.04.001
dizziness, and a paper on any condition!). Reviews wereselectively chosen from the years 2000 to May 2005. Thepaper included no reference to a methodologicalapproach used in the appraisal in the systematic reviewscontained within the article and did not go back tosource. No critical appraisal of the pre-existing systema-tic reviews was offered. In addition, the interventionswere poorly described and defined and were hetero-geneous i.e., the reviews included a range of manualtherapy techniques, although the article title referred tospinal manipulation only. The authors comments on theincluded systematic reviews unfortunately appeared tobe selectively biased. One is left asking the questionwhy?
As mentioned we know that the accepted hierarchy ofevidence as it is held currently in the health arena, showssystematic reviews to be the gold standard level of
evidence, but in view of the recent developments howwill the hierarchy will look in a few years time if we donot take stock now?
ARTICLE IN PRESSEditorial / Manual Therapy 11 (2006) 91–9292
It is worth reflecting on issues with the model in thecurrent context. Systematic reviews are recognized as thehighest level of evidence, but to be at this highest levelthey must include multiple, well designed randomizedcontrol trials. Hence the systematic review is only asgood as the original studies it includes and thesystematic reviewers interpretation of the studies. Thereare several additional known problems associated withsystematic reviews. The primary studies do not necessa-rily reflect contemporary practice largely due to the timeat which they were published and also due to the factthat the research base is evolving all the time.Frequently we see the pooled studies include hetero-geneous patients, professions and interventions whichare often poorly defined and described.
The quality of interventional approach is neverassessed in systematic reviews. The only assessment thattakes place is with regards to the quality of therandomized control trial design. Following the CON-SORT statement (Moher et al., 2001) can help to rectifythis. Worryingly many of the author teams associatedwith systematic reviews do not include an expert in theinterventions used, therefore the interpretation of rando-mized control trials including interventions with whichthe author team are not strictly familiar can be flawed ifclear definitions of terms have not been included in theoriginal randomized control trial write up.
Generally speaking, the systematic review can for anumber of reasons muddy the waters of understandingand foil attempts to get to the truth rather thanclarifying the evidence base picture.
The systematic review of systematic reviews is anuntested methodology at best and might take us evenfurther away from the truth. It has become popular inmedicine but it is worth reflecting on the fact that manyhard science subject areas, for example chemistry,believe that the systematic review itself is poor science.
If anything positive has come out of this newpublication, it must be that it raises the need for allthe professions engaged in manual therapy to do severalthings:
Increase the amount of research in the field to includestandardized data collection, qualitative, experimentaland randomized control trial studies which are welldesigned.
Refine and closely define the terms used in manualtherapy so that techniques are not subject to misinter-pretation.
Clearly publish the probable and possible knowneffects and side effects of these individual techniques.
Increase the development work on clinical sub-grouping.
Develop a system for assessing the quality ofinterventions included in randomized control trials.
So lets get back to proper science and discover the realtruth if we can.
References
Ernst E, Canter PH. A systematic reviews of systematic reviews of
spinal manipulation. Journal of the Royal Society of Medicine
2006;99:189–93.
Moher D, Schulz KF, Altman DG. The CONSORT Statement:
revised recommendations for improving the quality of reports of
parallel group randomised control trials. BMC Medical Research
Methodology 2001;1(2):1186–97.
(Editors)Ann Moore, Gwen Jull
University of Brighton, Clinical Centre for Health
Professions, 49, Darley Road, Eastbourne BN20 7UR,
UK
E-mail address: [email protected] (A. Moore)