bmj open...30 nicole wertheim college of nursing & health science 31 florida international...

For peer review only

WHAT IS THE INFLUENCE OF RANDOMIZATION SEQUENCE

GENERATION AND ALLOCATION CONCEALMENT ON

TREATMENT EFFECTS OF PHYSICAL THERAPY TRIALS? A

META-EPIDEMIOLOGICAL STUDY

Journal: BMJ Open

Manuscript ID: bmjopen-2015-008562

Article Type: Research

Date Submitted by the Author: 21-Apr-2015

Complete List of Authors: Armijo-Olivo, Susan; University of Alberta, faculty of rehabilitation Medicine Saltaji, Humam; University of Alberta, Department of Dentistry da Costa, Bruno; University of Bern, Institute of Social and Preventive Medicine Fuentes, Jorge; University of Alberta, Physical Therapy Ha, Christine; University of Alberta, Rehabilitation Medicine Cummings, Greta; University of Alberta, Faculty of Nursing

<b>Primary Subject Heading</b>:

Epidemiology

Secondary Subject Heading: Epidemiology, Evidence based practice, Rehabilitation medicine

Keywords: allocation concealment, sequence generation, physical therapy, meta-epidemiological, risk of bias

For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml

BMJ Open on M

arch 26, 2020 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2015-008562 on 3 Septem

ber 2015. Dow

nloaded from

http://bmjopen.bmj.com/


Allocation concealment in physiotherapy 21/04/2015

Armijo-Olivo Page 1 of 29

Title: WHAT IS THE INFLUENCE OF RANDOMIZATION SEQUENCE 1

GENERATION AND ALLOCATION CONCEALMENT ON TREATMENT EFFECTS 2

OF PHYSICAL THERAPY TRIALS? A META-EPIDEMIOLOGICAL STUDY 3 4

5

Authors: 6

Susan Armijo-Olivo, BSc. PT, MSc PT, PhD 7 Postdoctoral Fellow 8

CLEAR Outcomes (Connecting Leadership, Education and Research) Research Program 9

University of Alberta 10

5-115A Edmonton Clinic Health Academy (ECHA) 11

11405 – 87 Avenue 12

Edmonton, Alberta 13

T6G 1C9 14 Faculty of Rehabilitation Medicine, 15

3-48 Corbett Hall, Department of Physical Therapy, 16

University of Alberta, Edmonton, Canada T6G 2G4 17

Email: [email protected] / [email protected] 18

19

Humam Saltaji, DDS, MSc (Ortho) 20 PhD Candidate and Resident 21

Orthodontic Graduate Program 22

School of Dentistry 23


Edmonton, Alberta, Canada 25

E-mail: [email protected] 26

27

Bruno R. da Costa PT, BSc. PT, MSc PT, PhD 28

Department of Physical Therapy 29

Nicole Wertheim College of Nursing & Health Science 30

Florida International University 31

Miami, USA 32

[email protected] 33

34

35

Jorge Fuentes BPT, MSc PhD 36 Faculty of Rehabilitation Medicine 37


Edmonton, Alberta. Canada 39

Email: [email protected] 40

Catholic University of Maule 41


Talca, Chile 43

44

Christine Ha, BSc 45 Rehabilitation Research Center 46

Faculty of Rehabilitation Medicine, 47

3-62 Corbett Hall, 48

University of Alberta, 49

Edmonton, Canada T6G 2G4 50

Page 1 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on March 26, 2020 by guest. P

rotected by copyright.http://bm

jopen.bmj.com

/B

MJ O

pen: first published as 10.1136/bmjopen-2015-008562 on 3 S

eptember 2015. D

ownloaded from






52

53

Greta G. Cummings, RN PhD FCAHS 54 Principal, CLEAR Outcomes (Connecting Leadership Education & Research) Research 55

Program 56

Centennial Professor, Faculty of Nursing, University of Alberta 57

Edmonton Clinic Health Academy | University of Alberta 58

11405-87 Ave, Edmonton, AB | Canada T6G 1C9 59


61

62

Corresponding author: 63 Susan Armijo-Olivo, BSc.PT, MSc PT, PhD 64

Postdoctoral Fellow 65




11405 – 87 Avenue 69






76

77

Correspondence: 78

Name Susan Armijo-Olivo, BSc. PT, MSc PT, PhD

Department Faculty of Rehabilitation Medicine,

3-48 Corbett Hall, Department of Physical Therapy

Institution University of Alberta,

Country Canada T6G 2G4

Tel 780-437-3521

Mob --

Fax 780-492-1626

Email [email protected] / [email protected]

79

80

Abbreviated title: Allocation concealment and randomization in physical therapy 81

Key words: Physical therapy; allocation concealment; randomization; meta-82

epidemiological approach; risk of bias 83 84

List of abbreviations: 85 RCTs: Randomized controlled trials 86

PT: Physical therapy 87

88

89

90

Page 2 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





Word Count: 275 words (Abstract) 91

3050 words (Introduction, Method, Results, and Discussion) 92

References: 58 93

Tables: 3 94

Figures: 6 95

Ethics approval: The University of Alberta Ethics Committee(s) approved this study. 96

Running head: Allocation concealment in physiotherapy 97 Study performed: CLEAR Outcomes (Connecting Leadership, Education and Research) 98

and Faculty of Rehabilitation Medicine, Department of Physical Therapy, University of 99

Alberta, Edmonton, Canada. 100

101

102

103 104

105

Page 3 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





ABSTRACT 106

Objective: to determine if adequacy of randomization and allocation concealment is 107

associated with changes in effect sizes when comparing physical therapy (PT) trials with and 108

without these methodological characteristics. 109

Design: Meta-epidemiological study. 110

Setting: NA 111

Participants: A random sample of randomized controlled trials (RCTs) included in meta-112

analyses in the PT discipline were identified. 113

Intervention: NA. Data extraction including assessments of random sequence generation and 114

allocation concealment was conducted independently by 2 reviewers. To determine the 115

association between sequence generation, and allocation concealment and effect sizes, a 2-116

level analysis was conducted using a meta-meta-analytic approach. 117

Primary and secondary outcome measures: association between random sequence 118

generation and allocation concealment and effect sizes in PT trials. 119

Results: 393 trials included in 43 meta-analyses, analyzing 44,622 patients contributed to this 120

study. Adequate random sequence generation and appropriate allocation concealment were 121

accomplished in only 39.7% and 11.5% of PT trials respectively. Although trials with 122

inappropriate allocation concealment tended to have an overestimate treatment effect when 123

compared with trials with adequate concealment of allocation, the difference was non-124

statistically significant (ES=0.12; 95%CI -0.06; 0.30). When pooling our results with those of 125

Nuesch et al., we obtained a pooled statistically significant value (ES= 0.14; 95%CI 0.02; 126

0.26). There was no difference in ES in trials with appropriate or inappropriate random 127

sequence generation (ES=0.02; 95%CI -0.12; 0.15). 128

Conclusion: Our results suggest that when evaluating risk of bias of primary RCTs in PT 129

area, systematic reviewers and clinicians implementing research into practice should pay 130

attention to these biases since they could exaggerate treatment effects. Systematic reviewers 131

Page 4 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





should perform sensitivity analysis including trials with low risk of bias in these domains as 132

primary analysis and/or in combination with less restrictive analyses. Authors and editors 133

should make sure that allocation concealment and random sequence generation are properly 134

reported in trial reports. (298 words) 135

136

Strengths and limitations of this study 137

Strengths 138

• As far of our knowledge, this is the first meta-epidemiological study using continuous 139

outcomes in allied health disciplines such as physical therapy (PT). 140

• This large meta-epidemiological study, which was based on 43 Cochrane reviews, 393 141

trials and 44,622 patients makes an important contribution to the field. 142

• This study provides with novel evidence regarding the association between adequacy 143

of randomization and allocation concealment methodological factors and treatment 144

estimates in PT. 145

Limitations 146

• We restricted our analysis to Cochrane systematic reviews in PT and results might not 147

be applicable to all Cochrane reviews or other reviews conducted in other areas of 148

research. 149

• For determining the association between random sequence generation and allocation 150

concealment, we limited our analysis to trials describing a true control group, or 151

placebo intervention, reducing our statistical power. 152

• It could be possible that other biases present in the studies interact with the 153

investigated biases and have an influence in the treatment effects. This should include 154

a multivariate analysis where several biases should be taken into consideration. Future 155

research should look into this. 156

157

Page 5 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





INTRODUCTION 158

Randomization and allocation concealment have been extensively investigated in the medical 159

literature as key methodological characteristics of randomized controlled trials (RCTs) in 160

medical research.1-3

Allocation concealment is the most commonly evaluated quality 161

characteristic in reviews of RCTs due to its crucial role as a key marker of internal validity of 162

RCT.4 Moreover, it is firmly established the conventional wisdom that adequate 163

randomization sequence generation is an essential part of a valid clinical trial.5 164

165

There has been an extensive body of empirical research looking at the influence of random 166

sequence generation and allocation concealment on treatment effect estimates of health RCTs. 167

Several meta-epidemiological studies investigating the association between trial 168

characteristics and treatment effects have found an association between inadequate 169

randomization and/or allocation concealment and an overestimation of treatment effects.1-3,6-9

170

For example, inadequate random sequence generation can overestimate treatment effects by 171

12% (ROR 0.88, 95% CI 0.79,0.99).10

Inadequate allocation concealment can overestimate 172

treatment effects on average by 18%.3,8,11 In addition, Pidal et al. found that 2/3 of the 173

conclusions from meta-analyses were no longer supported if only trials with adequate 174

allocation concealment were included 12

and that 69% of meta-analyses lost statistical 175

significance when trials with unclear or inadequate allocation concealment were excluded.12

176

Over-estimates or underestimates of treatment effects can lead to biased or inaccurate results 177

and conclusions in systematic reviews and meta-analyses.3,12-15

These factors can ultimately 178

have repercussions on decision-making and quality of patient care since different assessments 179

could lead to different decisions for clinical practice. 180

181

Page 6 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





Most of the empirical evidence regarding the relationship between trial components, and 182

specifically random sequence generation and allocation concealment, and treatment effect 183

estimates comes from RCTs in medicine and is based mainly on dichotomous outcomes.3,8,11

184

No such studies using continuous outcomes have been conducted in other health areas such as 185

the allied health professions, including PT. Furthermore, although it has been reported that 186

the quality of reporting of PT trials has improved over time,16

a finer analysis of types of 187

random sequence generation and allocation concealment in PT trials is lacking. 188

189

Therefore, it is necessary to determine the extent to which randomization and allocation 190

concealment affect treatment effect estimates in PT trials to provide accurate results to the PT 191

clinical community. This information is urgently needed to develop guidelines for designing, 192

conducting, and implementing PT trials as well as providing clear benchmarks to assess the 193

quality and/or risk of bias of PT trials in systematic reviews and meta-analyses and ultimately 194

the strength of evidence for decision-making in PT. Therefore, our research questions were: 1) 195

are random sequence generation and allocation concealment adequately used and reported in 196

RCTs of PT; 2) do random sequence generation and allocation concealment have an effect on 197

estimates of treatment in PT trials; and 3) do effects sizes on PT trials differ depending on 198

some characteristics of the meta-analyses analyzed such as magnitude of the effect size, meta-199

analysis heterogeneity, type of outcome (subjective or objective), and whether the meta-200

analysis involved the musculoskeletal area. 201

METHOD 202

Design 203

Meta-epidemiological approach 204

205

206

Page 7 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





Study selection 207

A random sample of Randomized Controlled Trials (RCTs) included in meta-analyses in the 208

Physical therapy (PT) discipline were identified by searching the Cochrane Database of 209

Systematic Reviews from Jan 2005 to May 25 2011 on PT interventions. The search strategy 210

can be found elsewhere and can be provided upon request.17,18

Meta-analyses and RCTs 211

included in these meta-analyses were included if they met the following eligibility criteria: 1) 212

the meta-analysis included at least 3 RCTs comparing at least two interventions, with at least 213

one of the interventions being part of PT scope of practice according to the World 214

Confederation for Physical Therapy (WCPT)19

; and 2) the main outcome or the outcome of 215

the largest meta-analysis conducted in the review was continuous. 216

217

Assessment of Random sequence generation and Allocation Concealment domains 218

Assessments of random sequence generation and allocation concealment domains were 219

performed by two independent reviewers following the Cochrane collaboration guidelines.20

220

221

Random sequence generation assessment 222

In addition to the general evaluation of adequacy of random sequence generation, different 223

methods of random sequence generation were determined. We grouped these methods into 4 224

categories as follows: Category 1 included trials where random sequence generation was 225

unclear or not reported; category 2 included trials that had adequate randomization (e.g. use 226

of a computer software, random number table, and minimization); category 3 included trials 227

using acceptable methods of randomization, but less efficient than the previous category (e.g., 228

drawing lots, envelopes, shuffling cards, throwing a dice); category 4 involved trials used 229

inappropriate methods of sequence generation (e.g., date of birth, day of admission, hospital 230

record number). Categorization into one of these four categories was conducted in duplicate. 231

Page 8 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





To facilitate comparison with previous meta-analyses, these four categories were combined to 232

create two main groups: an "adequate sequence generation" group (combining categories 2 233

and 3) and an "inadequate or unclear sequence generation" group (combining categories 1 and 234

4). 235

236

Allocation Concealment assessment 237

As for allocation concealment assessment, we divided the methods of allocation concealment 238

used in the trials into the following categories: Category 1 comprised trials that used any type 239

of central randomization that (e.g., a remote telephone service or a central office); category 2 240

comprised trials that used sequentially numbered, opaque and sealed envelopes; category 3 241

comprised trials that used sealed envelopes without reporting any further details; and 242

category 4 comprised trials where allocation was clearly not hidden (e.g., being based on an 243

open list, odd or even days of the week, participant’s birth date, or the team on duty at 244

enrolment); and category 5 comprised trials were concealment of allocation was not reported 245

or unclear. Categorization into one of these five categories was conducted by two independent 246

assessors. To facilitate comparison with previous meta-analyses, these five categories were 247

combined to create two main groups: an "adequate concealment" group (combining categories 248

1 and 2) and an "inadequate or unclear concealment" group (combining categories 3, 4, and 249

5). 250

251

Data extraction of treatment estimates and trial characteristics 252

Two independent reviewers extracted specific data (e.g. type of interventions, type of 253

outcomes (i.e. objective, subjective), PT area) for all trials included in the meta-analyses as 254

well as data on means, standard deviations, and sample sizes. The primary outcome chosen 255

for the analysis was the main outcome of interest reported in the review or determined from 256

Page 9 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





the meta-analysis that contained the largest number of trials in the review. Details on the 257

reviewers’ panel and training process can be found elsewhere.17,18

258

259

Data Analysis 260

Data on sequence generation and allocation concealment were analyzed descriptively based 261

on the categories described previously. In order to determine whether random sequence 262

generation and allocation concealment domains affect treatment effect estimates, a 2-level 263

analysis was conducted using a meta-meta-analytic approach with a random-effects model to 264

allow for within and between meta-analyses heterogeneity as suggested by Sterne.21

265

We additionally stratified analyses accompanied by interaction tests according to the pre-266

specified characteristics: 1) treatment benefit in overall meta-analysis: small [ES greater that -267

0.5] versus large [ES ≤ to -0.5]; between-trial heterogeneity in overall meta-analysis (low [τ2 268

<0.06] versus high [τ2 ≥0.06]), nature of the outcome (subjective or objective) and if the 269

intervention was classified as musculoskeletal or other PT area. 270

271

In order to evaluate the effect of random sequence generation and allocation concealment on 272

treatment effect estimates, we limited the analyses to studies describing a true control group, 273

or placebo intervention as well as studies in which the direction of the expected treatment 274

effect was evident (i.e. standard care vs. standard care plus active intervention; and active 275

intervention 1 plus active intervention 2 vs. active intervention 1). 276

277

Finally, results of our analyses were pooled with results from the meta-epidemiological study 278

performed by Nuesch et al.1, which also investigated the effect of concealment of allocation 279

on treatment effects on osteoarthritis measured on a continuous scale. STATA statistical 280

software version 12 was used to perform these analyses. 281

Page 10 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





RESULTS 282

Selection and characteristics of meta-analyses and RCTs 283

The search identified 3901 Cochrane reviews, with 271 reviews potentially relevant to PT. Of 284

these, 68 reviews included a meta-analysis of at least three studies of PT interventions and 285

used a continuous outcome. We randomly selected 44 meta-analyses but excluded one 22

286

because it used follow-up data from the same group rather than a control group for 287

comparison (Figure 1). Forty-three meta-analyses including 393 trials and analyzing 44,622 288

patients contributed to this study. Table 1 summarizes the characteristics of the 43 Cochrane 289

reviews. Briefly, the reviews were published between 2008 and 2011 and included meta-290

analyses of the effectiveness of PT interventions for musculoskeletal (22 reviews)23-31

, 291

cardiorespiratory (9 reviews)32-40

, neurological (6 reviews)41-47

, and other areas of physical 292

therapy (6 reviews).47-52

A median number of 6 trials were included in the meta-analyses 293

(interquartile range 5-8). Most trials were parallel group trials (367; 93.4%), single-center 294

studies (298; 76%) and had active control interventions (362; 92%). The most common 295

intervention was exercise (n=282, 71.8%). Supplementary Table S1 (Appendix 1) lists the 296

characteristics of each of the 43 meta-analyses. 297

Sequence generation descriptive 298

From the 393 trials included in the 43 meta-analyses, 156 trials (39.7%) had appropriate 299

sequence generation and 237 (60.3%) had inappropriate sequence generation (229 were 300

classified as “unclear” and 8 as “high” risk of bias) according to the Cochrane Collaboration 301

guidelines. 302

When analyzing the sequence generation categories, we found that 229 trials (58.27%) did not 303

clearly report the mechanism of how the sequence was generated. One hundred and thirty two 304

trials (33.6%) generated the sequence through computer software, a table of random numbers, 305

or by minimization method. For a more specific sequence generation methods description, see 306

Page 11 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





Table 2. 307

In addition, simple randomization was reported in 34 trials, block randomization in 54 trials, 308

stratification in 62 trials, and unclear or not reported in 218 trials. The rest of the trials 309

reported other methods of random sequence generation. 310

311

Allocation Concealment descriptive 312

From the 43 meta-analysis and 393 trials, 45 trials (11.5%) had appropriate allocation 313

concealment and 348 (88.6%) had inappropriate concealment of allocation according to the 314

Cochrane Collaboration guidelines. Results of analysis of the allocation concealment within 315

each category were as follows: 282 trials (71.8%) had unclear method of allocation 316

concealment, 21 (5.34%) trials used central allocation, 24 (6.11%) trials used envelopes with 317

all 3 safe wards (opaque, sealed, and sequentially numbered), 15 (3.82%) trials used 318

envelopes without safeguards, and 51 (16.8%) trials used any other non-concealed methods. 319

Further details on allocation concealment methods are displayed in Table 3. 320

321

Sequence generation and treatment effects in PT trials 322

For the purpose of analyzing the effect of sequence generation on treatment effects, 22 meta-323

analyses including 257 trials and analyzing 30,287 patients contributed to this analysis. 324

Figure 2 shows the forest plot of the differences in effect sizes between trial with adequate 325

and inadequate random sequence generation. There was no statistically significant difference 326

between the ES of trials with adequate or inadequate random sequence generation (ES=0.02; 327

95%CI -0.12; 0.15). 328

The results of the stratified analyses are displayed in Figure 3. None of the meta-analyses 329

characteristics had a statistically significant interaction. 330

331

Page 12 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





Allocation Concealment and treatment effects in PT trials 332

For the purpose of analyzing the effect of allocation concealment on treatment effects, 17 333

meta-analyses including 198 trials and analyzing 27,011 patients contributed to this analysis. 334


and inadequate allocation concealment. Although trials with inappropriate allocation 336

concealment tended to have an overestimate treatment effect when compared with trials with 337

adequate concealment of allocation, the difference was non-statistically significant (ES=0.12; 338

95%CI -0.06; 0.30). The results of the stratified analyses are displayed in Figure 5. None of 339

the meta-analyses characteristics had a statistically significant interaction. 340

341

Pooling results with a previous meta-epidemiological study 1 342

When pooling our results with those of Nuesch et al.1, who performed a meta-epidemiological 343

study with continuous outcomes as well, we obtained a pooled statistically significant value 344

(ES= 0.14; 95%CI 0.02; 0.26), meaning that trials with inappropriate concealment of 345

allocation had more beneficial effect than trials with appropriate concealment of allocation 346

(Figure 6). 347

348

DISCUSSION 349

Our findings showed that adequate allocation concealment and appropriate allocation 350

concealment were only accomplished in a low percentage of PT trials (39.7% and 11.5%, 351

respectively) despite the fact that both of the above-mentioned quality domains can be always 352

appropriately performed when conducting an RCT in any field. In addition, trials with 353

inappropriate concealment of allocation had an overestimation of treatment effects when 354

compared with trials with adequate concealment of allocation. However, no difference in 355

treatment effects in trials with appropriate or inappropriate sequence generation was found. 356

Page 13 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





These results confirm previous results obtained by several meta-epidemiological studies 357

investigating the influence of sequence generation and allocation concealment on several 358

areas of medicine using dichotomous outcomes.7,8,11,12,53

In addition, the pooled estimate 359

obtained from our study and that of Nuesch et al.,1 indicated that trials with inappropriate 360

allocation concealment had a more beneficial effect than those with appropriate. 361

These results have important implications for the research community in general as well as for 362

the discipline of PT. To our knowledge; this study is the first of its kind conducted in PT that 363

examined continuous outcomes. Thus, it provides novel evidence in a very specific area of 364

health. Most of the previous studies were looking at medical areas, such as pregnancy and 365

childbirth, circulatory conditions, infectious disease, surgery, and mental health trials. All of 366

these medical areas certainly differ from PT with respect to: type of intervention (being 367

mostly drugs), type of outcomes used, and specific area of study. 368

369

First, it is not surprising that a large number of trials did not clearly report either 370

randomization sequence or allocation concealment. Our findings showed that approximately 371

62% and 70% of studies were classified as ‘unclear/not reported’ in their reporting of 372

sequence generation and allocation concealment, respectively. This has also been observed in 373

a meta-epidemiological study investigating meta-analyses before 2005.2 The study found a 374

64% and 69% of “unclear” reporting of sequence generation and allocation concealment, 375

respectively.2 Thus, despite the efforts made to improve reporting of RCTs such as 376

introducing the CONSORT Statement, there is still inappropriate reporting and 377

implementation of these important characteristics in RCTs. This is in agreement with results 378

reported by Moseley et al., 16

who found little or no effect of the CONSORT Statement on the 379

quality of reports of physiotherapy trials. Quality of reporting has always been an issue when 380

trying to understand the association between trial characteristics and treatments effects since 381

Page 14 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





most of authors do not properly report their methods. Although inappropriate reporting does 382

not always reflect on weak trial conduct, 54

some studies such as Pildal et al.,12

found that 383

most trials with unclear allocation concealment had also unclear allocation concealment in 384

their protocol. Therefore, quality of reporting is related to assessment of risk of bias, but the 385

direction of this relationship is still unclear. Future studies should assess how, how much, and 386

in which direction trial reporting affects treatment effects. 387

388

The magnitude and direction of the effect found in our study is consistent and almost identical 389

with a previously published study performed in osteoarthritic trials investigating allocation 390

concealment. In our study, the average bias associated with lack of allocation concealment 391

corresponds to ¼ or ½ of a typical treatment effect found in PT interventions.55-58

As stated by 392

others,2,8

if allocation concealment is not accomplished, it is more likely that the results of the 393

trial are inaccurate because, intentionally or unintentionally, investigators could have 394

interfered with assigning subjects to different groups, favoring the effect of the intervention. 395

In addition, allocation concealment could be a proxy measure for other aspects of trial design 396

besides selection bias.2,8

397

398

Random sequence generation was found not to be significantly associated with distortion of 399

treatment effects in this study. Thus, it appears that random sequence generation does not 400

have such a strong influence on treatment effects in comparison with concealment of 401

allocation. Other meta-epidemiological studies have found similar results. 8,11,59

One of these 402

studies, performed by Balk et al., 59

has been questioned because of the high heterogeneity 403

between trials included in meta-analysis. This could introduce a higher noise to be able to 404

detect any effect of the trials characteristics. The other studies reported the same results argue 405

that sequence generation is still an important characteristic that ensures rigor in the trial by 406

Page 15 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





allowing a trial to be concealed.8,11

In addition, other authors have suggested that the 407

relationship between certain methodological characteristics and treatment effects varies 408

according to each health area 59,60

and more work should be done in other health areas to 409

confirm these results.59,60

410

411

Our stratified analysis for sequence generation and allocation concealment showed that none 412

of the meta-analysis characteristics presented with significant interaction. This is somehow 413

contrary to other meta-epidemiological studies. For example, Wood et al.,3 and Savovic et 414

al.,2 found that there was an overestimation of treatment effects that ranged between 22%-415

31% for subjective outcomes. We followed exactly the same definition for determining 416

outcomes as “objective and subjective” used by Wood et al., 3 to facilitate comparisons and 417

avoid misclassifications. Thus, it seems that for PT area, there is no evidence of bias 418

regarding random sequence generation or allocation concealment when subjective or 419

objective outcome are used. This could be attributed to the fact that both domains can be 420

performed in a trial regardless of the outcome analyzed. Thus, whether the outcome is 421

objective or subjective does not affect whether it is easier or not to predict patient’s prognosis 422

at the time of recruitment. 423

424

Study limitations 425

We analyzed the influence of random sequence generation and allocation concealment on 426

treatment effect estimates. It could be possible that other biases interact with these biases and 427

have an influence in the treatment effects. This should include a multivariate analysis where 428

several biases should be taken into consideration. Future research should look into this. 429

We limited our analysis to trials describing a true control group, or placebo intervention as 430

well as those studies in which the direction of expected treatment effect was evident. In this 431

Page 16 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





way, we could anticipate the direction of the bias in analyses. Combining effects of all trials 432

without clear direction of what would be the effect (i.e. in case of 2 similar active 433

interventions) would have increased noise in the analyses and heterogeneity limiting our 434

ability to find any significant effect of trials characteristics.61

However, limiting our analyses 435

to these trials, we also reduced the power of our study. 436

437

CONCLUSIONS 438

Trials with inappropriate concealment of allocation had an overestimation of treatment effects 439

when compared with trials with adequate concealment of allocation. Our results suggest that 440

when evaluating risk of bias of primary RCTs in PT area, systematic reviewers and clinicians 441

implementing trial findings into their clinical practice should pay attention to these 442

characteristics, especially allocation concealment since it can be associated with an 443

exaggeration of a treatment effect. Systematic reviewers should perform sensitivity analysis 444

including trials with low risk of bias in these domains as primary analysis and/or in 445

combination with less restrictive analyses. Authors and editors should make sure that 446

allocation concealment and random sequence generation are properly reported in trial reports. 447

(3160 words) 448

449

450

451

452

453

454

455

456

Page 17 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





457

Acknowledgements: This project is funded the Canadian Institutes of Health Research 458

(CIHR), Alberta Innovates Health Solutions through a knowledge translation initiative grant, 459

the Knowledge Translation (KT) Canada research Stipend program, and the Physiotherapy 460

Foundation of Canada (PFC) through a B.E. Schnurr Memorial Fund Award. 461

The funding bodies had no input in the design, collection, analysis, and interpretation of data; 462

in the writing of the manuscript; and in the decision to submit the manuscript for publication. 463

This Research protocol has been approved by the Ethics Board of the University of Alberta 464

(Pro00038172). 465

Dr. Susan Armijo-Olivo is supported by the Canadian Institutes of Health Research (CIHR) 466

through a full-time Banting fellowship, the Alberta Innovates Health Solution through an 467

incentive award, the STIHR Training Program from Knowledge Translation (KT) Canada, the 468

University of Alberta, and by the Faculty of rehabilitation Medicine, at the University of 469

Alberta, through the “Music and Motion Fellowship”. 470

Dr. Greta Cummings is holds a Centennial Professorship at the University of Alberta (2013-471

2020). 472

Dr. Fuentes is supported by the Government of Chile, University of Alberta through a 473

Dissertation fellowship, and the University Catholic of Maule. 474

Dr. Humam Saltaji is supported through a Clinician Fellowship Award by Alberta Innovates - 475

Health Solutions (AIHS), the Honorary Izaak Walton Killam Memorial Scholarship by the 476

University of Alberta, and the Honorary WCHRI Award by the Women and Children’s 477

Health Research Institute (WCHRI). 478

Competing Interest 479

The author(s) declare that they have no competing interests. 480

481

482

Page 18 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





Ethics approval: The University of Alberta Ethics Committee(s) approved this study 483

484

Authors’ contributions 485

486 SAO provided concept/idea/research design and writing. Performed the experiments: SAO, 487

CH, JF, HS. Interpretation of the results and review of the final manuscript: SAO, BRdaC, 488

GGC, CH, JF, HS. Analyzed the data: SAO, BRdaC. Wrote the paper: All authors critically 489

revised and provided final approval of the version to be published. 490

491

Justification that this study does not require a checklist: 492

This study is considered a meta-epidemiological research study. We feel that none of the 493

checklists is applicable for this type of study. 494

495

Figure Legends 496

Figure 1. Diagram for identification of studies 497

Figure 2. Forest plot of the Differences in Effect sizes between Trials with and without 498

Adequate Sequence Generation 499

Figure 3. Forest Plot of the Differences in Effect sizes between Trials with and without 500

Sequence Generation Stratified by Meta-analyses characteristics (Effect size Magnitude, 501

Heterogeneity, Type outcome, and PT area). 502

503


Adequate Concealment of Allocation 505


Adequate Concealment of Allocation Stratified by Meta-analyses characteristics (Effect 507

size Magnitude, Heterogeneity, Type outcome, and PT area) 508

Figure 6. Pooled Data of the Effect of Concealment of Allocation on Treatment Effect 509

Estimates using Continuous Outcomes 510

511

512

513

514

515

Page 19 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





REFERENCES 516 517

518

1. Nuesch E, Reichenbach S, Trelle S, et al. The importance of allocation concealment 519

and patient blinding in osteoarthritis trials: A meta-epidemiologic study. Arthritis 520

Care and Research. 2009;61(12):1633-1641. 521

2. Savovic J, Jones HE, Altman DG, et al. Influence of reported study design 522

characteristics on intervention effect estimates from randomized, controlled trials. 523

Annals of Internal Medicine. 2012;157(6):429-438. 524

3. Wood L, Egger M, Gluud LL, et al. Empirical evidence of bias in treatment effect 525

estimates in controlled trials with different interventions and outcomes: Meta-526

epidemiological study. BMJ. 2008;336(7644):601-605. 527

4. Falagas ME, Grigori T, Ioannidou E. A systematic review of trends in the 528

methodological quality of randomized controlled trials in various research fields. 529

Journal of Clinical Epidemiology. 2009;62(3):227-231.e229. 530

5. Aickin M. Randomization, balance, and the validity and efficiency of design-adaptive 531

allocation methods. Journal of Statistical Planning and Inference. 2001;94(1):97-119. 532

6. Bausell RB, Lee WL, Soeken KL, Li YF, Berman BM. Larger effect sizes were 533

associated with higher quality ratings in complementary and alternative medicine 534

randomized controlled trials. Journal of Clinical Epidemiology. May 2004;57(5):438-535

446. 536

7. Kjaergard LL, Villumsen J, Gluud C. Reported methodologic quality and 537

discrepancies between large and small randomized trials in meta-analyses. Annals of 538

Internal Medicine. 2001;135(11):982-989. 539

8. Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias: 540

Dimensions of methodological quality associated with estimates of treatment effects in 541

controlled trials. Journal of the American Medical Association. 1995;273(5):408-412. 542

9. Siersma V, Als-Nielsen B, Chen W, Hilden J, Gluud LL, Gluud C. Multivariable 543

modelling for meta-epidemiological assessment of the association between trial 544

quality and treatment effects estimated in randomized clinical trials. Statistics in 545

Medicine. 2007;26(14):2745-2758. 546

10. Als-Nielsen B, Gluud L, Gluud C. Methodological quality and treatment effects in 547

randomised trials: a review of six empirical studies. Paper presented at: 12th Cochrane 548

Colloquium 2004 2004; Oct 2-6; Ottawa, Ontario, Canada. . 549

11. Moher D, Pham B, Jones A, et al. Does quality of reports of randomised trials affect 550

estimates of intervention efficacy reported in meta-analyses?[see comment]. Lancet. 551

Aug 22 1998;352(9128):609-613. 552

12. Pildal J, Hrobjartsson A, Jorgensen KJ, Hilden J, Altman DG, Gotzsche PC. Impact of 553

allocation concealment on conclusions drawn from meta-analyses of randomized 554

trials. International Journal of Epidemiology. 2007;36(4):847-857. 555

13. Hewitt CE, Kumaravel B, Dumville JC, Torgerson DJ. Assessing the impact of 556

attrition in randomized controlled trials. Journal of Clinical Epidemiology. 557

2010;63(11):1264-1270. 558

14. Kjaergard LL, Als-Nielsen B. Association between competing interests and authors' 559

conclusions: Epidemiological study of randomised clinical trials published in the 560

BMJ. British Medical Journal. 2002;325(7358):249-252. 561

15. Trowman R, Dumville JC, Torgerson DJ, Cranny G. The impact of trial baseline 562

imbalances should be considered in systematic reviews: a methodological case study. 563

Journal of Clinical Epidemiology. 2007;60(12):1229-1233. 564

Page 20 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





16. Moseley AM, Herbert RD, Maher CG, Sherrington C, Elkins MR. Reported quality of 565

randomized controlled trials of physiotherapy interventions has improved over time. 566


17. Armijo-Olivo S, Cummings G, Fuentes CJ, et al. Identifying Items to Assess 568

Methodological Quality In Physical Therapy Trials: A Factor Analysis. Physical 569

Therapy. 2014;94 (9):1272-1284. 570

18. da Costa B, Resta N, Beckett B, et al. Effect of standardized training on the reliability 571

of the Cochrane risk of bias assessment tool: a study protocol. Systematic Reviews. 572

2014/12/13 2014;3(1):1-6. 573

19. World Confederation for Physical Therapy. Position statement: standards of physical 574

therapy practice. World Confederation for Physical Therapy;2011. 575

20. Higgins JPT, Altman DG, Goetzsche PC, et al. The Cochrane Collaboration's tool for 576

assessing risk of bias in randomised trials. BMJ. 2011;343(7829). 577

21. Sterne JAC, Juni P, Schulz KF, Altman DG, Bartlett C, Egger M. Statistical methods 578

for assessing the influence of study characteristics on treatment effects in 'meta-579

epidemiological' research. Statistics in Medicine. 2002;21(11):1513-1524. 580

22. Spittle A, Orton J, Doyle LW, Boyd R. Early developmental intervention programs 581

post hospital discharge to prevent motor and cognitive impairments in preterm infants. 582

Database of Systematic.Reviews.Issue:2. John.Wiley & Sons, Ltd. Chichester. 2007. 583

23. Fransen M, McConnell S, Hernandez MG, Reichenbach S. Exercise for osteoarthritis 584

of the hip. Cochrane Database of Systematic Reviews: Issue 3. John.Wiley.& Sons., 585

Ltd. Chichester. 2009. 586

24. Handoll-Helen HG, Cameron ID, Mak-Jenson CS, Finnegan TP. Multidisciplinary 587

rehabilitation for older people with hip fractures. Cochrane Database of Systematic 588

Reviews. Issue 4. John.Wiley.& Sons., Ltd.Chichester. 2009 2009. 589

25. Harvey LA, Brosseau L, Herbert RD. Continuous passive motion following total knee 590

arthroplasty in people with arthritis. Cochrane Database of Systematic Reviews. Issue 591

3. John.Wiley.& Sons., Ltd.Chichester. 2010 2010. 592

26. Katalinic OM, Harvey LA, Herbert RD, Moseley AM, Lannin NA, Schurr K. Stretch 593

for the treatment and prevention of contractures. Cochrane Database of Systematic 594


27. Manheimer E, Cheng K, Linde K, et al. Acupuncture for peripheral joint osteoarthritis. 596

Cochrane Database of Systematic Reviews. Issue 1. John Wiley& Sons, Ltd. 597

Chichester, UK. 2010 2010. 598

28. Ostelo-Raymond WJG, Costa-Leonardo OP, Maher CG, de-Vet-Henrica CW, van-599

Tulder MW. Rehabilitation after lumbar disc surgery. Cochrane Database of 600

Systematic Reviews. Issue 4. John Wiley & Sons., Ltd.Chichester, UK. 2008. 601

29. Rutjes-Anne WS, Nüesch E, Sterchi R, Jüni P. Therapeutic ultrasound for 602

osteoarthritis of the knee or hip. Cochrane Database of Systematic Reviews. Issue 1. 603

John Wiley & Sons., Ltd.Chichester. 2010. 604

30. Rutjes-Anne WS, Nüesch E, Sterchi R, et al. Transcutaneous electrostimulation for 605

osteoarthritis of the knee. Cochrane Database of Systematic Reviews. Issue 4. John 606

Wiley & Sons., Ltd. Chichester. 2009. 607

31. Schaafsma F, Schonstein E, Whelan KM, Ulvestad E, Kenny DT, Verbeek JH. 608

Physical conditioning programs for improving work outcomes in workers with back 609

pain. Cochrane Database of Systematic Reviews. Issue 1. John Wiley & Sons, Ltd 610

Chichester, UK. 2010. 611

32. Davies P, Taylor F, Beswick A, et al. Promoting patient uptake and adherence in 612

cardiac rehabilitation. Cochrane Database of Systematic Reviews. Issue 7. John Wiley 613

& Sons Ltd. Chichester, UK. 2010. 614

Page 21 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





33. Effing T, Monninkhof-Evelyn EM, Valk-Paul PDLP, et al. Self-management 615

education for patients with chronic obstructive pulmonary disease. Cochrane 616

Database of Systematic Reviews. Issue 4. John Wiley & Sons Ltd. Chichester, UK. 617

2007. 618

34. Puhan MA, Gimeno SE, Scharplatz M, Troosters T, Walters EH, Steurer J. Pulmonary 619

rehabilitation following exacerbations of chronic obstructive pulmonary disease. 620

Cochrane Database of Systematic Reviews. Issue 1. John Wiley & Sons Ltd. 621


35. Taylor RS, Dalal H, Jolly K, Moxham T, Zawada A. Home-based versus centre-based 623


& Sons., Ltd.Chichester, UK. 2010 2010. 625

36. Main E, Prasad A, van der Schans CP. Conventional chest physiotherapy compared to 626

other airway clearance techniques for cystic fibrosis. Cochrane.Database.of 627

Systematic.Reviews.Issue.1 John Wiley & Sons., Ltd.Chichester, UK. 2005. 628

37. O'Brien K, Nixon S, Tynan AM, Glazier R. Aerobic exercise interventions for adults 629

living with HIV/AIDS. Cochrane.Database.of Systematic.Reviews.Issue.8.John Wiley 630


38. Jolliffe J, Rees K, Taylor-Rod RS, Thompson DR, Oldridge N, Ebrahim S. Exercise-632

based rehabilitation for coronary heart disease. Cochrane.Database.of 633

Systematic.Reviews.Issue.1 John Wiley & Sons., Ltd.Chichester, UK. 2001 2001. 634

39. Shaw KA, Gennat HC, O'Rourke P, Del MC. Exercise for overweight or obesity. 635

Cochrane.Database.of Systematic.Reviews.Issue.4 John Wiley & Sons., 636

Ltd.Chichester, UK. 2006. 637

40. Lacasse Y, Goldstein R, Lasserson TJ, Martin S. Pulmonary rehabilitation for chronic 638

obstructive pulmonary disease. Cochrane Database of Systematic Reviews. Issue.3. 639

John Wiley & Sons., Ltd.Chichester, UK. 2009. 640

41. Sirtori V, Corbetta D, Moja L, Gatti R. Constraint-induced movement therapy for 641

upper extremities in stroke patients. Cochrane Database of Systematic Reviews. Issue 642

4. John Wiley & Sons., Ltd.Chichester, UK. 2009. 643

42. States RA, Pappas E, Salem Y. Overground physical therapy gait training for chronic 644

stroke patients with mobility deficits. Cochrane Database of Systematic Reviews. 645

Issue 3.John Wiley & Sons. Ltd.Chichester, UK. 2009. 646

43. Mead GE, Morley W, Campbell P, Greig CA, McMurdo M, Lawlor DA. Exercise for 647

depression. Mead Gillian.E., Morley.Wendy., Campbell.Paul, Greig.Carolyn.A, 648

McMurdo.Marion., Lawlor.Debbie.A.Exercise.for.depression.Cochrane.Database.of 649

Systematic.Reviews.: Reviews.2009.Issue.3 John.Wiley.& Sons., Ltd.Chichester, UK 650

DOI.: 10.1002./14651858.CD0043. 2009 2009. 651

44. Pollock A, Baer G, Pomeroy VM, Langhorne P. Physiotherapy treatment approaches 652

for the recovery of postural control and lower limb function following stroke. 653

Cochrane Database of Systematic Reviews. Issue.1 John Wiley& Sons Ltd. Chichester, 654

UK. 655

. 2007. 656

45. Mehrholz J, Friis R, Kugler J, Twork S, Storch A, Pohl M. Treadmill training for 657

patients with Parkinson's disease. Cochrane Database of Systematic Reviews.Issue.1 658

John Wiley& Sons., Ltd.Chichester, UK. 2010 2010. 659

46. Woodford HJ, Price C, I. EMG biofeedback for the recovery of motor function after 660

stroke. Cochrane.Database.of Systematic.Reviews. Issue.2 John Wiley & Sons., 661

Ltd.Chichester, UK. 2007 2007. 662

Page 22 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





47. Watson L, Ellis B, Leng GC. Exercise for intermittent claudication. 663



48. Kramer MS, McDonald SW. Aerobic exercise for women during pregnancy. 666

Cochrane.Database.of Systematic.Reviews. Issue.3 John Wiley & Sons., 667


49. Orozco LJ, Buchleitner AM, Gimenez PG, Figuls M, Richter B, Mauricio D. Exercise 669

or exercise and diet for preventing type 2 diabetes mellitus. Cochrane Database of 670

Systematic Reviews. Issue 3. John Wiley & Sons Ltd. Chichester, UK. 2008. 671

50. Bendermacher-Bianca LW, Willigendael EM, Teijink-Joep AW, Prins MH. 672

Supervised exercise therapy versus non-supervised exercise therapy for intermittent 673

claudication. Cochrane.Database.of Systematic.Reviews. Issue.2 John.Wiley & Sons., 674


51. Forster A, Lambley R, Hardy J, et al. Rehabilitation for older people in long-term care. 676

Cochrane.Database.of Systematic.Reviews.Issue.1 John.Wiley.& Sons., 677


52. Cramp F, Daniel J. Exercise for the management of cancer-related fatigue in adults. 679

Cochrane Database of Systematic Reviews.Issue 2. John Wiley & Sons., 680


53. Egger M, Juni P, Bartlett C, Holenstein F, Sterne J. How important are comprehensive 682

literature searches and the assessment of trial quality in systematic reviews? Empirical 683

study. Health technology assessment. 2003;7(1):1-76. 684

54. Soares HP, Daniels S, Kumar A, et al. Bad reporting does not mean bad methods for 685

randomised trials: Observational study of randomised controlled trials performed by 686

the Radiation Therapy Oncology Group. British Medical Journal. 2004;328(7430):22-687

24. 688

55. Abdul Latif L, Daud Amadera JE, Pimentel D, Pimentel T, Fregni F. Sample size 689

calculation in physical medicine and rehabilitation: A systematic review of reporting, 690

characteristics, and results in randomized controlled trials. Arch Phys Med Rehabil. 691

2011;92(2):306-315. 692

56. Dorstyn D, Mathias J, Denson L. Applications of telecounselling in spinal cord injury 693

rehabilitation: A systematic review with effect sizes. Clinical Rehabilitation. 694

2013;27(12):1072-1083. 695

57. Barker AL, Talevski J, Morello RT, Brand CA, Rahmann AE, Urquhart DM. 696

Effectiveness of aquatic exercise for musculoskeletal conditions: A meta-analysis. 697

Arch Phys Med Rehabil. 2014;95(9):1776-1786. 698

58. Pollock A, Baer G, Langhorne P, Pomeroy V. Physiotherapy treatment approaches for 699

the recovery of postural control and lower limb function following stroke: A 700

systematic review. Clinical Rehabilitation. 2007;21(5):395-410. 701

59. Balk EM, Bonis PAL, Moskowitz H, et al. Correlation of quality measures with 702

estimates of treatment effect in meta-analyses of randomized controlled trials. Journal 703

of the American Medical Association. 2002;287(22):2973-2982. 704

60. Verhagen AP, de Vet HCW, Willemsen S, Stijnen T. A meta-regression analysis 705

shows no impact of design characteristics on outcome in trials on tension-type 706

headaches. Journal of Clinical Epidemiology. 2008;61(8):813-818. 707

61. Hempel S. Detection of associations between trial quality and effect sizes. 2012; 708

http://www.ncbi.nlm.nih.gov/books/NBK84249/. 709

710 711

712

Page 23 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





713

714

715

716

717

718

719

720

721

722

723

724

725

726

Table 1. Characteristics of the Selected Meta-analysis within Physical Therapy Areas 727 728

729

Musculoskeletal Cardio respiratory Neurology Other

Characteristics Number of meta-analyses 22 9 6 6 Median year of publication 2009 2010 2009 2009 Total number of trials included 194 78 52 69 Total number of patients included

19861 9015 2138 13608

Main intervention Exercise 13 7 3 5 Physical Agents 1 0 1 0 Acupuncture 2 0 0 0 Manual Therapy 1 0 0 0 Other 1 2 2 1 Outcomes Clinician assessed outcome 8 4 6 3 Self-reported outcome 11 4 0 2 Administrative data or automated outcome

3 1 0 1

730

731

732

733

Table 2. Methods of Sequence Generation in Physical therapy Trials 734

735

Sequence Generation Methods Number of trials (%)

Unclear/ Not reported 232 (59.03%)

Computer Software 65 (16.54%)

Random Number table 60 (13.49%)

Drawing of lots 12 (3.05%)

Shuffling cards or envelopes 9 (2.29%)

Minimization 6 (1.52%)

Other non-random methods 4 (4.33%)

Page 24 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





Coin Tossing 1 (0.25%)

Date of birth 1 (0.25%)

Day of admission 1 (0.25%)

Hospital/institution records number 1 (0.25%)

Throwing a dice 1 (0.25%)

Total 393 (100%)

736

737

738

739

740

741

742

Table 3. Methods of Allocation Concealment in Physical therapy Trials 743

744 Allocation Concealment Methods Number of trials (%)


Central Concealment 21 (5.34%)

Sequentially numbered, opaque and sealed envelopes

24 (6.11%)

Unsafe envelopes 15 (3.82%)

Non-safe methods of allocation 51 (12.98%)

Total 393 (100%)

745 746 747

748

Page 25 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





749 750

751 752

753

754

755

756

757

758

759

760

761

762 763 764 765

766 767 768

769


771

772

773

774

775

776

Total number of Meta-analysis (MA) identified from Cochrane Library search

relevant to physical therapy = 271 MAs

Total number Meta-analysis (MA) and Randomized Controlled Trials (RCT)

Selected from Cochrane Library search (Inclusion and Exclusion criteria)

78 MAs and 720 RCT

68 MAs (631 trials) had only continuous data and had > 3 trials in the main comparison

Total number of titles identified from Cochrane Library search =

3901

1 MA (8 trials) was excluded since it did not provide a control group

44 randomly selected MAs with 401 trials

43 MAs and 393 trials Finally Included

Page 26 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





Insert Figure 2 here 777

778


Adequate Sequence Generation 780 781

782 783

784

785

786

787

788

789

790

791

792

793

794

795

796

797

798

799

800

801

802

803

804

805

Page 27 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from







Heterogeneity, Type outcome, and PT area) 808

809 Figure 4. Forest plot of the Differences in Effect sizes between Trials with and without 810


812

813

Page 28 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





814




818

819 820



823

Page 29 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



58x44mm (300 x 300 DPI)

Page 30 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



56x42mm (300 x 300 DPI)

Page 31 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



52x39mm (300 x 300 DPI)

Page 32 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



58x43mm (300 x 300 DPI)

Page 33 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



54x41mm (300 x 300 DPI)

Page 34 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



1

Appendix 1. Characteristics of Selected Meta-analyses

Author Year # of trials

Pt in tx

Pt in control

# of Pt included

Outcome Outcome Source

Type of outcome

Area Specific Area

Pollock, A 2009 5 213 209 422 Functional

Independence Scale Clinician

assessment Ob Neuro Other

States, R 2009 6 195 186 381 Gait speed (m/s) (higher is better)

Clinician assessment

Ob Neuro Exercise

Schaafsma, F 2011 5 1057 Time to return to work (lower is better)

Administrative data

Ob MSK Exercise

Markes, M 2009 4 127 80 207 CR fitness (12 min

walking test), higher is better


Ob MSK Exercise

McNeely, M 2010 6 166 158 324 Shoulder flexion ROM

in degrees Clinician

assessment Ob MSK Exercise

Cramp, F 2010 27 1662 742 940 Fatigue Self-report Sub Other Exercise

Main, E 2010 7 201 Pulmonary function

(FEV1) Clinician

assessment Ob CR Chest PT

Davies, E 2010 9 1554 1555 3109 Health-related QoL Self-report Sub CR Exercise

Busch, A 2008 6 188 161 349 Tender points number

Clinician assessment (self-reported

pain at examination]

Sub MSK Exercise

Liu, C 2009 33 1076 1096 2172 Main function measure (higher is better)

Self-report (clinician

assessment] Sub MSK Exercise

Furlan, A 2011 3 188 168 356 Pain (higher is worse) Self-report Sub MSK Acupuncture

Fransen, M 2009 5 102 102 204 Pain (higher is worse) Self-report Sub MSK Exercise

Ostelo, R 2011 3 77 45 122 Pain (higher is worse) Self-report Sub MSK Exercise

Taylor, R 2010 12 817 740 1557 Exercise capacity 3-12 mo (higher is better)


Ob CR Exercise

Harvey, L 2010 8 193 186 379 Active knee flexion

(ROM) Clinician


Mead, GE 2010 23 476 431 907 Reduction of

depression symptoms post-tx (lower is better)


Sub Neuro Exercise

Edmonds, M 2010 5 143 143 286 Chalder fatigue scale (higher is worse)

Self-report Sub MSK, educa

Exercise

Page 35 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on March 26, 2020 by guest. Protected by copyright. http://bmjopen.bmj.com/ BMJ Open: first published as 10.1136/bmjopen-2015-008562 on 3 September 2015. Downloaded from



2

tion

Howe, TE 2008 5 140 164 304 Gait speed (20 min

walk test) (s), (lower is better)


Ob MSK Exercise

Fransen, M 2009 31 2074 1542 3616 Pain (higher is worse) Self-Report Sub MSK Exercise

Lin, CH 2008 8 513 Activity limitation

questionnaire (activity level) higher is better

Self-Report/performa

nce test Sub MSK

Exercise; other

Rutjes, AW 2010 5 177 143 320 Pain (higher is worse) Self-Report Sub MSK Exercise, other

Woodford, HJ 2009 5 47 44 91 Change in ROM (higher

is better) Clinician

Assessment Ob Neuro PA, exercise

Saunders, DH 2009 7 194 177 371 Gait speed (m/min) Clinician

Assessment Ob MSK Exercise

O'Brien , K 2010 5 145 131 276 Vo2 max (ml/kg/min) Clinician

Assessment Ob CR Exercise

Sirtoti, V 2009 6 96 88 184 Disability post-

intervention. (higher is better]


Sub Neuro Other

Hayden, J 2011 13 704 669 1373

Function measure (Oswestry low back pain disability, lower is

better)

Self-report Sub MSK Exercise, other

Orozco, LJ 2008 6 1668 1647 3315 Change in fasting

plasma glucose (mg/dl)

Laboratory measure

(physiological measures)

Ob Other Exercise, education

De Morton, N 2009 5 1621 1857 3478 Acute hospital length of

stay Administrative

data Ob MSK Other

Mehrholz, J 2010 7 76 77 153 Gait speed Clinician

assessment Ob Neuro Exercise

Shaw, K 2009 15 595 484 1079 Wt change in kilograms

(higher is worse) Clinician

assessment Ob CR Exercise

Handholl, H 2009 8 817 846 1663 Length of hospital stay Administrative

data Ob MSK Other

Effing, T 2009 6 381 317 698 HRQOL: SGRQ total (0-100, lower is better)

Self-report Sub

CR education, prevention

Education

Page 36 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




3

Bendermacher, B

2009 6 118 118 236 Maximal treadmill walking distance


Ob

Other (cardiovascular)

Exercise

Bonaiuti D, 2009 8 166 170 336 BMD Clinician


Foster, C 2009 17 4395 3203 7598

Change in self-reported physical activity

between BL and follow-up

Self-report Sub Other (GYN)

Other

Jolliffe, J 2009 8 610 588 1198 Total cholesterol Lab measure (physiological measures)

Ob CR Exercise

Katalinic, O 2010 7 96 97 193 Joint mobility (ROM) Clinician

assessment Ob MSK

Manual Therapy, exercise

Lacasse, Y 2009 11 326 292 618 Quality of Life (Fatigue) Self-report Sub CR Exercise

Puhan, M 2010 5 279 CRQ (higher is better] Self-report Ob CR Exercise

Kramer, M 2010 6 280 276 556 Birth weight (higher is

better] Clinician

assessment Ob Other Exercise

Rutjes, AW 2010 11 275 190 465 Pain (higher is worse) Self-report Sub MSK PA

Watson 2008 6 129 112 241 Treadmill walking

distance Clinician

assessment Ob Other Exercise

Manheimer E, 2010 7 902 871 1773 Pain (higher is worse) Self-report Sub MSK Acupuncture, exercise

BL = baseline; BMD = bone mineral density; CR = cardiorespiratory; CRQ = Chronic respiratory disease questionnaire; FEV1 = ; GYN = gynaecological; Lab = laboratory; min = minute; mo = month(s); MSK = musculoskeletal; Neuro = neurology; Ob = objective; PA = physical agents; PT = physiotherapy; ROM = range of motion; Sub = subjective; tx = treatment; wt = weight; pt=patients

Page 37 of 37


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




WHAT IS THE INFLUENCE OF RANDOMIZATION SEQUENCE

GENERATION AND ALLOCATION CONCEALMENT ON

TREATMENT EFFECTS OF PHYSICAL THERAPY TRIALS? A

META-EPIDEMIOLOGICAL STUDY

Journal: BMJ Open

Manuscript ID: bmjopen-2015-008562.R1

Article Type: Research

Date Submitted by the Author: 27-Jul-2015

Complete List of Authors: Armijo-Olivo, Susan; University of Alberta, faculty of rehabilitation Medicine Saltaji, Humam; University of Alberta, Department of Dentistry da Costa, Bruno; University of Bern, Institute of Social and Preventive Medicine Fuentes, Jorge; University of Alberta, Physical Therapy Ha, Christine; University of Alberta, Rehabilitation Medicine Cummings, Greta; University of Alberta, Faculty of Nursing

<b>Primary Subject Heading</b>:

Epidemiology

Secondary Subject Heading: Epidemiology, Evidence based practice, Rehabilitation medicine

Keywords: allocation concealment, sequence generation, physical therapy, meta-epidemiological, risk of bias


BMJ Open on M

arch 26, 2020 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2015-008562 on 3 Septem

ber 2015. Dow

nloaded from





Title: WHAT IS THE INFLUENCE OF RANDOMIZATION SEQUENCE 1

GENERATION AND ALLOCATION CONCEALMENT ON TREATMENT EFFECTS 2

OF PHYSICAL THERAPY TRIALS? A META-EPIDEMIOLOGICAL STUDY 3 4

5

Authors: 6

Susan Armijo-Olivo, BSc. PT, MSc PT, PhD 7 Postdoctoral Fellow 8




11405 – 87 Avenue 12






19

Humam Saltaji, DDS, MSc (Ortho) 20 PhD Candidate and Resident 21

Orthodontic Graduate Clinic 22

School of Dentistry 23


Edmonton, Alberta, Canada 25


27

Bruno R. da Costa PT, BSc. PT, MSc PT, PhD 28


Nicole Wertheim College of Nursing & Health Science 30

Florida International University 31

Miami, USA 32

[email protected] 33

34

35

Jorge Fuentes BPT, MSc PhD 36 Faculty of Rehabilitation Medicine 37


Edmonton, Alberta. Canada 39


Catholic University of Maule 41


Talca, Chile 43

44

Christine Ha, BSc 45 Rehabilitation Research Center 46

Faculty of Rehabilitation Medicine, 47

3-62 Corbett Hall, 48

University of Alberta, 49

Edmonton, Canada T6G 2G4 50

Page 1 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from






52

53

Greta G. Cummings, RN PhD FCAHS 54 Principal, CLEAR Outcomes (Connecting Leadership Education & Research) Research 55

Program 56

Centennial Professor, Faculty of Nursing, University of Alberta 57

Edmonton Clinic Health Academy | University of Alberta 58

11405-87 Ave, Edmonton, AB | Canada T6G 1C9 59


61

62

Corresponding author: 63 Susan Armijo-Olivo, BSc.PT, MSc PT, PhD 64

Postdoctoral Fellow 65




11405 – 87 Avenue 69






76

77

Correspondence: 78

Name Susan Armijo-Olivo, BSc. PT, MSc PT, PhD

Department Faculty of Rehabilitation Medicine,

3-48 Corbett Hall, Department of Physical Therapy

Institution University of Alberta,

Country Canada T6G 2G4

Tel 780-437-3521

Mob --

Fax 780-492-1626

Email [email protected] / [email protected]

79

80

Abbreviated title: Allocation concealment and randomization in physical therapy 81

Key words: Physical therapy; allocation concealment; randomization; meta-82

epidemiological approach; risk of bias 83 84

List of abbreviations: 85 RCTs: Randomized controlled trials 86

PT: Physical therapy 87

88

89

90

Page 2 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





Word Count: 275 words (Abstract) 91

3050 words (Introduction, Method, Results, and Discussion) 92

References: 58 93

Tables: 3 94

Figures: 6 95

Ethics approval: The University of Alberta Ethics Committee(s) approved this study. 96

Running head: Allocation concealment in physiotherapy 97 Study performed: CLEAR Outcomes (Connecting Leadership, Education and Research) 98

and Faculty of Rehabilitation Medicine, Department of Physical Therapy, University of 99

Alberta, Edmonton, Canada. 100

101

102

103 104

105

Page 3 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





ABSTRACT 106

Objective: to determine if adequacy of randomization and allocation concealment is 107

associated with changes in effect sizes when comparing physical therapy (PT) trials with and 108

without these methodological characteristics. 109

Design: Meta-epidemiological study. 110

Setting: NA 111

Participants: A random sample of randomized controlled trials (RCTs) included in meta-112

analyses in the PT discipline were identified. 113

Intervention: NA. Data extraction including assessments of random sequence generation and 114

allocation concealment was conducted independently by 2 reviewers. To determine the 115

association between sequence generation, and allocation concealment and effect sizes, a 2-116

level analysis was conducted using a meta-meta-analytic approach. 117

Primary and secondary outcome measures: association between random sequence 118

generation and allocation concealment and effect sizes in PT trials. 119

Results: 393 trials included in 43 meta-analyses, analyzing 44,622 patients contributed to this 120

study. Adequate random sequence generation and appropriate allocation concealment were 121

accomplished in only 39.7% and 11.5% of PT trials respectively. Although trials with 122

inappropriate allocation concealment tended to have an overestimate treatment effect when 123

compared with trials with adequate concealment of allocation, the difference was non-124

statistically significant (Effect size (ES) =0.12; 95%CI -0.06; 0.30). When pooling our results 125

with those of Nuesch et al., we obtained a pooled statistically significant value (ES= 0.14; 126

95%CI 0.02; 0.26). There was no difference in ES in trials with appropriate or inappropriate 127

random sequence generation (ES=0.02; 95%CI -0.12; 0.15). 128

Conclusion: Our results suggest that when evaluating risk of bias of primary RCTs in PT 129

area, systematic reviewers and clinicians implementing research into practice should pay 130

attention to these biases since they could exaggerate treatment effects. Systematic reviewers 131

Page 4 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





should perform sensitivity analysis including trials with low risk of bias in these domains as 132

primary analysis and/or in combination with less restrictive analyses. Authors and editors 133


reported in trial reports. 135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

Page 5 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





Strengths and limitations of this study 157

Strengths 158

• As far of our knowledge, this is the first meta-epidemiological study using continuous 159

outcomes in allied health disciplines such as physical therapy (PT). 160

• This large meta-epidemiological study, which was based on 43 Cochrane reviews, 393 161

trials and 44,622 patients makes an important contribution to the field. 162

• This study provides with novel evidence regarding the association between adequacy 163

of randomization and allocation concealment methodological factors and treatment 164

estimates in PT. 165

Limitations 166

• We restricted our analysis to Cochrane systematic reviews in PT and results might not 167

be applicable to all Cochrane reviews or other reviews conducted in other areas of 168

research. 169

• For determining the association between random sequence generation and allocation 170

concealment, we limited our analysis to trials describing a true control group, or 171

placebo intervention, reducing our statistical power. 172

• It could be possible that other biases present in the studies interact with the 173

investigated biases and have an influence in the treatment effects. This should include 174

a multivariate analysis where several biases should be taken into consideration. Future 175

research should look into this. 176

177

Page 6 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





INTRODUCTION 178

Randomization and allocation concealment have been extensively investigated in the medical 179

literature as key methodological characteristics of randomized controlled trials (RCTs) in 180

medical research.1-3

Allocation concealment is the most commonly evaluated quality 181

characteristic in reviews of RCTs due to its crucial role as a key marker of internal validity of 182

RCT.4 Moreover, it is firmly established the conventional wisdom that adequate 183

randomization sequence generation is an essential part of a valid clinical trial.5 184

185

There has been an extensive body of empirical research looking at the influence of random 186

sequence generation and allocation concealment on treatment effect estimates of health RCTs. 187

Several meta-epidemiological studies investigating the association between trial 188

characteristics and treatment effects have found an association between inadequate 189

randomization and/or allocation concealment and an overestimation of treatment effects.1-3,6-9

190

For example, inadequate random sequence generation can overestimate treatment effects by 191

12% (Ratio of Odds Ratios (ROR) 0.88, 95% CI 0.79,0.99).10

Inadequate allocation 192

concealment can overestimate treatment effects on average by 18%.3,8,11 In addition, Pidal et 193

al. found that 2/3 of the conclusions from meta-analyses were no longer supported if only 194

trials with adequate allocation concealment were included 12

and that 69% of meta-analyses 195

lost statistical significance when trials with unclear or inadequate allocation concealment were 196

excluded.12

Over-estimates or underestimates of treatment effects can lead to biased or 197

inaccurate results and conclusions in systematic reviews and meta-analyses.3,12-15

These 198

factors can ultimately have repercussions on decision-making and quality of patient care since 199

different assessments could lead to different decisions for clinical practice. 200

201

Page 7 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





Most of the empirical evidence regarding the relationship between trial components, and 202

specifically random sequence generation and allocation concealment, and treatment effect 203

estimates comes from RCTs in medicine and is based mainly on dichotomous outcomes.3,8,11

204

No such studies using continuous outcomes have been conducted in other health areas such as 205

the allied health professions, including Physical Therapy (PT). Furthermore, although it has 206

been reported that the quality of reporting of PT trials has improved over time,16

a finer 207

analysis of types of random sequence generation and allocation concealment in PT trials is 208

lacking. 209

210

Therefore, it is necessary to determine the extent to which sequence generation and allocation 211

concealment affect treatment effect estimates in PT trials to provide accurate results to the PT 212

clinical community. This information is urgently needed to develop guidelines for designing, 213

conducting, and implementing PT trials as well as providing clear benchmarks to assess the 214

quality and/or risk of bias of PT trials in systematic reviews and meta-analyses and ultimately 215

the strength of evidence for decision-making in PT. Therefore, our research questions were: 1) 216

are random sequence generation and allocation concealment adequately used and reported in 217

RCTs of PT; 2) do random sequence generation and allocation concealment have an effect on 218

estimates of treatment in PT trials; and 3) do effects sizes on PT trials differ depending on 219

some characteristics of the meta-analyses analyzed such as magnitude of the effect size, meta-220

analysis heterogeneity, type of outcome (subjective or objective), and whether the meta-221

analysis involved the musculoskeletal area. 222

METHOD 223

Design 224

Meta-epidemiological approach 225

226

Page 8 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





Study selection 227

A random sample of Randomized Controlled Trials (RCTs) included in meta-analyses in the 228

PT discipline were identified by searching the Cochrane Database of Systematic Reviews 229

from Jan 2005 to May 25 2011 on PT interventions. The search strategy can be found 230

elsewhere and can be provided upon request.17,18

Meta-analyses and RCTs included in these 231

meta-analyses were included if they met the following eligibility criteria: 1) the meta-analysis 232

included at least 3 RCTs comparing at least two interventions, with at least one of the 233

interventions being part of PT scope of practice according to the World Confederation for 234

Physical Therapy (WCPT)19

; and 2) the main outcome or the outcome of the meta-analysis 235

with the largest number of trials conducted in the review was continuous. 236

237

Assessment of Random sequence generation and Allocation Concealment domains 238

Assessments of random sequence generation and allocation concealment domains were 239

performed by two independent reviewers following the Cochrane collaboration guidelines.20

240

241

Random sequence generation assessment 242

In addition to the general evaluation of adequacy of random sequence generation, different 243

methods of random sequence generation were determined. We grouped these methods into 4 244

categories as follows: Category 1 included trials where random sequence generation was 245

unclear or not reported; category 2 included trials that had adequate randomization (e.g. use 246

of a computer software, random number table, and minimization); category 3 included trials 247

using acceptable methods of randomization, but less efficient than the previous category (e.g., 248

drawing lots, envelopes, shuffling cards, throwing a dice); category 4 involved trials using 249

inappropriate methods of sequence generation (e.g., date of birth, day of admission, hospital 250

record number). Categorization into one of these four categories was conducted in duplicate. 251

Page 9 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





To facilitate comparison with previous meta-analyses, these four categories were combined to 252

create two main groups: an "adequate sequence generation" group (combining categories 2 253

and 3) and an "inadequate or unclear sequence generation" group (combining categories 1 and 254

4). 255

256

Allocation Concealment assessment 257

We divided the methods of allocation concealment used in the trials into the following 258

categories: Category 1 comprised trials that used any type of central randomization (e.g., a 259

remote telephone service or a central office); category 2 comprised trials that used 260

sequentially numbered, opaque and sealed envelopes; category 3 comprised trials that used 261

sealed envelopes without reporting any further details; and category 4 comprised trials where 262

allocation was clearly not hidden (e.g., being based on an open list, odd or even days of the 263

week, participant’s birth date, or the team on duty at enrolment); and category 5 comprised 264

trials were concealment of allocation was not reported or unclear. Categorization into one of 265

these five categories was conducted by two independent assessors. To facilitate comparison 266

with previous meta-analyses, these five categories were combined to create two main groups: 267

an "adequate concealment" group (combining categories 1 and 2) and an "inadequate or 268

unclear concealment" group (combining categories 3, 4, and 5). 269

270

Data extraction of treatment estimates and trial characteristics 271

Two independent reviewers extracted specific data (e.g. type of interventions, type of 272

outcomes (i.e. objective, subjective), PT area) for all trials included in the meta-analyses as 273

well as data on means, standard deviations, and sample sizes. The primary outcome chosen 274

for the analysis was the main outcome of interest reported in the review or determined from 275

the meta-analysis that contained the largest number of trials in the review. Details on the 276

Page 10 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





reviewers’ panel and training process can be found elsewhere.17,18

277

278

Data Analysis 279

Data on sequence generation and allocation concealment were analyzed descriptively based 280

on the categories described previously. In order to determine whether random sequence 281

generation and allocation concealment domains affect treatment effect estimates, a 2-level 282

analysis was conducted using a meta-meta-analytic approach with a random-effects model to 283

allow for within and between meta-analyses heterogeneity as suggested by Sterne.21

284

The first level analysis (within meta-analysis) was as follows: we derived effect sizes (ES) for 285

each trial by dividing the between-group difference in mean values by the pooled standard 286

deviation .22

A negative ES indicates a beneficial effect of the experimental intervention. If 287

some required data were unavailable, we used approximations as previously described.23

The 288

data from each trial were obtained from the meta-analyses included in our study. We followed 289

the classification used in the Cochrane reviews to classify the treatment arms as the 290

experimental treatment of interest or as the control group. In the case of studies appearing in 291

more than one review, the study was only considered once in the meta-analysis with the fewer 292

number of overall studies. We then calculated two pooled effect sizes for each meta-analysis: 293

one corresponding to the pooled effect size from studies having the characteristic of interest 294

(e.g., allocation concealment) and the other for studies that did not (e.g., no or unclear 295

allocation concealment). We used standard random-effects meta-analyses to combine ES 296

across trials and calculated the DerSimonian and Laird estimate of the variance to determine 297

heterogeneity between trials.1,24

Then, for each meta-analysis, we derived the difference 298

between pooled ES estimates from trials with and without the characteristic of interest (e.g., 299

allocation concealment). A negative difference in ES indicates that trials without the 300

characteristic of interest show a more beneficial effect for the experimental group. 301

Page 11 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





The second level analysis (between meta-analyses) involved pooling the results of the 302

previous analysis to describe the effect of each trial component across all meta-analyses. The 303

effect sizes were also combined at this stage using the DerSimonian and Laird random-effects 304

models 25

to allow for between meta-analysis heterogeneity. 305

Formal tests of interaction between adequate sequence generation and concealment of 306

allocation and estimated treatment benefits were performed separately for each meta-analysis 307

based on Z scores using the estimated difference in ES between trials with and without 308

adequate sequence and concealment of allocation and the corresponding standard error (SE). 309

We additionally stratified analyses accompanied by interaction tests according to the pre-310

specified characteristics as reported by Nuesch et al1 1) treatment benefit in overall meta-311

analysis: small [ES greater than -0.5] versus large [ES ≤ to -0.5]; between-trial heterogeneity 312

in overall meta-analysis (low [τ2 <0.06] versus high [τ

2 ≥0.06]), nature of the outcome 313

(subjective or objective) and if the intervention was classified as musculoskeletal or other PT 314

area. 315

316

In order to evaluate the effect of random sequence generation and allocation concealment on 317

treatment effect estimates, we limited the analyses to studies describing a true control group, 318

or placebo intervention as well as studies in which the direction of the expected treatment 319

effect was evident (i.e. standard care vs. standard care plus active intervention; and active 320

intervention 1 plus active intervention 2 vs. active intervention 1). 321

322

Finally, results of our analyses were pooled with results from the meta-epidemiological study 323

performed by Nuesch et al.1, which also investigated the effect of concealment of allocation 324

on treatment effects on osteoarthritis measured on a continuous scale. Stata statistical software 325

version 12 was used to perform these analyses. 326

Page 12 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





RESULTS 327

Selection and characteristics of meta-analyses and RCTs 328

The search identified 3901 Cochrane reviews, with 271 reviews potentially relevant to PT. Of 329

these, 68 reviews included a meta-analysis of at least three studies of PT interventions and 330

used a continuous outcome. We randomly selected 44 meta-analyses but excluded one 26

331

because it used follow-up data from the same group rather than a control group for 332

comparison (Figure 1). Forty-three meta-analyses including 393 trials and analyzing 44,622 333

patients contributed to this study. Table 1 summarizes the characteristics of the 43 Cochrane 334

reviews. Briefly, the reviews were published between 2008 and 2011 and included meta-335

analyses of the effectiveness of PT interventions for musculoskeletal (22 reviews)27-35

, 336

cardiorespiratory (9 reviews)36-44

, neurological (6 reviews)45-51

, and other areas of physical 337

therapy (6 reviews).51-56

A median number of 6 trials were included in the meta-analyses 338

(interquartile range 5-8). Most trials were parallel group trials (367; 93.4%), single-center 339

studies (298; 76%) and had active control interventions (362; 92%). The most common 340

intervention was exercise (n=282, 71.8%). Supplementary Table S1 (Appendix 1) lists the 341

characteristics of each of the 43 meta-analyses. 342

343

Sequence generation descriptive 344

From the 393 trials included in the 43 meta-analyses, 156 trials (39.7%) had appropriate 345

sequence generation and 237 (60.3%) had inappropriate sequence generation (229 were 346

classified as “unclear” and 8 as “high” risk of bias) according to the Cochrane Collaboration 347

guidelines. 348

When analyzing the sequence generation categories, we found that 229 trials (58.27%) did not 349

clearly report the mechanism of how the sequence was generated. One hundred and thirty two 350

trials (33.6%) generated the sequence through computer software, a table of random numbers, 351

Page 13 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





or by minimization method. For a more specific sequence generation methods description, see 352

Table 2. 353

In addition, simple randomization was reported in 34 trials, block randomization in 54 trials, 354

stratification in 62 trials, and unclear or not reported in 218 trials. The rest of the trials 355

reported other methods of random sequence generation. 356

357

Allocation Concealment descriptive 358

From the 43 meta-analysis and 393 trials, 45 trials (11.5%) had appropriate allocation 359

concealment and 348 (88.6%) had inappropriate concealment of allocation according to the 360

Cochrane Collaboration guidelines. Results of analysis of the allocation concealment within 361

each category were as follows: 282 trials (71.8%) had unclear method of allocation 362

concealment, 21 (5.34%) trials used central allocation, 24 (6.11%) trials used envelopes with 363

all 3 safe wards (opaque, sealed, and sequentially numbered), 15 (3.82%) trials used 364

envelopes without safeguards, and 51 (16.8%) trials used any other non-concealed methods. 365

Further details on allocation concealment methods are displayed in Table 3. 366

Sequence Generation and allocation concealment descriptive 367

Only 8.9% of the trials (n=35) had both appropriate sequence generation and appropriate 368

allocation concealment. A great percentage (51%; n=199) of the trials did not have either 369

appropriate sequence generation or appropriate concealment of allocation. 370

371

Sequence generation and treatment effects in PT trials 372

For the purpose of analyzing the effect of sequence generation on treatment effects, 22 meta-373

analyses including 257 trials and analyzing 30,287 patients contributed to this analysis. 374

Figure 2 shows the forest plot of the differences in effect sizes between trials with adequate 375

and inadequate random sequence generation. There was no statistically significant difference 376

Page 14 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





between the ES of trials with adequate or inadequate random sequence generation (ES=0.02; 377


the meta-analyses characteristics had a statistically significant interaction. 379

380

When analyzing trials belonging only to category 2 which included trials that had adequate 381

sequence generation (e.g. use of a computer software, random number table, and 382

minimization) vs. unclear or not reported sequence generation, no statistically significant 383

difference between the ESs of these trials was found (ES=0.10; 95%CI -0.04; 0.23). However, 384

trials with unclear or not reported sequence generation tended to overestimate the treatment 385

effect when compared with trials with adequate sequence generation from category 2. It was 386

not possible to conduct the subgroup analysis between trials of category 2 vs trials from 387

category 4 (using inappropriate methods to perform sequence generation such as date of 388

birth, day of admission, hospital record number) since there were not enough trials for such 389

analysis. 390

391

Allocation Concealment and treatment effects in PT trials 392

For the purpose of analyzing the effect of allocation concealment on treatment effects, 17 393

meta-analyses including 198 trials and analyzing 27,011 patients contributed to this analysis. 394


and inadequate allocation concealment. Although trials with inappropriate allocation 396

concealment tended to have an overestimate treatment effect when compared with trials with 397

adequate concealment of allocation, the difference was non-statistically significant (ES=0.12; 398


the meta-analyses characteristics had a statistically significant interaction. When focusing on 400

trials with appropriate allocation concealment (category 1 and 2) vs. trials with clearly 401

Page 15 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





inappropriate methods of concealment (category 4), the difference was not statistically 402

significant (ES=0.26; 95%CI -0.04; 0.55). However, trials with clearly inappropriate methods 403

of concealment (category 4) tended to overestimate the treatment effects. The same was the 404

case when comparing the trials with appropriate allocation concealment (category 1 and 2) vs. 405

trials with unclear or unreported concealment of allocation (ES=0.20; 95%CI 0.05; 0.34). This 406

time, the difference between these trials was statistically significant. 407

408

Pooling results with a previous meta-epidemiological study 1 409

When pooling our results with those of Nuesch et al.1, who performed a meta-epidemiological 410

study with continuous outcomes as well, we obtained a pooled statistically significant value 411

(ES= 0.14; 95%CI 0.02; 0.26), meaning that trials with inappropriate concealment of 412

allocation had more beneficial effect than trials with appropriate concealment of allocation 413

(Figure 6). 414

415

DISCUSSION 416

Our findings showed that adequate sequence generation and appropriate allocation 417

concealment were only accomplished in a low percentage of PT trials (39.7% and 11.5%, 418

respectively). In addition only 8.9% of the trials (n=35) had both appropriate sequence 419

generation and appropriate allocation concealment despite the fact that both of the above-420

mentioned risk of bias domains can be always appropriately performed when conducting an 421

RCT in any field.57

In addition, trials with inappropriate concealment of allocation had an 422

overestimation of treatment effects when compared with trials with adequate concealment of 423

allocation. However, no difference in treatment effects in trials with appropriate or 424

inappropriate sequence generation was found. These results confirm previous results obtained 425

by several meta-epidemiological studies investigating the influence of sequence generation 426

Page 16 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





and allocation concealment on several areas of medicine using dichotomous 427

outcomes.7,8,11,12,58

In addition, the pooled estimate obtained from our study and that of 428

Nuesch et al.,1 indicated that trials with inappropriate allocation concealment had a more 429

beneficial effect than those with appropriate allocation concealment. 430

These results have important implications for the research community in general as well as for 431

the discipline of PT. To our knowledge; this study is the first of its kind conducted in PT that 432

examined continuous outcomes. Thus, it provides novel evidence in a very specific area of 433

health. Most of the previous studies were looking at medical areas, such as pregnancy and 434

childbirth, circulatory conditions, infectious disease, surgery, and mental health trials. All of 435

these medical areas certainly differ from PT with respect to: type of intervention (being 436

mostly drugs), type of outcomes used, and specific area of study. 437

438

First, it is not surprising that a large number of trials did not clearly report either 439

randomization sequence or allocation concealment or both. Our findings showed that 440

approximately 62% and 70% of studies were classified as ‘unclear/not reported’ in their 441

reporting of sequence generation and allocation concealment, respectively. This has also been 442

observed in a meta-epidemiological study investigating meta-analyses before 2005.2 The 443

study found a 64% and 69% of “unclear” reporting of sequence generation and allocation 444

concealment, respectively.2 Thus, despite the efforts made to improve reporting of RCTs such 445

as introducing the CONSORT Statement, there is still inappropriate reporting and 446

implementation of these important characteristics in RCTs. This is in agreement with results 447

reported by Moseley et al., 16

who found little or no effect of the CONSORT Statement on the 448

quality of reports of physiotherapy trials. Quality of reporting has always been an issue when 449

trying to understand the association between trial characteristics and treatments effects since 450

most of authors do not properly report their methods. Although inappropriate reporting does 451

Page 17 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





not always reflect on weak trial conduct, 59

some studies such as Pildal et al.,12

found that 452

most trials with unclear allocation concealment had also unclear allocation concealment in 453

their protocol. Therefore, quality of reporting is related to assessment of risk of bias, but the 454

direction of this relationship is still unclear. Future studies should assess how, how much, and 455

in which direction trial reporting affects treatment effects. 456

457

The magnitude and direction of the effect found in our study is consistent and almost identical 458

with a previously published study performed in osteoarthritic trials investigating allocation 459

concealment. In our study, the average bias associated with lack of allocation concealment 460

corresponds to ¼ or ½ of a typical treatment effect found in PT interventions.60-63

461

Furthermore, the overestimation of a treatment effect was even higher when only focusing the 462

analysis to trials with clearly inappropriate methods of allocation concealment. As stated by 463

others,2,8

if allocation concealment is not accomplished, it is more likely that the results of the 464

trial are inaccurate because, intentionally or unintentionally, investigators could have 465

interfered with assigning subjects to different groups, favoring the effect of the intervention. 466

In addition, allocation concealment could be a proxy measure for other aspects of trial design 467

besides selection bias.2,8

468

469

Random sequence generation was found not to be significantly associated with distortion of 470

treatment effects in this study. Thus, it appears that random sequence generation does not 471

have such a strong influence on treatment effects in comparison with concealment of 472

allocation based on the results of the present study. Other meta-epidemiological studies have 473

found similar results. 8,11,64

One of these studies, performed by Balk et al., 64

has been 474

questioned because of the high heterogeneity between trials included in meta-analysis. This 475

could introduce a higher noise to be able to detect any effect of the trials characteristics. The 476

Page 18 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





other studies reported the same results argue that sequence generation is still an important 477

characteristic that ensures rigor in the trial by allowing a trial to be concealed.8,11

In addition, 478

other authors have suggested that the relationship between certain methodological 479

characteristics and treatment effects varies according to each health area 64,65

and more work 480

should be done in other health areas to confirm these results.64,65

481

482

Our stratified analysis for sequence generation and allocation concealment showed that none 483

of the meta-analysis characteristics presented with significant interaction. This is somehow 484

contrary to other meta-epidemiological studies. For example, Wood et al.,3 and Savovic et 485

al.,2 found that there was an overestimation of treatment effects that ranged between 22%-486

31% for subjective outcomes. We followed exactly the same definition for determining 487

outcomes as “objective and subjective” used by Wood et al., 3 to facilitate comparisons and 488

avoid misclassifications. Thus, it seems that for PT area, there is no evidence of bias 489

regarding random sequence generation or allocation concealment when subjective or 490

objective outcome are used. This could be attributed to the fact that both domains can be 491

performed in a trial regardless of the outcome analyzed. Thus, whether the outcome is 492

objective or subjective does not affect whether it is easier or not to predict patient’s prognosis 493

at the time of recruitment. 494

495

Study limitations 496

We analyzed the influence of random sequence generation and allocation concealment on 497

treatment effect estimates. It could be possible that other biases interact with these biases and 498

have an influence in the treatment effects. This should include a multivariate analysis where 499

several biases should be taken into consideration. Future research should look into this. 500

Page 19 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





However, such analysis would require meta-analyses with a large number of included studies, 501

which are extremely rare; this makes such an analysis hard to be performed. 502

503

We limited our analysis to trials describing a true control group (i.e. group receiving no 504

treatment, or a waiting list), or placebo intervention as well as those studies in which the 505

direction of expected treatment effect was evident. In this way, we could anticipate the 506

direction of the bias in analyses. Combining effects of all trials without clear direction of what 507

would be the effect (i.e. in case of 2 similar active interventions) would have increased noise 508

in the analyses and heterogeneity limiting our ability to find any significant effect of trials 509

characteristics.66

However, limiting our analyses to these trials, we also reduced the power of 510

our study. 511

512

CONCLUSIONS 513

Trials with inappropriate concealment of allocation had an overestimation of treatment effects 514

when compared with trials with adequate concealment of allocation. Our results suggest that 515

when evaluating risk of bias of primary RCTs in PT area, systematic reviewers and clinicians 516

implementing trial findings into their clinical practice should pay attention to these 517

characteristics, especially allocation concealment since it can be associated with an 518

exaggeration of a treatment effect. Systematic reviewers should perform sensitivity analysis 519

including trials with low risk of bias in these domains as primary analysis and/or in 520

combination with less restrictive analyses. In addition, appropriate methods of sequence 521

generation as well as allocation concealment should be implemented. Authors and editors 522


reported in trial reports. 524

525

Page 20 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





Acknowledgements: This project is funded the Canadian Institutes of Health Research 526

(CIHR), Alberta Innovates Health Solutions through a knowledge translation initiative grant, 527

the Knowledge Translation (KT) Canada research Stipend program, and the Physiotherapy 528

Foundation of Canada (PFC) through a B.E. Schnurr Memorial Fund Award. 529

The funding bodies had no input in the design, collection, analysis, and interpretation of data; 530

in the writing of the manuscript; and in the decision to submit the manuscript for publication. 531

This Research protocol has been approved by the Ethics Board of the University of Alberta 532

(Pro00038172). 533

Dr. Susan Armijo-Olivo is supported by the Canadian Institutes of Health Research (CIHR) 534

through a full-time Banting fellowship, the Alberta Innovates Health Solution through an 535

incentive award, the STIHR Training Program from Knowledge Translation (KT) Canada, the 536

University of Alberta, and by the Faculty of rehabilitation Medicine, at the University of 537

Alberta, through the “Music and Motion Fellowship”. 538

Dr. Greta Cummings is holds a Centennial Professorship at the University of Alberta (2013-539

2020). 540

Dr. Fuentes is supported by the Government of Chile, University of Alberta through a 541

Dissertation fellowship, and the University Catholic of Maule. 542

Dr. Humam Saltaji is supported through a Clinician Fellowship Award by Alberta Innovates - 543

Health Solutions (AIHS), the Honorary Izaak Walton Killam Memorial Scholarship by the 544

University of Alberta, and the Honorary WCHRI Award by the Women and Children’s 545

Health Research Institute (WCHRI). 546

In addition, the authors of this study thank the Alberta Research Center for Health Evidence 547

(ARCHE) at the University of Alberta and all research assistants who helped with data 548

collection 549

550

Page 21 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





Competing Interest 551

The author(s) declare that they have no competing interests. 552

553 Ethics approval: The University of Alberta Ethics Committee(s) approved this study 554

555

556

Authors’ contributions 557

558 SAO provided concept/idea/research design and writing. Performed the experiments: SAO, 559

CH, JF, HS. Interpretation of the results and review of the final manuscript: SAO, BRdaC, 560

GGC, CH, JF, HS. Analyzed the data: SAO, BRdaC. Wrote the paper: All authors critically 561

revised and provided final approval of the version to be published. 562

563

Data Sharing Statement 564

No additional data available 565

Justification that this study does not require a checklist: 566

This study is considered a meta-epidemiological research study. We feel that none of the 567

checklists is applicable for this type of study. 568

Figure Legends 569



Adequate Sequence Generation 572



Heterogeneity, Type outcome, and PT area). 575

576








Page 22 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





REFERENCES 584 585

586

1. Nuesch E, Reichenbach S, Trelle S, et al. The importance of allocation concealment 587

and patient blinding in osteoarthritis trials: A meta-epidemiologic study. Arthritis 588

Care and Research. 2009;61(12):1633-1641. 589

2. Savovic J, Jones HE, Altman DG, et al. Influence of reported study design 590

characteristics on intervention effect estimates from randomized, controlled trials. 591

Annals of Internal Medicine. 2012;157(6):429-438. 592

3. Wood L, Egger M, Gluud LL, et al. Empirical evidence of bias in treatment effect 593

estimates in controlled trials with different interventions and outcomes: Meta-594

epidemiological study. BMJ. 2008;336(7644):601-605. 595

4. Falagas ME, Grigori T, Ioannidou E. A systematic review of trends in the 596

methodological quality of randomized controlled trials in various research fields. 597

Journal of Clinical Epidemiology. 2009;62(3):227-231.e229. 598

5. Aickin M. Randomization, balance, and the validity and efficiency of design-adaptive 599

allocation methods. Journal of Statistical Planning and Inference. 2001;94(1):97-119. 600

6. Bausell RB, Lee WL, Soeken KL, Li YF, Berman BM. Larger effect sizes were 601

associated with higher quality ratings in complementary and alternative medicine 602

randomized controlled trials. Journal of Clinical Epidemiology. May 2004;57(5):438-603

446. 604

7. Kjaergard LL, Villumsen J, Gluud C. Reported methodologic quality and 605

discrepancies between large and small randomized trials in meta-analyses. Annals of 606

Internal Medicine. 2001;135(11):982-989. 607

8. Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias: 608

Dimensions of methodological quality associated with estimates of treatment effects in 609

controlled trials. Journal of the American Medical Association. 1995;273(5):408-412. 610

9. Siersma V, Als-Nielsen B, Chen W, Hilden J, Gluud LL, Gluud C. Multivariable 611

modelling for meta-epidemiological assessment of the association between trial 612

quality and treatment effects estimated in randomized clinical trials. Statistics in 613

Medicine. 2007;26(14):2745-2758. 614

10. Als-Nielsen B, Gluud L, Gluud C. Methodological quality and treatment effects in 615

randomised trials: a review of six empirical studies. Paper presented at: 12th Cochrane 616

Colloquium 2004 2004; Oct 2-6; Ottawa, Ontario, Canada. . 617

11. Moher D, Pham B, Jones A, et al. Does quality of reports of randomised trials affect 618

estimates of intervention efficacy reported in meta-analyses?[see comment]. Lancet. 619

Aug 22 1998;352(9128):609-613. 620

12. Pildal J, Hrobjartsson A, Jorgensen KJ, Hilden J, Altman DG, Gotzsche PC. Impact of 621

allocation concealment on conclusions drawn from meta-analyses of randomized 622

trials. International Journal of Epidemiology. 2007;36(4):847-857. 623

13. Hewitt CE, Kumaravel B, Dumville JC, Torgerson DJ. Assessing the impact of 624

attrition in randomized controlled trials. Journal of Clinical Epidemiology. 625

2010;63(11):1264-1270. 626

14. Kjaergard LL, Als-Nielsen B. Association between competing interests and authors' 627

conclusions: Epidemiological study of randomised clinical trials published in the 628

BMJ. British Medical Journal. 2002;325(7358):249-252. 629

15. Trowman R, Dumville JC, Torgerson DJ, Cranny G. The impact of trial baseline 630

imbalances should be considered in systematic reviews: a methodological case study. 631


Page 23 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





16. Moseley AM, Herbert RD, Maher CG, Sherrington C, Elkins MR. Reported quality of 633

randomized controlled trials of physiotherapy interventions has improved over time. 634


17. Armijo-Olivo S, Cummings G, Fuentes CJ, et al. Identifying Items to Assess 636

Methodological Quality In Physical Therapy Trials: A Factor Analysis. Physical 637

Therapy. 2014;94 (9):1272-1284. 638

18. da Costa B, Resta N, Beckett B, et al. Effect of standardized training on the reliability 639

of the Cochrane risk of bias assessment tool: a study protocol. Systematic Reviews. 640

2014/12/13 2014;3(1):1-6. 641

19. World Confederation for Physical Therapy. Position statement: standards of physical 642

therapy practice. World Confederation for Physical Therapy;2011. 643

20. Higgins JPT, Altman DG, Goetzsche PC, et al. The Cochrane Collaboration's tool for 644

assessing risk of bias in randomised trials. BMJ. 2011;343(7829). 645

21. Sterne JAC, Juni P, Schulz KF, Altman DG, Bartlett C, Egger M. Statistical methods 646

for assessing the influence of study characteristics on treatment effects in 'meta-647

epidemiological' research. Statistics in Medicine. 2002;21(11):1513-1524. 648

22. Cohen J. Chapter 1: The Concepts of Power Analysis. In: Cohen J, ed. Statistical 649

Power Analysis for the Behavioral Sciences. Second ed. Hillsdale, New jersey: 650

Academic Press , INc; 1988:1-17. 651

23. Reichenbach S, Sterchi R, Scherer M, et al. Meta-analysis: Chondroitin for 652

osteoarthritis of the knee or hip. Annals of Internal Medicine. 2007;146(8):580-590. 653

24. Nuesch E, Trelle S, Reichenbach S, et al. The effects of excluding patients from the 654

analysis in randomised controlled trials: Meta-epidemiological study. BMJ. 655

2009;339(7722):679-683. 656

25. DerSimonian R, Laird N. Meta-analysis in clinical trials. Controlled Clinical Trials. 657

1986;7(3):177-188. 658

26. Spittle A, Orton J, Doyle LW, Boyd R. Early developmental intervention programs 659

post hospital discharge to prevent motor and cognitive impairments in preterm infants. 660

Database of Systematic.Reviews.Issue:2. John.Wiley & Sons, Ltd. Chichester. 2007. 661

27. Fransen M, McConnell S, Hernandez MG, Reichenbach S. Exercise for osteoarthritis 662

of the hip. Cochrane Database of Systematic Reviews: Issue 3. John.Wiley.& Sons., 663

Ltd. Chichester. 2009. 664

28. Handoll-Helen HG, Cameron ID, Mak-Jenson CS, Finnegan TP. Multidisciplinary 665

rehabilitation for older people with hip fractures. Cochrane Database of Systematic 666


29. Harvey LA, Brosseau L, Herbert RD. Continuous passive motion following total knee 668

arthroplasty in people with arthritis. Cochrane Database of Systematic Reviews. Issue 669

3. John.Wiley.& Sons., Ltd.Chichester. 2010 2010. 670

30. Katalinic OM, Harvey LA, Herbert RD, Moseley AM, Lannin NA, Schurr K. Stretch 671

for the treatment and prevention of contractures. Cochrane Database of Systematic 672


31. Manheimer E, Cheng K, Linde K, et al. Acupuncture for peripheral joint osteoarthritis. 674

Cochrane Database of Systematic Reviews. Issue 1. John Wiley& Sons, Ltd. 675

Chichester, UK. 2010 2010. 676

32. Ostelo-Raymond WJG, Costa-Leonardo OP, Maher CG, de-Vet-Henrica CW, van-677

Tulder MW. Rehabilitation after lumbar disc surgery. Cochrane Database of 678

Systematic Reviews. Issue 4. John Wiley & Sons., Ltd.Chichester, UK. 2008. 679

33. Rutjes-Anne WS, Nüesch E, Sterchi R, Jüni P. Therapeutic ultrasound for 680

osteoarthritis of the knee or hip. Cochrane Database of Systematic Reviews. Issue 1. 681

John Wiley & Sons., Ltd.Chichester. 2010. 682

Page 24 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





34. Rutjes-Anne WS, Nüesch E, Sterchi R, et al. Transcutaneous electrostimulation for 683

osteoarthritis of the knee. Cochrane Database of Systematic Reviews. Issue 4. John 684

Wiley & Sons., Ltd. Chichester. 2009. 685

35. Schaafsma F, Schonstein E, Whelan KM, Ulvestad E, Kenny DT, Verbeek JH. 686

Physical conditioning programs for improving work outcomes in workers with back 687

pain. Cochrane Database of Systematic Reviews. Issue 1. John Wiley & Sons, Ltd 688


36. Davies P, Taylor F, Beswick A, et al. Promoting patient uptake and adherence in 690


& Sons Ltd. Chichester, UK. 2010. 692

37. Effing T, Monninkhof-Evelyn EM, Valk-Paul PDLP, et al. Self-management 693

education for patients with chronic obstructive pulmonary disease. Cochrane 694

Database of Systematic Reviews. Issue 4. John Wiley & Sons Ltd. Chichester, UK. 695

2007. 696

38. Puhan MA, Gimeno SE, Scharplatz M, Troosters T, Walters EH, Steurer J. Pulmonary 697

rehabilitation following exacerbations of chronic obstructive pulmonary disease. 698

Cochrane Database of Systematic Reviews. Issue 1. John Wiley & Sons Ltd. 699


39. Taylor RS, Dalal H, Jolly K, Moxham T, Zawada A. Home-based versus centre-based 701



40. Main E, Prasad A, van der Schans CP. Conventional chest physiotherapy compared to 704

other airway clearance techniques for cystic fibrosis. Cochrane.Database.of 705

Systematic.Reviews.Issue.1 John Wiley & Sons., Ltd.Chichester, UK. 2005. 706

41. O'Brien K, Nixon S, Tynan AM, Glazier R. Aerobic exercise interventions for adults 707

living with HIV/AIDS. Cochrane.Database.of Systematic.Reviews.Issue.8.John Wiley 708


42. Jolliffe J, Rees K, Taylor-Rod RS, Thompson DR, Oldridge N, Ebrahim S. Exercise-710

based rehabilitation for coronary heart disease. Cochrane.Database.of 711

Systematic.Reviews.Issue.1 John Wiley & Sons., Ltd.Chichester, UK. 2001 2001. 712

43. Shaw KA, Gennat HC, O'Rourke P, Del MC. Exercise for overweight or obesity. 713



44. Lacasse Y, Goldstein R, Lasserson TJ, Martin S. Pulmonary rehabilitation for chronic 716

obstructive pulmonary disease. Cochrane Database of Systematic Reviews. Issue.3. 717

John Wiley & Sons., Ltd.Chichester, UK. 2009. 718

45. Sirtori V, Corbetta D, Moja L, Gatti R. Constraint-induced movement therapy for 719

upper extremities in stroke patients. Cochrane Database of Systematic Reviews. Issue 720

4. John Wiley & Sons., Ltd.Chichester, UK. 2009. 721

46. States RA, Pappas E, Salem Y. Overground physical therapy gait training for chronic 722

stroke patients with mobility deficits. Cochrane Database of Systematic Reviews. 723

Issue 3.John Wiley & Sons. Ltd.Chichester, UK. 2009. 724

47. Mead GE, Morley W, Campbell P, Greig CA, McMurdo M, Lawlor DA. Exercise for 725

depression. Mead Gillian.E., Morley.Wendy., Campbell.Paul, Greig.Carolyn.A, 726

McMurdo.Marion., Lawlor.Debbie.A.Exercise.for.depression.Cochrane.Database.of 727

Systematic.Reviews.: Reviews.2009.Issue.3 John.Wiley.& Sons., Ltd.Chichester, UK 728

DOI.: 10.1002./14651858.CD0043. 2009 2009. 729

48. Pollock A, Baer G, Pomeroy VM, Langhorne P. Physiotherapy treatment approaches 730

for the recovery of postural control and lower limb function following stroke. 731

Page 25 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





Cochrane Database of Systematic Reviews. Issue.1 John Wiley& Sons Ltd. Chichester, 732

UK. 733

. 2007. 734

49. Mehrholz J, Friis R, Kugler J, Twork S, Storch A, Pohl M. Treadmill training for 735

patients with Parkinson's disease. Cochrane Database of Systematic Reviews.Issue.1 736

John Wiley& Sons., Ltd.Chichester, UK. 2010 2010. 737

50. Woodford HJ, Price C, I. EMG biofeedback for the recovery of motor function after 738

stroke. Cochrane.Database.of Systematic.Reviews. Issue.2 John Wiley & Sons., 739


51. Watson L, Ellis B, Leng GC. Exercise for intermittent claudication. 741



52. Kramer MS, McDonald SW. Aerobic exercise for women during pregnancy. 744

Cochrane.Database.of Systematic.Reviews. Issue.3 John Wiley & Sons., 745


53. Orozco LJ, Buchleitner AM, Gimenez PG, Figuls M, Richter B, Mauricio D. Exercise 747

or exercise and diet for preventing type 2 diabetes mellitus. Cochrane Database of 748

Systematic Reviews. Issue 3. John Wiley & Sons Ltd. Chichester, UK. 2008. 749

54. Bendermacher-Bianca LW, Willigendael EM, Teijink-Joep AW, Prins MH. 750

Supervised exercise therapy versus non-supervised exercise therapy for intermittent 751

claudication. Cochrane.Database.of Systematic.Reviews. Issue.2 John.Wiley & Sons., 752


55. Forster A, Lambley R, Hardy J, et al. Rehabilitation for older people in long-term care. 754

Cochrane.Database.of Systematic.Reviews.Issue.1 John.Wiley.& Sons., 755


56. Cramp F, Daniel J. Exercise for the management of cancer-related fatigue in adults. 757

Cochrane Database of Systematic Reviews.Issue 2. John Wiley & Sons., 758


57. Jadad AR. Bias in Randomized Controlled Trials. In: Jadad AR, Enkin M, eds. 760

Randomized controlled trials : questions, answers, and musings 761

Second ed. Malden, Mass.: Blackwell Pub.; 2007:29-47. 762

58. Egger M, Juni P, Bartlett C, Holenstein F, Sterne J. How important are comprehensive 763

literature searches and the assessment of trial quality in systematic reviews? Empirical 764

study. Health technology assessment. 2003;7(1):1-76. 765

59. Soares HP, Daniels S, Kumar A, et al. Bad reporting does not mean bad methods for 766

randomised trials: Observational study of randomised controlled trials performed by 767

the Radiation Therapy Oncology Group. British Medical Journal. 2004;328(7430):22-768

24. 769

60. Abdul Latif L, Daud Amadera JE, Pimentel D, Pimentel T, Fregni F. Sample size 770

calculation in physical medicine and rehabilitation: A systematic review of reporting, 771

characteristics, and results in randomized controlled trials. Arch Phys Med Rehabil. 772

2011;92(2):306-315. 773

61. Dorstyn D, Mathias J, Denson L. Applications of telecounselling in spinal cord injury 774

rehabilitation: A systematic review with effect sizes. Clinical Rehabilitation. 775

2013;27(12):1072-1083. 776

62. Barker AL, Talevski J, Morello RT, Brand CA, Rahmann AE, Urquhart DM. 777

Effectiveness of aquatic exercise for musculoskeletal conditions: A meta-analysis. 778

Arch Phys Med Rehabil. 2014;95(9):1776-1786. 779

Page 26 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





63. Pollock A, Baer G, Langhorne P, Pomeroy V. Physiotherapy treatment approaches for 780

the recovery of postural control and lower limb function following stroke: A 781

systematic review. Clinical Rehabilitation. 2007;21(5):395-410. 782

64. Balk EM, Bonis PAL, Moskowitz H, et al. Correlation of quality measures with 783

estimates of treatment effect in meta-analyses of randomized controlled trials. Journal 784

of the American Medical Association. 2002;287(22):2973-2982. 785

65. Verhagen AP, de Vet HCW, Willemsen S, Stijnen T. A meta-regression analysis 786

shows no impact of design characteristics on outcome in trials on tension-type 787

headaches. Journal of Clinical Epidemiology. 2008;61(8):813-818. 788

66. Hempel S. Detection of associations between trial quality and effect sizes. 2012; 789

http://www.ncbi.nlm.nih.gov/books/NBK84249/. 790

791 792

793

794

795

796

797

798

799

800

801

802

803

804

805

806

807

808

809

810

811

812

813

814

815

816

817

818

819

820

821

822

823

824

825

826

827

828

829

Page 27 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





Table 1. Characteristics of the Selected Meta-analysis within Physical Therapy Areas 830 831

832

Musculoskeletal Cardio respiratory Neurology Other

Characteristics Number of meta-analyses 22 9 6 6 Median year of publication 2009 2010 2009 2009 Total number of trials included 194 78 52 69 Total number of patients included

19861 9015 2138 13608

Main intervention Exercise 13 7 3 5 Physical Agents 1 0 1 0 Acupuncture 2 0 0 0 Manual Therapy 1 0 0 0 Other 1 2 2 1 Outcomes Clinician assessed outcome 8 4 6 3 Self-reported outcome 11 4 0 2 Administrative data or automated outcome

3 1 0 1

833

834

835

836

Table 2. Methods of Sequence Generation in Physical therapy Trials 837

838

Sequence Generation Methods Number of trials (%)


Computer Software 65 (16.54%)

Random Number table 60 (13.49%)

Drawing of lots 12 (3.05%)

Shuffling cards or envelopes 9 (2.29%)

Minimization 6 (1.52%)

Other non-random methods 4 (4.33%)

Coin Tossing 1 (0.25%)

Date of birth 1 (0.25%)

Day of admission 1 (0.25%)

Hospital/institution records number 1 (0.25%)

Throwing a dice 1 (0.25%)

Total 393 (100%)

839

840

841

842

843

844

845

Page 28 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





Table 3. Methods of Allocation Concealment in Physical therapy Trials 846

847 Allocation Concealment Methods Number of trials (%)


Central Concealment 21 (5.34%)

Sequentially numbered, opaque and sealed envelopes

24 (6.11%)

Unsafe envelopes 15 (3.82%)

Non-safe methods of allocation 51 (12.98%)

Total 393 (100%)

848 849 850

851

852 853

854

855

856

857

858

859

860

861

862

863

864

865

866

867

868

869

870

871

872

873

874

875

876

877

878

879

880

881

882

883

884

885

886

Page 29 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





887

888

889

890

891

892 893

894

895

896

897 898

899

900

901

902

903

904

905

906

907

908

909

910

911

912

913

914

915

916

917

918

919

920

921 922

923

924

925

926

927

928

929

930

931

932

933

934

935

936

Page 30 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





937

938

939

940

941

942

943

944

945

946

947

948

949

Page 31 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



90x102mm (300 x 300 DPI)

Page 32 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



55x41mm (300 x 300 DPI)

Page 33 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



59x44mm (300 x 300 DPI)

Page 34 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



58x43mm (300 x 300 DPI)

Page 35 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



59x44mm (300 x 300 DPI)

Page 36 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



57x43mm (300 x 300 DPI)

Page 37 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



1

Appendix 1. Characteristics of Selected Meta-analyses Author Year # of

trials Pt in

tx Pt in

control # of Pt

included Outcome Outcome Source

Type of outcome Area Specific

Area

Pollock, A 2009 5 213 209 422 Functional Independence Scale

Clinician assessment Ob Neuro Other

States, R 2009 6 195 186 381 Gait speed (m/s) (higher is better)

Clinician assessment Ob Neuro Exercise

Schaafsma, F 2011 5 1057 Time to return to work (lower is better)

Administrative data Ob MSK Exercise

Markes, M 2009 4 127 80 207 CR fitness (12 min

walking test), higher is better

Clinician assessment Ob MSK Exercise

McNeely, M 2010 6 166 158 324 Shoulder flexion ROM in degrees


Cramp, F 2010 27 1662 742 940 Fatigue Self-report Sub Other Exercise

Main, E 2010 7 201 Pulmonary function (FEV1)

Clinician assessment Ob CR Chest PT

Davies, E 2010 9 1554 1555 3109 Health-related QoL Self-report Sub CR Exercise

Busch, A 2008 6 188 161 349 Tender points number


(self-reported pain at

examination]

Sub MSK Exercise

Liu, C 2009 33 1076 1096 2172 Main function measure (higher is better)

Self-report (clinician

assessment] Sub MSK Exercise

Furlan, A 2011 3 188 168 356 Pain (higher is worse) Self-report Sub MSK Acupuncture

Fransen, M 2009 5 102 102 204 Pain (higher is worse) Self-report Sub MSK Exercise

Ostelo, R 2011 3 77 45 122 Pain (higher is worse) Self-report Sub MSK Exercise

Taylor, R 2010 12 817 740 1557 Exercise capacity 3-12 mo (higher is better)

Clinician assessment Ob CR Exercise

Harvey, L 2010 8 193 186 379 Active knee flexion (ROM)


Mead, GE 2010 23 476 431 907 Reduction of

depression symptoms post-tx (lower is better)

Clinician assessment Sub Neuro Exercise

Edmonds, M 2010 5 143 143 286 Chalder fatigue scale (higher is worse) Self-report Sub MSK,

educa Exercise

Page 38 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



2

tion

Howe, TE 2008 5 140 164 304 Gait speed (20 min

walk test) (s), (lower is better)


Fransen, M 2009 31 2074 1542 3616 Pain (higher is worse) Self-Report Sub MSK Exercise

Lin, CH 2008 8 513 Activity limitation

questionnaire (activity level) higher is better

Self-Report/performa

nce test Sub MSK Exercise;

other

Rutjes, AW 2010 5 177 143 320 Pain (higher is worse) Self-Report Sub MSK Exercise, other

Woodford, HJ 2009 5 47 44 91 Change in ROM (higher is better)

Clinician Assessment Ob Neuro PA, exercise

Saunders, DH 2009 7 194 177 371 Gait speed (m/min) Clinician Assessment Ob MSK Exercise

O'Brien , K 2010 5 145 131 276 Vo2 max (ml/kg/min) Clinician Assessment Ob CR Exercise

Sirtoti, V 2009 6 96 88 184 Disability post-

intervention. (higher is better]

Clinician assessment Sub Neuro Other

Hayden, J 2011 13 704 669 1373

Function measure (Oswestry low back

pain disability, lower is better)

Self-report Sub MSK Exercise, other

Orozco, LJ 2008 6 1668 1647 3315 Change in fasting plasma glucose (mg/dl)

Laboratory measure

(physiological measures)

Ob Other Exercise, education

De Morton, N 2009 5 1621 1857 3478 Acute hospital length of stay

Administrative data Ob MSK Other

Mehrholz, J 2010 7 76 77 153 Gait speed Clinician assessment Ob Neuro Exercise

Shaw, K 2009 15 595 484 1079 Wt change in kilograms (higher is worse)

Clinician assessment Ob CR Exercise

Handholl, H 2009 8 817 846 1663 Length of hospital stay Administrative data Ob MSK Other

Effing, T 2009 6 381 317 698 HRQOL: SGRQ total (0-100, lower is better) Self-report Sub

CR education,

prevention

Education

Page 39 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



3

Bendermacher, B 2009 6 118 118 236 Maximal treadmill

walking distance Clinician

assessment Ob

Other (cardiovascular)

Exercise

Bonaiuti D, 2009 8 166 170 336 BMD Clinician assessment Ob MSK Exercise

Foster, C 2009 17 4395 3203 7598

Change in self-reported physical activity

between BL and follow-up

Self-report Sub Other (GYN

) Other

Jolliffe, J 2009 8 610 588 1198 Total cholesterol Lab measure (physiological

measures) Ob CR Exercise

Katalinic, O 2010 7 96 97 193 Joint mobility (ROM) Clinician assessment Ob MSK

Manual Therapy, exercise

Lacasse, Y 2009 11 326 292 618 Quality of Life (Fatigue) Self-report Sub CR Exercise

Puhan, M 2010 5 279 CRQ (higher is better] Self-report Ob CR Exercise

Kramer, M 2010 6 280 276 556 Birth weight (higher is better]

Clinician assessment Ob Other Exercise

Rutjes, AW 2010 11 275 190 465 Pain (higher is worse) Self-report Sub MSK PA

Watson 2008 6 129 112 241 Treadmill walking distance

Clinician assessment Ob Other Exercise

Manheimer E, 2010 7 902 871 1773 Pain (higher is worse) Self-report Sub MSK Acupuncture, exercise

BL = baseline; BMD = bone mineral density; CR = cardiorespiratory; CRQ = Chronic respiratory disease questionnaire; FEV1 = ; GYN = gynaecological; Lab = laboratory; min = minute; mo = month(s); MSK = musculoskeletal; Neuro = neurology; Ob = objective; PA = physical agents; PT = physiotherapy; ROM = range of motion; Sub = subjective; tx = treatment; wt = weight; pt=patients

Page 40 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from


bmj open...30 nicole wertheim college of nursing & health science 31 florida international...

Documents