Title Page

A CONSORT analysis of randomised controlled trials for the treatment of invasive

aspergillosis

Brian L Jones*1, Malcolm D Richardson2, Patricia M Ingram3, and Samir G Agrawal4

Address: 1Department of Clinical Microbiology, Royal Infirmary, Glasgow, United

Kingdom and University of Glasgow, Glasgow, United Kingdom, 2Mycology

Reference Centre, and Manchester Academic Health Science Centre, Education and

Research Centre, University Hospital of South Manchester, Manchester, United

Kingdom, 347 Tring Avenue, London, United Kingdom and 4Division of Haemato-

Oncology, St Bartholomew's Hospital, Barts Health NHS Trust and Queen Mary

University of London, London, United Kingdom

Email: Brian L Jones* - Brian.Jones@glasgow.ac.uk; Malcolm D Richardson -

Malcolm.Richardson@manchester.ac.uk; Patricia M Ingram –

pmingram@btinternet.com; Samir G Agrawal - s.g.agrawal@qmul.ac.uk

* Corresponding author

Keywords:

CONSORT, invasive aspergillosis, haemato-oncology, RCT

Page 1 of 29

Abstract

There is no assessment of the reporting quality of antifungal randomised, controlled

trials (RCT), upon which guidelines for the treatment of invasive aspergillosis (IA) in

patients with haematological malignancy are based. Trial reports were identified

through Trip, Cochrane, Medline, and Embase database searches. Report quality

was assessed using the 25-item CONSORT checklist and a rating scale of 1

(strongly disagree) to 4 (strongly agree). The primary endpoint was quality as

assessed by mean group-scores among papers published at the time of the most

recent IA treatment guidelines. Seven RCTs were identified for analysis. Overall

mean group-score for all seven papers was 2·44 (out of a total of four). There were

significant differences between publications regarding overall reporting quality

(p<0·001) and specifically for the Methods and Results (p = 0·004 and p = 0·010,

respectively), which best reflect data quality. The Cornely trial report achieved the

highest mean group-score overall (3·15 + 0·93; 95% CI, 2·82, 3·47), as well as for

Methods (3·36) and Results (3·40). Mean group scores also showed that it was of

significantly higher overall quality than the other six publications (p value range;

0·012 to <0·001), and of higher quality for Methods than five publications (p value

range; 0·013 to <0·001). Incorporating this CONSORT analysis into the evidence-

based grading systems in North American (IDSA), European (ECIL and ESCMID) IA

guidelines could alter the value placed on these RCTs, thereby impacting on clinical

recommendations.

Page 2 of 29

Introduction

Invasive aspergillosis (IA) has emerged as an important cause of morbidity and

mortality in immunocompromised hosts,1 including recipients of haematopoietic

stem-cell transplantation (HSCT) and solid organ transplantation, patients with

advanced human immunodeficiency virus (HIV) infection and inherited

immunodeficiency, and those with prolonged neutropaenia.2 IA is associated with

unacceptably high mortality.3 Early diagnosis is associated with improved patient

survival but is rarely achieved with current methods.4 The lack of definitive diagnostic

criteria has also complicated studies of the treatment of IA and few randomised

controlled trials (RCTs) have been published.2

While RCTs are the most rigorous way of determining whether a cause-effect

relation exists between treatment and outcome, and for assessing the cost

effectiveness of a treatment,5 studies with inadequate methodology are associated

with bias, especially exaggerated treatment effects.6 In light of overwhelming

evidence of poor-quality reporting of RCTs, an international group of clinical trialists,

statisticians, epidemiologists and journal editors, methodologists developed and

published the CONsolidated Standards Of Reporting Trials (CONSORT) Statement

in 1996.7 The CONSORT Statement is an evidence-based, minimum set of

recommendations for writing RCT reports, which aims to facilitate their complete and

transparent reporting and, thus, their critical appraisal and interpretation. The current

version published in 2010 consists of a 25-item checklist of the key items that should

be included in an RCT report, with the items grouped according to the general format

of a journal publication; namely Title and Abstract, Methods, Results, Discussion,

and Other Information such as funding.8 However, since many of the 25 items are

subdivided into two categories, the checklist actually comprises 37 items.

Page 3 of 29

Furthermore, most of the 37 items consist of multiple statements that require

independent evaluation. The CONSORT Statement is currently endorsed by 585

journals,9 and a recent systematic review shows that while most RCTs published in

these journals are still incompletely reported, the CONSORT Statement beneficially

influences their reporting quality.10

Although numerous publications have used the CONSORT statement to evaluate

the quality of reports of RCTs in various sub-specialities of medicine, to our

knowledge no publication has evaluated the quality of RCTs focusing on the

treatment of IA. Since current guidelines for the treatment of IA are based on a

paucity of RCT data, we considered that a critique of the published clinical data

according to CONSORT was warranted. The CONSORT tool published in 2010 was

used for this purpose, regardless of the study’s year of publication, as the aim of the

analysis was to assess the quality of reporting of data that are used to inform current

treatment guidelines and, hence, influence clinical practice.

Materials and Methods

Data sources, search strategies and studies selection

Trip, Cochrane, Medline and Embase searches were conducted in March 2016 for

reports on RCTs involving patients with proven or probable invasive aspergillosis.

The following filters and search criteria were used: “randomised/randomized

controlled trial”, “proven or probable invasive aspergillosis”, “proven or probable

invasive mould/mold infection”, “acute leukaemia/leukemia”, “haematopoietic/

hematopoietic stem-cell transplantation”, and “myelodysplastic syndrome”. The

searches were limited to English language articles from 1976 to present and did not

include conference abstracts. Identification of reports for inclusion in the study was

Page 4 of 29

also based on the authors’ knowledge, and a review of references cited in the

current aspergillosis treatment guidelines of the Infectious Diseases Society of

America (IDSA), European Conference on Infections in Leukemia (ECIL) and the

European Society for Clinical Microbiology and Infectious Diseases (ESCMID).2,11,12

Reporting assessment tools

The quality of reporting was evaluated according to the main CONSORT 2010

checklist and accompanying Explanation and Elaboration guidelines,8 except for the

abstract, which was evaluated according to the Extended CONSORT Statement for

Reporting RCTs in Journal and Conference Abstracts.13 The extent to which each

item fulfilled the CONSORT requirements was assessed using the four-point rating

scale associated with the Appraisal of Guidelines for Research and Evaluation

(AGREE) I Instrument, which ranges from 1 “Strongly Disagree” to 4 “Strongly

Agree” (Table 1).14 Following its publication in 2003, the AGREE I Instrument quickly

became the international gold standard for evaluating the quality of reporting of

practice guidelines (http://www.agreetrust.org).

Methodological evaluation

The four authors independently assessed all the RCT publications. For Item 17b,

which relates to presentation of both absolute and relative effect sizes for binary

outcomes, a statistician was consulted to aid interpretation of the CONSORT

Explanation and Elaboration guidelines. Following collation of the individual results,

the authors met to agree a group-score for each of the 37 CONSORT items. In

situations where all the individual scores were positive (scores 3 and 4), the

predominant score was assigned; if the scores were equally balanced between 3

Page 5 of 29

and 4, then the lower score was assigned. Similar principles were applied if all the

individual scores were negative (scores 1 and 2). In situations where the individual

scores were a mixture of positive and negative scores, each author provided

evidence to support his/her chosen score, and a group-score was agreed for each

item. This discussion identified several difficulties associated with interpreting the

CONSORT requirements and, thus, assigning a score. The authors, therefore,

developed a set of “scoring principles”. Individual scores for all 37 items were

checked against these principles and modified where appropriate, and the previously

agreed group-score was also modified if required. The “scoring principles” were as

follows:

For information that was given in the publication but not in the section designated

by CONSORT, an individual score was reduced by one point.

“Not applicable” (NA) was used when it was clear from the information provided

that the CONSORT item was not applicable.

“Not stated/not applicable” (NS/NA) was used when it was not possible from the

information provided to explicitly determine if the CONSORT item was applicable,

but the authors considered it reasonable to assume that it was not applicable, i.e.

the information would have been provided if relevant. This principle was applied

to the following items:

Item 3b: “Important changes to methods after trial commencement (such

as eligibility criteria), with reasons”

Item 6b: “Any changes to trial outcomes after the trial commenced, with

reasons”

Item 14b: “Why the trial ended or was stopped”

Page 6 of 29

Endpoints and statistical analysis

The primary study endpoint consisted of the quality of publications (overall and

according to each section within CONSORT) as assessed by mean group-scores.

The overall and by-section quality of the publications as assessed according to the

proportion of group scores for which there was a positive result (scores 3 and 4) was

a secondary endpoint. For all analyses, greater emphasis was placed on results

obtained for the Methods and Results sections within the CONSORT checklist since

these sections address data quality.

The Mann-Whitney U test, which is an extension of the Mann-Whitney U test to

three or more groups, was used to compare the publication found to be of the

highest quality based on primary and secondary endpoint mean group scores with

the other publications included in the analysis.

Since the aim of the study was to assess the quality of reporting of data that are

used to inform current treatment guidelines, the primary and secondary endpoints,

as well as the comparison of the highest quality paper with the other papers, were

based on those publications that had been published prior to May 2014; the date

upon which the most recent guidelines (ESCMID) were made available.

Results

Eligible studies

The Trip, Cochrane, Medline and Embase searches identified a total of 7

publications that met the study inclusion criteria, one of which was excluded because

it was published after the date of the most recent current treatment guidelines. Two

additional qualifying publications were known by the authors, but one of these was

Page 7 of 29

also excluded due to its date of publication. A review of the current IDSA, ECIL and

ESCMID aspergillosis treatment guidelines identified no further publications for

inclusion in the analysis. Consequently, the study included a total of seven

publications (Figure 1). All papers were published in peer-reviewed journals, with six

of the journals specialising in infectious diseases, cancer, or haematology (Table

2).15-21

Primary and secondary outcomes

The group-scores for each of the 37 items within the CONSORT checklist according

to the publication are shown in Table S1 (available at www.XXXX). The overall and

by-section quality of the publications based on mean group-scores (primary

endpoint) and the proportion of group scores for which there was a positive result

(secondary endpoint) are summarised in Tables 3 and 4, respectively.

In general, the quality of reporting was markedly below that required by

CONSORT. Out of a possible score of 4 for each item, the mean group-score for all

seven papers was 2.44 overall, 2.27 for Methods and 2.78 for Results.

Corresponding results for the proportion of positive group-scores (3 or 4) were

54.2%, 48.5%, and 67.2%. Statistical analyses of the primary and secondary

endpoint data showed that there were significant differences between the seven

publications in terms of the overall quality of reporting (p<0.001 and p<0.002,

respectively), as well as the quality of reporting of Methods (p = 0.004 and p = 0.007,

respectively) and Results (p = 0.010 and p = 0.010, respectively).

Primary endpoint data showed that the Cornely publication was of the highest

quality in terms of the overall mean group score (3.15 + 0.93; 95% CI, 2.82, 3.47),

and mean group scores for Methods (3.36) and Results (3.40). Verweij ranked

lowest in terms of mean group scores overall (1.73 + 0.87; 95% CI, 1.41, 2.06), for

Page 8 of 29

Methods (1.45) and Results (1.78). The remaining five papers (Bowden, Herbrecht,

Ellis, Leenders, and Caillot) were tightly grouped with overall scores ranging from

2.35 to 2.53, and those for Methods and Results ranging from 1.83 to 2.43 and 2.67

to 3.22, respectively.

Assessment according to the proportion of positive scores also showed that

Cornely ranked highest overall (82.4%; 95% CI, 65.5, 93.2) and for Methods (92.9%)

It rated second highest for Results (90%), with Herbrecht scoring the highest for this

section (100%). The publication by Verweij had the lowest proportion of positive

scores overall (26.7%, 95% CI, 12.3, 45.9), for Methods (18.2%) and Results

(22.2%). With the exception of the aforementioned result for Herbrecht, secondary

endpoint data showed that the other five publications were of comparable quality.

Comparison between the Cornely report and the six other publications showed

that it was of significantly higher quality overall and for Methods as assessed by

mean group-scores (Table 5). Similar results were seen for secondary endpoint data

except for the comparison between the Cornely and Leenders reports, which just

failed to reach statistical significance (92.9% vs 61.5%; p = 0.055). Primary and

secondary endpoint data showed that the Cornely report was also of significantly

higher quality than the Verweij publication for Results, and of comparable quality to

the five other publications.

Discussion

Primary and secondary endpoint data revealed significant differences in the quality

of reporting between the seven publications, overall and for the CONSORT sub-

sections addressing Methods and Results. The Cornely publication achieved the

highest mean group scores (overall, Methods and Results) and the highest

Page 9 of 29

proportion of positive scores (overall and Methods). The Herbrecht publication

scored the highest for the proportion of positive scores for Results. Conversely, the

quality of the Verweij publication was rated as being lower than the other

publications.

A key reason for performing this analysis was to critique the RCT reports from a

clinical perspective. Current IDSA, ECIL and ESCMID guidelines (dated 2008, 2013

and 2014, respectively),2,11,12 recommend voriconazole for the primary treatment of IA

in based on the results of the Herbrecht study; with the ESCMID guidelines also

supporting the recommendation based on a conference abstract (Marr KA, et al. A

randomised, double-blind study of combination antifungal therapy with voriconazole

and anidulafungin versus voriconazole monotherapy for primary treatment of

invasive aspergillosis. 22nd European Congress of Clinical Microbiology and

Infectious Diseases (ECCMID); London, UK; March 31–April 3, 2012. Abstract

LB2812). The Herbrecht study showed that voriconazole was associated with better

survival, improved responses, and fewer severe drug-related adverse events than

deoxycholate amphotericin B (D-AMB).19 Liposomal amphotericin B (L-AMB) is

recommended as alternative therapy to voriconazole2,11,12 based on the results of the

study by Cornely, which compared two doses of L-AMB in the primary treatment of

IA.17 The IDSA grading system for ranking recommendations classes both the

Herbrecht and Cornely studies as AI, with A (the strength of recommendation)

defined as “Good evidence to support the recommendation for use” and I (quality of

the evidence) defined as “Evidence from >1 properly randomised, controlled trial”.2

ECIL and ESCMID also grades the Herbrecht study as AI, with A defined as “Good

evidence to support a recommendation for use” and “Strongly support a

recommendation for use”, respectively, and I defined as “Evidence from at least one

Page 10 of 29

properly randomized, controlled trial” and “Evidence from at least 1 properly

designed randomised controlled trial (orientated on the primary endpoint of the

trial”.11,12 However, ECIL grades the Cornely study as BI, with B defined as

“Moderate evidence to support a recommendation for use”.11 ESCMID also grades

the Cornely study as B (Moderately support a recommendation for use) but assigns it

a grade of II for quality of evidence (Evidence from at least 1 well-designed clinical

trial [incl. secondary endpoints], without randomisation…) even though the study was

randomised.12 Conversely, our study showed that the Cornely report was of higher

quality overall and for Methods than that by Herbrecht. This finding suggests that a

more rigorous assessment of published reports, such as this CONSORT analysis,

can provide further useful evaluation of the quality of papers over and above the

commonly used grading system for ranking recommendations that are solely based

on trial design and the magnitude of reported outcomes.

Two of the publications identified in our database searches, Maertens et al,

201522 and Marr et al, 2015,23 were published after the current IDSA, ECIL and

ESCMID guidelines were produced. Both publications reported on prospective,

double-blind, multicentre studies that evaluated voriconazole in the primary

treatment of IA or other invasive mould disease; the Maertens study comparing

voriconazole with isavuconazole,22 and the Marr study comparing voriconazole

monotherapy with voriconazole plus anidulafungin.23 The Maertens study received

financial support from Astellas and Basilea, and the Marr study from Pfizer.

Isaconazole was shown to be non-inferior to voriconazole in terms of survival and

associated with fewer study-drug-related adverse events.22 Voriconazole plus

anidulafungin failed to show superiority in 6-week survival compared with

voriconazole monotherapy and was associated with more treatment-emergent

Page 11 of 29

hepatobiliary adverse events.23 IDSA, ECIL and ESCMID will undoubtedly review the

Marr and Maertens publications, as well as other current evidence, when producing

updated treatment guidelines. As mentioned above, current ESCMID guidelines

reference Marr study data presented as an abstract at ECCMID in 2012 to support

their recommendation of voriconazole for the primary treatment of IA, giving the

evidence a grade of AI.12 Current ESCMID guidelines also reference Maertens study

data presented as an abstract at ECCMID in 2014 to support their recommendation

of itraconazole, giving the evidence as AII (Maertens JA, et al. A phase 3

randomised, double-blind trial evaluating isavuconazole vs. voriconazole for the

primary treatment of invasive fungal disease caused by Aspergillus spp. or other

filamentous fungi (SECURE). 24th European Congress of Clinical Microbiology and

Infectious Diseases, Barcelona, Spain, 10 - 13 May 2014. Oral presentation O230a).

Current ECIL guidelines only include the Marr abstract data, and rate the evidence

as CI; “Poor evidence to support a recommendation for use” of voriconazole plus

anidulafungin.

We decided to repeat our CONSORT analysis including the Maertens and Marr

publications, results of which are shown in Tables S2, S3 and S4 (available at

www.XXXX). Ranking the papers by overall mean group scores, Maertens was

highest (3.43 + 0.95; 95% CI, 3.10, 3.75), closely followed by Marr (3.21 + 0.81; 95%

CI, 2.92, 3.49) and Cornely (3.15 + 0.93; 95% CI, 2.82, 3.47. These three papers

also ranked highest in terms of mean group scores for Methods (3.36, 3.19, and 3.07

for Cornely, Maertens, and Marr, respectively) and for Results (3.78, 3.44, and 3.40

for Maertens, Marr and Cornely, respectively).

Our assessment of the publications according to mean group-scores (primary

endpoint) provided a more accurate reflection of their reporting quality compared

Page 12 of 29

with assessment according to the proportion of positive results (secondary endpoint).

For example, primary endpoint point data showed that the quality of reporting of

Results was higher for the Cornely publication than the Herbrecht publication (3.40

vs 3.22). Conversely, secondary endpoint data showed that a positive score (score 3

or 4) was achieved for nine of the 10 (90%) assessable items in the Cornely report

and all of the nine (100%) assessable items in the Herbrecht report. However, closer

inspection of the secondary endpoint data revealed that a score of 4 (item fully

reported) was achieved for only two of the sub-items in the Herbrecht paper

compared with five sub-items in the Cornely paper.

The difference in quality between the reports included in our study and the

supplementary analysis may be partly due to their year of publication. The report by

Verweij, which was assessed as being of the poorest quality, was published two

years before release of the first CONSORT Statement in 1996 and its adoption by

peer-reviewed journals. The other reports were all published after the 1996

CONSORT Statement, with the years of publication ranging from 1998 to 2007 in our

study and 1998 to 2015 in our supplementary analysis. Our study assessed the

quality of reporting using the latest 2010 CONSORT Statement, when it may have

been more appropriate to use the 1996 or 2001 CONSORT Statements that were

applicable at the time the reports were published,7,24 Indeed, some CONSORT

analyses have analysed quality of reporting using the CONSORT Statement relevant

to the given article’s year of publication, and have excluded articles before 1996. 25

However, we believe it is appropriate to use the most recent CONSORT tool as we

are assessing the quality of reporting of available RCTs that are used to inform the

latest clinical guidelines.

Page 13 of 29

As noted by authors of other CONSORT analyses, a problem in using the

CONSORT checklist is that each of the items listed consists of multiple statements,

each of which can be evaluated independently.26 This presents difficulties if a

“pass/failure” method is used to determine whether an item fulfils all the CONSORT

Statement requirements. If all components within an item must be fulfilled to certify it

as a “pass”, then an article that complied with none of the components would be

equated with one that complied with all but one.26 To address this issue, we

assessed each item using the four-point rating scale associated with the AGREE I

Instrument published in 2003.14 However, the specificity afforded by the four-point

scale remained problematic for items with a high number of components.

Assessment using the seven-point rating scale (1-strongly disagree to 7-strongly

agree) associated with the latest version of AGREE (AGREE II) may have proven

valuable in this study.27 An alternative method for quantitative use of CONSORT,

which has been adopted in other CONSORT analyses, has been to divide each

CONSORT item into sub-items.26, 28, 29

As described by other authors, we encountered many problems of logic, validity,

and interpretability relating to the CONSORT requirements and developed a set of

“scoring principles” to enhance the consistency of our individual scores.26 Some of

these problems related to the way in which an item was worded. For example, Item

6b states, “Any changes to trial outcomes after the trial commenced, with reasons”.

We decided that it was reasonable to assume the item was non-applicable if no

mention of changes to the trial were mentioned in the report; not stated/not

applicable (NS/NA) was assigned in such a situation. Changing the wording of Item

6b to, “Where there were changes to trial outcomes after the trial started, were

reasons given?”, would aid its interpretation and assessment.

Page 14 of 29

Whether an article has fulfilled all the CONSORT requirements may also be

influenced by the word-count imposed by some journals. Abstracts are often limited

to 250 to 300 words, making it difficult to include all the information required by the

Extended CONSORT Statement for Reporting RCTs in Journal and Conference

Abstracts. Restrictions to an article’s total word-count may also result in loss of

important detail, especially the introductory section.26

In summary, this is the first application of CONSORT analysis to RCTs for

treatment of IA. As with previous CONSORT work, a number of issues of

methodology and interpretation were identified, which should be addressed in future

iterations of CONSORT. In an attempt to standardise the assessment of CONSORT

items and improve objectivity between authors, a rating scale was used. This

approach showed significant quality differences between the seven RCTs analysed,

with the Cornely report being of the highest quality and the Verweij report being rated

least well compared with the other studies. It is reasonable to place greater value on

data from well-designed, statistically robust, and well-reported studies as evaluated

by CONSORT. However, exactly how this value is apportioned remains unresolved.

The limitations of CONSORT must also be considered: for example, two studies may

score similarly in CONSORT even if one shows markedly better outcomes than the

other, as CONSORT does not evaluate the magnitude of reported outcomes. Given

these inconsistencies, does the CONSORT statement have any clinical relevance?

Should the evidence grading used by American (IDSA) or European (ECIL and

ESCMID) IA treatment guidelines include an assessment of quality of reporting

according to the CONSORT statement? If so, it seems likely that an impact on

clinical recommendations would follow.

Page 15 of 29

Acknowledgements

We thank Corinne LeReun for her valuable statistical advice and for performing the

statistical analyses. We also thank Gilead Sciences Ltd., Cambridge, United

Kingdom for financially supporting logistical aspects of this study. Funding was

restricted to providing support to one of the authors (PMI) and to Corinne LeReun,

and to the reimbursement of authors’ travel expenses for face-to-face meetings.

Gilead Sciences Ltd had no influence on the study design; collection, analysis and

interpretation of data; writing of the report; or the decision to submit the paper for

publication.

Conflict of interest

BLJ has received educational and/or scientific grants from and served on advisory

boards for Gilead, Pfizer, Astellas, and MSD. MDR has received educational and/or

scientific grants from and served on advisory boards for Gilead, Pfizer, Astellas, and

MSD. PMI received financial support from Gilead Sciences Ltd., Cambridge, UK for

her role in this article. SGA has received educational and scientific grants from

Gilead, Pfizer, Astellas, MSD, Bio-Rad, and Associates of Cape Cod. He is a

member of the trial steering committee on the AmBiGuard study (a Gilead-sponsored

study).

Page 16 of 29

References

1. Richardson M, Lass-Flőrl C. Changing epidemiology of systemic fungal

infections. Clin Microbiol Infect 2008; 14(Suppl. 4): 5-24.

2. Walsh TJ, Anaissie EJ, Denning DW et al. Treatment of aspergillosis: clinical

practice guidelines of the Infectious Diseases Society of America. Clin Inf Dis

2008; 46: 327-360.

3. Baddley JW, Andes DR, Marr KA et al. Factors associated with mortality in

transplant patients with invasive aspergillosis. Clin Infect Dis 2010; 50(12): 1559-

1567.

4. Hope WW, Walsh TJ, Denning DW. Laboratory diagnosis of invasive

aspergillosis. Lancet Infect Dis 2005; 5: 609-622.

5. Sibbald B, Roland M. Understanding controlled trials: Why are randomised

controlled trials important? BMJ 1998; 316: 201.

6. Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias.

Dimensions of methodological quality associated with estimates of treatment

effects in controlled trials. JAMA 1995; 273(5): 408-412.

7. Begg C, Cho M, Eastwood S et al. Improving the quality of reporting of

randomised controlled trials. The CONSORT statement. JAMA 1996; 276(8):

637-639.

8. Moher D, Hopewell S, Schulz KF et al. CONSORT 2010 explanation and

elaboration: updated guidelines for reporting parallel group randomisation trials.

BMJ 2010; 340: c869. doi:10.1136;bmj.c869.

9. CONSORT website. http://www.consort-statement.org/about-consort/endorsers

(accessed 28 April 2016).

Page 17 of 29

10.Turner L, Shamseer L, Altman DG et al. Consolidated standards of reporting

trials (CONSORT) and the completeness of reporting of randomised controlled

trials (RCTs) published in medical journals (Review). Cochrane Database of

Systematic Reviews 2012, Issue 11. Art. No.: MR000030. DOI:

10.1002/14651858.MR000030.pub2.

11.Herbrecht R, Tissot F, Agrawal S et al. 2013-update of the ECIL Guidelines for

Antifungal Therapy in Leukemia and HSCT Patients (ECIL-5). Presented at the

5th European Conference on Infections in Leukaemia. Sophia Antipolis, France,

20–21 September 2013.

http://www.kobe.fr/ecil/telechargements2013/ECIL5%20Antifungal

%20Therapy.pdf (accessed 28.04.16).

12.Ullmann AJ, et al. ESCMID, ESFIG and ECCMM guidelines for the diagnosis and

treatment of Aspergillus disease. Presented at the European Congress of Clinical

Microbiology and Infectious Diseases 2014. Barcelona, Spain 10-13 May 2014.

https://www.escmid.org/escmid_publications/escmid_elibrary/?q=ullmann&tx_solr

%5Bfilter%5D%5B0%5D=main_filter_eccmid%253Atrue&tx_solr%5Bfilter%5D

%5B1%5D=entry_type%253AWorkshop%2Blecture&tx_solr%5Bfilter%5D

%5B2%5D=author%253AAndrew%2BJ.%2BUllmann&tx_solr%5Bsort

%5D=created_desc%2BDESC (accessed 17.06.16).

13. Hopewell S, Clarke M, Moher D et al. CONSORT for reporting randomised

controlled trials in journal and conference abstracts: explanation and elaboration.

PLoS Med 2008; 5(1): e20. doi:10.1371/journal. pmed.0050020.

14.The AGREE Collaboration. Development and validation of an international

appraisal instrument for assessing the quality of clinical practice guidelines: the

AGREE project. Qual Saf Health Care 2003; 12(1): 18–23.

Page 18 of 29

15.Bowden R, Chandrasekar P, White MH et al. A double-blind, randomised,

controlled trial of amphotericin B colloidal dispersion versus amphotericin B for

treatment of invasive aspergillosis in immunocompromised patients. Clin Infect

Dis 2002; 35(4): 359-366.

16.Caillot D, Thiébaut A, Herbrecht R et al. Liposomal amphotericin B in combination

with caspofungin for invasive aspergillosis in patients with hematologic

malignancies: a randomized pilot study (Combistrat trial). Cancer 2007; 110(12):

2740-2746.

17.Cornely OA, Maertens J, Bresnik M et al. Liposomal amphotericin B as initial

therapy for invasive mold infection: a randomized trial comparing a high-loading

dose regimen with standard dosing (AmBiLoad trial). Clin Inf Dis 2007; 44(10):

1289-1297.

18.Ellis M, Spence D, de Pauw B et al. An EORTC International multicentre

randomised trial (EORTC Number 19923) comparing two dosages of liposomal

amphotericin B for treatment of invasive aspergillosis. Clin Infect Dis 1998; 27(6):

1406-1412.

19.Herbrecht R, Denning DW, Patterson TF et al. Voriconazole versus amphotericin

B for primary therapy of IA. NEJM 2002; 347(6): 407-415.

20.Leenders ACAP, Daenen S, Jansen RLH et al. Liposomal amphotericin B

compared with amphotericin B deoxycholate in the treatment of documented and

suspected neutropenia-associated invasive fungal infections. Br J Haematol

1998; 103(1): 205-212.

21. Verweij PE, Donnelly JP, Kullberg BJ, Meis JFGM, De Pauw BE. Amphotericin B

versus amphotericin B plus 5-fluorocystosine: poor results in the treatment of

proven systemic mycoses in neutropenic patients. Infection 1994; 22(2): 81-85.

Page 19 of 29

22.Maertens JA, Raad II, Marr KA et al. Isavuconazole versus voriconazole for

primary treatment of invasive mould disease caused by Aspergillus and other

filamentous fungi (SECURE): a phase 3, randomised-controlled, non-inferiority

trial. Lancet 2015 Dec 9; [e-pub]. (http://dx.doi.org/10.1016/S0140-

6736(15)01159-9).

23.Marr KA, Schlamm HT, Herbrecht R et al. Combination antifungal therapy for

invasive aspergillosis: a randomized trial. Ann Intern Med 2015;162(2): 81-9. doi:

10.7326/M13-2508.

24.Moher D, Schulz KF, Altman DG. The CONSORT statement: revised

recommendations for improving the quality of reports of parallel group

randomized trials. Lancet 2001; 357(9263): 1191-1194.

25. Bousquet PJ, Calderón MA, Demoly P et al. The Consolidated Standards of

Reporting Trials (CONSORT) Statement applied to allergen-specific

immunotherapy with inhalant allergens: a Global Allergy and Asthma European

Network (GA(2)LEN) article. J Allergy Clin Immunol 2011; 127(1): 49-56.

26. Moberg-Mogren E, Nelson DL. Research concepts in clinical scholarship:

Evaluating the quality of reporting occupational therapy randomized controlled

trials by expanding the CONSORT criteria. Am J Occup Ther 2006; 60(2): 226-

235.

27.AGREE Next Steps Consortium (2009). The AGREE II Instrument. AGREE,

http://www.agreetrust.org (accessed 28 April 2016).

28.Stinson JN, McGrath PJ, Yamada JT. Clinical trials in the Journal of Pediatric

Psychology: applying the CONSORT Statement. J Pediatr Psychol 2003; 28(3):

159-167.

Page 20 of 29

29.Moher D, Jones AJ, Lepage L for the CONSORT Group. Use of CONSORT

Statement and quality of reports of randomized trials: a comparative before and

after evaluation. JAMA 2001; 285(15): 1992-1995.

Page 21 of 29

Table 1. AGREE instrument rating scale used to assess whether the RCT reports

fulfilled the CONSORT checklist.14

Rating scale Definition

1: Strongly Disagree Assessor is confident that the item has not been fulfilled

at all or no information is available

2: Disagree

3: Agree

Assessor is unsure whether the item has been fulfilled;

with ‘Agree’ or ‘Disagree’ being used depending on the

extent to which the issue has been addressed.

4: Strongly Agree Assessor is confident that the item has been fully met

Page 22 of 29

Abbreviations: AGREE: Appraisal of Guidelines for Research and Evaluation.

Page 23 of 29

Table 2. Details of the seven RCT reports included in the analysis.

Reference Type of RCT Treatments Financial

support?

Formal study

group?

Bowden R et al,

Clin Infect Dis 200215

Prospective, double-blind,

multicentre

ABCDa

d-AmB (1 or 5 mg/kg/day)

Sequus No

Caillot D et al,

Cancer 200716

Prospective, open, multicentre L-AmBa + caspofungina L-AmB

10 mg/kg/day

Gilead No

Cornely OA et al,

Clin Inf Dis 200717

Prospective, double-blind,

multicentre

L-AmB 3 mg/kg/day

L-AmB 10 mg/kg/day

Gilead No

Ellis M et al,

Clin Infect Dis 199818

Prospective, multicentreb L-AmB 1 mg/kg/day

L-AmB 4 mg/kg/day

Nexstar EORTC

Herbrecht R et al,

NEJM 200219

Prospective, open, multicentre d-AmB (1 – 1.5 mg/kg/day)

Voriconazolea

Pfizer EORTC

Leenders ACAP et al,

Br J Haematol 199820

Prospective, open, multicentre L-AmB 5 mg/kg/day

d-AmB (1 mg/kg/day)

Nexstar No

Page 24 of 29

Verweij PE et al,

Infection 199421

Prospective, single-centreb d-AmB (1 mg/kg/day)

d-AmB + 5-FC (150 mg/kg/day)

No No

a. Standard dose.

b. The publication does not state whether the study was open or blinded.

Abbreviations: ABCD: amphotericin B colloidal dispersion, d-AmB: L-AmB: liposomal amphotericin B, EORTC: European Organisation for Research and

Treatment of Cancer, 5-FC: 5-Fluorocytosine

Page 25 of 29

Table 3. Report quality as measured by mean group scores: overall and by section.a,b

Individual reportsc All reports

Data items Cornely Bowden Herbrecht Ellis Leenders Caillot Verweij

Overalld 1 - 25 3.15 ± 0.93

(2.82, 3.47)

2.53 ± 1.05

(2.16, 2.90)

2.42 ± 1.03

(2.06, 2.·79)

2.39 ± 1.17

(1.96, 2.82)

2.38 ± 0.98

(2.02, 2.73)

2.35 ± 1.08

(1.96, 2.75)

1.73 ± 0.87

(1.41, 2.06)

2.44

Title and abstract 1a - 1b 3.00

(2 - 4)

3.00

(2 - 4)

1.50

(1 - 2)

3.00

(2 - 4)

1.50

(1 - 2)

3.00

(2 - 4)

1.50

(1 - 2)

2.36

Introduction 2a - 2b 3.50

(3 - 4)

3.50

(3 - 4)

3.00

(3 - 3)

2.50

(2 - 3)

2.00

(2 - 2)

3.00

(3 - 3)

3.00

(3 - 3)

2.93

Methodse 3a – 12b 3.36

(1 - 4)

2.36

(1 - 4)

2.43

(1 - 4)

2.08

(1 - 4)

2.38

(1 - 4)

1.83

(1 - 4)

1.45

(1 - 3)

2.27

Resultsf 13a – 19 3.40

(2 - 4)

2.90

(2 - 4)

3.22

(3 - 4)

2.67

(1 - 4)

2.78

(1 - 4)

2.67

(1 - 4)

1.78

(1 - 3)

2.78

Discussion 20 – 22 2.67

(2 - 3)

2.33

(1 - 3)

1.00

(1 - 1)

3.00

(3 - 3)

3.00

(3 - 3)

2.67

(2 - 3)

2.67

(2 - 3)

2.48

Other information 23 - 25 1.67

(1 - 3)

1.33

(1 - 2)

1.67

(1 - 3)

1.67

(1 - 3)

1.33

(1 - 2)

2.33

(1 - 4)

1.00

(1 - 1)

1.57

Overall results and sections addressing data quality are shown in bold.

a. Overall results are given as mean + SD (95% CI) values, and results by section are expressed as mean (range) values.

b. Items considered as not applicable or not applicable/not stated were excluded from the calculations.

c. Reports are presented according to ranking based on overall scores.

d. P <0.001 for the difference between the seven reports (Kruskal-Wallis test).

e. P = 0.004 for the difference between the seven reports (Kruskal-Wallis test).

Page 26 of 29

f. P = 0.010 for the difference between the seven reports (Kruskal-Wallis test).

Page 27 of 29

Table 4. Report quality as measured by the proportion of group-scores classified as positive: overall and by section.a,b,c

Individual reportsd All reports

Data items Cornely Bowden Herbrecht Leenders Caillot Ellis Verweij

Overalle 1 - 25 82.4

(65.5, 93.2)

58.8

(40·7, 75·4)

57.6

(39.2, 74.5)

53.1

(34.7, 70.9)

48.4

(30.2, 66.9)

48.4

(30.2, 66.9)

26·7

(12·3, 45·9)

54·2

Title and abstract 1a - 1b 50.0 50.0 0 0 50.0 50.0 0 28·6

Introduction 2a - 2b 100 100 100 0 100 50.0 100 78·6

Methodsf 3a – 12b 92.9 50.0 50.0 61.5 33.3 33.3 18·2 48·5

Resultsg 13a – 19 90.0 80.0 100 66.7 55.6 55.6 22·2 67·2

Discussion 20 – 22 66.7 66.7 0 100 66.7 100 66·7 66·7

Other information 23 - 25 33.3 0 33.3 0 33.3 33.3 0 19·0

Overall results and sections addressing data quality are shown in bold.

a. Positive results were defined as those that scored 3 or 4.

b. 95% CI values are given in parenthesis.

c. Items considered as not applicable or not applicable/not stated were excluded from the calculations.

d. Reports are presented according to ranking based on overall scores.

e. P <0.002 for the difference between the seven reports (Kruskal-Wallis test).

f. P = 0.007 for the difference between the seven reports (Kruskal-Wallis test).

g. P = 0.010 for the difference between the seven reports (Kruskal-Wallis test).

Page 28 of 29

Table 5. Difference in quality between the Cornely report and the six other reports.a

Data items Cornely vs. (p value)b, c

Bowden Caillot Ellis Herbrecht Leenders Verweij

Mean group-scores

- Overall 1 - 25 0.012 0.003 0.009 0.003 0.001 <0.001

- Methods 3a – 12b 0.026 0.002 0.040 0.013 0.001 <0.001

- Results 13a – 19 NS NS NS NS NS 0.002

Proportion of group-scores classified as positive

- Overall 1 - 25 0.035 0.004 0.004 0.028 0.012 <0.001

- Methods 3a – 12b 0.014 0.002 0.002 0.014 NS <0.001

- Results 13a – 19 NS NS NS NS NS 0.004

Abbreviations: NS: not significant

a. Items considered as not applicable or not applicable/not stated were excluded from the calculations.

b. P values represent a statistically significant difference in quality in favour of the Cornely report.

c. Mann-Whitney U test.

Page 29 of 29