transparent and accurate reporting increases reliability, utility and impact of your research:...

Transparent and accurate reporting increases reliability, utility and impact

of your research:

Reporting guidelines and the EQUATOR Network

Iveta Simera

The EQUATOR NetworkCentre for Statistics in Medicine, Oxford, UK

13 November 2010

2

Biomedical research & its publication

• Biomedical research should advance scientific knowledge and – directly or indirectly – lead to improvements in prevention or management of illness

• Publications are usually the only tangible evidence that a study was done, how it was done, and what the findings were– PubMed: more then 20 mil citations

(500,000 added each year on average = around 1 per minute)

3

Purpose of research publication

• Large number of articles but are they reliable? Can they be used?

• Articles are written for multiple readerships:– Clinicians

•To learn how to treat their patients better– Researchers:

•To inform their own research•To help plan a similar study •To include the study in a systematic review

– Patients/consumers:•To aid personal decision-making

– Policy makers/purchasers: •To aid policy decision-making

• … should present sufficiently detailed information to allow assessment of study reliability and relevance and comparison across studies (important for SR and CPG)

4

Much evidence of poor reporting

• Hundreds of reviews of methodology of published research articles– Highlighting severe

deficiencies in reporting (biased or unusable research reports)

– These deficiencies limit or prevent use of many of the published findings further in research or clinical practice

5

Examples: Poor reporting of RCT methodolgoy

• 519 RCTs published in Dec 2000: failing to report key aspects of methodology

– 79% did not report the method of random sequence generation *

– 82% did not report the method of allocation concealment *

– 73% did not report a sample size calculation *– 55% did not report a defined primary outcome (s) – 60% did not report whether blinded

(Chan & Altman Lancet 2005)

• 616 RCTs published in Dec 2006: – Reporting of several important aspects of trial methods

improved between 2000 and 2006 (*); however, the quality of reporting remains well below an acceptable level.

(Hopewell BMJ 2010)

6

Example: Poor description of intervention

• Glasziou et al. (BMJ 2008) – assessed descriptions of treatments in 80 published

articles(55 randomised trials & 25 systematic reviews)

– crucial elements of the interventions were missing in 41 of those studies (of 25 SR only 3 provided intervention description sufficient for implementation)

• Herbert & Bo (BMJ 2005)– Discuss effect of quality of intervention on SR– “Assessment of the quality of the intervention relies on

sufficient detail in trial reports, but many reports provide only superficial description of complex interventions. .. Interventions should be described in sufficient detail to enable reader to assess if the intervention was administered well as the quality of intervention can affect the results of clinical trials.”

7

Systematic reviews and clinical guidelines

• Poor reporting is a serious problem for SR and CPG

• In the UK – Cochrane reviews are highly regarded and yet ..– “The biggest problem was the quality of reporting, which

did not allow us to judge the important methodological items ...”

– “Data reporting was poor. 15 trials met the inclusion criteria for this review but only 4 could be included as data were impossible to use in the other 11.”

– “If the CONSORT recommendations were followed in the reporting of future studies, the effects of Morita therapy would be clearer. Much important data within the included studies were so poorly reported that clinicians, funders and recipients of care might have reason to feel let down by the research community.”

(Cochrane Library, accessed on 18 Sept 07)

8

Examples: Poor reporting of systematic reviews

• Reviews are not immune to the problems of poor reporting– Epidemiological and reporting characteristics and

bias-related aspects of 300 systematic reviews (of which 125 were Cochrane reviews). The overall quality of reporting of key aspects of methodology was very inconsistent with particularly discouraging findings for non-Cochrane reviews.

(Moher; PLoS Med, 2007, 4(3), e78)

9

Common problems in research reporting

• Unethical reporting practices with serious adverse consequences– Non-reporting or delayed reporting of whole studies– Omissions or misinterpretation of results in abstracts– Omission of crucial information in the description of

research methods and interventions– Inconsistencies between study protocol and publication– Selective reporting of only certain outcomes– Inadequate statistical reporting– Presenting data (graphs) in confusing or misleading ways

(particularly important for presenting benefits and harms)– General misinterpretation of study results (spin)

10

How to improve research reporting?

11

How to improve research reporting?

12

How to improve research reporting?

13

Reporting guidelines (RG)

• RG provide structured advice on what to include in a research report

• RG specify a minimum set of items required for a clear and transparent account of what was done and what was found in a research study, reflecting in particular issues that might introduce bias into the research

• Benefits of using RG:– Improved accuracy and transparency of publications – Easier appraisal of reports for research quality and relevance– Better further use of presented findings – Improved efficiency of literature searching

14

RG example: CONSORT Statement

15

http://www.consort-statement.org/

16

17

Other reporting guidelines

• PRISMA (SR/meta-analyses of RCTs) • STARD (diagnostic studies) • STROBE (observational studies)

... and many others (over 100 RG identified)

• As yet most guidelines have had limited impact– Passive dissemination through publication only - not

widely known– Complience not required by journals

18

EQUATOR Network

• EQUATOR Network is an international initiative set up to improve reliability and value of medical research literature

• EQUATOR promotestransparent accuratecompleteand timely

reporting of health research studies

19

EQUATOR main goals and activites

Website launched in Oct 2007

www.equator-network.org

EQUATOR seven major goals

• Promote responsible research reporting in practice (wider use of RG)

• Develop a comprehensive online resource centre

• Develop and establish an education and training programme

• Assist in the development, dissemination and implementation of

robust reporting guidelines

• Expand EQUATOR activities globally

• Assess use of reporting guidelines

• Audit reporting quality across the health research literature

20

EQUATOR online resources

All collated resources are available in our Library

Some of the resources are grouped according to relevance to our main user groups

21

Resources for editors

22

Guidance on reporting in journals’ I to A:examples

• General medical journals:

• Prakticky lekar– http://www.prolekare.cz/prakticky-lekar-pokyny– Very brief and general instructions on research

reporting

• BMJ (British Medical Journal)– http://resources.bmj.com/bmj/authors/types-of-

article/research – Very detailed, clearly worded requirements and links

to internationally accepted guidelines

23

Prakt Lekar – I to A

24

BMJ - I to A

25

Guidance on reporting in journals’ I to A:examples

• Specialty (Association) journals:

• Cesko-Slovenska Dermatologie– http://www.prolekare.cz/cesko-slovenska-

dermatologie-pokyny – Some instructions on reporting scientific content, no

reference to any internationally accepted RG

• British Journal of Dermatology– http://www.wiley.com/bw/submit.asp?ref=0007-

0963&site=1

26

Cesko-Slovenska Dermatologie

27

BJD

28

How to shift the ‘reporting culture’

• Collaboration of all parties involved in research publishing needed on a global scale– Scientists, research organisations, funders and regulators– Journals (editors, peer reviewers, publishers)– Other organisations (higher education, REC, ..)

• Working towards:– .. accurate, complete, transparent and timely reporting of

research studies is considered a ‘norm’ (not something extra)

• How to achieve this?– Clearly defined policies, requirements and expectations– Provision of tools and other resources– Education and training– Motivation and incentives– Application of safeguards and checks

• See ‘Steps to consider’ in Simera et al. BMC Medicine 2010,8:24

29

EQUATOR on a global scale

• Last year, our website has been visited from most countries in the world

• Traffic doubled compared with the previous year

30

EQUATOR Spanish website – launched July 2010

We are looking for collaborators to establish local centres of activities supporting better reporting of research studies

Signed collaboration agreement with PAHO to raise standards of research reporting in South America and Caribbean

31

Czech medical research contributes to SR

• More difficult to write a good paper (although many really bad papers were written by English native speakers!)– Important is a good understanding of principles of

good research methodology (including terminology) and principles of responsible reporting

– Translating educational materials (e.g. E&E papers) into ‘local’ languages can improve understanding

-Selected for inclusion in Cochrane SR of RCTs

-No information about pts allocation into groups

-6 emails with the authors

-Final answer:

-“It was a double-blind study”

www.equator-network.org

Dr Iveta Simera, Head of Programme DevelopmentEQUATOR Network, Centre for Statistics in Medicine, Oxford, UK

iveta.simera@csm.ox.ac.uk

33

EQUATOR Steering Group

Doug Altman, Centre for Statistics in Medicine, University of Oxford, UK

John Hoey, University of Toronto, Canada

Ana Marusic, University of Split, Croatia

David Moher, Ottawa Health Research Institute, Canada

Kenneth F. Schulz, Family Health International, Chapel Hill, USA