EXPERIMENTAL PAPERS IN THE BJS – LESSONS LEARNED AND ROOM FOR

IMPROVEMENT

Editors Assistant ProjectMalin Sund

2011

Background

• The importance, relevance and quality of experimental papers in the BJS are under debate – Lack of clinical relevance – lack of interest among the BJS

readership?– Do experimental papers reach the correct readership?– Are the experimental papers a “burden” for the journal

and its IF?– What experimental papers are successful in a clinical

journal?

Definition of an experimental paper

All papers in which a novel (not in clinical practice) topic is explored using in vitro, animal

and/or human samples

Materials & Methods• Experimental papers in the BJS 2004-2008 were reviewed in detail, and a

database constructed (145 original articles + 21 leaders/reviews)

– Basic article descriptors– Field of surgery, disease/topic, aim, set-up, ethics, statistics– Evaluation of research question – clinical vs. basic science– Citation metrics

• Total number and toward IF• Total number and toward IF in surgical journals• Total number and toward IF in non-surgical journals

• Similar database for ASO and Annals 2007-2008 (188 original articles)

• Citation metrics for J Surg Res 2007-2008

• Statistical analysis using non-parametric ANOVA (Kruskal-Wallis) for multiple groups and Mann-Whitney when comparing two groups

BJS 2004-2008

• Number of experimental papers stable• Number of leaders/reviews fluctuate more• Average (SD) number of

figures 3 (1,95)tables 2 (1,64)references 30 (8,70)authors 6 (1,45)disciplines 2 (1,07)

Recommendations to authors 2011:A maximum of 5 figures/tables and30 references!

2008

2004200520062007

Where do experimental papers come from?

BJS 2004-2008

• As expected the total average citations 2004 are significantly higher than 2007-2008• No significant differences in citations within 2 years 2004-2008

***

ns

BJS 2004-2008

• No significant differences among citations among papers in relation to field of surgery(ANOVA)• In pairwise comparison both GI groups were significantly more cited than Breast/Endo (p = 0.02)

• Most citations total and within 2 years in non-surgical journals• Surgical journals cite experimental papers less • Most surgical citations – general, transplantation, CIT < 2 groups

BJS vs. the competition

• The number of experimental papers is not significantly different between BJS, ASO and Annals• J Surg Res has a focus on experimental surgical research and the majority of papersare experimental

BJS vs. the competition

• Both total citations and citations within 2 years (toward IF) are significantly higher for experimental papers in Annals, when compared to BJS, ASO and J Surg Res• ASO has higher citations on experimental papers than BJS

***

***

*

0,052

BJS vs the competition

• Half of all citations within 2 years. Note J Surg Res!

• Most citations total and within 2 years in non-surgical journals

• Surgical journals cite experimental papers less

BJS CIT ≤ 2• Only 9 (6.2%) “0-citation” papers among experimental papers in the

BJS 2004-2008 • 24 (16%) papers with 2 or less citations (CIT ≤ 2)• 68% of these citations were within 2 years (count toward IF)

• Top 10 - Average total citations 41,3 (range 28-80), 27% within 2 years

• 53 papers have ≥ 10 citations (37%), 101 ≥ 5 citations (70%)• Most citations in non-surgical journals (74%)• Less citations in surgical journals (26%)

BEST IN SHOW

BJS Top 10, CIT ≥ 10 vs. CIT ≤ 2

Field of surgerySuccessful experimental papers TEND to be….

“GI related translational papers on cancer, with a topic related to biomarkers, prognosisprediction or pathogenesis”

BJS Top 10

BJS Top 10, CIT ≥ 10 vs. CIT ≤ 2

Top 10

 

Europe (n=113)

Asia (n=29)

North America

(n=2)

Top 10 50% 40% 10%

CIT ≥ 10 70% 28% 2%

CIT ≤ 2 79% 21% 0%

Experimental paper IF

Do the experimental papers have a negative impact in the IF?

What would the IF of BJS be based on experimental papers only?

Year IF Experimental IF Difference

2006 4,092 3,397 ns

2007 4,304 3,985 ns

2008 4,921 4,148 ns

An IF factor constructed only on experimental papers is not significantly different than the actual IF of BJS

Questions

• IF of experimental papers similar to that of the BJS – but would it be better without these?– Likely not…

• How to predict what will be highly cited?– GI, surgical device/intervention, cancer, biomarker, translational

• Why is there such a difference in papers related to field of surgery?– Low numbers and/or quality on trauma, breast/endo– Does this reflect the overall low number/quality of papers within the

field published in the BJS?

• How to cut the tail of poor papers?

Conclusions

• The overall quality of experimental papers is decent • 70% have ≥ 5 citations total, and 27% ≥ 5 citations within 2

years• The papers attract the majority of citations from outside the

surgical readership – good/bad?• Citations totally and within two years are low in surgical

journals• Translational papers attract most citations – perceived higher

clinical relevance?

Possibilities

• Focus on translational papers with a clear clinical relevance – clinical relevance box

• Attract more experimental research on trauma, breast/endo• GI papers highly cited - maintain position • Cancer and surgical interventions/device related papers are

highly cited • The experiments and set-up needs to be scrutinized - Editors

with specific interest in experimental research?

Acknowledgements

• Supervisor Kjetil Søreide – Tack, tack!• Bibliometrics Gavin Stewart /Wiley• Editorial Team for good discussions