key topics in surgical research and methodology || how to read a paper

545T. Athanasiou (eds.), Key Topics in Surgical Research and Methodology,DOI:10.1007/978-3-540-71915-1_40, © Springer-Verlag Berlin Heidelberg 2010

How to Read a Paper

Hutan Ashrafi an and Thanos Athanasiou

40

Abstract To adequately equip surgeons with the required skills necessary to successfully read a paper, one needs not only time and energy but also a core level of experience. This chapter aims to classify the components that make up a scientifi c paper with the goal of presenting the audience to some of the analyti-cal concepts that will enable the successful reading of a surgical paper.

40.1 Introduction

Scientific papers are the most favoured vehicles through which research is communicated. Each manu-script has been specifi cally designed to allow the reader to understand why a research question was addressed, how this was done and what the implica-tions are for the newly discovered results. As a result, unlike the text of a novel, wherein a story develops in sequential order, the text of a scientifi c manuscript is totally different, wherein it objectively states a prob-lem that needs solving, and states how the authors went about solving it. Thus, the process of reading a scientifi c text varies signifi cantly from normal prose, and requires a consistent application of both analytical and critical faculties.

Although surgical papers in print would have under-gone a process of peer-review, the ultimate responsibil-ity of assessing published material lies with the reader. To adequately equip surgeons with the required skills necessary to successfully read a paper, one needs not only time and energy but also a core level of experi-ence. This chapter aims to classify the components that make up a scientifi c paper with the goal of presenting the audience to some of the analytical concepts that will enable the successful reading of a surgical paper.

H. Ashrafi an (�)Department of Biosurgery and Surgical Technology, Imperial College London, Queen Elizabeth the Queen Mother (QEQM) Building, Imperial College Healthcare NHS Trust, 10th Floor, St. Mary’s Hospital Campus, Praed Street, London W2 1NY, UKe-mail: h.ashrafi [email protected]

Contents

40.1 Introduction ............................................................ 545

40.2 The Conceptual Basis of A Scientifi c Paper ......... 546

40.3 Reasons to Read a Research Journal .................... 546

40.4 The Psychology of Reading a Paper ..................... 546

40.5 Originality ............................................................... 547

40.6 Types of Paper and Quality of Evidence .............. 548

40.7 Core Components ................................................... 549

40.8 Title .......................................................................... 550

40.9 Authorship and Ancillary Information ................ 550

40.10 Abstract/Summary ................................................. 551

40.11 Introduction/Backround ........................................ 551

40.12 Materials and Methods .......................................... 551

40.13 Results, Tables, Figures .......................................... 552

40.14 Discussion ................................................................ 552

40.15 Acknowledgements and Declarations ................... 553

40.16 References and Bibliography ................................ 553

40.17 Supplementary fi les ................................................ 553

40.18 Conference Discussion ........................................... 553

40.19 Editorial .................................................................. 553

40.20 The importance of assessing a paper .................... 553

40.21 Conclusions ............................................................. 554

References ........................................................................... 554

546 H. Ashrafi an and T. Athanasiou

40.2 The Conceptual Basis of A Scientifi c Paper

Some authors suggest that not only are research papers a simple method by which to communicate scientifi c discoveries, but they themselves can be considered an innovation that permitted scientifi c progress to occur through the transmission and interaction of information [8]. This concept is not unlike the theory that human thought capacity increased following a developmental improvement in our speech ability (vocalisation) through the evolutionary migration of our larynx further down in the neck when compared with other ape species [15]. Thus, the more we develop and read our scientifi c papers, the more advances we can make in research.

It can be considered that “all knowledge is the result of imposing some kind of order upon the reactions of the psychic system as they fl ow into our consciousness” [13]. Therefore, an imposition of order and decrease in chaos on our sensory perception is what leads to mean-ingful information. In keeping with this concept, we read scientifi c literature in order to decrease the uncer-tainty and chaos inherent in our current state of knowl-edge and therefore increase our personal information.

Claude E. Shannon in his 1948 paper “A Mathematical Theory of Communication” [19] introduced the con-cept of information entropy (a measure of chaos) sum-marised in the equation below:

2

1

logn

s j jj

H K p p=

= - å

where Hs provides a mathematical measure of disorder

that may exist in a quantity of given information. K is a constant, and p

j is the probability of fi nding one par-

ticular piece of information from among a subset of data. The role of a research paper is to alter the con-stant K in order to minimise and decrease the uncer-tainty and chaos in our current state of understanding to a state of better “more meaningful” understanding.

40.3 Reasons to Read a Research Journal

Although research journals are a means of communi-cating scientifi c papers, and thereby information, we read these in surgery not for academic interest alone, but more importantly for how we can apply these fi nd-ings to treat our patients. These reasons can be classi-fi ed into the following (Fig. 40.1):

40.4 The Psychology of Reading a Paper

In his seminal work on studying how scientists read sci-entifi c papers, educationalist Charles Bazerman closely studied and interviewed seven physicists on how they discerned information from the academic papers that they read [4].

His work was based on the premise that scientifi c reading habits are affected by psychological and

Fig. 40 .1 Reasons for reading a surgical journal

Translational medicine

Other

Academic vanity

As a news source within subspeciality

Advertising/applying for employment

Learning

Comparing

Technical

Improving patient managenentIdentifying ‘Best Care’

Identifying ‘Gold Standard’ procedures

New surgical procedures

New diagnostic procedures

Treatments

Diagnostic Modalities

Individuals and Units

from top units

from experienced individuals

from the mistakes of others

Reading a Surgical Journal

Molecular biology

Pathophysiology

Disease aetiology

To Understand DiseaseMechanism

To improve Clinical Treatment

40 How to Read a Paper 547

sociological variables, and further came up with the concept that all scientists have a dynamic knowledge-based mind-map (or schema) that can be built upon and expanded by information and data from new papers.

Here, he analysed each individual’s choice of paper (Fig. 40.2), identifying that these are picked by per-sonal research needs and the necessary self-updating for each scientist’s own particular speciality. “Must reading” was found to be proportional to the amount of research available in the relevant fi eld.

Understanding a paper (Fig. 40.3) relies on whether the manuscript’s subject is close to that of the reader’s own speciality. Increased familiarity of a subject to a reader will allow faster information gathering and a more complete understanding of the paper. However, if the paper is poorly written, then it is obvious that an increased effort for reading will be required, and an increased time requirement to assimilate the informa-tion presented.

40.5 Originality

Important scientifi c discoveries are frequently a result of their originality, although this is a notoriously

diffi cult concept to measure. To help quantify the degree of originality of the data from a new publication, Lynn Dirk, a specialist in science communication, has pro-posed a method to measure and score originality in a scientifi c manuscript [9]. Using this technique, each paper is broken down into three component units of hypothesis–methods–results. Each of the three compo-nents is assigned a value of originality of “P” – Previously reported or “N” – New. This then allows each paper to have an originality score for each of the hypothesis–methods–results subsections that put together can concisely refl ect the originality of the paper subsections. For example, if all three components of hypothesis–methods–results were new, then the paper would be scored as N–N–N, whereas if all three compo-nents were previously known, then they would be scored as P–P–P.

Using this typology, eight combinations of original-ity can be assigned to a scientifi c paper. Dirk went on to perform a mail survey on 301 scientists, 68% of whom responded. They rated papers selected from the “Citation Classics” in Current Contents® – Life Sciences over a 5 year period (Table 40.1), demonstrating that this tech-nique can be used to attain useful insights into assessing the originality of scientifi c papers.

Fig. 40.3 Mechanisms of understanding a scientifi c paper

In-depth reading of whole paper

In-depth reading of whole paperUnderstanding a Paper

Research close to researcher’s own field

If poorly written

Research not directly related toresearcher’s own

Increased effort of reading

Numerous re-reading of manuscript

Unexpected finding triggers more in-depth reading

Speedy selective reading to reveal new facts

Reading relies heavily on personalmethodological experience

Fig. 40.2 Choosing a scientifi c paper

directed by scientist’s own research needs

Necessary periodic scanning of relevant sources

Trigger words noticed during ‘scanning’ of manuscripts

Word of mouth

Reasons to read manuscripts

Mechanism of choice

Choice of Paper

25% from single word in title

75% from other triggers in the manuscript

Directly found in relevant database (e.g. PubMed)

Individuals or Institution who carried out research


40.6 Types of Paper and Quality of Evidence

Scientifi c research is not a homogeneous entity and can be broadly categorised into four main types (Fig. 40.4):

Many papers combine these for research elements to varying extents, and as a result, a number of research paper types are used to try and communicate this varied data.

Types of surgical research include topics that can:

Assess or improve upon surgical treatments• Assess or improve upon surgical disease diagnosis • and screeningElucidate underlying surgical disease aetiology and • pathophysiologyAssess or improve upon surgical skills and training• Assess or reduce surgical errors•

Typical types of surgical paper are catalogued below in Fig. 40.5.

In clinical research, scientifi c papers can be assessed by their “quality of evidence”, which can improve with increased subject numbers and randomisation of both patients and treatments (thereby decreasing the likeli-hood of false results). The traditional hierarchy of evi-dence in clinical papers has been (in descending order, with the most important fi rst) [20]:

Fig. 40.4 Types of scientifi c research

Testing a specific Hypothesis

Analytical

Methodological

Descriptive

Comparative

Scientific Research

Improving research techniques

Introducing research techniques

Listing the findings of a study

Formulating a hypothesis

Statistical analysis

Defining an ‘effect size’

Breaks down a postulation into itscomponent parts

Fig. 40.5 Catalogue of surgical research papers

Local

NationalInternational

Qualitative

Analysis Surgical Papers

Reviews

Prospective (Clinical Trial)

Retrospective (Survey)

Guidelines

Experimental

Clinical safetyFailure Mode and Effect Analysis (FMEA)

Other safety assessment tool studies

Interview-based studies

Decision

Observational study

Behavioural/Psychological studies

Error

EconomicalClinical Skills Assessment

Simulator-based assessment

Behavioural/Psychological studies

Quantitative

SystematicMolecular biology

PhysiologyNew surgical technique

Novel/New surgical technology (e.g. Robotics)

Animal experimentation

Non-ComparativeCase seriesCross-sectional survey

Randomised control trials (RCTs)

Cohort studies

Case-control studies

Comparative (with 2 or more groups)

Cross-sectional survey

Case seriesCase report

Non-Comparative

Comparative (with 2 or more groups)Case-control studies

Parallel group comparison

Matched comparison

Within-participant comparisonSingle Blinded

Double Blinded

Crossover

Placebo controlFactorial design

Non-systematicMeta-analysis

Cohort studies (most)

Originality score: hypothesis + methods + results

Frequency (%)

N + N + N 15

N + N + P 1

N + P + P 4

N + P + N 43

P + N + P 3

P + N + N 11

P + P + N 11

P + P + P 13

N, new; P, previous (Adapted from Dirk [9]).

Table 40.1 Originality scoring and frequency of highly cited papers from Current Contents® – life sciences


1. Systematic Reviews and Meta-analyses2. Randomised Controlled Trials (RCTs) with Defi n-

itive Results (confi dence intervals that do not over-lap the threshold clinically signifi cant effect)

3. RCTs with Non-Defi nitive Results (confi dence inter-vals that do overlap the threshold clinically signifi -cant effect)

4. Cohort Studies5. Case-controlled studies6. Cross-sectional surveys7. Case reports (only one or two patients)

To further reveal the level of quality of a scientifi c paper, the Centre for Evidence-Based Medicine at Oxford has

come up wisth a classifi cation for papers that grades them according to the level of evidence (Table 40.2) and subsequent grade of recommendation.

40.7 Core Components

Before starting to read a paper, it is important not to miss vital “self-evident” information. This includes in which journal or internet site is the paper published, and what audience is it aimed for? Is for instance in a purely a surgical journal where the contents are intended

Table 40.2 Levels of evidence and grades of recommendation modifi ed from the Oxford Centre for Evidence-based Medicine (May 2001) [17]

Level Therapy/prevention, aetiology/harm Prognosis Diagnosis

1a Systematic review (with homogeneity) of RCTs

Systematic review (with homogeneity) of inception cohort studies

Systematic review (with homogene-ity) of level 1 diagnostic studies

1b Individual RCT (with narrow confi dence interval)

Individual inception cohort study with > 80% follow-up

Validating cohort study with good reference standards

1c All or none studies All or none case-series Absolute SpPins and SnNouts

2a Systematic review (with homogeneity) of cohort studies

Systematic review (with homogeneity) of either retrospective cohort studies or untreated control groups in RCTs

Systematic review (with homogene-ity) of level greater than two diagnostic studies

2b Individual cohort study (including low-quality RCT; e.g. < 80% follow-up)

Retrospective cohort study or follow-up of untreated control patients in an RCT

Exploratory cohort study with good reference standards

2c “Outcomes” Research; Ecological studies

“Outcomes” Research

3a Systematic Review (with homogeneity) of case-control studies

Systematic Review (with homoge-neity) of 3b and better studies

3b Individual Case-Control Study Non-consecutive study; or without consistently applied reference standards

4 Case-series (and poor quality cohort and case-control studies)

Case-series (and poor-quality prognostic cohort studies)

Case-control study, poor or non-independent reference standard

5 Expert opinion without explicit critical appraisal, or based on physiology, bench research or “fi rst principles”

Expert opinion without explicit critical appraisal, or based on physiology, bench research or “fi rst principles”

Expert opinion without explicit critical appraisal, or based on physiology, bench research or “fi rst principles”

Grades of Recommendation

A Consistent level 1 studies

B Consistent level 2 or 3 studies or extrapolations from level 1 studies

C Level 4 studies or extrapolations from level 2 or 3 studies

D Level 5 evidence or troublingly inconsistent or inconclusive studies of any level


to be read only by one specialist group, or is it pub-lished in one of the internationally renowned medical journals such as The Lancet or The New England Journal of Medicine, whereby it might deliver a research message that carries a broader scope of interest.

Furthermore, the section under which the paper is published alludes to type of research being presented. Common sections include case reports, clinical trials, reviews and meta-analyses. There are, however, some exceptions, and thus, for example, in the journal Nature, original research papers are divided into the categories of Articles or Letters, although the latter should not be con-fused with the totally separate Correspondence section.

Scientifi c papers reporting empirical fi ndings are traditionally structured by the IMRD system: Intro-duction, Methods, Results and Discussion [1]. This system is still currently in use, but has been expanded on, to add a variety of extra information for the scien-tifi c reader.

To assess international reading strategies of IMRD articles, delegates at the 6th General Assembly and Conference of the European Association of Science Editors (EASE) were surveyed on their reading-order of paper subsections [7]. It was demonstrated that peo-ple rarely followed IMRD when reading as scientists (15%), but were more likely to use it if reading as reviewers (42%), and even more likely when reading as editors (56%). “Hard” scientists (physicists and chem-ists) used IMRD the most, incorporating this sequence in 48% of their reading strategies, biomedical scientists in 33.3% and social scientists the least at 17.8%. Although native-speakerhood can affect reading strat-egy, age does not seem to be a signifi cant factor.

A typical surgical paper is broken down into the fol-lowing subsections:

Title• Author(s)• List of Departments and Institutions involved in the • projectAbstract• Introduction or Background• Materials and Methods• Results• Figures and tables• Conclusion or Discussion• Acknowledgements• References or Bibliography• Declaration of confl icts of interest or sources of • funding

Supplementary data/documents/fi les• Conference Discussion (at a meeting where the • paper may have been presented)

The majority of surgical journals use the above format in the sequence listed, though some variation does exist. Thus, for example, the Results and Conclusion sections are sometimes combined, or the declaration of the confl icts of interest may appear earlier in the manu-script, or an overall summary may appear at the end of the paper. Furthermore, each journal is characterised by its own unique use of fonts, printing style and refer-ence format.

Once the identifying details of a paper have been elicited, one can discern a fair amount regarding the information that can be derived from that paper. For example, is it original research, has it been by invita-tion only, is it in a high-impact journal and what is the reputation of the author or the institution writing the manuscript? As tools to equip the reader to a scientifi c framework in which to appraise a paper, they can apply the well-known SQ3R (Survey, Question, Read, Recite and Review) [18] and PQRST (Preview, Question, Read, Summary, Test) [21] reading strategies.

40.8 Title

This is a carefully chosen succinct “sound-bite” that has the dual purpose of attracting the reader’s attention to the manuscript’s topic, stating with the fewest pos-sible words, the contents of the paper and the type of research carried out.

40.9 Authorship and Ancillary Information

Each manuscript clearly identifi es the contributing authors. The ancillary information further specifi es the time and the academic unit from which the research was conducted and written. This helps the reader discern a number of factors. The date gives the reader a perspec-tive on the modernity of the research. Furthermore, some authors and units may have built up a reputation in a particular fi eld and thus their research can be con-sidered in the context of their previous academic work, whilst also standing as an independent source of data.


For surgical authorship, the most widely used author order is the classical “sequence-determines-credit” approach, where the fi rst author is the individual who has participated most in the compilation and composi-tion of the paper, whereas the last author is the person who has the most supervisory role on the work. However, the literature is suffused with controversy over author-ship, as increasingly this has become synonymous with de facto “academic ownership” of the ideas expressed in the research paper. A recent study examined the instruc-tions to contributors from 234 biomedical journals and revealed that only 21 (9%) journals specifi ed individual authorship contributions to be described [24]. The International Committee of Medical Journal Editors (ICMJE) request authors to disclose their contributions.

The ICMJE list three conditions that authorship credit should be based on [12]. They state that authors should meet conditions 1, 2 and 3:

1. Substantial contributions to conception and design, or acquisition of data, or analysis and interpretation of data

2. Drafting the article or revising it critically for important intellectual content

3. Final approval of the version to be published.

However, many manuscripts in the medical literature list “Honorary authors”, who do not fulfi l all the ICMJE criteria for authorship, and are cited in up to 60% of articles in some journals, accounting for up to 21.5% of all authors listed [3]. This has led to some confusion as how to credit each individual author on the work pre-sented. Other factors to take into account include the communicating (or “corresponding”) author often, but not always taking the role of the lead author, and the fact that the author sequence might be allocated on the basis of non-scientifi c values. For example, in the United Kingdom, the Research Assessment Exercise (RAE), which determines governmental research funding, gives more credit to the fi nal author than the second or some-times co-author to a paper, which has led to further uncertainty in the listed author sequence, as these may sometimes be changed so as to favour individual and unit funding values [14].

There are a number of alternatives to the sequence-determines-credit authorship: these include “equal con-tribution”, where all authors are listed alphabetically as equals; “percent-contribution-indicated”, where each author has his/her contribution quantifi ed; and the “fi rst-last-author-emphasis”, where the fi rst author gets the whole impact, the last gets 50%, and the remainder

divide the remaining impact [22]. There is currently no universally accepted authorship sequence in the scien-tifi c literature, and as a consequence, this has led to numerous propositions [23] for a unanimously accepted ranked sequence so as to specify individual contribution for each piece of work.

40.10 Abstract/Summary

The abstract contains a succinct (one or two para-graphs) summary of the paper. It leaves out much of the technicalities of the paper, but identifi es the research background and hypothesis, going on to highlight the most prominent results and conclusions. It therefore communicates the major points of a manuscript, and is helpful in distinguishing whether a paper is relevant to one’s fi eld, but is also a good summary to re-read after having read the whole manuscript.

40.11 Introduction/Backround

This section places the research question posed by the paper into context. In the fi rst few lines, it will identify the fundamental knowledge in any particular fi eld, and it will then focus on the particular area that the research is being performed. It essentially states the broad cur-rent knowledge to-date in that particular fi eld, and will then go on to specify what still needs to be known, and how the subsequent research of the paper is relevant to this. It is in the latter part of the Introduction that the research hypothesis should be stated and this should be a logical progression from the pre-requisite facts detailed in the earlier part of the introduction.Important questions to ask are:

1. Do the stated facts follow a logical sequence?2. Has a hypothesis of research question been clearly

stated?

40.12 Materials and Methods

Once the research question has been specifi ed by the Introduction, the Materials and Methods go on to spec-ify in detail how the question was answered. Depending on the research, whether scientifi c or statistical, each


step needs to be clearly outlined, detailing all the tools, techniques and instruments used. Typical examples would include listing, operative manoeuvres, methods of patient randomisation, power calculations, literature searches employed, laboratory equipment utilised or statistical software applied. As a result, this section explains what was done, and how this was archived. It also gives an idea of time period taken, and should give the reader enough data with which to repeat and repli-cate the exact experiments, should the reader choose to. Reproducibility of data is one of the cornerstones of modern science, and therefore as a result, it is vital that the Materials and Methods section be as accurate as possible in communication on how the experiments chosen were preformed.

Important questions to ask are as follows:

1. Are the methods and experiments salient to the questions asked?

2. How good are the chosen methods at answering the questions posed by the authors?

3. Are there better methods that the authors could have considered?

40.13 Results, Tables, Figures

This section reveals the results of the questions posed in the Introduction. These data will be communicated in the form of words, tables and fi gures. This informa-tion is only stated in this section, and will be analysed in more depth in the Discussion section that follows.

The results will list those of the experiments per-formed in order, and if the paper is clinical, it will begin with stating the patient demographics and base-line features.

Text, tables and fi gures need to be carefully scru-tinised, and taken into account while considering the methods used to achieve the results. All tables and fi g-ures should be put into a contextual framework in the corresponding text. Important questions to ask are as enlisted below:

1. Is the data clearly presented?2. Is the data relevant to the study question?3. What are the major fi ndings of the study?4. Are the results expected?5. Does the data support or confl ict with the authors’

claims?

6. Was adequate data presented to answer the research question?

7. What is the “quality” of the data/evidence? 8. Are the results valid? 9. Are there adequate controls?10. How do the results compare with other similar

studies?11. Are there trends in the data that are not mentioned

by the author?12. Are there data that were not presented?13. What are the practical implications?14. Do the results offer areas for future research?

40.14 Discussion

This section performs several functions. It allows the authors to analyse and interpret their results for the audience while also openly specifying any limitations to their study. The results can be placed in the context of the existing fi eld, and they allow for specifi c conclu-sions to be made on the study. The work can also be compared with previously published work in the same fi eld, so as to assess whether it agrees or contrasts with them. A broader meaning can be drawn from the stated conclusions and any further necessary research can be listed as a result of the study’s fi ndings.

For readers, however, the following questions need to be asked:

1. What are the “real” conclusions of the work? 2. Is the study signifi cant? 3. Are the conclusions logical and adequately

stated? 4. Do the results stated support the conclusions? 5. Do the conclusions, methods and results all make

sense when assessed together? 6. Do the results support or refute the stated hypothesis? 7. Are there any limitations to the study that are not

listed? 8. Is the study relevant to other populations? 9. How do the conclusions from this study fi t in with

previous studies?10. Is there suffi cient data to back up the interpreta-

tion in the discussion section?11. Are there any inconsistencies or unsubstantiated

claims?12. How reliable is the evidence in support of the

discussion?


13. What are the conclusions that can be applied to one’s own surgical practice?

14. How could the work have been done better?15. What new hypotheses and further experiments can

be suggested following what was found?

40.15 Acknowledgements and Declarations

Many papers have a short Acknowledgements section, where various contributions of colleagues and assis-tances who are recognised without having to list them as formal authors. This section is usually optional and typi-cally considers individuals who have helped the research with technical advice or even artistic help. However, it also a location where one can specify sources of mone-tary support, and increasingly this section specifi es the source of funding of a particular area of work.

Sources of fi nancial support can also be listed in the Declarations section, where any potential confl icts of interest are also mandatory. This, for example, one can openly discern whether a particular researcher bene-fi ted from one particular source such as a surgical equipment company, whilst also publishing positive results as to the effi cacy of that equipment.

40.16 References and Bibliography

This section lists the sources of information that were used in the paper. These are generally other published manuscripts, but occasionally can also be “unpublished data” or “personal communications”. The majority of these would have been used in the Introduction and Discussion sections, although they can also be mentioned in the Materials and Methods. They are rarely used in the Results section, as this part of the paper is by defi nition new. The references are typically listed at the end of a paper, and are usually displayed alphabetically or numer-ically in a format set by each individual journal.

40.17 Supplementary fi les

This section allows researchers to communicate infor-mation and data that may be relevant to the paper, but may be too lengthy or complex for placement in the

results, or maybe interpretable by only the super-spe-cialist. These fi les are put at the end of the paper, and typically include long records of statistical analysis or long lists of primary data.

40.18 Conference Discussion

Some papers are presented at national or international meetings before publication, and therefore once the paper is published, a short precis of the discussion by an expert panel that followed the presentation is also published alongside the paper itself. This allows the reader to gain insights from the thoughts and comm-nets on the paper by experts in the fi eld.

40.19 Editorial

Occasionally, a paper communicates results that are of notable importance or carry signifi cant medical impli-cations. In these cases, the editors of a paper may place a short accompanying piece to the paper, placing it in an appropriate context to the readers, whilst also high-lighting its salient features. This can be written by the editors themselves, or may be written following an invitation by the editors for an independent expert in the fi eld to comment on the manuscript.

40.20 The importance of assessing a paper

Not all journals are considered equal in the scientifi c literature, as they do not all publish papers with the same scientifi c rigour. While some journals set strin-gent rules for scientifi c precision, others have less demanding regulations. This is refl ected by the general rule that papers in stringent journals are cited more often than papers in the less demanding journals. This has led to journals being ranked according to their cita-tion numbers per article, otherwise known as “impact factor”. Since its introduction in the 1960s, these rank-ings have to some extent modifi ed editorship, as jour-nals with higher impact factors are generally considered to be of higher academic standing [6].


In 2005, CA was the highest ranked impact factor journal with a score of 49.74, The New England Journal of Medicine had a score of 44.016, Science and Nature had scores between 29 and 31, and the highest ranked purely surgical journal was the Annals of Surgery at 6.33.

In essence, therefore, much of the papers in these journals have been scrutinised before publication by the editorial staff, so as to ensure high levels of aca-demic accuracy and integrity. This is coupled with the journals’ desire to achieve high-impact factors, many of the papers that are submitted go through an arduous process of review, assessment and modifi cation before they are available for us to read.

Nevertheless, the responsibility of reading a paper and making important conclusions in response to the results lies fi rmly with the individual readers. Some papers can give misleading information that has seri-ous medical consequences. For example, a paper pub-lished in the high-impact journal The Lancet led to signifi cant confusion and subsequent decrease in the use of the MMR (measles, mumps and rubella) vac-cine, and as result of the subsequent disarray, the Editorial Board of The Lancet apologised for publish-ing a paper that misled its readers [10]. Although rare, other such cases may occur, and thus it is incumbent on the readers of any paper not to lose an air objectivity and scientifi c precision when reading manuscripts. Indeed, if fl aws are picked up, or there are any potential objections, then these can be communicated through the medical literature (such as in the correspondence section of journals) or even the non-medical media should the need arise.

When reading a paper, it is important to take into consideration that the manuscript may have been sub-jected to bias. This can be of three main types:

Methodological bias• – papers are published where the data is fl awed due to inaccuracies or mistakes in the research methods applied.Submission bias• – papers are only submitted when the scientists discern “positive” results from their work. For example, a surgical unit may only publish its data if there was a statistical difference between the end points of two compared procedures.Publication bias• – papers are only accepted by edi-tors when there are “positive” results from the research presented. This would result in a wider readership, increased citations and therefore a higher impact for the journal.

Finally, when assessing a paper, it is important to con-sider if the results presented as “signifi cant” are truly so. This requires knowledge of statistics, and a deeper understanding of the p value and confi dence intervals (not in the scope of this chapter). If these values are not fully understood in a paper, then it is important for sur-geons to make further endeavours in order to under-stand this, so as to better grasp the result of the paper.

40.21 Conclusions

It has sadly become increasingly common that some doctors do not read research papers [2]. They cite dif-fi cult statistics, editorial inadequacy [11] and poorly written manuscripts [16] as the reasons behind this. Indeed, one former editor of the British Medical Journal is famed for quoting that only 5% of published research papers attain the standards of scientifi c soundness, and in most journals this fi gure would be less than 1% [5]. In order for scientifi c information to be adequately dis-seminated and used, an indispensable equilibrium exists between both readers and authors to participate in the communicative process. Each group needs to attain insight into the others’ role, but readers specifi -cally need to place concerted effort into attaining the necessary skills with which to read, and successfully discern information from scientifi c papers in the ever expanding fi elds of medicine and surgery. These skills can be acquired by regular reading of scientifi c articles, the attendance of local and occasionally international conferences, and the prodigious application of objec-tive thought and scientifi c reason.

References

1. Albert T (1995) How “Imradiation” has ruined the writing of scientists. Eur Sci

2. Barraclough K (2004) Why doctors don’t read. Br Med J 329:1411

3. Bates T, Anic A, Marusic M et al (2004) Authorship criteria and disclosure of contributions: comparison of 3 general medical journals with different author contribution forms. JAMA 292:86–88

4. Bazerman C (1985) Physicists reading physics: schema-laden purposes and purpose-laden schema. Written Commun 2:3–24

5. Boseley S. Medical studies 'rubbish'. The Guardian 1998, 24 June, p5


6. Brown H (2007) How impact factors changed medical pub-lishing–and science. Br Med J 334:561–564

7. Burrough-Boenisch J (1999) International reading strategies for IMRD articles. Written Commun 16:296–315

8. de Solla Price D (1981) The development and structure of the biomedical literature. In: Warren KS (ed) Coping with the biomedical literature: A primer for the scientist and the clinician. Praeger, New York, pp 3–16

9. Dirk L (1999) A measure of originality: the elements of sci-ence. Soc Stud Sci 29:765–776

10. Horton R (2004) A statement by the editors of The Lancet. Lancet 363:820–821

11. Ide CW (2005) Why doctors don’t read research papers: edi-tors’ behaviour might have something to do with it. Br Med J 330:256

12. International_Committee_of_Medical_Journal_Editors_(ICMJE) (2007) Publication Ethics: Sponsorship, Authorship, and Accountability. Available at: http://www.icmje.org/spon-sor.htm

13. Jung CG (1981) The structure and dynamics of the psyche. Princeton University Press, Princeton

14. Laurance WF (2006) Second thoughts on who goes where in author lists. Nature 442:26

15. Lieberman P, Laitman JT, Reidenberg JS et al (1992) The anatomy, physiology, acoustics and perception of speech:

essential elements in analysis of the evolution of human speech. J Hum Evol 23:447–467

16. O’Donnell M (2005) Why doctors don’t read research papers: scientifi c papers are not written to disseminate infor-mation. Br Med J 330:256

17. Oxford_Centre_for_Evidence-based_Medicine (2001) Levels of Evidence. Available at: http://www.cebm.net/index.aspx? o = 1025

18. Robinson FP (1970) Effective study. Harper & Row, New York

19. Shannon CE (1948) A mathematical theory of communica-tion. Bell Syst Tech J 27:379–423, 623–656

20. Slade M, Priebe S (2001) Are randomised controlled trials the only gold that glitters? Br J Psychiatry 179:286–287

21. Spache GD, Berg PC (1966) The art of effi cient reading. Macmillan, New York

22. Tscharntke T, Hochberg ME, Rand TA et al (2007) Author sequence and credit for contributions in multiauthored pub-lications. PLoS Biol 5:e18

23. Vital MV (2006) Author lists: specify who did what to aid assessment. Nature 443:26

24. Wager E (2007) Do medical journals provide clear and con-sistent guidelines on authorship? MedGenMed 9:16

http://www.icmje.org/spon-sor.htm

http://www.icmje.org/spon-sor.htm

http://www.cebm.net/index.aspx? o = 1025

http://www.cebm.net/index.aspx? o = 1025

key topics in surgical research and methodology || how to read a paper

Documents