evaluation of scientific literature
DESCRIPTION
This is a rapid guide for assessment of literature (EBM)TRANSCRIPT
EVALUATION OF RELEVANT SCIENTIFIC LITERATURE
Ahmed ZeeneldinAssociate Prof of Medical Oncology
Master of Clinical Research, Liverpool, UK
Why should we evaluate the literature?• PubMed
• 2011: ~ 1000 000 new articles added• Dec 2011: >100 000 new articles added• Anthracycline AND chemotherapy
AND breast cancer: >8000 articles
• Thus: • little time and huge literature volume• Prioritize• Evaluate what you read/hear
Acquiring information• The ‘push’ method
• lectures, seminars, • reading journals • no control over the content
• The ‘pull’ Method• You search for answers
for your questions
Evidence 3-dimensions (apq)
• Author (a)• Known• Expert
• Publisher (p)• Peer-reviewed• Non-reviewed
• Quality of evidence (q)• On the fly: EBM guides• Comprehensive
Literature assessment/appraisal
• Pre-appraised sources • Up-to-date• Trip database• Rehab+ database• Risk lies on expert authority (a & P)
• Appraise it yourself (q)- Rapidly: EBM tools- Extensively: critical appraisal tools
Types of Information Sources• Primary Sources: original research
• Not appraised
• Secondary Sources: about primary• Appraised
• Tertiary Sources: about secondary• Appraised
Primary Sources • Original materials • Not been filtered through interpretation, condensation• Not evaluated by a second party; • Examples:
• Original journal articles• Theses
Secondary Sources • Sources about primary, or original, information, • Modified, selected, or rearranged for a specific purpose or
audience• Examples:
• Textbooks• Review articles
Tertiary Sources • Sources twice removed from original sources• Examples:
• Handbooks• Encyclopedias
Steps of reading literature• You may start by 3ry or 2ry information recourses
• Textbook• Review article• Encyclopedia• handbook
• To get overview of the topic• Then you use the bibliography
to locate 1ry sources• Or search directly for 1ry sources
Evidence-based medicine (EBM) tools
•EBM: is a process to systematically • FIND, • APPRAISE, and • APPLY• research findings to clinical decisions
Evidence Pyramid• I - Controlled and randomized • II-I Controlled but not randomized • II-2 Cohort or case control • II-3 Multiple time series • III- Expert opinion or case study
2011 OCEBM Levels of Evidence (Evidence Boxes)
If you have limited time, where do you begin searching for evidence?
- hierarchy of the likely best evidence- for busy people (few minutes to few hours)
PubMed search for “ Anthracycline AND chemotherapy AND (breast cancer) ” plus some filters
Type Term used No. of articlesAll articles (no filter) 8066
SR Systematic review 32
RCT "random allocation" [MeSH] 120
DBRCT double blind method [mh] 58
Cohort "cohort studies" [MeSH] 1379
Case-control "Case-Control Studies"[Mesh] 521
Case report Case Reports [Publication Type] 568
Evidence according to question• Incidence
(How common is the problem?)• Local surveys with random sampling OR consensus
• Diagnosis(Is this diagnostic or monitoring test accurate?)• SR of cross sectional studies with reference standard
• Prognosis(What will happen if we do not add a therapy?)• SR of inception cohort studies
• Treatment Benefits(Does this intervention help or harm?)• SR of RT
• Screening(Is this (early detection) test worthwhile• SR of RT
Systematic Review Appraisal
• What is the question (PICO)?• Were important and relevant studies missed?• Were criteria to select articles appropriate?• Were the quality of included articles assessed?• Were the results similar from study to study?• What were the results? and how they are presented?
Diagnostic test study appraisal• Was the diagnostic test evaluated in a representative
spectrum of patients?• Was the reference standard applied regardless of the
index test result?• Was there an independent, blind comparison between
the index test and an appropriate reference ('gold') standard of diagnosis?
• Are test characteristics presented?• Accuracy: sensitivity and specificity• Performance in population: PPV and NPV
• Details to replicate the test?
Critical appraisal of Prognostic studies • Were patients representative and assembled at a
common point?• Was patient follow-up sufficiently long and complete?• Were outcome criteria either objective or applied in a
‘blind’ fashion? • If subgroups with different prognoses are identified, did
adjustment for important prognostic factors take place?
• How likely are the outcomes over time?• How precise are the prognostic estimated?• Are the results applicable to my patients?
RCT appraisal• What question did the study ask (PICO)?• Was the assignment of patients to treatments
randomised?• Were the groups similar at the start of the trial?• Aside from the allocated treatment, were groups treated
equally?• Were all patients who entered the trial accounted for?
and were they analysed in the groups to which they were randomised?
RCT appraisal• Were measures objective or were the patients and
clinicians kept “blind” to which treatment was being received?
• How large was the treatment effect?• How precise was the estimate of the treatment effect?• Will the results help me in caring for my patient?
Steps of EBM
1. asking questions2. searching for answers3. critically appraising the results
_____________________________4. determining applicability to practice5. integrating data with experience6. involving patient in decisions
Asking questions1) what is the question about?
• Therapy, diagnosis, prognosis, aetiology/harm, screening etc2) what is the type of question?
- background Q (topic)- foreground Q (patient)
3) how can questions be structured to facilitate a search for precise answers?
PICO’ components:• Patient group/Population• Intervention • Comparison• Outcome
Examples of Focused Questions1 2 3 4
Patient or Problem
Intervention(a cause, prognostic
factor, treatment, etc.)
Comparison Intervention
(if necessary)
Outcomes
Med OncologyIn patients with MBC does FEC regimen Compared to CMF
regimenYield better OS rates?
Surgical Oncology
In patients with early BC Does BCS Compared to MRM Lead to higherrecurrences and lower OS?
RadiotherapyIn patients with operable BC
Does IO RT Compared to PO RT Yield higher recurrences?
Clinical Pathology
In patients with ovarian cancer
Does HEP4 Compared to CA125 Leads to earlier diagnosis of recurrence?
EpidemiologyIn normal people Does smoking Compared to no
smokingIncrease risk of lung cancer
How to Conduct a Computer Search 1. Specify the research problem2. Select the databases3. Select the key concepts,
operators, limits4. Conduct the search5. Increase or decrease your total
results 6. Review the citation list
1. Honey and cancer2. PubMed3. Honey AND cancer4. ENTER: 198 records5. Honey OR bees
AND cancer: 281Honey AND cancer AND treatment: 129
Determining the Level of Evidence • Quickly:
• Evidence box according to question
• Thoroughly (critically appraise according to study type): • Relevance (related):
• applicability of the evidence to my problem • Validity (correct):
• the extent to which a concept, conclusion or measurement is well-founded and corresponds accurately to the real world
Trial Validity • Internal: proper study
• Did the study measure what it is meant to?• Do we trust the results?
• External: generalizability of results
Making decisions• Define the population and intervention
• Patients’ characteristics• Intervention: price and availability, training
• Search for and understand the biases• Critical appraisal
• Interpret the findings and apply them to your patient• Sample size• Summary statistic: • Probablity (p) and confidence of the results (CI)
Conclusion• Pre-appraised literature may be the
main source of information for undergraduate students
• However, graduates and faculty staff should be able to appraise information both quickly (EBM tools) and thoroughly (critical appraisal tools)
• APQ 3-dimension concept
Recommendation• I suggest that principles of EBM be taught to all NCI
students• By whom?
• Concerned Departments? • Epidemiology/Biostatistics Department?