![Page 1: Critical Appraisal - KSU · Critical Appraisal = assessment of evidence by systematically reviewing its: • Relevance • Validity • Applicability of its results to specific questions](https://reader033.vdocument.in/reader033/viewer/2022041604/5e32b5a11c048a102a08924b/html5/thumbnails/1.jpg)
Critical Appraisal
Einas Al-Eisa, MSc, PhDKing Saud University
![Page 2: Critical Appraisal - KSU · Critical Appraisal = assessment of evidence by systematically reviewing its: • Relevance • Validity • Applicability of its results to specific questions](https://reader033.vdocument.in/reader033/viewer/2022041604/5e32b5a11c048a102a08924b/html5/thumbnails/2.jpg)
Critical Appraisal
= assessment of evidence by systematicallyreviewing its:
• Relevance• Validity• Applicability of its results to specific questions
![Page 3: Critical Appraisal - KSU · Critical Appraisal = assessment of evidence by systematically reviewing its: • Relevance • Validity • Applicability of its results to specific questions](https://reader033.vdocument.in/reader033/viewer/2022041604/5e32b5a11c048a102a08924b/html5/thumbnails/3.jpg)
Why? • Medical practice is constantly changing
• Rate of change is accelerating
• “Medline” contains: – 6 million references from 4000 journals– To keep a head of this information, one would
need to read 6000 article each day
![Page 4: Critical Appraisal - KSU · Critical Appraisal = assessment of evidence by systematically reviewing its: • Relevance • Validity • Applicability of its results to specific questions](https://reader033.vdocument.in/reader033/viewer/2022041604/5e32b5a11c048a102a08924b/html5/thumbnails/4.jpg)
Why?• You can not read every
article
• Some articles are more important than others
• You need to divide the way of reading to:– Scanning– Exhaustive
![Page 5: Critical Appraisal - KSU · Critical Appraisal = assessment of evidence by systematically reviewing its: • Relevance • Validity • Applicability of its results to specific questions](https://reader033.vdocument.in/reader033/viewer/2022041604/5e32b5a11c048a102a08924b/html5/thumbnails/5.jpg)
AppraisedAppraised TopicTopicTopicTopic::A subject ofdiscussion
AppraiseAppraise::To evaluate & estimate thequality & amountof validity,
results and applicability
CriticalCritical::careful, exact evaluationand judgment
CriticallyCritically
![Page 6: Critical Appraisal - KSU · Critical Appraisal = assessment of evidence by systematically reviewing its: • Relevance • Validity • Applicability of its results to specific questions](https://reader033.vdocument.in/reader033/viewer/2022041604/5e32b5a11c048a102a08924b/html5/thumbnails/6.jpg)
All Evidence is notcreated equal!!
![Page 7: Critical Appraisal - KSU · Critical Appraisal = assessment of evidence by systematically reviewing its: • Relevance • Validity • Applicability of its results to specific questions](https://reader033.vdocument.in/reader033/viewer/2022041604/5e32b5a11c048a102a08924b/html5/thumbnails/7.jpg)
Hierarchy of EvidenceMeta-analysis of RCTsSystematic review of RCTs
Individual RCT
Observational studies
Basic research
Clinical experience
![Page 8: Critical Appraisal - KSU · Critical Appraisal = assessment of evidence by systematically reviewing its: • Relevance • Validity • Applicability of its results to specific questions](https://reader033.vdocument.in/reader033/viewer/2022041604/5e32b5a11c048a102a08924b/html5/thumbnails/8.jpg)
Adapted from Slawson et al, J Fam Pract 1994; 38:505-513
Usefulness ofUsefulness ofMedical InformationMedical Information= RelevanceRelevance x Validity
Work
![Page 9: Critical Appraisal - KSU · Critical Appraisal = assessment of evidence by systematically reviewing its: • Relevance • Validity • Applicability of its results to specific questions](https://reader033.vdocument.in/reader033/viewer/2022041604/5e32b5a11c048a102a08924b/html5/thumbnails/9.jpg)
Work :• Evidence Search (= MEDLINE)• Retrieval• Appraisal• Application to patient
Usefulness ofMedical Information
= Relevance x ValidityWorkWork
![Page 10: Critical Appraisal - KSU · Critical Appraisal = assessment of evidence by systematically reviewing its: • Relevance • Validity • Applicability of its results to specific questions](https://reader033.vdocument.in/reader033/viewer/2022041604/5e32b5a11c048a102a08924b/html5/thumbnails/10.jpg)
VAR system
1. Validity: Are the results of the study valid?
2. Applicability: Will the results help in caring for our patients?
3. Relevance: Are the results important? Impact and precision?
![Page 11: Critical Appraisal - KSU · Critical Appraisal = assessment of evidence by systematically reviewing its: • Relevance • Validity • Applicability of its results to specific questions](https://reader033.vdocument.in/reader033/viewer/2022041604/5e32b5a11c048a102a08924b/html5/thumbnails/11.jpg)
RResultsesultsmagnitude & magnitude &
precisionprecisionAApplicabilitypplicability
VValidityalidity
VAR VAR systemsystem
![Page 12: Critical Appraisal - KSU · Critical Appraisal = assessment of evidence by systematically reviewing its: • Relevance • Validity • Applicability of its results to specific questions](https://reader033.vdocument.in/reader033/viewer/2022041604/5e32b5a11c048a102a08924b/html5/thumbnails/12.jpg)
3 pillarsV. Are the results of the study Valid?
Closeness to the truth
A. Will the results be Applied to my patient? InterventionPopulationPreferences
R. Are the Results clinically important?Magnitude and precision
![Page 14: Critical Appraisal - KSU · Critical Appraisal = assessment of evidence by systematically reviewing its: • Relevance • Validity • Applicability of its results to specific questions](https://reader033.vdocument.in/reader033/viewer/2022041604/5e32b5a11c048a102a08924b/html5/thumbnails/14.jpg)
Critical questions
• Are the findings clinically important?
• Is the study valid?
• Do the findings apply to my patient?
![Page 15: Critical Appraisal - KSU · Critical Appraisal = assessment of evidence by systematically reviewing its: • Relevance • Validity • Applicability of its results to specific questions](https://reader033.vdocument.in/reader033/viewer/2022041604/5e32b5a11c048a102a08924b/html5/thumbnails/15.jpg)
Is the study valid?
• Validity = how close is the evidence to the truth
• Was the study retrospective or prospective?
• Was it randomized?
![Page 16: Critical Appraisal - KSU · Critical Appraisal = assessment of evidence by systematically reviewing its: • Relevance • Validity • Applicability of its results to specific questions](https://reader033.vdocument.in/reader033/viewer/2022041604/5e32b5a11c048a102a08924b/html5/thumbnails/16.jpg)
Is the study valid?
• Diagnostic / screening tests: was there an independent, blind comparison with a “gold standard” of diagnosis?
• Prognostic markers: was there an inception cohort?
• Therapy / Intervention: was assignment to treatments randomized and blinded?
![Page 17: Critical Appraisal - KSU · Critical Appraisal = assessment of evidence by systematically reviewing its: • Relevance • Validity • Applicability of its results to specific questions](https://reader033.vdocument.in/reader033/viewer/2022041604/5e32b5a11c048a102a08924b/html5/thumbnails/17.jpg)
Screening questions
• Did the study address a clearly focused issue?– Clearly stated purpose and research question– Clear explanation of the population studied
(inclusion / exclusion criteria)– Clear overview of interventions– Clear overview and justification for outcomes
![Page 18: Critical Appraisal - KSU · Critical Appraisal = assessment of evidence by systematically reviewing its: • Relevance • Validity • Applicability of its results to specific questions](https://reader033.vdocument.in/reader033/viewer/2022041604/5e32b5a11c048a102a08924b/html5/thumbnails/18.jpg)
Screening questions
• Was the assignment of participants to treatment randomized?
• Were all participants in the study accounted for?– The results are invalid in case of 15%
dropouts
![Page 19: Critical Appraisal - KSU · Critical Appraisal = assessment of evidence by systematically reviewing its: • Relevance • Validity • Applicability of its results to specific questions](https://reader033.vdocument.in/reader033/viewer/2022041604/5e32b5a11c048a102a08924b/html5/thumbnails/19.jpg)
Applicability
![Page 20: Critical Appraisal - KSU · Critical Appraisal = assessment of evidence by systematically reviewing its: • Relevance • Validity • Applicability of its results to specific questions](https://reader033.vdocument.in/reader033/viewer/2022041604/5e32b5a11c048a102a08924b/html5/thumbnails/20.jpg)
Applicability
• Compare your patient to the subjects used in the study
• Consider the feasibility, risks, and benefits of the intervention
• Determine if it is in line with the client’s preferences
![Page 21: Critical Appraisal - KSU · Critical Appraisal = assessment of evidence by systematically reviewing its: • Relevance • Validity • Applicability of its results to specific questions](https://reader033.vdocument.in/reader033/viewer/2022041604/5e32b5a11c048a102a08924b/html5/thumbnails/21.jpg)
Applicability
• Will the results help me in caring for my patients?
• Can the results be applied to my patients?
• Rx/intervention AVAILABLE in our setting
• Were all clinically relevant outcomes considered?
• Are the benefits worth the harm and cost?
![Page 22: Critical Appraisal - KSU · Critical Appraisal = assessment of evidence by systematically reviewing its: • Relevance • Validity • Applicability of its results to specific questions](https://reader033.vdocument.in/reader033/viewer/2022041604/5e32b5a11c048a102a08924b/html5/thumbnails/22.jpg)
Remember:
• CAT is a serious meticulous structured piece of hard work
• It can NOT be done in hurry or while doing other tasks simultaneously
• Bring your pencil and yellow highlighter pen
• Be ready to close the door of your office for nearly one hour
![Page 23: Critical Appraisal - KSU · Critical Appraisal = assessment of evidence by systematically reviewing its: • Relevance • Validity • Applicability of its results to specific questions](https://reader033.vdocument.in/reader033/viewer/2022041604/5e32b5a11c048a102a08924b/html5/thumbnails/23.jpg)
Read the conclusions
• Are the objectives met?
• Is there any recommendations?
• Any mention about limitations of the study?
![Page 24: Critical Appraisal - KSU · Critical Appraisal = assessment of evidence by systematically reviewing its: • Relevance • Validity • Applicability of its results to specific questions](https://reader033.vdocument.in/reader033/viewer/2022041604/5e32b5a11c048a102a08924b/html5/thumbnails/24.jpg)
Read the discussion
• Did they make good reasoning for the findings
• Other studies that support or contradict the study results
![Page 25: Critical Appraisal - KSU · Critical Appraisal = assessment of evidence by systematically reviewing its: • Relevance • Validity • Applicability of its results to specific questions](https://reader033.vdocument.in/reader033/viewer/2022041604/5e32b5a11c048a102a08924b/html5/thumbnails/25.jpg)
Remember, CAT not always conclusive
It may highlight lack of good evidence
It can be disappointing
It Does not always provide the “easy answer”
![Page 26: Critical Appraisal - KSU · Critical Appraisal = assessment of evidence by systematically reviewing its: • Relevance • Validity • Applicability of its results to specific questions](https://reader033.vdocument.in/reader033/viewer/2022041604/5e32b5a11c048a102a08924b/html5/thumbnails/26.jpg)
PatientPatientPatient
ClinicalClinicalQuestionQuestion
AnswerableAnswerableQuestionsQuestions
EvidenceEvidence
SearchSearch
AnalyzeAnalyzeApply new knowledgeApply new knowledgeto the patientto the patient
InformationInformation
Critically Appraised Topic LoopCritically Appraised Topic Loop
CCAA
TT
EvidenceEvidence
AnalyzeAnalyze
InformationInformation
![Page 27: Critical Appraisal - KSU · Critical Appraisal = assessment of evidence by systematically reviewing its: • Relevance • Validity • Applicability of its results to specific questions](https://reader033.vdocument.in/reader033/viewer/2022041604/5e32b5a11c048a102a08924b/html5/thumbnails/27.jpg)
Sensitivity and Specificity
• When a test has a very high sensitivity, a negative result effectively rules out the diagnosis.
• When a test has a very high Specificity, a positive result effectively rules in the diagnosis.
• However, we can be misled by the old sensitivity–specificity approach that restricts us to just two levels (positive and negative) of the test result.
![Page 28: Critical Appraisal - KSU · Critical Appraisal = assessment of evidence by systematically reviewing its: • Relevance • Validity • Applicability of its results to specific questions](https://reader033.vdocument.in/reader033/viewer/2022041604/5e32b5a11c048a102a08924b/html5/thumbnails/28.jpg)
Tools Used in Diagnostic Study
• Sensitivity
TPTP + FP
D-D+
FPTPT+
TNFNT-True is
always UP
![Page 29: Critical Appraisal - KSU · Critical Appraisal = assessment of evidence by systematically reviewing its: • Relevance • Validity • Applicability of its results to specific questions](https://reader033.vdocument.in/reader033/viewer/2022041604/5e32b5a11c048a102a08924b/html5/thumbnails/29.jpg)
• Specificity
TNTN + FN
D-D+
FPTPT+
TNFNT-True is
always UP
Tools Used in Diagnostic Study