talk on critical appraisal 17.08.11

8/6/2019 Talk on Critical Appraisal 17.08.11

1/22

An introduction toCritical Appraisal for Psychiatry

Wednesday 17thAug 2011In an Academic Meeting at Department of Psychiatry,

Eastbourne District General Hospital

Akira NAITO MD PhD

CT1 to Dr. A. GALEA


2/22


3/22

Evidence-Based Practice (EBP)

Definition ofEvidence Based Medicine:

The conscientious, explicit and judicious use of

current best evidence in making decisions about the

care of patients (Sackett et all 2000)

The goal ofEBPis the integration of:

1.Clinical expertise

2.Scientific evidence, and

3.Client/patient/caregiver values

3


4/22

Information solutions

Useful sources of information for clinical practice

The Cochrane Library: http://www.thecochranelibrary.com/

The Evidence-Based Mental Health (EBMH):

http://www.rcpsych.ac.uk/publications/journals/ebmhinfo1.aspx

Other internet resources

Clinical Evidence: http://clinicalevidence.bmj.com

PubMed(MEDLINE): http://www.ncbi.nlm.nih.gov/pubmed

National Institute of Clinical Excellence (NICE):http://www.nice.org.uk/

4


5/22

Why doing critical appraisal?

Uncertainty is universal in clinical medicine.

It takes practice to routinely identify areas of

uncertainty and to ask clinical questions in aproductive way.

Such questions will be brief summaries of the

clinical scenario and the information required,constructed in a way that makes searching

for evidence likely to succeed.

5


6/22

Different types of question require

different study designs Aetiology: What caused this illness? Cross-sectional surveys

Diagnosis:

What does this test result mean?

RCTs Prognosis:

What is likely to happen? Inception cohort studies

Harm:

Is being exposed to it likely to do harm? RCTs Effectiveness of therapy/treatment:

Is this treatment likely to help? RCTs

Qualitative:

What are the outcomes that are most important? 6


7/22

Evidence levels (from NICE)

Ia. Systematic review / Meta-analysis of RCTs

Ib. Randomised Controlled Trial (RCT)

IIa. Non-randomised controlled study

IIb. Quasi-experimental study

III. Non-experimental descriptive study

Comparative studies, Correlation studies

Case controlled studies

IV. Clinical experience

Expert-committee reports or opinions

Evidence from expected authorities7


8/22

Main types of bias

Selection bias Sampling / Performance bias

non-representative identification/recruitment of subjects

Unequal provision of care apart from treatment under evaluation

Response/ Attrition bias (non-/late- responders) unrepresentative participation differs from responders

Biased occurrence and loss to follow up

Observation / Detection bias Interviewer bias (blind to subject status or not):

differential data recording in subject groups by researcher

Recall bias historical data is selectively filtered

Confounding correlations/associations (not causal links)

8


9/22

What is critical appraisal?

The process by which a reader can assess

whether a written material possesses:

V

alidity(Is it close to the truth?), andApplicability(Is it clinically useful?).

The process of carefully & systematically

examining research to judge its:

Trustworthiness, and

Value and Relevance in a particular context.

9


10/22

Common ground inquiries

1. Is the study/research/guideline valid?

Validity / Trustworthiness

(Internal validity = free/minimum bias)

2. What does it show? & Is it important?

Value and Relevance

3. Can I use this in caring for my patients?

Applicability(for ones own decision making)

10


11/22

Structure of a paper

Introduction/Background

Clinical importance (Why do the study?)

A review of previous work in the subject area Hypotheses to be tested: a null hypothesis

Methods Results

Discussion / Conclusions

11


12/22


13/22

Results

Check whether original research question

or hypothesis was properly assessed?

Appropriate tables and figures? Logical manner?

Key findings?

Any inconsistencies in the data?

13


14/22

Discussion / Conclusions

Value and relevance of the study results

Limitations of the study

Implications of the studys findings

Generalisability of positive findings

Implication of negative results

14


15/22

Any question so far?

(References for writing a paper)

Writing a case report

http://careerfocus.bmjjournals.com/cgi/content/full/327/7424/2153-a

Tips andC

heck-points f

or academic writing in Englishhttp://homepage2.nifty.com/akira_naito/english.doc

Using English for Academic Purposes

http://www.uefap.com/writing/writfram.htm

15


16/22

Example questions

for therapy/treatment articles

16


17/22

What question did the study ask?+ PICO

Patients

Intervention

Comparison

Outcome(s)

17


18/22

Are the results of the trial valid?

Was the assignment of patients to treatments

randomised?

Were the groups similar at the start of the trial?

Aside from the allocated treatment, were groups

treated equally?

Were all patients who entered the trial

accounted for? and were they analysed in thegroups to which they were randomised?

Were measures objective or were the patients

and clinicians kept blind to which treatment

was being received? 18


19/22

What were the results?

How large was the treatment effect?

Relative Risk (RR)

Relative Risk Reduction (RRR) Number Needed to Treat (NNT) = 1 / ARR

Coefficient of Intra-Cluster Correlation (ICC)

How precise was the estimate of the

treatment effect?

(confidence intervals for each estimate)

19


20/22

Will the results help me in caringfor my patient?

The questions that you should ask before

you decide to apply the results of the study

to your patient are:

Is my patient so different to those in the study

that the results cannot apply?

Is the treatment feasible in my setting?

Will the potential benefits of treatmentoutweigh the potential harms of treatment for

my patient?

20


21/22

References

EBM Tools for Critical Appraisal

http://www.cebm.net/index.aspx?o=1157

What is critical appraisal?

http://www.medicine.ox.ac.uk/bandolier/painres/download/whatis/What_is_critical_appraisal.pdf

EBP tutorials - Critical Appraisal Skills

http://www.brighton.ac.uk/ncor/tutorials/EBP_Tutorial_intro_Critical_Appraisal_Skills.pdf

Clinical Study Design and Methods Terminologyhttp://www.vetmed.wsu.edu/courses-jmgay/glossclinstudy.htm

Critical Appraisal for Psychiatry (MRCPsych Guides)

http://www.amazon.co.uk/dp/0443070172

21


22/22

talk on critical appraisal 17.08.11

Documents