introduction to best evidence medical education

Dr. hossein aghily

Introduction toBest Evidence Medical Education

The Truth Is Rarely Pure And Never Simple

Oscar wild 1895

2

Systematic Review OR synthetic research

Is main procedure for synthesis of clinical evidence

3

A ‘review’ is the generic term for any attempt to synthesis the results and conclusions of two or more publications on a given topic.

Systematic Review: What is it?

A summary of several research articles on one topic presented in table and narrative formats

Qualitative rather than quantitative analysis

Subjective interpretation Time consuming process Strongest form of evidence

What are the advantages of a systematic review?

Condensed: allowing the reader to access consolidated results of huge volumes of information;

Objective: reducing (though not eliminating) the risk of bias and error

Balanced: including a broad range of studies which are identified via a thorough and systematic

search strategy;

Verifiable: incorporating transparent processes that allow the reader to know exactly how the

conclusions were reached;

Replicable: using a structured methodology.

Flexible: can be updated on a regular basis

Dynamic: in identifying areas that are under-researched or in identifying new

research questions;

Readable: presented in a format that is easy to read and understand.

Rationale for systematic reviews

We need reliable information There is too much information around We need reviews of existing information Reviews can be unscientific and biased in the

way they collect, appraise and summarize information

A pilot systematic review and meta-a

nalysis on the effectiveness of Proble

m Based Learning

A systematic review of computer-assisted learning in endodontics education

Increasing Knowledge

0

2000

4000

6000

8000

1966 1976 1986 1996

Number of articles on Hypertension cited in Medline by Year

Articles

Not all of this information is valid or useful for patient care

A little more than 1% is both rigorous and clinically relevant.

Most research published in medical journals is

too poorly done

or

insufficiently relevant

to be clinically useful

Too much information, too little time

There is simply too much information around for people to keep up to date.

On top of this, high quality information is often not easy to find.

Review articles

Some reviews are usually based on narrative or commentary and are produced by a

‘content expert’

Professor Paul Knipschild has described how Nobel prize winning biochemist Linus Pauling

used selective quotes from the medical literature to "prove" his theory that

vitamin C helps you live longer and feel better.

When Knipschild and his colleagues searched the literature systematically for evidence

They found that

One or two trials did strongly suggest that vitamin C could prevent the onset of the common cold

There were far more studies which did not show any beneficial effect.

Unfortunately, expert reviewers often :

Make conflicting recommendations

Their advice frequently lags behind

Inconsistent with the best available evidence.

Minimizing bias

We need to do as much as possible to minimize the effects of anything that will

cause the results to deviate from the truth.

GOALS

Briefly Describe Synthetic Research

Systematic Review Teaching Package

Meta-analyses

Medical EducationUME GME CMEContext, Content, Method, Setting

Improved Learning

Knowledge, Skills, Attitudes

Task-specific Competencies, Meta-competenciesEssential Personal Characteristics

Improved PracticeSafe, Timely, Patient-centeredEffective, Efficient, Equitable

Improved OutcomesLevel of Health, Distribution of Health

Responsiveness, Fair Finance

UME knowledge-focused GME apprenticeship, better differentiatedCME discrete educational interventions

The concept of best evidence medical education A more helpful view of evidence-based teaching

is of it as a continuum between 100% opinion-based education at one end of the spectrum where no useful evidence is available, and 100% evidence-based education at the other where decisions can be taken on the basis of detailed evidence.

Opinion Based Teaching

Evidence Based Teaching

Hart (1999) has suggested that Taking a best evidence-based approach to medical education questions forces educators to:1. comprehensively and critically appraise the literature

that already exists in the area, and categorize the power of the evidence available

2. identify the gaps and flaws in the existing literature and suggest (and if possible carry out) appropriately planned studies to optimize the evidence necessary to make the proposed educational intervention truly evidence based.

What is the methodology behind a systematic review?

1. Formulating the question Should address the Patient group, the

Intervention, the Comparison intervention, and the Outcome – PICO

PICO exa:

Among children under the age of five living in rural settings (patient), does the use of insecticide treated bed nets (intervention) lead to lower malaria prevalence rates (outcome) as compared to when nontreated nets are used (comparison intervention)?

www.cochrane.org/resources/handbook www.cochrane.org/resources/openlearning


2. Search and inclusion of primary studies

Restricting a search to a general database(such as MEDLINE) is clearly insufficient; a review group found that this tends to identify only about half of all randomized-controlled trials (RCTs).

Where to locate primary studies?

Major databases Reference of studies identified in

databases Unpublished studies (in order to avoid

publication bias) Interviews with the authors of original

studies when data is missing Non-English language studies


3. Quality assessment and data extraction

studies are appraised for their methodological error, particularly to identify any possible sources of bias

usually involves two independent reviewers

What is Critical Appraisal?

The process starts with careful examination of all aspects of the studies selected for inclusion in the systematic review.

This breaks down the components of the study to evaluate characteristics of participants, outcome measures used, completeness of study follow up, and appropriateness of statistical measures.

Critical appraisal – which lies at the very heart of a systematic review – requires dedicated time and expertise


4. Synthesis and summary of study results

If applicable, this is the stage where a meta-analysis may be used to pool statistical results from the various studies under review.



The aim is to combine the multiple findings in order to reach a conclusion on the clinical effectiveness of the intervention under consideration..



When a difference of effect is detected across various studies, the source of this divergence can then be analyzed



“limitations of the primary studies … may include issues relating to design flaws. Limitations of the review itself may include issues such as inclusion of only English language studies or inability to accurately interpret the summary estimates due to heterogeneity.”


5. Interpretation synthesized study results are interpreted

and the limitations of both the review and the studies within it are discussed


5. Interpretation This allows the reader to judge personally

the value of the review and its conclusions.

Finally, the review discusses the practical implications and applicability of the findings.

Where can I find systematic reviews?

The Cochrane Collaboration

Reviews are available in two formats: on-line (www.cochranelibrary.com) or on CD-ROM

Writing the review

Guidelines on how to write reviews & meta-analyses: QUORUM statement*

For meta-analysis of RCTs MOOSE guidelines**

For meta-analysis of observational studies

The complete list of groups, which are hosted in centers around the world, is available at www.cochrane.org/contacts/entities.htm#centres.

Where can I find systematic reviews?

2-The Campbell Collaboration its focus on the healthcare sector

http://www.campbellcollaboration.org

Campbell Collaboration consists of six Coordinating Groups, each with an area of specialization.

The Campbell Collaboration Crime and Justice; The Campbell Collaboration Education; The Campbell Collaboration Social Welfare; The Campbell Collaboration Methods; The Campbell Collaboration Communication and

Internationalization The Campbell Collaboration Users Group.

A systematic review of systematic reviews of homeopathy

Dr. hossein aghily

SYNTHESIS AND SUMMARY OF STUDY RESULTS(META-ANALYSIS)

Pooling and pooled estimate

Forest plots display effect estimates from each study with their CI, and provide a visual summary of the data

Statistical models for pooling

random effects model fixed effects model

The Fixed Effects Model

“The fixed-effects model assumes that the true effect of treatment is the same for every study.”

Kevin C. Chung, MD, Patricia B. Burns, MPH, H. Myra Kim, ScD. “Clinical Perspective: A Practical Guide to Meta-Analysis.” The Journal of Hand Surgery. Vol.31A No.10 December 2006. p. 1675

The Random Effects Model

“The random effects model assumes that the true effect estimate for each study vary.”

Kevin C. Chung, MD, Patricia B. Burns, MPH, H. Myra Kim, ScD. “Clinical Perspective: A Practical Guide to Meta-Analysis.” The Journal of Hand Surgery. Vol.31A No.10 December 2006. p. 1672

homogenity

Heterogeneity : Meta-analysis of randomized controlled trials on oral zinc for common cold

Subgroup analyses

Meta-analysis on beta-carotene intake and cardiovascular mortality: example of subgroup analysis and exploration of heterogeneity

Here is one possible interpretation of Kappa

Poor agreement = Less than 0.20 Fair agreement = 0.20 to 0.40 Moderate agreement = 0.40 to 0.60 Good agreement = 0.60 to 0.80 Very good agreement = 0.80 to 1.00

SYSTEMATIC REVIEWS AND META-ANALYSES: AN ILLUSTRATED, STEP-BY-STEP GUIDE

Reference:

Thank You !

Any Question ?

introduction to best evidence medical education

Documents

systematic reviewswe

review articlessome

synthesis of clinical

evidencebased medicine

best available evidence

high quality information

research articles

medical literature