roadmap tutorial

29
2 0 1 0 E D I T I O N R OADMAP Tutorial Suzana Alves da Silva 1 and Peter Wyer 2 1 Teaching and Research Center of Pró-Cardíaco, Rio de Janeiro, BR and 2 College of Physicians and Surgeons, Columbia University Medical Center, New York, US A GUIDE FOR THE CONTENT OF EVIDENCE-BASED HEALTH CARE “ SIMPLE”, illustrated on the following page, is a comprehensive model of clinical practice which incorporates routine use of scientific evidence to inform health care decisions. The term “roadmap” refers to the pathway that leads across transition points connecting the subjective, individualised, domain of clinical interaction, interpretation and shared decisions (represented by the relational field in the figure), and the objective, generalised, world of scientific knowledge, including information derived from clinical research (represented by the information literacy field in the figure). Clinical actions are divided into four domains: Therapy, Diagnosis, Prognosis and Harm. Four “information literacy” skills, “ask”, “acquire”, “appraise” and “apply”, define a transitional process subordinated to each of the 4 action domains. The result is 4 categories of clinical action linked to the external realm of scientific information. This model retains and extends the skill sets developed from EBM and also explicitly acknowledges the general relevance of the full range and scope of scientifically derived medical knowledge. 1 Silva, S., MD and Wyer P, MD, 2010 The z SIMPLE z Scientifically Informed Medical Practice and Learning “Things must be as simple as possible but not any simpler” Einstein Premisses The premisses for the practice of evidence-based medicine Premise 1 Practice based skills and competency allows a health professional, to connect questions pertaining to clinical decision making to relevant and useful clinical research. Premise 2 Decision making needs to be shared unless the patient prefers otherwise. In all cases decisions need to be guided by explicit assessments of patient- importance.

Upload: suzana-silva

Post on 08-Mar-2016

214 views

Category:

Documents


0 download

DESCRIPTION

Roadmap for the content of EBCP

TRANSCRIPT

Page 1: Roadmap Tutorial

2 0 1 0 E D I T I O N

ROADMAP TutorialSuzana Alves da Silva1 and Peter Wyer2

1Teaching and Research Center of Pró-Cardíaco, Rio de Janeiro, BR and 2College of Physicians and Surgeons, Columbia University Medical Center, New York, US

A G U I D E F O R T H E C O N T E N T O F E V I D E N C E - B A S E D H E A L T H C A R E

“ SIMPLE”, illustrated on the following page,  is a comprehensive model of clinical practice which incorporates routine use of scientific evidence to inform health care decisions. The term “roadmap” refers to the pathway that leads across transition points connecting the subjective, individualised, domain of clinical interaction, interpretation and shared decisions (represented by the relational field in the figure), and the objective, generalised, world of scientific knowledge, including information derived from clinical research (represented by the information literacy field in the figure).

Clinical actions are divided into

four domains: Therapy, Diagnosis, Prognosis and Harm.

Four “information literacy” skills, “ask”, “acquire”, “appraise” and

“apply”, define a transitional

process subordinated to each of the 4 action domains. The result

is 4 categories of clinical action linked to the external realm of

scientific information. This

model retains and extends the skill sets developed from EBM

and also explicitly acknowledges the general relevance of the full

range and scope of scientifically

derived medical knowledge.

1 Silva, S., MD and Wyer P, MD, 2010

The

z SIMPLE zScientifically Informed Medical Practice and Learning

“Things must be as simple as possible but not any simpler”Einstein

PremissesThe premisses for the practice of evidence-based medicine

Premise 1Practice based skills and competency allows a health professional, to connect questions pertaining to clinical decision making to relevant and useful clinical research.

Premise 2Decision making needs to be shared unless the patient prefers otherwise. In all cases decisions need to be guided by explicit assessments of patient-importance.

Page 2: Roadmap Tutorial

The Relational FieldThe figure below exemplifies the representation of individualized health care subsuming identification and integration of relevant external scientific evidence into the substance of clinical action. There is a clear dichotomy between the realm of practice where decisions are taken and the realm of literature where the scientific information exists. The upper field in the figure is composed by relations, circumstances, policies and a dynamic interplay of values, preferences and priorities of all involved in the process. This principal for individualized care can be applied in the other layers of health care delivery including health management at local level, dealing with institutions and departments, and health management at government level, dealing with with public health policies. The dichotomy between the relational field and the information literacy field is constant across all these layers and a bridge between these two fields needed to be constructed.

THE REPRESENTATION OF SCIENTIFICALLY INFORMED HEALTH CARE

2 Silva, S., MD and Wyer P, MD, 2010

Page 3: Roadmap Tutorial

UTILITY, PERFORMANCE AND LIKELIHOOD

Utility According to the english dictionary utility means “the amount of usefulness that something has”.

For the purposes of EBM we use utility for estimating the magnitude of benefit offered or magnitude of harm caused by a therapeutic, diagnostic or prognostic intervention. Thus, utility issues pertain to all sub-domains: Therapy, Diagnosis, Prognosis and Harm.

PerformanceAccording to the english dictionary performance means “how well or badly someone or something do a particular activity”.

For the purposes of EBM we use performance

1) to refer to how well or badly a diagnostic test or prediction rule performs compared to a criterion standard (diagnosis and prognosis).

2) to refer to the estimate of effect of presence of one or more predictors on the likelihood of a clinical outcome (prognosis).

3) to refer to the estimate of effect of presence of one or more predictors to assess response to an intervention (therapy and harm).

Thus performance refers not only to the accuracy of a sign, test or rule to predict or establish if a patient has or has not a disease, condition or injury but also to the effect of a sign, test or rule to predict if the patient will or will not respond to or will or will not be harmed by an intervention.

LikelihoodAccording to the english dictionary likelihood means “the state or fact of something’s being likely”.

For the purpose of EBM we use the term likelihood

1) to refer to the estimate of likelihood of clinical outcomes over time (prognosis frequency)

2) to refer to the estimate of likelihood or probabilities of possible diagnoses for patients condition (differential diagnosis).

3) to refer to the estimate of likelihood of clinical outcomes over time if the patient is submitted to an intervention or exposure (therapy and harm).

3 Silva, S., MD and Wyer P, MD, 2010

Page 4: Roadmap Tutorial

THE BRIDGEPP A C T

Therapy Likelihood Share consideration of what would happen if submitted to a therapeutic intervention

No relevant choice Estimate of likelihood of important clinical outcomes attributable to the disease, condition or injury if treatment accepted

Therapy

Performance Share consideration of the performance of a predictor or rule to predict response to a therapeutic intervention

No relevant choice Estimate of effect of presence of one or more predictors on the likelihood of improved outcomes

Therapy

Utility Share consideration of therapeutic intervention

Share consideration of alternative or standard treatments

Estimate of magnitude of effect on important clinical outcomes attributable to the disease, condition or injury

Diagnosis Likelihood Share consideration of a set of signs and symptoms

No relevant choice Estimate of likelihood of possible diagnosis

Diagnosis

Performance Share consideration of the performance of a test, rule or clinical assessment

Share consideration of a criterion assessment

Estimate of effect of the test, rule or clinical assessment on patientʼs likelihood of the disease, condition or injury

Diagnosis

Utility Share consideration of a Utility de um teste diagnosis ou de uma regra de predição clínica

Share consideration of alternative assessments

Estimate of magnitude of effect on important clinical outcomes attributable either to the disease, condition or injury or to the process of care

Prognosis Likelihood Share consideration of clinical outcomes over time

No relevant choice Estimate of likelihood of important clinical outcomes attributable to the disease, condition or injury

Prognosis

Performance Share consideration of the performance of one or more predictors or rule

No relevant choice Estimate of effect of presence of one or more predictors on the likelihood of such outcomes

Prognosis

Utility Share consideration of the utility of clinical use of prognostic information

Share consideration of alternative or standard practice

Estimate of magnitude of effect of clinical use or sharing of prognostic information on patient important outcomes

Harm Likelihood Share consideration of what would happen if submitted to a harmful intervention or to an incidental exposure

No relevant choice Estimate of likelihood of important adverse outcomes attributable to the therapeutic, diagnostic or prognostic intervention or to the environmental or behavioral exposure but not to the patientʼs disease, condition or injury

Harm

Performance Share consideration of a performance of a predictor or rule to predict the likelihood of harm from a therapeutic, diagnostic or prognostic intervention or behavioral or environmental exposure

No relevant choice Estimate of effect of presence of one or more predictors on the likelihood of adverse outcomes attributable to the intervention or to the environmental or behavioral exposure but not to the patientʼs disease, condition or injury

4 Silva, S., MD and Wyer P, MD, 2010

Page 5: Roadmap Tutorial

PP A C TUtility Share consideration of

an accidental or incidental harm due to a therapeutic, diagnostic or prognostic intervention or behavioral or environmental exposure, respectively

Share consideration of alternative choices or exposures

Estimate of magnitude of effect on important adverse outcomes attributable to the therapeutic, diagnostic or prognostic intervention or to the behavioral or environmental exposure but not to the patientʼs disease condition or injury

PACT is an acronym that conveys the mutual understanding of the Problem, Clinical Actions that need to be taken, Choices that are available and Targets to be achieved. The pact between individuals requires shared understanding of all aspects of decision-making and thus creates a direct bridge between shared “problematization” of patient needs and acquisition and assessment of relevant scientific knowledge. Therapy, Diagnosis, Prognosis and Harm, are the actions generated through these shared deliberation and interpretation.

5 Silva, S., MD and Wyer P, MD, 2010

Page 6: Roadmap Tutorial

The Information Literacy FieldThe Roadmap is the pathway through the information literacy field and helps to address questions defined once a problem is delineated. To address those questions a health professional requires the full spectrum of information skills that makes possible the bridge between practice and scientific research. These skills are conventionally defined as pertaining to the elements of an information cycle: “ask”, ”acquire”, “appraise” and “apply” and were originally derived from the concept of information literacy. The process of moving through these four steps of the cycle is not directly represented in this figure.

6 Silva, S., MD and Wyer P, MD, 2010

Action Domains

THERAPY DIAGNOSIS PROGNOSIS HARM

Ask

Acquire

Appraise

Ask

Acquire

Appraise

Ask

Acquire

Appraise

Ask

Acquire

Appraise

Apply Apply Apply Apply

Page 7: Roadmap Tutorial

The Information Literacy FieldThe Roadmap is the pathway through the information literacy field and helps to address questions defined once a problem is delineated. To address those questions a health professional requires the full spectrum of information skills that makes possible the bridge between practice and scientific research. These skills are conventionally defined as pertaining to the elements of an information cycle: “ask”, ”acquire”, “appraise” and “apply” and were originally derived from the concept of information literacy.1 The process of moving through these four steps of the cycle is not directly represented in this figure.

The Pathways Within the Roadmap

7 Silva, S., MD and Wyer P, MD, 2010

Page 8: Roadmap Tutorial

Deriving Questions from ProblemsIf the problem is not validly delineated otherwise “right” questions may address the wrong problem. The consequence is that patients may be evaluated and treated for problems they don’t have.In order to link structured questions to the right problems the health professional needs to:

1. Identify the appropriate target of the question as dictated by a validly delineated problem and its priority

2. Identify the anatomy of the question by putting it into PICO format

The Anatomy of the QuestionThe steps in practicing EBM are centered on the patient and involve asking well-focused questions, searching for the best available evidence, appraising that evidence for validity, and then applying the results to the care of the patient. To accomplish this objectives the health professional must know firstly how to differentiate background questions from foreground questions. Secondly, must know how to build a focused foreground question.

Background versus Foreground questionsA background question usually refers to the need for basic information. It is not normally asked because of a need to make a clinical decision about a specific patient (foreground question).

How to ask a well built clinical question?What makes a clinical question well built? According to Richardson, 1995,2 the question should be directly relevant to the problem at hand and should be phrased to facilitate searching for a precise answer. To achieve these aims it has been proposed that the question must be well articulated for all 4 parts of its 'anatomy':

1. the patient or problem being addressed;2. the intervention or exposure being considered;3. the comparison intervention or exposure, when relevant;4. the clinical outcomes of interest.

8 Silva, S., MD and Wyer P, MD, 2010

Page 9: Roadmap Tutorial

Examples of PICO QuestionsThe tables below display the general framework for the anatomy of a foreground question within each action

domain. Several examples of the ‘anatomy’ of well-built clinical questions are demonstrated:

ASKING QUESTIONS UNDER THERAPY

P I C O

CASE 1FREQUENCY

Patient disease, condition or injury

If submitted to a Therapeutic intervention (medical, surgical or preventive)

Not Applicable Likelihood of clinical outcomes attributable to the disease, condition or injury when treatment accepted

Example In patients with myocardial infarction requiring anticoagulation

if they take warfarin

what is the probability of having a stroke anyway?

CASE 2PERFORMANCE

Patient disease, condition or injury

Performance of a predictor or rule on the response to a Therapeutic intervention (medical, surgical or preventive)

Not Applicable Effect of presence of one or more predictors on the likelihood of improved outcomes attributable to the disease, condition or injury

Example In patients with myocardial infarction requiring anticoagulation

for how much the genetic variation of genes related to warfarin metabolism

will change the load and maintenance dose?

CASE 3UTILITY

Patient disease, condition or injury

Therapeutic intervention (medical, surgical or preventive)

Compared to an alternative or standard treatment

Magnitude of benefit on clinical outcomes attributable to the disease, condition or injury

Example 1 In communicating with patients with septic shock and other severe co morbidity's and their families…

…does the use of quantitative information regarding effectiveness of care options…

…compared to qualitative summary only…

…result in decisions more or less in keeping with independently stated patient values and preferences?

Example 2 In adult patients with septic shock and multiple organ dysfunction…

…does the treatment with alfa-dotrecogin activated…

…compared to placebo…

…improve clinical outcomes?

9 Silva, S., MD and Wyer P, MD, 2010

Page 10: Roadmap Tutorial

ASKING QUESTIONS UNDER PROGNOSIS

P I (C) O

CASE 1LIKELIHOOD

Patient disease, condition or injury

Prognostic intervention:clinical observation over time

Not applicable Likelihood of clinical outcomes*

Example In adult patients presenting with obstructive nephropathy and infection…

…during the hospitalization phase…

Not applicable …what is the mortality rate?

P I (C) O

CASE 2PERFORMANCE

Patient disease, condition or injury

Prognostic intervention:one or more predictors or a prediction rule

Not applicable Effect of the presence of one or more predictors on the likelihood of clinical outcomes*

Example 1 In adult patients presenting with obstructive nephropathy and infection…

…what are the predictors...

Not applicable …and their effect on the likelihood of acute renal failure?

Example 2 … does the presence of renal failure…

Not applicable …increase mortality?

Example 3 … does the prediction rule ‘APPACHE’...

Not applicable …correctly classify high risk patients?

P I (C) O

CASE 3UTILITY

Patient disease, condition or injury

Prognostic intervention:prognostic information

Compared to an alternative or standard practice

Magnitude of effect of clinical use or sharing of prognostic information on clinical outcome*

Example In adult patients with urinary sepsis…

…does the information obtained with the use of a rule to predict mortality…

…compared to an alternative or standard assessment…

…affect the likelihood of clinical outcomes?

10 Silva, S., MD and Wyer P, MD, 2010

Page 11: Roadmap Tutorial

ASKING QUESTIONS UNDER DIAGNOSIS

P I (C) O

CASE 1

LIKELIHOOD

(Differential Diagnosis)

Patient disease, condition or injury

Diagnostic intervention: Evaluation of signals and symptoms or of a syndrome

Not applicable Possible diagnoses

Example In older adult patients …

…presenting with renal colic and sepsis…

…what are the possible diagnoses and their probabilities?

P I (C) O

CASE 2 PERFORMANCE

Patient disease, condition or injury

Diagnostic intervention: -screening test -diagnostic test- prediction rule-clinical assessment

Compared to a criterion assessment

Patient’s likelihood of the disease, condition or injury

Example In older adult patients with diabetes, and ischemic cardiomyopathy…

…does the use of CT scan…

…compared to a criterion assessment…

…correctly identify urolithiasis?

P I (C) O

CASE 3UTILITY

Patient disease, condition or injury

Diagnostic intervention: -screening test -diagnostic test-prediction rule-clinical assessment

Compared to an alternative assessment

Clinical Outcomes*

Example In older adult patients with possible prostate cancer…

…does screening with PSA …

…compared to no screening…

…decrease mortality from prostate cancer over a 20 year period?

11 Silva, S., MD and Wyer P, MD, 2010

Page 12: Roadmap Tutorial

ASKING QUESTIONS UNDER HARM FOR LIKELIHOOD

P I C O

AccidentalLIKELIHOOD

Patient disease, condition or injury

If submitted to a Therapeutic, Diagnostic or Prognostic intervention

Not applicable Likelihood of adverse outcomes attributable to the intervention but not to the patient disease, condition or injury for which the intervention is being recommended

Example In adult patients presenting to an emergency department with obstructive nephropathy…

…if they were submitted to a surgical intervention…

…what is the mortality anyway?

IncidentalLIKELIHOOD

Patient disease, condition or injury

if submitted to a Behavioral or environmental exposure

Not applicable Likelihood of adverse outcomes when experiencing a behavioral or an environmental exposure

Example In adult patients with severe heart failure…

…if keep drinking a glass of wine daily…

what is the likelihood of decompensation and death?

12 Silva, S., MD and Wyer P, MD, 2010

Page 13: Roadmap Tutorial

ASKING QUESTIONS UNDER HARM FOR PERFORMANCE

P I C O

AccidentalPERFORMANCE

Patient disease, condition or injury

Performance of a predictor or rule on the likelihood of a harm from a Therapeutic, Diagnostic or Prognostic intervention

Not applicable Effect on the likelihood of adverse outcome attributable to the intervention but not to the patient disease, condition or injury for which the intervention is being recommended

Example In adult patients with HIV…

...does the presence of the gene HLA-B+5701 if patient is submitted to abacavir...

...predict hypersensitivity reaction related to the drug?

IncidentalPERFORMANCE

Patient disease, condition or injury

Performance of a predictor or rule on the likelihood of a harm from a behavioral or environmental exposure

Not applicable Effect on the likelihood of an adverse outcome attributable to the behavioral or environmental exposure

Example In health adults candidates for a stressful job..

…does the presence of genetic variation for serotonin transporter…

...predict depression related to the exposure?

13 Silva, S., MD and Wyer P, MD, 2010

Page 14: Roadmap Tutorial

ASKING QUESTIONS UNDER HARM FOR UTILITY

P I C O

AccidentalUTILITY

Patient disease, condition or injury

Therapeutic, Diagnostic or Prognostic intervention

Compared to a control

Magnitude of effect on the likelihood of an adverse outcome attributable to the intervention but not to the patient disease, condition or injury for which the intervention is being recommended

Example In adult patients presenting to an emergency department with obstructive nephropathy…

…does the use of CT scan with isosmolar iodine contrast…

…compared to the use of CT scan without contrast…

Increase the likelihood of acute renal failure?

IncidentalUTILITY

Patient disease, condition or injury

Behavioral or environmental exposure

Compared to a control

Magnitude of effect on the likelihood of an adverse outcome attributable to the behavioral or environmental exposure

Example In adult patients with heart and renal dysfunction…

…does the low ingestion of liquids…

…compared to a control…

Increase the likelihood of urinary tract obstruction?

14 Silva, S., MD and Wyer P, MD, 2010

Page 15: Roadmap Tutorial

Acquire the best available evidenceTo get skilled on how to acquire the best available evidence the health professional needs:

1. To know how to identify the type of study design that best addresses his/her question

2. To know how to identify the best source of the information he/she is looking for

3. To know how to build a search strategy within the selected source in order to find the best and most relevant evidence

To achieve these 3 steps is necessary to know how to relate study designs to sources and questions. The following figures demonstrate 1) how single studies relate to more complex designs, 2) how the hierarchy of study designs is related to a hierarchy of sources organized according to user friendliness, and 3) how the designs and sources are linked to the question type.

RELATIONSHIP BETWEEN SOURCES AND DESIGNS

15 Silva, S., MD and Wyer P, MD, 2010

Page 16: Roadmap Tutorial

THE HIERARCHICAL ORGANIZATION

This ‘Sources vs Studies’ pyramid constitutes a modification of a scheme originally developed by Haynes et al. (Haynes RB. Of studies, syntheses, synopses, summaries and systems: The “5S” evolution of information services for evidence-based health care decisions. ACP J Club 2006; 145: A8-A9.)3, 4

Differently from Haynes model, this is a 3D formulation of the pyramids connecting sources, study designs and type of information. The hierarchy of sources is organized according to their user friendliness. Within each cell in the “Sources” panel on the left, the higher the source, the more user friendly it is. In the design panel on the right, the hierarchy of the study designs within each cell is organized according to their complexity, with the more complex designs at the top. As you move upwards across the layers of the pyramid, each successive level includes the elements of the preceding level within it.

16 Silva, S., MD and Wyer P, MD, 2010

Page 17: Roadmap Tutorial

Linking Study Designs to QuestionsConnecting a correctly classified and well formulated clinical question to a preferred study design is one of the most challenging aspect of Evidence-Based Clinical Practice. In most cases, the relationship between a question and the best study design is straightforward, in some case it is not. The first step is to understand the target of the question: utility, performance and likelihood.

UTILITY, PERFORMANCE AND LIKELIHOOD

Utility Utility issues pertain to all sub-domains: Therapy, Diagnosis, Prognosis and Harm. Randomized trials have the strongest methodology for evaluating the utility of interventions. Observational studies maybe used for this purpose also, but the likelihood of bias in the latter types of study is much higher.

PerformanceUnder diagnosis or prognosis, a question regarding the performance of a test or rule calls for a different type of study from a question regarding the utility of the test or rule to improve patient outcomes. Cross-sectional studies are the mostly used to evaluate diagnosis performance. Case-control studies may also be used for this purpose.

Cohort studies are the mostly used to evaluate prognosis performance and to predict patient’s response to an intervention based on patient’s characteristics.Other types of studies may also be used to achieve this aim as case-control studies. Cross-sectional studies are sometimes used to evaluate association between prognostic factors and outcomes.

LikelihoodAs for utility and performance, likelihood issues also pertain to all sub-domains: Therapy, Diagnosis, Prognosis and Harm. Under prognosis, therapy and harm a question regarding the likelihood of a clinical outcome over time calls for cohort studies while under diagnosis a question regarding the likelihood of possible diagnoses calls for other types of observational studies.

17 Silva, S., MD and Wyer P, MD, 2010

Page 18: Roadmap Tutorial

Study Designs that MatterRANDOMIZED

TRIALSCOHORT STUDIES

CROSS-SECTIONAL

STUDIES

CASE-CONTROL STUDIES

The basic ingredients in the information literacy field are the four types of study designs that are used to answer questions under

Therapy, Diagnosis, Prognosis and Harm:

1. Randomized Trials

2. Cohort Studies

3. Cross-Sectional Studies

4. Case-Control Studies

These studies have some specificities regarding the type of question they are addressing as detailed below.

They can be organized in more complex structures resulting in designs as syntheses and summaries of the literature.

Utility of Therapeutic, Diagnostic or Prognostic Interventions

Randomized Controlled TrialsRandomized Controlled Trials (RCT) are mostly used to evaluate the utility of therapeutic, diagnostic or prognostic interventions. The use of RCT for diagnosis and prognosis is relatively new, but some good examples can be found in the literature.

Ex. The disclosure of APOE genotype for risk of Alzheimer disease;5 the use of BNP to guide therapy,6 the use of capsule endoscopy for obscure gastrointestinal bleeding.7

18 Silva, S., MD and Wyer P, MD, 2010

Page 19: Roadmap Tutorial

Performance of Diagnostic Tests or Prediction Rules for Diagnosis

Cross-Sectional Studies

Cross-Sectional studies are mostly used to evaluate the performance of diagnostic tests or prediction rules for diagnosis.

Ex. The performance of BNP for patients admitted with dyspnea in the emergency room was evaluated in a cross-sectional study. In this study BNP was compared to criterion standard for the diagnosis of congestive heart failure.8, 9

Case-Control StudiesCase-control studies may also be used to evaluate the performance of a diagnostic test or a prediction rule.

Ex: Eletrocardiographic diagnosis of evolving acute myocardial infarction in the presence of left bundle-branch block.10

19 Silva, S., MD and Wyer P, MD, 2010

Page 20: Roadmap Tutorial

Evaluating the Likelihood of Outcomes or Diseases

Cohort StudiesCohort studies are mostly used to evaluate patients destiny which means to assess the likelihood of an event over a period of time. The classical example is the Framinghan Heart Study.

20 Silva, S., MD and Wyer P, MD, 2010

Page 21: Roadmap Tutorial

Observational Studies for the Evaluation of Performance and Utility

Case-Control StudiesCase-control studies are some times used to evaluate if there is association between the benefit (utility question under therapy, diagnosis or prognosis) or the harm caused by an intervention or exposure.

Ex: The use of 1 compared to 2 mammary arteries during surgical myocardial revascularization for preventing cardiovascular events over time was evaluated in a case-control study.11, 12 Vitamin E intake to prevent Parkinson disease.13

Case-control studies may also be used to evaluate the performance of prognostic factors, which means to assess if there is or there isn’t association between some prognostic factor and the outcome of interest.

Cohort StudiesCohort studies may also be used for means of evaluating the utility of a therapeutic, diagnostic or prognostic intervention or the harm caused by any of these interventions (accidental harm). They also can be used to evaluate the association between exposures and outcomes (incidental harm) or some factor that is intrinsic to the patient and outcomes (prognosis performance).

Ex: Prognostic Significance of Deep Vein Thrombosis in Patients Presenting with Acute Symptomatic Pulmonary Embolism.14

21 Silva, S., MD and Wyer P, MD, 2010

Page 22: Roadmap Tutorial

22 Silva, S., MD and Wyer P, MD, 2010

GO

ING

FR

OM

SIN

GL

E S

TU

DIE

S T

O C

OM

PLE

X D

ESI

GN

S A

ND

LIN

KIN

G T

HE

M T

O Q

UE

STIO

N T

YPE

Page 23: Roadmap Tutorial

Critical Appraisal of the LiteratureThe two tables below summarizes the main points that should be addressed while appraising the literature. Critical Appraisal within Simple Model is divided into:

1. Appraisal of the validity of the methodology

2. Appraisal of the results

APPRAISE THE VALIDITY OF THE METHODOLOGY

A Summary in one TableEach moment in the course of a study represents a point where systematic errors may be introduced. Dividing the temporal course of studies into three conceptual moments-‘beginning’, ‘middle’ and ‘end’- helps clarify the nature of the different criteria commonly used to evaluate bias and facilitates a clinician’s ability to understand the distinctions between them.

The graph above represents the design of 5 commonly encountered study types, and also demonstrates the application of a temporal scheme as a conceptual aide to understanding critical appraisal. The scheme has precedents. Jackson15 and Hulley16 propose a temporal approach to understanding study design in contexts aimed primarily at research and epidemiological oriented learners. The Users’ Guides introduced this approach in a fashion more appropriate to clinical learners but limited its use to the context of randomized trials.17 We have extended the principle to each of the 4 basic study designs and also to systematic reviews. The critical appraisal criteria listed within the temporal scheme for each of these study types, including systematic reviews, have been drawn largely from the Users’ Guides and carry forward the rigorous approach used to select them that was introduced in that source.17

23 Silva, S., MD and Wyer P, MD, 2010

Page 24: Roadmap Tutorial

APPRAISE THE MAGNITUDE AND IMPORTANCE OF THE RESULTS

Conceptual Framework to evaluate importance and precision of study results for the domains of Therapy, Diagnosis, Prognosis and Harm

Frequency According to the English dictionary, frequency means “the number of times that something happens”.

For the purpose of SIMPLE we use the term “measures of frequency” to refer to the risk or odds of occurrence of clinical outcomes over time or to the probability of having some condition causing a set of signs or symptoms (pre-test probability or differential diagnosis).

EffectAccording to the English dictionary, effect means “a change that is the result or consequence of an action”.

For the purpose of SIMPLE we use the term “measures of effect” to refer to the effect of some intervention or factor in one group in relation to the effect of the same intervention or factor in another group i.e. the measure in one group related to the same measure in a comparison group. We call this relative measures: relative risk, odds ratio and hazard ratio.

ImpactAccording to the English dictionary, impact means “the effect of one thing on another”.

For the purpose of SIMPLE we use the term “measures of impact” to refer to the magnitude of effect added by some intervention in one group in relation to a comparison group i.e. the absolute difference of the effect of some intervention between groups. We call this risk difference (absolute risk reduction or absolute risk increase). Number Needed to Treat, to Harm, To Diagnosis, To Screen, etc are calculated out of the risk difference.

PrecisionAccording to the English dictionary, precision means “the quality of being exact and accurate”.

For the purpose of SIMPLE we use the term “measures of precision” to refer the confidence interval of measures of frequency, effect or impact.

24 Silva, S., MD and Wyer P, MD, 2010

Page 25: Roadmap Tutorial

APPRAISE THE MAGNITUDE AND IMPORTANCE OF THE RESULTS

Conceptual Framework to evaluate importance and precision of study results for the domains of Therapy, Diagnosis, Prognosis and Harm

NA: Not applicable; RR: Relative Risk; HR: Hazard Ratio; OR: Odds Ratio; RD: Risk Difference; LR: Likelihood Ratio; CI: Confidence Interval

The table above is for purposes of illustration, confined to measures of dichotomous outcome, such as death, or presence or absence of disease. Continuous measures are also frequently important.

_______________________________________________

∂∂ In the context of trials of utility, i.e. effectiveness, of use of diagnostic information∂ When prognostic information, either as tests, simple estimates of risk, predictors or predictive instruments are used as an intervention, i.e. a ‘prognostic utility study’+ Cohort studies can be used to assess the effect (utility) of a diagnostic or prognostic intervention or the effect of a therapeutic or harmful exposure. They can also be used to investigate the predictive power of individual or multiple predictors on likelihood of outcome. In the latter case, the relationship between the predictor and the likelihood of outcome is not, strictly, an ‘effect’. Rather the predictor is purely a marker of destiny, there being no imputation of cause.

25 Silva, S., MD and Wyer P, MD, 2010

Page 26: Roadmap Tutorial

Apply1. Evaluate the applicability of the study in your patient

2. Integrate relevant knowledge within the relational construct

EVALUATE THE APPLICABILITY OF THE STUDY

Values = the unique concerns and expectations each one brings to an encounter.

Best research evidence = research from patient-centered clinical research into the accuracy and precision of diagnostic tests (including the clinical examination), the power of prognostic markers, and the efficacy and safety of therapeutic, rehabilitative and preventive regimens.

Circumstances = the full range of social, intellectual, economic and environmental factors that determine what is possible and feasible among hypothetical options available to a patient.

26 Silva, S., MD and Wyer P, MD, 2010

Page 27: Roadmap Tutorial

Clinical expertise = the ability to draw from our medical knowledge, clinical skills and our experience in applying knowledge to practice in the course of identifying each patient’s benefits of potential interventions, and their personal values, preferences and priorities.

KNOWLEDGE TRANSLATION

Clinical decisions should be informed by well-communicated assessments of the highest level of available evidence from research as well as by reflective interpretations of patient values, preferences and circumstances within the context of the patient-practitioner relationship. Patient and practitioner roles in such decisions are themselves a product of that relationship.

Enjoy the Workshop!

27 Silva, S., MD and Wyer P, MD, 2010

Page 28: Roadmap Tutorial

Bibliography1." Presidential Committee on Information Literacy. Final Report. 1989; http://

www.ala.org/ala/mgrps/divs/acrl/publications/whitepapers/presidential.cfm. Accessed May 05, 2010.

2." Richardson WS, Wilson MC, Nishikawa J, Hayward RS. The well-built clinical question: a key to evidence-based decisions. ACP J Club. Nov-Dec 1995;123(3):A12-13.

3." Haynes RB. Of studies, syntheses, synopses, and systems: the "4S" evolution of services for finding current best evidence. ACP J Club. Mar-Apr 2001;134(2):A11-13.

4." Haynes RB. Of studies, syntheses, synopses, summaries, and systems: the "5S" evolution of information services for evidence-based health care decisions. ACP J Club. Nov-Dec 2006;145(3):A8.

5." Green RC, Roberts JS, Cupples LA, et al. Disclosure of APOE genotype for risk of Alzheimer's disease. N Engl J Med. Jul 16 2009;361(3):245-254.

6." Pfisterer M, Buser P, Rickli H, et al. BNP-guided vs symptom-guided heart failure therapy: the Trial of Intensified vs Standard Medical Therapy in Elderly Patients With Congestive Heart Failure (TIME-CHF) randomized trial. JAMA. Jan 28 2009;301(4):383-392.

7." Laine L, Sahota A, Shah A. Does capsule endoscopy improve outcomes in obscure gastrointestinal bleeding? Randomized trial versus dedicated small bowel radiography. Gastroenterology. May 2010;138(5):1673-1680 e1671; quiz e1611-1672.

8." Maisel AS, Krishnaswamy P, Nowak RM, et al. Rapid measurement of B-type natriuretic peptide in the emergency diagnosis of heart failure. N Engl J Med. Jul 18 2002;347(3):161-167.

9." McCullough PA, Nowak RM, McCord J, et al. B-type natriuretic peptide and clinical judgment in emergency diagnosis of heart failure: analysis from Breathing Not Properly (BNP) Multinational Study. Circulation. Jul 23 2002;106(4):416-422.

10." Sgarbossa EB, Pinski SL, Barbagelata A, et al. Electrocardiographic diagnosis of evolving acute myocardial infarction in the presence of left bundle-branch block. GUSTO-1 (Global Utilization of Streptokinase and Tissue Plasminogen Activator for Occluded Coronary Arteries) Investigators. N Engl J Med. Feb 22 1996;334(8):481-487.

11." Calafiore AM, Di Giammarco G, Teodori G, et al. Late results of first myocardial revascularization in multiple vessel disease: single versus bilateral internal mammary artery with or without saphenous vein grafts. Eur J Cardiothorac Surg. Sep 2004;26(3):542-548.

12." Pick AW, Orszulak TA, Anderson BJ, Schaff HV. Single versus bilateral internal mammary artery grafts: 10-year outcome analysis. Ann Thorac Surg. Sep 1997;64(3):599-605.

13." Morens DM, Grandinetti A, Waslien CI, Park CB, Ross GW, White LR. Case-control study of idiopathic Parkinson's disease and dietary vitamin E intake. Neurology. May 1996;46(5):1270-1274.

14." Jimenez D, Aujesky D, Diaz G, et al. Prognostic significance of deep vein thrombosis in patients presenting with acute symptomatic pulmonary embolism. Am J Respir Crit Care Med. May 1 2010;181(9):983-991.

28 Silva, S., MD and Wyer P, MD, 2010

Page 29: Roadmap Tutorial

15." Jackson R, Ameratunga S, Broad J, et al. The GATE frame: critical appraisal with pictures. ACP J Club. Mar-Apr 2006;144(2):A8-11.

16." Hulley SB, Cummings SR, Browner WS, Grady DG, Newman TB. Designing Clinical Research. 3rd ed. Philadelphia: Lippincott, Williams & Wilkins; 2007.

17." Guyatt G, Rennie D, Meade M, Cook D. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice: McGraw Hill; 2008.

29 Silva, S., MD and Wyer P, MD, 2010