basic approaches in clinical practice guideline development

8/2/2019 Basic Approaches in Clinical Practice Guideline Development

1/11

Objectives:At the end of the session, the student should be able to:

Define Clinical Practice Guidelines Discuss the basic approaches to guidelines development Discuss the techniques in producing consensus statements Discuss the general steps in guideline development

Introduction

Proliferation of new diagnostic and therapeutic modalities

Patients may be exposed to wide variations in clinical care, even for the samecondition.

Competing obligations inhibit clinicians from critically appraising each newtechnology

CPGs help overwhelmed practitioners cope with the flood of new drugs anddevices.

There has been a widespread move toward developing clinical practiceguidelines

CPGs are designed to improve the quality of health care, to reduce the use ofunnecessary, ineffective or harmful interventions, and to facilitate thetreatment of patients with maximum change of benefit, with minimum risk of

harm, and at an acceptable cost.

DefinitionClinical Practice Guidelines (CPGs)

Systematically developed statements to assist practitioner and patientdecisions about appropriate health care for specific clinical circumstances.~~US Institute of Medicine, 1990

Field & Lohr, 1990

Otitis Media with Effusion in Young ChildrenNote: Dr. Frias just used the below as an example. No need to memorize. Justread whats written below to get an idea of the entire process behind the formation

of Clinical Practice Guidelines. He just wanted us to know how this all works.Basically, CPGs are created to inform physicians as to which treatment modalitiesare effective and in which scenarios. Also, to aid patients in deciding whichtreatment options are best for their situation. Eg, the scenario he gives is how tomanage a patient who arrives for the first time with otitis media. Does thephysician treat or observe? The CPGs below will aid the physician in coming to theappropriate treatment plan.Initial Management

Observation OR Antibiotic TherapyOption: observation OR antibiotictherapy may be chosen for management of otitis media with effusion in anotherwise healthy child. [Option based on limited scientific evidence andPanel consensus].

Note: Panel consensus is based on expert opinion and not on scientificresearch or evidence. Most of the time, the panel is multidisciplinary andare listed on the CPG. Evidence is not considered as strong as scientificevidence via randomized control study or clinical trial.

Environmental Risk Factor ControlOption: Parents should beencouraged to control environmental risk factors [Option based on limitedscientific evidence and strong Panel consensus].

Subj

ect: Comm Med 2Topic: Basic Approaches in ClinicalPractice Guideline DevelopmentLecturer: Dr. Frias and Dr. SosaDate of Lecture: March 2, 2012

Transcriptionist: Chocoholic andBonbonEditor: JojoPages: 10

SY2011-2012


2/11

MyringotomyRecommendation: Myringotomy with or without insertion oftympanostomy tubes should NOT be performed for initial management ofotitis media with effusion in an otherwise healthy child. [Strongrecommendation based on evidence that otitis media with effusion resolvesspontaneously in most cases and lack of conclusive evidence that a short

period of otitis media with effusion has deleterious effects on otherwise

healthy children]. Note: strong evidence implies that scientific research was done, usually

randomized control, whereas weak evidence is usually arrived at via casecontrol, cross sectional, or case studies.

After 3 monthsIf the child has hearing in the normal range, as indicated by hearing threshold levelbetter than 20 decibels in the better hearing ear, recommendations are as follows:

Observation OR Antibiotic TherapyOption: Observation OR antibiotictherapy may be chosen. [Option based on limited scientific evidence andPanel consensus].

Note: evidence based on panel consensus/expert evidence is the

weakest form of evidence as compared to clinical trial or meta analysis.

Environmental Risk Factor ControlOption: Parents should beencouraged to control environmental risk factors. [Option based on limitedscientific evidence and strong Panel consensus].

If the child has bilateral hearing deficits of 20 decibels hearing threshold level orworse, recommendations are as follows:

Antibiotic Therapy OR Myringotomy with TubesOption: Antibiotictherapy OR bilateral myringotomy with insertion of tympanostomy tubes maybe chosen to manage bilateral otitis media with effusion that has lasted atotal of 3 months in an otherwise healthy child age 1 through 3 years whohas a bilateral hearing deficit (defined as 20 decibels hearing threshold levelor worse in the better hearing ear). [Based on limited scientific evidenceand Panel consensus].

Environmental Risk Factor ControlOption: Parents should beencouraged to control environmental risk factors. [Option based on limitedscientific evidence and strong Panel consensus].

Not Recommended At Any Time Steroid TherapyRecommendation: steroid medications are not

recommended for treatment of otitis media with effusion in a child of anyage. [Based on limited scientific evidence and Panel majority opinion]. ThePanel makes no statement regarding the use of steroid medications forconditions other than otitis media with effusion.

Antihistamines/Decongestant TherapyRecommendation:Antihistamine and/or decongestant agents are not recommended fortreatment of otitis media with effusion [Strong recommendation based onevidence that can be generalized to a child of any age]. The Panel makes nostatement regarding the use of antihistamine and/or decongestantmedications for conditions other than otitis media with effusion.

AdenoidectomyRecommendation: Adenoidectomy is not recommendedfor treatment of otitis media with effusion in a child age 1 through 3 years inthe absence of specific adenoid pathology. [Based on limited scientificevidence and Panel majority opinion].

TonsillectomyRecommendation: Tonsillectomy should not be performed,either alone or with adenoidectomy, for the treatment of otitis media witheffusion in a child of any age. [Strong recommendation based on limited

scientific evidence and strong Panel consensus].

CPG Process Development: See below. This is the emphasis of this lecture. Dissemination: CPGs, once formed, should be dispersed among specialist (in

the case above, it would be EENT, pediatricians, infectious diseasespecialists, etc.)

Implementation


3/11

Evaluation: Usually done 3 5 years after a CPG is formed. The time intervalis decided upon by the framers of the CPG.

Revision: After development, it returns once again to development. Theentire process is cyclical.

Guideline Development

Three basic approaches Evidence based Consensus based Evidence and consensus based

~~Woolf, SH, 1993~~Audet, AM, 1990

Evidence-based approach Utilizes a systematic synthesis and appraisal of the literature and gives

recommendations according to the strength of evidence. Enhances scientific rigor. Inability to produce recommendations in the absence of acceptable

evidence. Thorough and systematic literature search Critical appraisal or rigorous review of the literature (based on a set of

criteria) strength of evidenced: level and quality. (Not all literature will beaccepted, only those that are valid and of good quality).

Systematic synthesis of the literature Evidence based report recommendations based on evidence are classified

according to a system of rating and grading.

Consensus based Approach Utilizes consensus building techniques to produce statements or

recommendations Informal Formal

Informal consensus building technique Results in guidelines reflecting the global subjective judgment of the

experts who are its framers. Advantages:

Easy Fast Free of complex analytic procedures

Disadvantages: Decibel factor: ie, those who speak the loudest may make their opinion

known over those who stay quiet. May pull recommendations in theirfavor.

Difficulty in the assessment of the guides validity (because it is not basedon scientific evidence).

Limitations to validity of experts opinion Dependence on personalities and affiliations (ie, a person may be biased

in favor of a test that they developed).Formal consensus building techniques

Results in guidelines reflecting the use of a structured methodology inconsensus building.

This type of approach is used in the general steps in guideline development.(See diagram below).

Problem identification: Using the article we read, it would be to determineif dipstick assay is just as accurate as blood culture in diagnosing typhoid.

Panel selection: Experts are selected. Should be multi disciplinary. Inour case, it would be infectious disease specialist, gastroenterologists,cardiologists, nurses, epidemiologists, even patients).

Data retrieval: If the problem is diagnosis, literature would be collectedon the diagnosis of typhoid.


4/11

Exchange of ideas/Definition of consensus: What is a consensus? 50% =1? This is determined by the panel. Exchange of ideas is usually done ina structured manner.

Construction of draft: Composition of topic forum would be same ascomposition of panel.

Feedback: Draft is reviewed. Final draft: Performed by topic forum. Schedule of review: This panel determines if the CPG is to be reviewed in

3 years, 5 years, etc. Also, it determines which portions are to bereviewed.

Then this all starts again with problem identification. Overall, it takes 1 2 years to create a CPG.

Advantages: Validity can be measured Personalities/affiliations exert less influence Evidence beyond experts opinions Acceptance is almost assured

Disadvantages: Time and cost constraints Difficulty of the process

Specific formal consensus building techniques:

Nominal group technique Provides for a structured meeting for obtaining information Equal participation among members Output is independent of personalities or affiliations Time constraints/reliance on skilled facilitators Usually done by epidemiologists as they are trained for this. Steps: Silent generation of ideas in writing Round robin recording of ideas (each idea is discussed) Serial discussion for clarification Preliminary vote on item importance Discussion of the preliminary vote

Final vote Delphi technique Seeks consensus anonymously through self administered

questionnaires (eg, in convention, meeting, etc.) Provides for easier access to many experts Provides for impersonal expression of views


5/11

Constrained by element of time (sometimes questionnaire may not beobtained immediately, in which case, the expert must be trackeddown).

May become complicated and exhaust the panelists Vote by mail Collects experts opinions by mail

Easy and produces fast results Very little interaction among experts Simple voting rather than a true consensus is achieved Steps: Panelists are asked to rate specific conditions for certain diagnostic

or therapeutic procedures Results are published in journal

Science Court

Gathers all the panelists in a closed session to present and discussissues (testimonies). Consensus statements are produced after themeeting

Multisectorial approach Absence of explicit criteria for consensus and voting process

Evidence based and Consensus based Approach Utilizes consensus building techniques in its methodology In the resulting document, distinction is made on recommendations based on

existing evidence and on experts opinions evidence is classified accordingto a system of rating and grading.

Table 1. Levels of evidence for rating studies on the effectiveness oftreatmentLevel1

An RCT that demonstrate a statistically significant difference in at least onemajor outcome OR if not statistically significant, an RCT of adequatesample size to exclude 25% diff in RR with 80% power

Level2

An RCT that does not meet the level 1 criteria

Level3

A non-randomized trial with concurrent controls selected by somesystematic method

Level4

Before-after study or case series (at least 10 patients) with historicalcontrols drawn from other studies

Level5

Case series (at least 10 patients) without controls

Level6

Case series (fewer than 10 patients) or case reports

Levels of evidence for rating studies on:1. Efficacy or effectiveness of treatment2. Accuracy of diagnostic tests3. Prognosis or causation4. Review of articles

Level of Evidence:Ia: evidence for meta analysis of randomized controlled trials (strongest formof evidence).Ib: evidence from at least on randomized controlled trialIIa: evidence from at least one controlled study without randomizationIIb: evidence from at least on other type of quasi experimental study

III: evidence from non experimental descriptive studies, such as comparativestudies, correlation studies, and case control studiesIV: evidence from expert committee reports or opinions or clinical experience ofrespected authorities, or both.

Strength of Recommendation (based on level of evidence)A: directly based on category I evidence


6/11

B: directly based on category II evidence or extrapolated recommendation fromcategory I evidenceC: directly based on category III evidence or extrapolated recommendation fromcategory I or II evidenceD: directly based on category IV evidence or extrapolated recommendation fromcategory I, II, or III evidence

Table 2. Grading System for Recommendation (please refer to Table 1above for an explanation of the levels below).GradeA

The recommendation is based on one or more studies at level 1

GradeB

The best evidence available is at level 2

GradeC

The best evidence is at level 3

GradeD

The best evidence available is lower than level 3, and included expertsopinions, clinical experience and common sense. These recommendationsaddress practical issues of implementation and other factors existing in the

local setting.

Note: If evidence based approach is used, then it only gets to the data retrievalor exchange of ideas only. The penultimate draft is already evidence based.Consensus based only goes from exchange of ideas to penultimate draft.

Summary (Know this for the exam. These are the types of questions thatwill be asked.)

Definition of Clinical Practice Guideline Discussed the three basic approaches in guideline development Discussed the various techniques in consensus building Discussed the general steps in guideline development

Dr. Sosa Lecture (This is the evaluation of the article):

SPECIFIC OBJECTIVESTo discuss the rationale for each of the users guides pertaining to the validity of

claims on accuracy.To define and differentiate the concepts of validity and reliability, as they pertain tothe results of a diagnostic test.

To learn the definitions of and calculations for sensitivity, specificity, positive andnegative predictive value, likelihood ration, pretest and post test probability, pretestand post test odds.


7/11

To recognize the advantages and disadvantages of using likelihood ratios,sensitivities, specificities and predictive values in measuring the accuracy ofdiagnostic tests.

To learn how to estimate posttest probabilities given the pretest probability and ameasure of accuracy.

APPRAISING DIAGNOSTIC TESTS: 3 EASY STEPS1 Are the results valid

Appropriate spectrum of patients?Does everyone get the gold standard?Is there an independent, blind or objective comparison with the gold

standard?2 What are the results?

Sensitivity, specificityLikelihood ratiosPredictive values

3 Will they help me look after my patients?Can I do the test in any setting?

Do results apply to the mix of patients I see?Will the result change my management?Costs to the patient/health service?

CLINICAL SCENARIO (handout given and scientific study given)

CLINICAL QUESTIONP: PopulationE/I: Exposure/ InterventionC: Comparison GroupO: OutcomeM: Methodology

In the scenario given: P Patients with typhoid feverE Dipstick assayC Blood c/sO Accuracy in the diagnosis of typhoid feverM

What is the accuracy of dipstick assay test compared with blood culture inthe diagnosis of typhoid fever?

EVALUATING DIRECTNESSDoes the study provide a direct enough answer to your clinical question in terms of :P: patient population with a certain disease?E: the exposure (or treatment) to be administeredO: the outcome (or condition) that the treatment are intended to prevent orpromoteM: methodology

Compare the clinical question with the research questionWhen will there be a mismatch in P?

Your cousin, who texted you in the scenario, is 25 years old and the studypopulation in the article is comprised of pediatric patients or pregnantfemales. Another example of mismatch in the population is when yourcousins case is mild, but the research focused on severe typhoid cases only.

What will happen if there is a mismatch in E? There would be a mismatch in exposure (or treatment) if the article were on

Typhidot and not on dipstick.When will there be a discrepancy in O?

Since your cousin asked about the accuracy of the test, the article shouldaddress this, instead of, lets say, the quality of life after typhoid infection.

That construes a mismatch on outcome.

APPRAISING VALIDITY


8/11

Was the reference standard an acceptable one? Yes. Its a blood culture, the gold standard for diagnosing typhoid.

Was a reference standard interpreted independently from the test in question?

ARE THE RESULTS IF THE STUDY VALID?

Primary GuidesWas there independent, blind comparison with a reference standard?

Accept the gold standard without sacrificing the patients If you cannot accept it --- results not valid Note: In this article, it is not specified that the researchers were blinded

when analyzing the blood cultures and dipstick assays.Did the patient sample include an appropriate spectrum of patients to whom thediagnostic test will be applied in clinical practice?

Should distinguished between the target disorders States that otherwise be confused Closely resembles clinical practice The article did not specify if there were a complete spectrum of illness

however.Validity Questions Answers

1. Was the reference standard anacceptable one?

Yes, blood culture and bone culture.

2. Was the reference standardinterpreted independently fromthe test in question?

Yes.

APPRAISING RESULTS?What likelihood ratios were associated with the range of possible results?

Measure of accuracy sensitivity, specificity, positive predictive value,negative predictive value

Estimate the likelihood ratios for each of the possible resultsEstimate the pretest and post test probability

SENSITIVITY AND SPECIFICITYSensitivity: probability of a positive test among people with the target diseaseSpecificity: probability of a negative test among people without the target disease

PREDICTIVE VALUEPositive predictive value probability of having the target disease among patientswith a positive testNegative predictive value probability of not having the target disease amongpatients with a negative test

Reference StandardTest result Disease present Disease absent TotalDisease present True Positive

(a)False Positive

(b)B+D

Disease absent False Negative(c)

True Negative(d)

C+D

Sensitivity= a/ (a+c)Specificity= d/(b+d)LR for positive test result= [a/(a+c) / [b/(b +d)]]LR for negative test result= [c/(a+c) / [d/(b+d)]]Positive predictive value= a/(a+b)Negative predictive value= d/(c+d)Note: The above equations will only apply when a 2 x 2 table will work, eg, testingonly for (+) and (-) results.

WHAT ARE THE RESULTS?


9/11

Are the likelihood ratios for the test results presented or data necessary for theircalculation or provided?

Of the test-determined by the accuracy the target diseaseAccuracy measure LIKELIHOOD RATIO

LIKELIHOOD RATIO (LR)

Indicates how much a given diagnostic test result will raise or lower the pre testprobability of a target disease

LR = 1: post test probability=pretest probabilityLR>1: increase the probability of a target disease is present

The higher the LR the greater the increaseLR


10/11

Note: The above equations work only if a 2 x 2 square is applicable. As wesaw with our article, since they also tested for varying degrees of test results,2 x 2 square did not apply in our case.Sensitivity = 70/91 = 0.77 or 77% of patients who tested positive for theblood culture also tested positive for the dipstick. This ratio is clearlyspecified in the table.

Specificity = 32/36 = 0.89 or 89% of patients who tested negative for theblood culture also tested negative for the dipstick. This evidence is notclearly specified. You have to look at the patients who tested negative forthe dipstick assay but who also tested negative for the culture.

2. Calculate for positive and negative likelihood ratios for the dipstick assay.Answer:Since we dont have test results that would give us a standard 2 x 2 square,we must use the following values:LR+= sens/(1-spec) LR-= (1-sens)/specSensitivity and specificity were given in the article. So.LR+ = 0.865/(1-0.889) = 7.792LR- = (1 0.865)/0.889 = 0.152

Probability of positive test = a/(1+a) = 7.792/(1+7.792) = 0.886 or 88.6%.Since our pre-test value was 50%, this shows a value >pre test value.

Therefore, this patient should be treated for typhoid.3. Calculate LR+ for +1 positive tests.

Answer:Again, since the researchers didnt do just + and test results, but instead

did varying grades oftest results, a standard 2 x 2 square did not apply, so the equations ofsensitivity= a/ (a+c) and specificity= d/(b+d) will also not apply. Hence, tocalculate sensitivity and specificity for +1 test:Sensitivity = # with +1/# total with + blood culture = 18/91 = 0.20Specificity = # who did not get +1 test/# total with blood culture = 114/116= 0.98 (Note: she added the control population to those who were S. typhi -)LR+ = 0.20/(1-0.98) = 10Probability = 10/(1+10) = 0.91

4. Calculate LR+ for +4 positive testsAnswer:Same rules as for #3 above apply.Sensitivity = # with +4 value/#total with +blood culture = 3/91 = 0.033Specificity = # who did not get +4 test/# total with blood culture = 115/116LR+ = 0.033/(1-0.99) = 3.3Probability = 3.3/(1+3.3) = 0.767

Note: since probabilities for +1 test > probability for +4 test. Either way, the posttest probability for both +1 and +4 tests > than pretest probability of 50%, so thistest should be recommended.

Addendum for Epidemic Investigation 1:Note: These are just some notes that we thought would be useful for the exam.

They may ask what type of study was performed for the epidemic investigation inthe question and these are some tips for recognizing them.

Cross Sectional A snapshot in time. Does not prove causality.

Measures prevalence ratio. Prevalence ratio:

Prevalence of disease among the exposedPrevalence of disease among the unexposed

A/(A + B)C/(C + D)

PR = 1 no association PR < 1 preventive association


11/11

PR > 1 causative association

Case Control Retrospective study Does not prove causality Measures odds ratio

Odds of exposure among those cases with disease (cases)___Odds of exposure among those without the disease (control)

(A/C)(B/D)

OR = 1 no association OR < 1 preventive association OR > 1 causative association

Cohort Study Retrospective or prospective study Does prove causality

Measures relative risk Incidence of disease among exposed______Incidence of disease among unexposed

A/(A + B)C/(C + D)

RR = 1 no association RR < 1 preventive association RR > 1 causative association

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~End ofTranscription~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

For I know the plans I have for you, plans for wholeness and not for evil, to giveyou a future and hope. Then you will call upon me and come and pray to me, and I

will hear you. You will seek me and find me. When you seek, seek Me with all your

heart. Jeremiah 29:11-13

basic approaches in clinical practice guideline development

Documents