Making Guidelines

Actionable

Richard Rosenfeld & Richard Shiffman

E-GAPPS Breakout SessionNY Academy of Medicine 12/12

Standards for DevelopingTrustworthy Clinical Practice Guidelines

Updated IOM Definition ofClinical Practice Guidelines

Guidelines are statements that include recommendations intended to optimizepatient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options

http://www.iom.edu/Reports/2011/Clinical-Practice-Guidelines-We-Can-Trust/Standards.aspx

AAO-HNS Clinical Practice

Guideline Development

Process

www.entnet.org

Clinical Practice Guideline Development: A Quality-Driven Approach for Translating Evidence into Action

Pragmatic, transparent approach to creating guidelines for performance assessment

Evidence-based, multidisciplinary process leading to publication in 12-18 months

Emphasizes a focused set of key action statements to promote quality improvement

Uses evidence profiles to summarize decisions and value judgments in recommendations

Rosenfeld & Shiffman, Otolaryngol HNS 2009

Otolaryngol Head Neck Surg 2009; 140(Suppl):S1-43

Two Approaches to Evidence and Guidelines

Evidence as Protagonist ModelDevelopment is driven by the literature search,

which takes center stage with exhaustive evidence tablesor textual discussions that rank and summarize citations.

Product is a Practice Parameter, EvidenceReport, or Evidence-Based Review

Evidence as Supporting Cast ModelDevelopment is driven by a priori considerations of

quality improvement, using the literature search as one of manyfactors that are used to translate evidence into action.

Product is a Guideline with Actionable Statements

Generating Topics for Action Statements

Ask “If we could only discuss a few aspects of this condition, what topics would we focus on most to improve quality of care?”

Ask “What should we focus on to minimize harm?” Consider high level evidence from systematic review and the

concept list generating when discussion scope. Remember: A quality-driven approach allows all

important topics to be included, even if evidence isweak or limited. Action statements may still bepossible based on the balance of benefit and harm.

Rosenfeld & Shiffman, Otolaryngol HNS 2009

Developing key action statements begins with asking the group to suggest topics that are opportunities for quality improvement within the scope

Otolaryngol Head Neck Surg 2009; 140(Suppl):S1-43

Ranked Topic List for Hoarseness Guideline

1. Promote appropriate care2. Reduce inappropriate or harmful care3. Reduce variations in delivery of care4. Improve access to care5. Facilitate ethical care6. Educate & empower clinicians & patients7. Facilitate coordination & continuity of

care8. Improve knowledge base across disciplines

Quality Improvement Opportunities

Eden J, Wheatley B, McNeil B, Sox H (eds).Washington, DC: Nat’l Academies Press

a.k.a. Potential topics for guideline action statements

Standards for DevelopingTrustworthy Clinical Practice Guidelines

Standard 6. Articulation of Recommendations

6.1 Recommendations should be articulated in a standardized form detailing precisely: what the recommended action is, and under what circumstances it should be performed.

6.2 Strong recommendations should be worded so that compliance with the recommendation(s) can be evaluated.

http://www.iom.edu/Reports/2011/Clinical-Practice-Guidelines-We-Can-Trust/Standards.aspx

Statements of Fact vs. Action

Clinicians should not routinely administer or prescribe perioperative antibiotics to children undergoing tonsillectomy.

Antibiotic therapy does not improve recovery after tonsillectomy

The management of acute otitis externa should include an assessment of pain.The clinician should recommend analgesic treatment based on the severity of pain.

Acute otitis externa (swimmer’s ear) is associated with moderate to severe pain.

Clinicians should advocate for voice therapy for patients diagnosed with hoarseness (dysphonia) that reduces voice-related quality of life.

Voice therapy has been shown to improve quality of life for patients with hoarseness (dysphonia).

Clinicians should use pneumatic otoscopy as the primary diagnostic method for otitis media with effusion.

Pneumatic otoscopy is the most accurate test for otitis media with effusion.

Statement of ActionStatement of Fact

Guidelines ARE NOT Review Articles!Guidelines contain key statements that are action-oriented

prescriptions of specific behavior from a clinician

Monitor

Test

Gather Interpret Perform Dispose

Action

Conclude Prescribe

Educate

Document

Procedure

Consult

Advocate

PrepareBeware of the dreaded “Consider…”

Key Action Statements

An ideal action statement describes: When (under what conditions) Who (specifically) Must, Should, or May

(e.g., the level of obligation) do What (precisely) to Whom

Anatomy of a Guideline Recommendation

Action Statement Profiles andGuideline Development1. Encourage an explicit and transparent

approach to guideline writing

2. Force guideline developers to discuss and document the decision making process

3. Create “organizational memory” to avoidre-discussing already agreed upon issues

4. Allow guideline users to rapidly understand how and why statements were developed

5. Facilitate identifying aspects of guideline best suited to performance assessment

Key action statement withrecommendation strengthand justification

Supporting text for keyaction statement

Action statement profile: Aggregate evidence quality: Confidence in evidence: Benefit: Risk, harm, cost: Benefit-harm assessment: Value judgments: Intentional vagueness: Role of patient preferences: Differences of opinion: Exclusions:

1. Diagnosis of acute rhinosinusitis: Clinicians should distinguish presumed acute bacterial rhinosinusitis (ABRS) from acute rhinosinusitis caused by viral upper respiratory infections and non-infectious conditions.

A clinician should diagnose ABRS when (a) symptoms or signs of acute rhinosinusitis are present 10 days or more beyond the onset of upper respiratory symptoms, or (b) symptoms or signs of acute rhinosinusitis worsen within 10 days after an initial improvement (double worsening). Strong recommendation based on diagnostic studies with minor limitations and a preponderance of benefit over harm.

AAO-HNS Adult Sinusitis Clinical Practice Guideline

Evidence profile (abbreviated): Aggregate evidence quality: Grade B, diagnostic studies with minor limitations

regarding signs and symptoms associated with ABRS Benefits: decrease inappropriate use of antibiotics for non-bacterial illness;

distinguish non-infectious conditions from rhinosinusitis Harms: risk of misclassifying bacterial rhinosinusitis as viral, or vice-versa Benefits-harm assessment: preponderance of benefit over harms Value judgments: importance of avoiding inappropriate antibiotics for treatment of

viral or non-bacterial illness; emphasis on clinical signs and symptoms for initial diagnosis; importance of avoiding unnecessary diagnostic tests

Otolaryngol Head Neck Surg 2007; 137(Suppl):S1-S31

Classifying Recommendations for Practice GuidelinesAAP Steering Committee on Quality Improvement and Management

Pediatrics 2004; 114:874-877

Action Statements as Behavior Constraints

MAY

SHOULD

MUST or SHOULD

Obligation level

Be flexible in decision making regarding appropriate practice, although bounds may be set on alternatives

Generally follow a recommendation, but remain alert to new information

Follow unless a clear and compelling rationale for alternative approach exists

Implication for clinicians

Option

Recommendation

Strong recommendation

Policy strength

Cross-sectional survey of 1,332 registrants of the 2008 annual AHRQ conference given a clinical scenario with recommendations and asked to

rate the level of obligation they believe the authors intended

Lomotan E, et al. How “should” we write guideline recommendations? Interpretation of deontic terminology. Quality Safety Health Care 2009

Standards for DevelopingTrustworthy Clinical Practice Guidelines

Standard 5. RecommendationsFor each recommendation provide: An explanation of the reasoning including:

benefits, harms, evidence summary (quality, quantity, consistency), and the role of values, opinion and experience

A rating of the level of confidence in (certainty regarding) the evidence

A rating of recommendation strength A description and explanation of any differences

of opinion regarding the recommendation

http://www.iom.edu/Reports/2011/Clinical-Practice-Guidelines-We-Can-Trust/Standards.aspx

Building Better Guidelines with BRIDGE-Wiz

Description of a software assistant for structured action statementcreation to promote clarity, transparency and implementability

Shiffman…Rosenfeld et al, JAMIA 2012

J Am Med Inform Assoc 2012; 19:94-101.

1. Choose an action type

2. Choose a verb

3. Define the object for the verb

4. Add actions

5. Check executability

6. Define conditions for the action

7. Check decidability

8. Describe benefits, risks, harms & costs

9. Judge the benefit-harms balance

10. Select aggregate evidence quality

11. Review proposed strength of recommendation and level of obligation

12. Define the actor

13. Choose recommendation style

14. Edit the final statement

8. Testing for allergy and immune function: Clinicians may obtain testing for allergy and immune function in evaluation a patient with chronic rhinosinusitis (CRS) or recurrent acute rhinosinusitis.

Option based on observational studies with an unclear balance of benefit vs. harm.

AAO-HNS Adult Sinusitis Clinical Practice Guideline

Evidence profile: Aggregate evidence quality: Grade C, observational studies Benefits: identify allergies or immunodeficient states that are potential modifying

factors for CRS or recurrent acute rhinosinusitis Harms: procedural discomfort; instituting therapy based on test results with limited

evidence of efficacy for CRS or recurrent acute rhinosinusitis; very rare chance of anaphylactic reactions during allergy testing

Cost: procedural and laboratory cost Benefits-harm assessment: unclear balance of benefit vs. harm Value judgments: need to balance detecting allergy in a population with high

prevalence vs. limited evidence showing benefits of allergy management outcomes Role of patient preferences: role for shared decision making

Otolaryngol Head Neck Surg 2007; 137(Suppl):S1-S31

Classifying Recommendations for Practice GuidelinesAAP Steering Committee on Quality Improvement and Management

Pediatrics 2004; 114:874-877

Clinicians and Options

Evidence quality is suspect or well-designed studies have demonstrated little clear advantage to one approach vs. another

Options offer flexibility in decision making about appropriate practice, although they may set boundaries on alternatives

Hard to hold clinicians accountable (performance measures) Patient preference should have a substantial role in influencing

clinical decision making

What Do They Mean?

…And Now It’s Your Turn…

Treatment & Prevention of the Common Cold

Cochrane Systematic Reviews

The Cochrane Library, 2010; John Wiley & Sons, Ltd

Intervention (update) Evidence Conclusion

Antibiotics (2009) 6 trials No benefits; more adverse events

Non-steroidal anti-inflammatory drugs (2009)

9 trials Reduced headache, ear pain, muscle & joint pain; no effect on duration or adverse events

Echinacea (2007) 16 trials Some early treatment benefit; no effect on prevention

Heated, humidified air (2006)

6 trials Benefit for symptom relief in 3 studies; overall effects equivocal; minor discomfort, irritation, congestion

Chinese medicinal herbs (2008)

17 trials Faster recovery 7 trials; no benefits in 10; problem with heterogeneity

Vitamin C (2010) 29 trials Reduced duration and severity in prophylaxis trials (but not treatment trials); no benefit for prevention

Garlic (2009) 1 trial Benefit for prevention in a single trial