Evidence-Based Practice Parameters   David G. Nickinovich PhD , Richard T. Connis PhD , Robert A. Caplan MD , James F. Arens MD and Jeffrey L. Apfelbaum MD

Evidence-Based Practice of Anesthesiology, Chapter 1, 2-6

  Get rights and content

Practice parameters developed by the American Society of Anesthesiologists (ASA) have been an important resource for physicians and other health-care workers for more than 20 years. The intention of the ASA evidence-based practice parameter is to enhance and promote safe medical practice as well as offer guidance for diagnosing, managing, or treating a variety of clinical conditions. ASA evidence-based practice parameters consist of a “broad body of documents developed on the basis of a systematic and standardized approach to the collection, assessment, analysis and reporting of: scientific literature, expert opinion, ASA member opinion, feasibility data and open forum commentary.”

Evidence-based practice parameters may take the form of guidelines or advisories.

Before the development of a policy for evidence-based practice parameters in 1991, ASA practice parameters were primarily consensus-based documents, and the majority of these documents were practice standards. Practice standards were typically declarative statements focusing on simple aspects of patient care applicable to virtually all relevant anesthetic situations.

The standards were well received within both the anesthesia community and allied medical professions and positioned the ASA and the Anesthesia Patient Safety Foundation of the ASA at the forefront of medical practice by demonstrating the benefits of a proactive approach to patient safety.

Many aspects of practice, however, could not be adequately covered by the relatively limited and prescriptive recommendations of practice standards. When broader and more flexible recommendations for practice were needed, the ASA broadened its scope to encompass practice guidelines. The practice guidelines were initially formulated on the basis of evidence generated by the same consensus-based methodology used in the development of standards. To effectively evaluate the increasing breadth and complexity of issues considered by practice guidelines, the ASA Committee on Standards and Practice Parameters (Committee) determined that a systematic evaluation of scientific evidence was necessary to fully support recommendations driven by expert opinion. Using a method that systematically combined a synthesis of the literature with opinions

from experts and other sources, the ASA produced the first two evidence-based practice guidelines in 1993.

In developing these guidelines, the Committee recognized the unique properties of both the anesthesia literature and the practice of anesthesiology and realized that further methodologic changes were needed. Over the next few years, a more elaborate multidimensional method to guideline development evolved. It contained four critical components: (1) a rigorous review and evaluation of all available published scientific evidence, (2) meta-analytic assessments of controlled clinical studies when appropriate, (3) a statistical assessment of expert and practitioner opinions obtained by formally developed surveys, and (4) the informal evaluation of opinions obtained from invited and public commentary.

Process of Parameter Development

The process used by the ASA to develop evidence-based practice parameters normally begins when the Committee identifies an issue or clinical problem. The Committee then appoints a task force of 8 to 12 anesthesiologists who are recognized experts on the issue or clinical problem to advise the Committee on the need for a practice parameter. Task force members are carefully chosen to not only provide representation from both private practice and academia but also ensure representation across major geographic areas of the United States. Occasionally, nonanesthesiologists may also be appointed to a task force if the Committee determines that their appointment would add specific subspecialty expertise (e.g., the appointment of a radiologist to the magnetic resonance imaging task force). Conflict of interest issues are fully evaluated before individuals are selected to serve on a task force, and such information is fully transparent to the reader.

If the task force determines that sufficient evidence is available, the process of defining goals and objectives within the mandate established by the Committee begins. During this conceptualization phase, approximately 75 to 150 peer-review consultants are identified as secondary external sources of opinion, practical knowledge, and expertise. Consultants typically are recognized experts in the subject matter and, like the task force members, represent a balance of practice settings and geographic locations. Depending on the clinical topic, individuals from nonanesthesia medical specialties or organizations may be selected as consultants.

An initial step in the development of an evidence-based practice parameter is to survey the task force members to identify target conditions, patient or clinical presentations, providers, interventions, practice settings, and other characteristics that help define or clarify the parameter. On the basis of the survey responses, members of the task force collectively develop a list of clinical interventions and expected outcomes. The list, typically referred to as “evidence linkages” between interventions and outcomes, forms the foundation on which evidence is collected and organized and provides structure for formulation of recommendations. When possible and appropriate, evidence linkages are designed to describe comparative relationships between interventions and outcomes. For example, the linkage statement “spinal opioids versus parenteral opioids improve

maternal analgesia for labor” identifies a specific intervention (spinal opioids), a comparison intervention (parenteral opioids), and a specific clinical outcome (maternal analgesia) thought to be affected by the intervention. Once all evidence linkages for the parameter are specified, the task force then begins the process of collecting evidence.

Sources of Evidence

The multiple sources of information used by a task force in developing an evidence-based practice parameter are displayed in Table 1-1 . During the search for evidence, the task force considers two major sources: literature-based evidence and opinion-based evidence. Within the domain of literature-based evidence, meta-analytic findings are reported when sufficient numbers of randomized controlled trials (RCTs) are available, and descriptive outcome data summaries (e.g., means, ranges, and sensitivity/specificity values) are reported for interventions not subject to evaluation by RCTs. For opinion-based evidence, consensus-based information obtained from formal surveys as well as informal sources (e.g., open forum commentary and Internet comments) is considered. The final determination of whether the document is a guideline or an advisory is based on the totality of evidence accumulated.

TABLE 1-1 Sources of Evidence for Practice Parameters Source of Evidence Type of Evidence Literature-Based Evidence Randomized controlled trials Comparative statistics Nonrandomized prospective studies Comparative statistics Controlled observational studies Correlation/regression Retrospective comparative studies Comparative statistics Uncontrolled observational studies Correlation/regression/descriptive statistics Case reports No statistical data Opinion-Based Evidence Consultants Survey findings/expert opinion ASA members Survey findings/opinion Invited sources Expert opinion Open forum commentary Public opinion Internet commentary Public opinion

The Literature Search

The initial literature search includes a computerized search of PubMed and other large reference sources/databases and usually yields 2000 to 5000 citations for each practice parameter. Manual searches are also conducted when supplemental references are supplied by the consultants and members of the task force.

In the selection of published studies, three conditions must be met. First, the study must assess one or more of the interventions being considered. Second, the study must report an anesthetic or clinical outcome or set of findings that can be tallied or quantified, thereby eliminating reports that contain only opinion (e.g., editorials and news reports). Third, the study must be an original investigation or report. Review articles, books or book chapters, and manuscripts that report findings from previous publications are not used as sources of evidence. After the initial electronic review, letters, editorials, commentaries, and other literature with no original data are removed from consideration. Typically, only 1000 to 2500 articles prove suitable for retrieval and further review.

Evaluating and Summarizing the Literature

The literature review process focuses on studies that report outcomes relevant to an identified intervention. A standard classification system separates findings by strength and quality of research design, statistical findings, and type of data. RCTs offer the strongest evidence; findings from studies using other research designs are separately categorized as observational. Observational studies contain critical information not necessarily found in RCTs. For example, a nonrandomized comparative study may provide evidence for the differential benefits or risks of select interventions. Observational studies may report frequency or incidence data revealing the scope of a problem, event, or condition or may report correlations that associate clinical interventions and outcomes. In addition, when case reports describe adverse events that are not normally reported in controlled studies, they can be a source of important cautionary notations within a recommendation or advisory. Case reports also may be the first indication that a new drug or new technique is associated with a previously unrecognized benefit or unwanted side effect.

One of the strengths of the ASA protocol for developing evidence-based practice parameters is that the primary search and evaluation of the literature are jointly conducted by the clinicians and methodologists of the task force. Consequently, the clinical and practical significance of a study, as well as its research design and statistical aspects, are appropriately and thoroughly evaluated. The protocol is evaluated with the use of formal reliability testing by task force members and methodologists. Interobserver agreement values for research design, type of analysis, linkage assignment, and study inclusion are calculated with both two-rater agreement pairs (kappa) and multirater chance-corrected agreement (Sav) calculations.

These values are reported in the final published document.

Evaluating and Summarizing Consensus Opinion

Although literature-based scientific evidence is a critical part of the process of developing an evidence-based practice parameter, the literature is never used as the sole source of evidence. Scientific findings are always supplemented by the practical knowledge and opinions of expert consultants. The consultants participate in formal surveys regarding

conceptualization, application, and feasibility, and they review and comment on the initial draft by the task force. Opinion surveys of the ASA membership also are conducted to obtain additional consensus-based information used in the final development of an evidence-based practice parameter. The evidence obtained from surveys of consultants and ASA members represents a valuable and quantifiable source, critical to the formulation of effective and useful practice parameters.

In addition to survey information and commentary obtained from consultants and practitioners, the task force continually attempts to maximize the amount of consensus-based information by obtaining opinions from a broader range of sources. These sources include comments made by readers of a draft of the practice parameter posted on the ASA website ( www.asahq.org ) and comments from attendees of public forum presentations of the practice parameters scheduled during major national meetings. After collection and analysis of all scientific and consensus-based information, the draft document is further revised, and additional commentary or opinion is solicited from invited sources, such as the ASA Board of Directors and presidents of ASA component societies.

Meta-Analytic Evidence

When sufficient numbers of controlled studies are found addressing a particular evidence linkage, a formal meta-analysis for each specific outcome is conducted. For studies containing continuous data, either general variance-based methods or combined probability tests are used. When studies report dichotomous outcomes, an odds-ratio procedure is applied. In summarizing findings, an acceptable significance level typically is set at p < 0.01 (one-tailed) and effect size estimates are determined.

Reported findings in the anesthesia literature often use common outcome measures, thereby enhancing the likelihood that aggregated (i.e., pooled) studies will be homogeneous. Because homogeneity is generally expected, a fixed-effects meta-analytic model is used for the initial analysis. If the pooled studies for an evidence linkage are subsequently found to be heterogeneous, a random-effects analysis is performed, and possible reasons for the heterogeneous findings are explored. The heterogeneous findings are reported and discussed as part of the literature summary for an evidence linkage.

Whenever possible, more than one test is used so that a better statistical profile of the evidence linkage can be evaluated. For example, when a set of studies allows for more than one meta-analysis (e.g., using both continuous and dichotomous findings), separate meta-analyses are conducted. To be conclusive, the separate findings for the results of the analysis must agree. Additionally, the results should be in agreement with the directional evaluation of the literature and with consensus opinion before an unequivocal supportive recommendation is offered. If the results do not agree, the disparity is fully reported in the summary of evidence and acknowledged in caveats or notations to the recommendation.

Distinction Between a Guideline and an Advisory

For an evidence-based practice parameter to become a guideline, all sources of evidence (meta-analytic findings, non–meta-analytic literature, responses from consultants, and responses from ASA members) must agree. If, given the nature of the topic, sufficient numbers of controlled studies are not available, a practice advisory is formulated to assist practitioners in clinical decision making and matters of patient safety.

Use of the evidence-based practice advisory was instituted by the Committee and authorized by the ASA in 1998 in response to the need for expansion of the process to areas for which RCTs were sparse or nonexistent. This innovation gave the ASA tremendous flexibility in applying the evidence-based process to a broader scope of topics.

The evidence-based protocol for a practice advisory is identical to that used in the creation of evidence-based practice guidelines. A systematic literature search and formal evaluation of the literature is conducted. Survey information is obtained from consultants and a sample of the ASA membership, and informal input is accepted from public postings regarding draft copies on the ASA website, open forum presentations, and other invited and public sources.

The available evidence is then synthesized, and a practice advisory document is prepared. The resultant document summarizes the current state of the literature, characterizes the current spectrum of clinical opinion, and provides interpretive commentary from the task force.

Guideline/Advisory Dissemination

A typical practice guideline or advisory requires approximately 2 years for completion at a cost of $200,000 to $300,000. Periodic updates occur 5 to 7 years after publication, unless circumstances require an earlier update. These documents are published in Anesthesiology and are available on the journal's website ( http://journals.lww.com/anesthesiology ) and are free of charge on the ASA website ( www.asahq.org ). Supporting material also is available on the journal's website or can be obtained, on request, from the ASA.

Since adopting the evidence-based model in 1991, the ASA has developed and approved 14 evidence-based practice guidelines, 10 guideline updates, 8 evidence-based practice advisories, and 5 advisory updates. Currently, no evidence-based practice standards are planned.

Anesthesiologists and other anesthesia care providers are generally interested in easily accessible, specific recommendations/advice about how to provide optimal care to their patients; therefore ASA evidence-based practice guidelines and advisories are presented in a format that emphasizes the clinical utility of the recommendations/advisory

statements. Detailed rationales or descriptions of techniques, exhaustive critiques of the literature, or elaborate cost–benefit analyses are usually of secondary concern and are made available in an appendix or from a separate source. Documents are brief and succinct. Supportive information is summarized within the guideline or advisory and can be studied in greater detail in an appendix, at the ASA website, or by request.

The general structure of an ASA practice guideline or advisory consists of an introductory section, a guidelines/advisory section, and supporting information (e.g., tables, figures, or appendices). The introductory section contains the ASA definition of practice guidelines or advisories and is followed by a discussion of the focus, application, and methodology used in the guideline/advisory development process. The guideline recommendations or advisory statements are serially divided into subsections, each based on a separate evidence linkage. Each evidence linkage subsection is, in turn, divided into two parts: (1) a summary of the evidence and (2) an articulation of the recommendations or advice.

The evidence summary subsection describes and classifies the literature, generally including statements concerning its availability, the strength of evidence obtained from the literature, and details about particular aspects of the literature necessary for a clear interpretation of the evidence linkage. Consultant and membership survey findings are also summarized, and other opinion-based information is discussed when warranted.

Because it is assumed that the intended readers of the document are knowledgeable regarding the topic, the recommendations or advisories subsections are concise, with explanations added only if required for clarification. Cautionary notations may accompany a recommendation or advisory when deemed necessary by the task force.

Summary

Evidence-based practice parameters are important decision-making tools for practitioners, and they are particularly helpful in providing guidance in areas of difficult or complex practice. These documents can be instrumental in identifying areas of practice that have not yet been clearly defined and can improve research in anesthesiology by (1) identifying areas in need of additional study, (2) suggesting direction for the development of more efficacious interventions, and (3) emphasizing the importance of robust outcome-based research methods. By recognizing the value of merging empirical evidence with the practical nature of opinion and consensus, the ASA has taken a leadership role in improving specific areas of clinical practice, patient care, and safety.

The ASA is committed to the development of practice guidelines and practice advisories by using an evidence-based process that examines testable relationships between specific clinical interventions and desired outcomes ( Box 1-1 ). The process recognizes that the quality of evidence is highly variable and that it comes from many sources, including scientific studies, case reports, expert opinion, and practitioner opinion. By providing a consistent and transparent framework for collecting evidence and for considering its

strengths and weaknesses, the ASA evidence-based process results in practice parameters that clinicians regard as scientifically valid and clinically applicable.

Box 1-1 Strengths of the ASA Evidence-Based Process

Specific outcome data related to a specific intervention are collected and evaluated

A broad-based literature search from a wide variety of published articles

Systematic evaluation of evidence from qualitatively different sources

o •

Randomized controlled studies used in meta-analyses to evaluate causal relationships

o •

Nonrandomized observational comparison studies to provide supplemental information

o •

Other observational literature (e.g., correlational, descriptive/incidence literature) to provide an indication of the scope of a problem

o •

Case reports to describe adverse events not normally found in controlled studies

o •

Opinion-based evidence to evaluate clinical and practical benefits

Evidence from the literature is directionally summarized to clarify and formalize evidence linkages and to reduce bias inherent in selective reviews

Reliance on randomized clinical trials to demonstrate causal relationships and reduce bias inherent in nonrandomized studies or case reports

General use of identical outcome measures, instead of pooling different measures

Consensus information obtained from both formal (e.g., surveys) and informal (e.g., open forums, Internet commentary) sources

One-to-one correspondence between evidence linkages and recommendations

Brevity in reporting evidence

o •

Simple summary statements of literature findings for each evidence linkage, thereby avoiding exhaustive literature reviews or critiques

o •

Specific clinical recommendations without lengthy discussion or detailed rationale

o •

Scientific documentation is provided in appendices or is available separately

o •

Bibliographic information is available separately

Periodic updating to reflect new medications, technologies, or techniques

Some physicians have voiced concern that guidelines and advisories will be treated as de facto standards, thereby increasing liability and creating unnecessary restraints on clinical practice. The ASA emphasizes the nonbinding nature of practice guidelines. It defines them as “recommendations that may be adopted, modified, or rejected according to clinical needs and constraints.” Because the process of evidence-based guideline and advisory development emphasizes consensus formation and communication throughout the practicing community, guidelines and advisories will continue to be relied on by anesthesiologists and other practitioners in their ongoing efforts to maintain a high quality of patient care and safety.