methodology handbook for ebm guidelines, its evidence ... tracking... · videos, 14 calculators or...

23
Methodology Handbook for EBM Guidelines, its Evidence Summaries, and EBMeDS Decision Support Rules 31.8.2016/Ilkka Kunnamo on behalf of the methodology handbook team of Duodecim (Jorma Komulainen, Jukkapekka Jousimaa, Raija Sipilä, Piia Vuorela, Antti Malmivaara, Ilkka Kunnamo). Overview of the contents EBM Guidelines includes not only concise clinical Guidelines, but also quality graded summaries of the evidence "Evidence Summaries", colour images and audio samples to aid in diagnosis, videos demonstrating clinical examinations, procedures and ultrasonographic examinations, and clinical calculators. Recommendations based on the Evidence Summaries have been further developed into a decision support tool Evidence-Based Medicine electronic Decision Support (EBMeDS). EBM Guidelines covers all specialties, with an emphasis on primary care and ambulatory secondary care. As of August 2016 EBM Guidelines contains 965 Guidelines, 4317 Evidence Summaries, 1455 images, 77 audio samples, 83 videos, 14 calculators or interactive forms, and 12 Quick Reference topics. EBMeDS contains 1349 decision support rules (scripts) in various phases of development (453 published for use by clinicians) that are able to elicit 1852different reminders, and 6737 links to Guidelines. In the Finnish version of EBMeDS there are also 15512 drug interactions, 5024 renal dosing alerts, 3251 drug contraindications, 3297 dosage alerts, 2182 pregnancy and lactation alerts, and 4302 drug therapy best practice recommendations these are being developed for international use. The existing contents are updated regularly. Year methodology for developing guidance product was introduced: 2000 Year methodology for developing guidance product was last updated: 2016 1. Scope and purpose 1.1 The overall objective of the guidance: The overall objective of EBM Guidelines is to cover clinically important evidence and provide guidance on diagnosis, screening, treatment and follow-up of all conditions encountered in primary and ambulatory care. EBM Guidelines is a comprehensive online database of evidence-based, concise and practical guidance on diagnosis and treatment, covering the large majority of clinical problems. The Editorial Team of EBM Guidelines makes decisions on new topics on the basis of users’ needs identified from log file statistics of searches in the database (search terms used) and on the basis of new evidence that available from sources that are systematically monitored and searched. The contents are targeted at physicians working in the following settings: (1) general practice, (2) emergency departments, (3) general hospital clinics, (4) specialists in training, (5) specialists who are asked questions outside their specialist areas, and (6) students. EBM Guidelines is not intended for tertiary care. EBM Guidelines is quick and easy to use during consultations. The EBMeDS decision support tools can be integrated with electronic health records (EHRs) and provide patient-specific guidance at the point of care both in primary and secondary care settings. EBMeDS provides links to EBM Guidelines via reminder notices, which are automatically created from the diagnoses or problems coded in the EHR. The guidance provided by EBMeDS in the form of reminders is based on EBM Guidelines and the Evidence Summaries (particularly those summarizing the results of Cochrane Reviews). The reminders are brief, actionable messages that can be shown within the user interface of the EHR. EBMeDS content management tools also provide an easy way of linking to local guidelines and developing decision support reminders based upon them.

Upload: others

Post on 05-Aug-2020

9 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Methodology Handbook for EBM Guidelines, its Evidence ... tracking... · videos, 14 calculators or interactive forms, and 12 Quick Reference topics. EBMeDS contains 1349 decision

Methodology Handbook for EBM Guidelines, its Evidence Summaries, and EBMeDS Decision Support Rules 31.8.2016/Ilkka Kunnamo on behalf of the methodology handbook team of Duodecim (Jorma Komulainen, Jukkapekka Jousimaa, Raija Sipilä, Piia Vuorela, Antti Malmivaara, Ilkka Kunnamo). Overview of the contents EBM Guidelines includes not only concise clinical Guidelines, but also quality graded summaries of the evidence — "Evidence Summaries", colour images and audio samples to aid in diagnosis, videos demonstrating clinical examinations, procedures and ultrasonographic examinations, and clinical calculators. Recommendations based on the Evidence Summaries have been further developed into a decision support tool — Evidence-Based Medicine electronic Decision Support (EBMeDS). EBM Guidelines covers all specialties, with an emphasis on primary care and ambulatory secondary care. As of August 2016 EBM Guidelines contains 965 Guidelines, 4317 Evidence Summaries, 1455 images, 77 audio samples, 83 videos, 14 calculators or interactive forms, and 12 Quick Reference topics. EBMeDS contains 1349 decision support rules (scripts) in various phases of development (453 published for use by clinicians) that are able to elicit 1852different reminders, and 6737 links to Guidelines. In the Finnish version of EBMeDS there are also 15512 drug interactions, 5024 renal dosing alerts, 3251 drug contraindications, 3297 dosage alerts, 2182 pregnancy and lactation alerts, and 4302 drug therapy best practice recommendations – these are being developed for international use. The existing contents are updated regularly. Year methodology for developing guidance product was introduced: 2000 Year methodology for developing guidance product was last updated: 2016

1. Scope and purpose

1.1 The overall objective of the guidance: The overall objective of EBM Guidelines is to cover clinically important evidence and provide guidance on diagnosis, screening, treatment and follow-up of all conditions encountered in primary and ambulatory care.

EBM Guidelines is a comprehensive online database of evidence-based, concise and practical guidance on diagnosis and treatment, covering the large majority of clinical problems. The Editorial Team of EBM Guidelines makes decisions on new topics on the basis of users’ needs identified from log file statistics of searches in the database (search terms used) and on the basis of new evidence that available from sources that are systematically monitored and searched. The contents are targeted at physicians working in the following settings: (1) general practice, (2) emergency departments, (3) general hospital clinics, (4) specialists in training, (5) specialists who are asked questions outside their specialist areas, and (6) students. EBM Guidelines is not intended for tertiary care. EBM Guidelines is quick and easy to use during consultations. The EBMeDS decision support tools can be integrated with electronic health records (EHRs) and provide patient-specific guidance at the point of care both in primary and secondary care settings. EBMeDS provides links to EBM Guidelines via reminder notices, which are automatically created from the diagnoses or problems coded in the EHR. The guidance provided by EBMeDS in the form of reminders is based on EBM Guidelines and the Evidence Summaries (particularly those summarizing the results of Cochrane Reviews). The reminders are brief, actionable messages that can be shown within the user interface of the EHR. EBMeDS content management tools also provide an easy way of linking to local guidelines and developing decision support reminders based upon them.

Page 2: Methodology Handbook for EBM Guidelines, its Evidence ... tracking... · videos, 14 calculators or interactive forms, and 12 Quick Reference topics. EBMeDS contains 1349 decision

1.2 The clinical questions covered by the guidance: The selection of topics is mainly evidence-driven at present. This means that we systematically search for high-quality evidence, appraise and grade its quality, and summarize it as briefly as possible.

The evidence summaries contain an evidence statement that contains the PICO elements, and the PICO structure is used in the recommendations attached to the evidence summaries. MAGICApp (www.magicapp.org) which is used a the tool for authoring evidence summaries and recommendations from August 2016 guides the authors to start with the definition of the clinical question by entering the PICO elements in structured format. The evidence summary basically gives an answer to a PICO question (patient group, intervention, control, outcome). In an ideal case, there is one intervention and a small number of patient-oriented outcomes.

As the evidence summaries are often based on one Cochrane review, their contents reflect the scope of that review. The following exceptions occur:

The results of several systematic reviews on the same topic and the latest studies not yet included in systematic rewievs are combined into one evidence summary. This is very common at present. The inclusion criteria for a Cochrane review are often very tight, and a systematic review cited by a DARE abstract may include also other study types than RCTs which may broaden the body of evidence.

The results of one systematic review assessing several interventions are divided into more than one evidence summary. This may be necessary e.g. with Cochrane reviews covering many different types of interventions for a certain condition.

Dividing results into more than one evidence summary according to outcome may be necessary in cases where there are several outcomes of interest, and the grade of evidence is different for these outcomes. However, the most common way of handling this situation is to express the different grades of evidence in the wording of the statement (see following paragraph).

We prefer to group interventions of the same class and of proven equal effectiveness into one evidence summary. Thus, our evidence summaries deal with drug classes such as ACE inhibitors, statins, or proton-pump inhibitors. In cases where systematic reviews are only available of single drugs, but not from the whole drug class, the evidence summary only deals with a single drug.

EBMeDS rules contain a section describing the aim, context, and method of implementing the recommendation. The aim paragraph specifies the general aim and desired outcomes of implementing the recommendation. The context section defines the health care context where the recommendation should be applied. The method paragraph specifies the inclusion and exclusion criteria to determine eligibility to and need of the intervention recommended by the rule (the patient groups covered). The EBMPracticeNet team checks the topics against a list of clinical questions created by topic experts. The feedback from the Belgian team is used for developing EBM Guidelines. EBM Guidelines covers prevention, screening, diagnosis, treatment, follow-up and rehabilitation, i.e. all aspects of comprehensive care provided in settings listed under Scope and purpose. For diagnosis, diagnostic strategies, and imaging, a large image library of dermatological diseases, eye conditions, electrocardiograms and other conditions are included. Non-invasive primary and secondary care, diagnostic and therapeutic procedures, and minor surgery are covered comprehensively. The Guidelines include

Page 3: Methodology Handbook for EBM Guidelines, its Evidence ... tracking... · videos, 14 calculators or interactive forms, and 12 Quick Reference topics. EBMeDS contains 1349 decision

procedures that are not commonly carried out in ambulatory care or general practice, but can be performed there after adequate training, and may need to be performed (e.g. in remote areas). The video library of EBM Guidelines demonstrates procedure skills and basic ultrasonographic examinations that can be performed by non-radiologists. The organisation of practice is covered where appropriate (e.g. team work and sharing of tasks) according to the evidence available. Social questions are also covered, with topics such as burnout, bullying at school, marginalization of adolescents, and the effect of parental illness on the child. EBMeDS covers health problems encountered in both primary and secondary care, wherever patient data from electronic health records or interactive forms and calculators can be accessed to meet both the knowledge and the process management needs of health care professionals. EBMeDS rules can be tailored to include local guidance. 1.3 The patient populations and/or target audience to whom the guidance applies: EBM Guidelines, Evidence Summaries and EBMeDS rules covers guidance and clinically important evidence on diagnosis, screening, treatment and follow-up of all conditions for patient populations encountered in the following settings:

- Unselected patients in primary care - Acute conditions treated by general physicians in emergency rooms - Managed care for chronic diseases - Treatment on community hospital wards by primary care physicians - Maternal and child health surveillance

The following settings are mostly excluded:

- Management by subspecialists in secondary hospitals - Neonatology - Techniques in anaesthesia and surgery on specialist level - Interpretation of diagnostic procedures only performed and interpreted by subspecialists.

However, subspecialist procedures and interventions are included when there is a possibility that primary care physicians have a role in referring patients to these investigations and treatments or counseling their patients on whether to undertake them.

EBM Guidelines does not cover tertiary care. Basic information on tertiary care procedures and interventions are included when primary care physicians may have a role in referring patients to these investigations and treatments or counselling their patients about them. EBMeDS decision support rules may, in the future, be extended to target treatment on the tertiary level and, in the meantime, rules for tertiary care may be written at a local level. EBMeDS rules are specifically constructed to suggest the recommended action to patients that are the intended target group of the intervention, determined by age and gender, conditions and comorbidities, medications, test results and measurements that are either inclusion or exclusion criteria for the intervention. During the development process, every element is checked independently by at least three members of the Editorial Team to ensure that inclucions and exclusions are applied correctly according to the evidence. These checks are documented in the Notes and Communication section of the authoring tool where they remain available for further review and audit. EBM Guidelines are intended for health care professionals. EBMeDS decision support rules also contain notices for citizens and patients which have been written in plain language.

Page 4: Methodology Handbook for EBM Guidelines, its Evidence ... tracking... · videos, 14 calculators or interactive forms, and 12 Quick Reference topics. EBMeDS contains 1349 decision

1.4 Ensuring that the guidance includes clear recommendations in reference to specific clinical circumstances: EBM Guidelines includes guidance in concise formats, including lists of recommended investigations in specific situations, sequence and timing of diagnostic procedures and treatment, selection of treatments, drug therapy instructions (including dosage and duration of treatment), and follow-up protocols for chronic conditions. The Evidence Summaries of EBM Guidelines contain a PICO statement defining the patient group, intervention, comparison intervention, and outcome studied. The Evidence Summaries are indexed by PICO category, making it easy and quick to search the database. Recommendations graded according to the GRADE system (see section 3.2 for further information) are included in new Evidence Summaries from 2010. See example in attachment. EBMeDS rules contain specific recommendations that are based on patient data derived from the electronic health record (using coding systems including ICD-10 and SNOMED CT, amongst others), and rule logic that describes cut-off values for recommending actions. EBMeDS notices are categorized into three levels according to importance: alerts, prompts, and reminders. Alerts address situations when ignoring the recommended action might cause serious harm to the patient; prompts address situations when the recommended action has considerable benefits for patient-important outcomes; and, reminders address situations when there is clear evidence of benefit, but the magnitude of the benefit is not large in all patients. In most cases, EBMeDS notices are only generated in situations when a strong recommendation (according to the GRADE system) can be given. The evidence summaries are linked to guidelines at the exact places where the interventions are mentioned whenever possible. The link contains a letter (A, B, C or D) indicating the quality of evidence, and (if a structured recommendation is found in the evidence summary) arrow symbols indicating the strength and direction of the recommendation:

In cases where the evidence summary deals with a condition discussed in the guideline, but the guideline does not contain a recommendation that exactly matches the evidence summary, the evidence summary statement is included in a Related resources article attached to the guideline. From 2016, such evidence summaries will also be linked to the subheading level of EBM Guidelines articles.

Each evidence summary contains link(s) to all guidelines to which the evidence summary is linked, and to decision support rules (see www.ebmeds.org) that are based on the evidence summary.

2. Stakeholder involvement

2.1 Individuals from all relevant professional groups: The Editorial Team of EBM Guidelines includes 10 members: 8 physicians (6 general practitioners, one neurologist, and one specialist in internal medicine and oncology. Permanent experts include one otorhinolaryngologist, one pulmonologist and one urologist. , and two technical editors. In addition, one software developer, and one marketing manager work with the team. The Editorial Board of EBM Guidelines includes the chief editors of the nurses’ database, the patient health portal, and the drug information database maintained by Duodecim Medical Publications Ltd. (a nurse, a specialist in internal medicine, and a specialist in clinical pharmacology, respectively), a pharmacist with special interest in patient education (patient representative), a representative of the Finnish national Current Care Guidelines Editorial Team, and a representative of public health authorities in Finland. The editorial board members attend editorial board meetings every two months when topic coordinators and external expert referees give their feedback on the contents of EBM Guidelines.

Page 5: Methodology Handbook for EBM Guidelines, its Evidence ... tracking... · videos, 14 calculators or interactive forms, and 12 Quick Reference topics. EBMeDS contains 1349 decision

The Editorial Team of EBMeDS includes 10 members, of which 4 are also members of the EBM Guidelines Editorial Team or Editorial Board. Four other members of the EBM Guidelines Editorial Team serve as advisors and referees for EBMeDS contents. In addition, the Editorial Team of EBMeDS includes one specialist in internal medicine and infectious diseases, and four general practitioners, and one nurse. The authors of EBM Guidelines are practicing clinicians in various specialities. The Current Care Editorial Team consists of one paediatrician, one general practitioner, one internist, one orthopedist, one specialist in physical medicine and rehabilitation, one neurologist, one dentist, two psychiatrists, one specialist in sports medicine, and one gynaecologist. Current Care guideline panels consist of experts from specialties that are involved in the care of the target condition of the guideline, at least one representative of primary care, and one methodologist from the Current Care Editorial team, and when indicated, a patient representative (see next paragraph). 2.2 Incorporating patients’ views and preferences: One member of the Editorial Board (a pharmacist by profession) serves as a patient representative (see above for functions of Editorial Board members). Patients are not directly involved in writing EBM Guidelines. EBMeDS patient reminders are evaluated by people who are not health care professionals and who serve as patient representatives. Duodecim Medical Publications Ltd. maintains a patient health portal in Finland (www.terveyskirjasto.fi) with a comprehensive collection of guidance for patients. User feedback is continuously collected and analysed, and used for improving both the patient portal and EBM Guidelines, which is as a content and evidence source for the patient portal. Referee comments from patient organizations are requested during the development of the Finnish national Current Care Guidelines, which are an important source of contents for EBM Guidelines. Statistics of search terms used by citizens when searching information from the citizens portal identify user needs that are not covered by the database, and are used for guiding content development. Meetings with patient association members are arranged on demand basis. Five focus groups for patients were held in 2012 discussing examples of EBM Guidelines contents, and how guidelines should be presented to serve patientss’ needs. The following paragraph contains instructions how Current Care Guidelines involves patients in guideline development: Aim

By involving patients and citizens in guideline development

the inclusion of topics important to patients in promoted patient views on outcome measures are considered patient views on care are considered patient adherence to guidelines is promoted.

Methods

The involvement of patients and citizens is actively pursued during the whole lifespan of a guideline.

Before starting a new guideline or updating a guideline

The editor in charge of the guideline and the chief editor decide how patient involvement is guaranteed fore each guideline. The following methods are used:

Page 6: Methodology Handbook for EBM Guidelines, its Evidence ... tracking... · videos, 14 calculators or interactive forms, and 12 Quick Reference topics. EBMeDS contains 1349 decision

o One or more patient organizations potentially interested in the guideline are identified, and their comments on the key questions that should be addressed and on the expected outcomes of treatment are requested. An on-line structured questionnaire is used. If a suitable organization is not identified, the questinnaire is sent to SOSTE

(http://www.soste.fi/soste/soste-in-english.html)

Information about the guideline is attached to articles in the Citizens’ Health Library addressing the guideline topic, and citizens’ input on key questions and outcomes measures is requested.

A literature search is made on patient views of the guideline topic In exceptional situations where the questionnaires and searches do not provide sufficient input, 1 –

2 focus group interviews can be performed in collaboration with hospital disctricts. When indicated, a patient representative is invited to participate the guideline authoring panel.

o The patient representatives have the same responsibilities as other members of the panel o The patient representatives are appointed using a similar procedure as for other members

of the panel o Suggestions for nominations of a patient representative can be requested from patient

organizations

During guideline development

Patient organizations or people suggested by patient organizations can be requested to comment on specific parts of the guideline or answering specified questions. o The guideline authoring panel makes decisions on consultation of patient representatives,

but the Current Care Editorial Team may require this as a mandatory procedure when indicated.

Peer review of the guideline is requested from the respective patient organizations and SOSTE, or both, before the guideline is published.

After the guideline has been published

The users of the guidelines can comment on the guideline during its full lifespan. The Current Care Editorial Team analyzes the comments and consults the chair of the guideline

panel when indicated. All comments are archived for the next scheduled update.

2.3 Representation of intended users in developing guidance: The Editor-in-Chief and 3 members of the Editorial Team in Finland are practicing primary care physicians who treat the types of patients covered by EBM Guidelines. For other members of the EBM Guidelines and EBMeDS Editorial Teams and Editorial Board, and Current Care Editorial Team and guideline panels see 2.1 The Austrian Editorial Team consists of 16 members, all of whom are practicing general practitioners. The specialist authors and referees are practicing specialists. The Austrian team evaluate all EBM Guidelines topics and suggest changes and updates to the Finnish Editorial Team. The Belgian Editorial Team consists of 7 members, most of whom are general practitioners. The team and external referees evaluate all topics in EBM Guidelines against a prespecified list of clinical questions for all topics.

Page 7: Methodology Handbook for EBM Guidelines, its Evidence ... tracking... · videos, 14 calculators or interactive forms, and 12 Quick Reference topics. EBMeDS contains 1349 decision

The Finnish language version of EBM Guidelines has been subject to continuous development and updating since 1989, and the international version since 2000. Development has been guided by feedback from users in the form of periodic questionnaires, online feedback via the website, and log file records of all searches and usage since 2002.

3. Rigour of development

3.1 Systematic methods to search for evidence and details of the search strategy: EBM Guidelines and Evidence Summaries use mainly secondary sources of evidence that have been selected on the basis of editorial quality. We search systematically for high-quality evidence from secondary sources (listed below), appraise and grade quality, and summarize it as succinctly as possible. The following sources are systematically followed by our editorial team and evidence summaries are produced of all reviews and topics relevant to EBM Guidelines as described above:

- Cochrane reviews - DynaMed Plus by EBSCO Health - DARE abstracts by the Centre for Reviews & Dissemination (CRD), University of York were

systematically followed until the production was stopped. - New England Journal of Medicine, JAMA, Lancet, BMJ

In addition, the following sources are followed regularly and used as supplementary sources when indicated:

- BMJUpdates - InfoPOEMs by Essential Evidence Plus - NIHR-HTA database

After having utilized these sources of evidence we specifically search evidence for decisive decision points in our guidelines not covered by the above-mentioned sources. We perform literature searches and use the guideline organizations in Finland (about 1000 experts) to appraise the evidence.

We prioritize Cochrane reviews and produce evidence summaries from them as soon as new reviews are published. We produce evidence summaries from other sources in batches, so that there may be some delay before new reviews or studies are included in the evidence summaries.

Search strategy for Current Care and EBM Guidelines

Introduction

The guidelines are developed using EBM principles. An adequately performed search is the basis of a high-

quality guideline.

This description of the search strategies aims at ensuring that the searches performed are of high and

uniform quality. The strategies are based on both domestic and international experience, particularly from

Page 8: Methodology Handbook for EBM Guidelines, its Evidence ... tracking... · videos, 14 calculators or interactive forms, and 12 Quick Reference topics. EBMeDS contains 1349 decision

Scotland and England. The strategies selected are in accordance with the resources of the groups and

information services.

Strategy

Basic search and additional searhes

The basic search is performed at the start of a guideline authoring group. Additional searches are

performed by request of group members/authors/editors. The searches are performed by a medical

informatician.

Databases

The search is always performed from Medline and the Cochrane Library. By request of the group, searches

can also be performed from Cinahl (nursing science), Psycinfo (psychiatry and psychology), Pedro and

Medic. Guidelines of other organizations are mainly searched from the Guidelines International Network

database.

Search terms

The search terms are selected by the informatician, in collaboration with the methodologist and group

chair.

Search filters

Filtering the search aims at optimizing the yield of the search by restricting the search to study types

relevant for the clinical questions. Filtering is performed on the basis of the filters tested by the Scottish

Intercollegiate Guidelines Network (SIGN) www.sign.ac.uk/methodology/filters.html, with the exception of

randomized trials where the Cochrane Highly Sensitive Search Strategy for identifying randomized trials in

MEDLINE is used: sensitivity- and precision-maximizing version (2008 revision); Ovid format

(http://handbook.cochrane.org/chapter_6/6_4_11_search_filters.htm

http://handbook.cochrane.org/chapter_6/6_4_11_search_filters.htm).

The searches are performed in four stages:

1. Basic search

2. Complementary searches for specific questions and evidence summaries

3. Search for evidence on rehabilitation issues

4. Update searches in situations where more than one year has elapsed from the basic search.

The search results are stored in the intranet of Current Care (accessible to editors of EBM Guidelines).

The searches provide the basis for the assessment and recommendations developed by the Current Care guideline authoring teams, that are subsequently used by the Editorial Teams of EBMG and EBMeDS. For topics not covered by Current Care Guidelines, searches of original literature are performed mainly on topics covered by systematic reviews. For topics covered by systematic reviewss, the evidence is based on secondary sources. The development of EBMeDS rules has initially been confined to topics where secondary sources of evidence or Current Care evidence are available. In addition to the sources mentioned above, national sources of the country where EBMeDS is used like the United States Preventive Task Force recommendations are used for creating nationally (and sometimes globally) applicable rules.

Page 9: Methodology Handbook for EBM Guidelines, its Evidence ... tracking... · videos, 14 calculators or interactive forms, and 12 Quick Reference topics. EBMeDS contains 1349 decision

3.2 Stating the criteria and reasons for inclusion or exclusion of recommendations identified by the evidence review: Inclusion and exclusion criteria for original studies are stated clearly in all Cochrane Reviews, DARE abstracts, and other systematic reviews that are cited by EBM Guidelines. Evidence Summaries contain links to Cochrane Reviews, DARE abstracts and PubMed abstracts (with links to full-text sources for the majority, including to Cochrane Reviews). As EBM Guidelines Evidence Summaries are very concise, the inclusion and exclusion criteria of the original sources (all of which are available to users via the links) are not repeated in the Evidence Summaries. As EBMeDS rules are based upon the same evidence, the EBM Guidelines criteria apply also to EBMeDS rules. The selection of topics on which the Evidence Summaries are produced is determined by the scope and purpose (see 1.2). The literature searches performed when systematic reviews are not available use tested filters (see above) The inclusion and exclusion criteria for the Current Care searches are determined collaboratively by the Current Care informatician and the expert (clinician) members of the guideline authoring teams. 3.3 Describing the strengths and limitations of the body of evidence and acknowledges any areas of uncertainty: All Evidence Summaries contain an evaluation of the quality of the evidence, determined according to the principles of the GRADE group (http://www.gradeworkinggroup.org). GRADE describes the quality of evidence as high, moderate, low, or very low. The corresponding codes in EBM Guidelines are A, B, C and D, respectively. The factors contributing to the grading of the quality of evidence are reported in the comments section of each Evidence Summary. Important information about the applicability of the results is also included here. The way we express the quality (level) of evidence and strength of recommendations is explained in this

document http://www.terveysportti.fi/ebmg/ltk.avaa?p_artikkeli=ebm01066

The MAGIC (www.magicproject.org) and its tool MAGICApp (www.magicapp.org) has been taken into use by Duodecim in summer 2016 as a tool for authoring evidence summaries and recommendations. The MAGICApp handbook (embedded in the product, and linked to the GRADE handbook) as well as the tool itself contains structured elements of each step in evidence summary production. Summary of findings tables can be automatically downloaded from Cochrane reviews into MAGICApp, or they can be developed using the tool. Authoring then follows the steps guided by MAGICApp where the reasons for downgrading or upgrading the quality of evidence are recorded in structured fields, and standard phrases are produced. In the development of the recommendations , the benefits and harms, values and preferences, and cost considerations are explicitly stated.

In the first stage, new evidence summaries will be produced using MAGICApp. In the next stage, existing evidence summaries will be uploaded into MAGICApp for updating.

Collaboration with EBSCO Health/DynaMed Plus has brought the resources of the DynaMed Plus team in

the use of EBM Guidelines and EBMeDS authoring teams. The evidence-based methodology of DynaMed

Plus is described in its methodology handbook www.dynamed.com/home/files/channel-assets/step-

documents/D-MP_Evidence-basedMethodology.pdf. DynaMed Plus uses a systematic literature

surveillance methodology. A list of journals http://www.dynamed.com/home/files/channel-assets/step-

Page 10: Methodology Handbook for EBM Guidelines, its Evidence ... tracking... · videos, 14 calculators or interactive forms, and 12 Quick Reference topics. EBMeDS contains 1349 decision

documents/DMPContentSources.pdf is continuously monitored, targeted MEDLINE searches for systematic

reviews and randomized controlled trials for high-yield journals, targeted MEDLINE searches for selected

subject areas (e.g., complementary therapies), and comprehensive MEDLINE searches for guidelines. More

than 120 guideline organizations are monitored.

DynaMed Plus develops recommendations using the following procedure. If a recommendation in EBM

Guidelines needs updating based on review of DynaMed Plus by the EBM Guidelines editorial team, the

applicability of the recommendation in DynaMed Plus is evaluated, and the recommendation can be

adopted. A direct link to evidence in DynaMed Plus is added. These links complement Evidence Summaries

of EBM Guidelines.

Synthesized Recommendation Grading System for DynaMed Plus

DynaMed systematically monitors clinical evidence to continuously provide a synthesis of the most valid relevant evidence to support clinical decision-making (See 7-Step Evidence-Based Methodology).

Guideline recommendations summarized in the body of a DynaMed topic are provided with the recommendation grading system used in the original guideline(s), and allow DynaMed users to quickly see where guidelines agree and where guidelines differ from each other and from the current evidence.

In DynaMed Plus (DMP), we synthesize the current evidence, current guidelines from leading authorities, and clinical expertise to provide recommendations to support clinical decision-making in the Overview & Recommendations section.

We use the Grading of Recommendations Assessment, Development and Evaluation (GRADE) to classify synthesized recommendations as Strong or Weak.

o Strong recommendations are used when, based on the available evidence, clinicians (without conflicts of interest) consistently have a high degree of confidence that the desirable consequences (health benefits, decreased costs and burdens) outweigh the undesirable consequences (harms, costs, burdens).

o Weak recommendations are used when, based on the available evidence, clinicians believe that desirable and undesirable consequences are finely balanced, or appreciable uncertainty exists about the magnitude of expected consequences (benefits and harms). Weak recommendations are used when clinicians disagree in judgments of relative benefit and harm, or have limited confidence in their judgments. Weak recommendations are also used when the range of patient values and preferences suggests that informed patients are likely to make different choices.

DynaMed Plus (DMP) synthesized recommendations (in the Overview & Recommendations section) are determined with a systematic methodology:

o Recommendations are initially drafted by clinical editors (including ≥ 1 with methodological expertise and ≥ 1 with content domain expertise) aware of the best current evidence for benefits and harms, and the recommendations from guidelines.

o Recommendations are phrased to match the strength of recommendation. Strong recommendations use "should do" phrasing, or phrasing implying an expectation to perform the recommended action for most patients. Weak recommendations use "consider" or "suggested" phrasing.

o Recommendations are explicitly labeled as Strong recommendations or Weak recommendations when a qualified group has explicitly deliberated on making such a recommendation. Group deliberation may occur during guideline development. When group deliberation occurs through DynaMed-initiated groups:

Page 11: Methodology Handbook for EBM Guidelines, its Evidence ... tracking... · videos, 14 calculators or interactive forms, and 12 Quick Reference topics. EBMeDS contains 1349 decision

Clinical questions will be formulated using the PICO (Population, Intervention, Comparison, Outcome) framework for all outcomes of interest specific to the recommendation to be developed.

Systematic searches will be conducted for any clinical questions where systematic searches were not already completed through DynaMed content development.

Evidence will be summarized for recommendation panel review including for each outcome the relative importance of the outcome, the estimated effects comparing intervention and comparison, the sample size, and overall quality rating for the body of evidence.

Recommendation panel members will be selected to include at least three members that together have sufficient clinical expertise for the subject(s) pertinent to the recommendation, methodological expertise for the evidence being considered, and experience with guideline development.

All recommendation panel members must disclose any potential conflicts of interest (professional, intellectual, and financial), and will not be included for the specific panel if a significant conflict exists for the recommendation in question.

Panel members will make Strong recommendations if and only if there is consistent agreement in a high confidence in the likelihood that desirable consequences outweigh undesirable consequences across the majority of expected patient values and preferences. Panel members will make Weak recommendations if there is limited confidence (or inconsistent assessment or dissenting opinions) that desirable consequences outweigh undesirable consequences across the majority of expected patient values and preferences. No recommendation will be made if there is insufficient confidence to make a recommendation.

All steps in this process (including evidence summaries which were shared with the panel, and identification of panel members) will be transparent and accessible in support of the recommendation.

o Recommendations are verified by ≥ 1 editor with methodological expertise, not involved in recommendation drafting or development, with explicit confirmation that Strong recommendations are adequately supported.

o Recommendations are published only after consensus is established with agreement in phrasing and strength of recommendation by all editors.

o If consensus cannot be reached then the recommendation can be published with a notation of "dissenting commentary" and the dissenting commentary is included in the topic details.

o If recommendations are questioned during peer review or post publication by a qualified individual, or reevaluation is warranted based on new information detected through systematic literature surveillance, the recommendation is subject to additional internal review.

3.4 Clarifying the method used to arrive at recommendations

The recommendations in EBM Guidelines Evidence Summaries (which have be included in new and

updated Evidence Summaries (if applicable) from 2011, and the reminders in EBMeDS rules are written by

salaried editors of the Editorial Teams. Recommendations are included in Evidence Summaries if the

evidence is sufficient for a recommendation to be given, according to the judgment of the editors who

apply the GRADE principles. The Editorial Team of EBM Guidelines discusses the Evidence Summaries at

monthly meetings when the grading of the quality of evidence, and the wording and grading of the

recommendations are checked. If a consensus cannot be reached, voting is used. A formal process (nominal

Page 12: Methodology Handbook for EBM Guidelines, its Evidence ... tracking... · videos, 14 calculators or interactive forms, and 12 Quick Reference topics. EBMeDS contains 1349 decision

group technique http://en.wikipedia.org/wiki/Nominal_group_technique) has been agreed upon with

Current Care, and piloting of the process is starting.

For EBMeDS reminders, the wording and importance of the reminders are discussed and checked by the EBMeDS Editorial Team, and their consistency with EBM Guidelines Evidence Summaries is ensured. 3.5 Balancing the health benefits against the side effects and risks: Information on benefits, side effects and risks are included in the Evidence Summaries, with numerical information in the form of odds ratios, relative risks, and, when applicable, numbers needed to treat (NNTs). The balance of benefits and harms is mentioned in both the evidence statements and in the comments describing the grading of the quality of evidence and the strength of recommendation according to the GRADE principles. The EBMeDS rule descriptions contain a specific paragraph for potential harms that might arise from implementing the rules contained in the rule. By the end of 2016, a new database containing effect estimates of benefits and harms from systematic reviews, baseline risk estimates for various patient groups or validated risk calculators, and assessment of importance of outcomes will be taken into use by EBMeDS for calculating and quantifying net benefits or recommended interventions for individual patients. 3.6 Details the processes of external peer review: Current Care guidelines are sent for peer review to about 30 organizations or stakeholder groups in Finland. The list of the groups (Finnish names) is available at http://www.terveysportti.fi/dtk/khk/koti?p_artikkeli=khk00008 EBM Guidelines uses individual peer reviewers who comment upon a group of Guidelines, related by subject area, at the Editorial Board (EB) meetings. Additional peer reviews are requested at the discretion of the Editorial Team (ET) on topics when the EB is not able to reach a consensus. All contents are peer reviewed, but the length of the peer-reviewing cycle varies by topic, depending on how quickly the evidence base changes. Before each EB meeting, the whole body of Guidelines in one speciality is evaluated by the nominated referee, who will also attend the meeting. The discussions in the meetings are tape-recorded, and notes are written and distributed to all members of the Editorial Team. In addition, the Austrian ET performs peer reviews among Austrian experts on topics where systematically appraised evidence is scarce — usually one expert per guideline topic, as updates appear in the English language version. This feedback (which we presently receive every 1 – 2 months) is sent to the Finnish ET for their consideration. The Austrian ET routinely reviews the content for applicability in Austria, and makes changes to the German language version of EBM Guidelines if necessary. In 2016, a new company (GELIAM) was established in Germany for updating EBM Guidelines German language version and for peer-reviewing the contents. In Belgium, the EBMPracticeNet team performs peer review during the adaptation process of EBM Guidelines. The adaptation process is described in this article: https://dl.dropboxusercontent.com/u/14785933/Documents/Delvaux_et_al-2015-Journal_of_Evaluation_in_Clinical_Practice.pdf The EBM Guidelines ET and the Current Care Guidelines ET always have at least one member that works in both teams to share the feedback from peer review. EBMeDS rules are peer reviewed within the EBMeDS ET. Rules developed on the basis of other sources than Current Care or EBM Guidelines are assessed by at least one expert during the initial review process, and undergo further review whenever the rules are revised or updated on the basis of recent evidence. This process is informed by the EBM Guidelines

Page 13: Methodology Handbook for EBM Guidelines, its Evidence ... tracking... · videos, 14 calculators or interactive forms, and 12 Quick Reference topics. EBMeDS contains 1349 decision

updating cycle. External experts include guideline developers, experts of appropriate medical specialties, other health care professionals, public authorities, physician or patient groups, or other applicable sources. In most cases these experts are either members of the Current Care Guidelines authoring groups or members of the international Cochrane Review groups. In addition, the EBMeDS authoring tool (the EBMeDS Script Description Editor, ESDE) enables continuous open peer-review by all users of ESDE, and the plain text versions of published rules that are also freely available on the EBMeDS website (http://www.ebmeds.org/ebmeds/ebmeds_home.asp?mode=scripts&lang=en) can be commented upon by anyone via the user feedback page. The EBMeDS reminders contain a feedback button to enable the user to leave a comment immediately. EBMeDS rules review includes an assessment of both the rule description (the clinical accuracy of the rule) and the technical implementation (the rule is deployed in response to the correct prompts). In particular the peer reviewers are asked to focus on the following: • clarity and consistency of the information contained within the rule description • expected costs versus benefits from the described functionality • that the rule functions as intended • that the functionality as a part of the whole EBMeDS system is meaningful. Each new rule is tested with the web-based EBMeDS Test Application, which enables the rule to be tested within the decision support engine using fictitious patient data to assess the feedback generated. The actual JavaScript code of the rule is also reviewed by other EBMeDS script programmers. After the rule has been reviewed by at least three members of the EBMeDS ET, the Editor-in-Chief reviews the rule together with assessments from external experts and decides if it is suitable for publication. The reasons for the publication decision are recorded in the notes within the Script Editor application. 3.7 Updating guidance and maintaining and improving guidance quality: Since 1989 the contents of the EBM Guidelines have been updated continuously. Over the years the Guidelines have been reviewed extensively and rewritten several times to include evidence from clinical studies, comments by referees, and feedback collected systematically from clinicians who use the database in their practice. There are four updating processes which complement each other: (1) Triannually — all Guidelines are sent to the original authors; those that are updated significantly, and where controversies exist among experts, are also sent to external reviewers; (2) Every 2 months — the editorial board meets to discuss one speciality or a group of topics with 1–3 top experts in the field, who are invited to attend; (3) Continuously — the Editorial Team (ET) produces and updates Evidence Summaries whenever new evidence is published in Cochrane Reviews or other sources of evidence; the updated Evidence Summaries inform the updating of the Guidelines; (4) Continuously — the Austrian ET and the Belgian EBMPracticeNet ET systematically check for Guidelines that need updating, and provide more general feedback. In EBM Guidelines, updated content appears in red font for 6 months after the update was made. There are two categories of updates: - A major update means that the contents of the Evidence Summary or Guideline have been changed, and the change is of clinical importance. - A minor update means that there is a minor change that does not affect the clinical decisions of the user. EBMeDS rule descriptions are reviewed and updated by the EBMeDS ET whenever the Evidence Summaries on which they are based are updated. The evidence and guidelines linked to rules are monitored

Page 14: Methodology Handbook for EBM Guidelines, its Evidence ... tracking... · videos, 14 calculators or interactive forms, and 12 Quick Reference topics. EBMeDS contains 1349 decision

continuously, and all rules are updated accordingly when necessary. All rules are reviewed at least annually by the ET. In addition, the quality assessment of the executable program code of EBMeDS includes testing within the EBMeDS application by three members of the ET. Maintenance needs for EBMeDS arise mainly via two routes: observations and feedback when a rule is employed in a real-world operating environment; and the medical knowledge on which a rule is based changes. Contact with users is necessary for detecting any unexpected problems that occur after rule publication. Even after careful planning and testing, a rule may function unexpectedly in a real-world environment. This is partly due to the protected nature of patient data, and partly due to the great variability of operating environments in which EBMeDS may be deployed. The simulation of EBMeDS output in a large patient population is feasible via “a virtual health check, VHC”, when all rules are executed for all patients. The log files contain information on the number of reminders elicited, and provide quality measure statistics based on the rules (an example: in a population of 16 000 people, 63 % of patients with type 2 diabetes reached the LDL cholesterol target of < 1.8 mmol/l if they had arterial disease, or < 2.5 mmol/l if they did not have arterial disease). Unexpected results in the VHC may reveal problems in patient data or rule logic. In addition to the evidence becoming obsolete, several other factors may also give rise to maintenance needs. These include changes to legislation, administrative processes, or standards governing structured data. After a need for maintenance is identified, the appropriate actions and their urgency are decided upon. A rule should be withdrawn if it has a serious problem that cannot be solved, or if working around it would require unreasonable efforts in relation to the achieved benefits. The urgency of maintenance action is decided based upon the gravity of the problem. All feedback from EBM Guidelines and EBMeDS users submitted via feedback buttons on the web pages is evaluated by the ETs as soon as it is received. A detailed description of the different measures to maintain the quality of EBMeDS is available in the attached document EBMeDS Quality Plan.

4. Clarity and presentation

4.1 Specific, unambiguous and clearly identifiable recommendations: The recommendations in EBM Guidelines that are based on systematically reported evidence contain links to Evidence Summaries. The Evidence Summaries describe the setting where the evidence applies, the target groups, interventions, and outcomes. The recommendations included in Evidence Summaries written or updated from 2010 onwards contain PICO (patient intervention comparison outcome) statements. The recommendations that are not linked to Evidence Summaries are based on expert opinion and assessment of the Editorial Team, backed by Finnish language evidence summaries of Current Care Guidelines when available, or by evidence-based review articles, which are always referenced in the Guidelines. The EBMeDS rules contain specific, clear and actionable recommendations. The rule descriptions contain sufficient detail to enable end-users (including non-professionals) to understand why a specific reminder was triggered and what evidence supports it.

How the evidence summaries are structured

Heading

The heading is neutral – it does not contain a message of effectiveness. When an evidence summary is based on a Cochrane review or a DARE abstract, the heading is often the same as that of the cited review, or (quite often) an abbreviated version of it.

The grade (level) of evidence

The grades of evidence are determined according to the principles of the GRADE group 1 that divides the quality of evidence into high, moderate, low, or very low. The corresponding codes in EBMG are A, B, C and D, and the definitions (according to the GRADE group) are as follows:

Page 15: Methodology Handbook for EBM Guidelines, its Evidence ... tracking... · videos, 14 calculators or interactive forms, and 12 Quick Reference topics. EBMeDS contains 1349 decision

High (A): We are very confident that the true effect lies close to that of the estimate of the effect.

- Several high-quality studies with consistent results - In special cases: one large, high-quality multi-centre trial Moderate (B): We are moderately confident in the effect estimate: The true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different.

- One high-quality study - Several studies with some limitations Low (C): Our confidence in the effect estimate is limited: The true effect may be substantially different from the estimate of the effect.

- One or more studies with severe limitations Very low (D): We have very little confidence in the effect estimate: The true effect is likely to be substantially different from the estimate of effect.

- Expert opinion - No direct research evidence - One or more studies with very severe limitations

Evidence statement

A PICO statement that describes the most important research finding(s) for the topic of the evidence summary, and gives a verbal description of the grade of evidence that corresponds to the letter code. The verbal expressions are given in article 1. The verbal expression of the main outcome is always reported first and its phrasing corresponds to the evidence code. Example (evd05438 Chest-compression only by bystanders for cardiac arrest):

Level of evidence = B

Cardiac-only resuscitation by bystanders appears to be more effective than conventional cardio-pulmonary resuscitation in patients with out-of-hospital cardiac arrest

Sometimes other outcomes of interest are mentioned in the statement. If the grade of evidence of these other outcomes is different from that of the main outcome, this is expressed in the wording. Example:

Level of evidence = A

Pre-operative radiotherapy followed by surgery is more effective than surgery alone for preventing local recurrence of rectal cancer, and may also improve survival.

In this example, the grade of evidence is A for local recurrence, and C for survival (which is indicated by the word may).

Recommendation

Newer evidence summaries contain structured recommendations. For wording of the recommendations see article http://www.terveysportti.fi/ebmg/ltk.avaa?p_artikkeli=ebm01066. Table 1 contains examples of recommended wordings.

Page 16: Methodology Handbook for EBM Guidelines, its Evidence ... tracking... · videos, 14 calculators or interactive forms, and 12 Quick Reference topics. EBMeDS contains 1349 decision

Table 1. Language for evidence statements. We produce evidence summaries that start with a statement presenting the key message of the topic for a clinician. To keep the statements clear, we use simple language and sentences that support the message.

Level A

High

Level B

Moderate

Level C

Low

Level D

Very Low

Effective … is effective

…are the most effective*

…appears to be effective

…may be effective

…might possibly be effective…but the evidence is insufficient** There is insufficient evidence on the effect... ***

Some effect

…has some effect*

…appears to have some effect

…appears to have more effect than

…may have limited effect

…might possibly have limited effect…but the evidence is insufficient** There is insufficient evidence on the effect... ***

Similar effect

... are both effective*

... both appear effective

... may both be effective

… might possibly have similar effect …but the evidence is insufficient** There is insufficient evidence on the effect... ***

No effect

…is not effective*

…appears not to be effective

…is probably not effective …probably has no effect

... might possibly have no effect…but the evidence is insufficient** There is insufficient evidence on the effect... ***

Harmful …is harmful*

…appears to be harmful

…may be harmful

might possibly be harmful…but the evidence is insufficient** There is insufficient evidence on the effect... ***

Side effects

…has side effects such as*

…appears to have side effects

…may have side effects

might possibly have side effects... but the evidence is insufficient** There is insufficient evidence on the effect... ***

* A suitable verb can be used instead of the word “effective”, e.g. “...appears to improve survival”; “...may not prevent

exacerbations”.

Page 17: Methodology Handbook for EBM Guidelines, its Evidence ... tracking... · videos, 14 calculators or interactive forms, and 12 Quick Reference topics. EBMeDS contains 1349 decision

** “Might possibly be effective/have limited effect/be harmful” can be used when the point estimate suggests clinically

meaningful/almost clinically meaningful beneficial or harmful effect.

*** This expression is used if no point estimate of the effect is available.

1. Example (use of low-dose aspirin in patients with type 2 diabetes): evd06320: B¤#x2191;

Summary of results

The most important outcomes are reported, with numerical information and confidence intervals. The summary is usually shorter than that of the original review or study abstract. Important adverse effects are reported.

Concepts like relative risk, odds ratio, weighted mean difference, confidence intervals are given in abbreviated form (RR, OR, WMD, CI) in the summary.

Comment

The factors downgrading or upgrading the quality of evidence are reported on the basis of the GRADE principles. Important comments on the applicability of the results may also be included.

Comment: The quality of evidence is upgraded by large magnitude of effect.

Clinical comment: The results are not applicable to patients who have primary respiratory arrest (e.g. drowning or suffocation).

The two situations where a comment on downgrading or upgrading is not needed are the following.

- The grade of evidence for RCTs is by default high (A). A comment on downgrading is added only if the given grade of evidence is lower.

- The grade of evidence for observational studies (cohort studies, case-control studies) is by default low (C). A comment on upgrading is added only if the given grade of evidence is higher.

References

The references are Vancouver style. At present the references of DARE abstracts have links to CRD website. All references have links to PubMed abstracts.

4.2 Presenting the different options for management of the condition: EBM Guidelines and EBMeDS include alternative treatment options in the management of specific conditions, based on proven effectiveness, patient characteristics, severity of disease, available resources, desired outcome, and patient preferences. The interactive graphical algorithms available within the EBMeDS interface are customized using patient data, and provide links to evidence resources. These algorithms are particularly suitable when there are several alternative treatment options, and choosing the right one is dependent upon specific features of the patient and setting. By the end of 2016, a new database containing effect estimates of benefits and harms from systematic reviews for different interventions, baseline risk estimates for various patient groups or validated risk calculators, and assessment of importance of outcomes will be taken into use by EBMeDS for calculating and quantifying net benefits or recommended interventions for individual patients.

Page 18: Methodology Handbook for EBM Guidelines, its Evidence ... tracking... · videos, 14 calculators or interactive forms, and 12 Quick Reference topics. EBMeDS contains 1349 decision

4.3 Stating the date of search, the date of publication or last update and the proposed date for review: The date of the last update is visible in every guideline and Evidence Summary. The search date is visible in the original sources (e.g. Cochrane Reviews) on which the Evidence Summaries are based. The EBMeDS rule descriptions include the date of most recent review, and comments by the editors. This date is included in the metadata of the documents for the Editorial Team and so is not visible in the published rule available for the users at present. More detailed versions of documentation can be developed if requested by the provider/EHR vendor. Recording the date of search is mandatory when any version of an evidence summary and recommendation is published using MAGICApp. In EBM Guidelines topics, two dates are currently given: the date of last reviewed up-to-date, and the date of latest change. The proposed date of next review is determined by the updating cycle, but will also be added in the metadata of EBM Guidelines topics in 2016.

See example from one evidence summary:

Date of latest search: 2014-01-20

References 1. Cao H, Yang G, Wang Y et al. Complementary therapies for acne vulgaris. Cochrane Database Syst Rev

2015;1():CD009436. PubMed

Last reviewed as up-to-date 15.4.2016 • Latest change 15.4.2016

Article ID: evd07461 (000.000)

© 2016 Duodecim Medical Publications Ltd

Update markings

When an evidence summary is created or updated, the date is changed to correspond the latest time of the update. The dates of the evidence summaries are shown at the beginning of each document.

There are two categories of updates.

A major update means that the contents of the evidence summary or guideline have been changed. The level of evidence may either remain the same or change during a major update.

A minor update means that there is a minor change that does not affect the clinical decisions of the user. Guidelines have minor updates quite often, evidence summaries only rarely.

A document with a major update during the last 6 months is indicated by a red dot in front of the heading in the search result. A document with a minor update within the last 6 months is indicated by a blue dot. Updated text within a guideline or evidence summary is shown in red colour.

4.4 Ensuring that the content of the guidance is suitable for the specified target audience: The Editorial Team of EBM Guidelines appraises content, scope, and language, and also takes into account feedback from users. The conciseness of the contents of EBM Guidelines determines its language and style that is suitable for quick reading. In addition to reminders for health care professionals, EBMeDS also includes reminders for patients. They use plain language that avoids medical terms.

Page 19: Methodology Handbook for EBM Guidelines, its Evidence ... tracking... · videos, 14 calculators or interactive forms, and 12 Quick Reference topics. EBMeDS contains 1349 decision

In Finland, Duodecim maintains a comprehensive Citizens’ Health Portal that contains patient versions of EBM Guidelines and Current Care guidelines (www.terveyskirjasto.fi).

5. Applicability details

5.1 Publishing support tools to aid implementation of guidance: EBM Guidelines contains a variety of tools, including images, audio, videos, interactive forms and calculators to aid the implementation of guidance. EBMeDS tools include a comprehensive implementation toolkit that integrates EBMeDS guidance with EHRs to produce patient-specific guidance at the point of care. This can be in the form of reminders, dynamic links to Guidelines, and interactive forms and algorithms. The reminders can be customised to aid implementation of guidance at a local level. The Finnish version of EBM Guidelines contains links to patient information on each topic.

Search functions to facilitate finding of the guidance

All evidence summaries are indexed by hand. There are two indexing fields:

- The browsing class indicates the specialities that deal with the topic of the evidence summary. One evidence summary can be indexed into several browsing classes.

- The search terms describe the contents of the evidence summary. The search terms are mainly MeSH terms with U.S. spelling, but other terms are also used in cases where a suitable MeSH term does not exist. The synonyms of the search terms are also included in our vocabulary, so that e.g. ”herpes zoster” and ”shingles” should give the same search result. The synonym vocabulary of EBMG will be updated and considerably enhanced during the next few months. The Unified Medical Language System (UMLS) by the National Library of Medicine is used for building the synonym vocabulary. Abbreviations are also included in the synonym vocabulary.

The search terms indexed by hand are assigned a category ”key” or ”non-key”. In the search result, the guidelines and evidence summaries containing the given search term as a key term are shown first, and the ones containing the given search term as a non-key term are shown lower in the search result. The third category of search terms are ”text words”. They are indexed automatically by the database software. The text word hits are shown at the end of the search result.

Each evidence summary is an independent document that can be searched and linked to other documents. Each document in EBM Guidelines has a unique 8-digit alpha-numeric identifier that can be used in pointing to the document in an URL or used as a search term to retrieve the document. The identifier is shown at the end of each document by EBMG search and browser software.

Summary of findings tables are included in all new evidence summaries that are developed using MAGICApp. From the tables, patient decision aids can be automatically created and published in MAGICApp.

5.2 Discussion of potential organisational and financial barriers in applying its recommendation: EBM Guidelines aims to help healthcare professionals identify and implement interventions that can be carried out with resources and equipment that are available or can be utilized in general practice or ambulatory care. The grading of the strength of recommendations (available for new and updated Evidence

Page 20: Methodology Handbook for EBM Guidelines, its Evidence ... tracking... · videos, 14 calculators or interactive forms, and 12 Quick Reference topics. EBMeDS contains 1349 decision

Summaries from 2010) includes the considered and appropriate use of resources as one of the criteria for a strong recommendation. Evidence Summaries with recommendations developed using MAGICApp contain a field Resources and

other considerations where the use of resources is described or discussed, as well as other organizational

aspects of implementing the recommendation.

EBM Guidelines Evidence Summaries contain a browsing class "Resource-poor settings" that contains

evidence applicable and relevant in settings with very limited resources in developing countries. EBMeDS

rules can be modified or written at a local level to ensure compliance with organisational and financial

preferences (e.g. for specific interventions).

EBMeDS rules can be modified or written at a local level to ensure compliance with organisational and financial preferences (e.g. for specific interventions).

5.3 Ensuring that the guidance is current, with review criteria for monitoring and/or audit purposes within each product: For EBM Guidelines, the appearance of new evidence in the systematically searched sources and subsequent authoring and updating of the Evidence Summaries, and the specified time points for scheduled review of the contents serve as internal audit points for the Editorial Team. Feedback from users and experts is dealt with and responded to quickly (within days or a few weeks at the latest). The maintenance plan for EBMeDS includes a checklist of events that should initiate an assessment of maintenance needs for the rule. These events may include: predetermined dates (e.g. an update of a Cochrane Review or an EBMeDS Editorial meeting); updates to evidence databases that are linked to the rule; user feedback relating to the rule; and, changes to the classification codes utilised in the rule. Automatic alerting services that monitor changes in specific databases or websites will be used in the future. For rules that have been tailored at the local level, the tracking of trigger events should be taken into account at the local level to be effective. All EBMeDS decision support rules contain a log-reporting feature, which identifies the patients eligible for a particular intervention, and the patients that have been (or have not been) treated according to the recommendation. Quality measures are thus automatically produced for every rule. From these rules, user organizations can select the quality measures they wish to use in monitoring and auditing. Because of the large number of evidence-based rules in the EBMeDS database, a variety of quality criteria can be developed, applied and tested using the same set of standard patient data contained in the EBMeDS query message. The quality measure rules can be edited to apply local rules, or they can be used to apply rules developed elsewhere (if available). This enables performance benchmarking against different health care providers or different divisions within a healthcare system. It is also possible to use the EBMeDS system for audit purposes only, without the need to implement the clinical decision support tools.

6. Editorial Independence

6.1 Ensuring editorial independence from the funding body: EBM Guidelines and EBMeDS Editorial Teams comprise salaried professionals whose task is to produce independent and unbiased guidance for improving clinical care and patient outcomes. The funding comes from license fees and subscriptions from customers, including a large number of publicly funded health-

Page 21: Methodology Handbook for EBM Guidelines, its Evidence ... tracking... · videos, 14 calculators or interactive forms, and 12 Quick Reference topics. EBMeDS contains 1349 decision

care organizations and health authorities in several countries. Thus, the funding is not dependent on any individual organizations or stakeholders, and the Editorial Teams have full independence.

Duodecim Medical Publications Ltd. (www.duodecimpublications.com) is 100% owned by the Finnish

Medical Society Duodecim (http://www.duodecim.fi/kotisivut/docs/f1908623236/esite_engl_web.pdf),

the scientific society of Finnish physicians.

The production of EBM Guidelines is totally funded by license sales from customers. In Finland the

customers include hospital districts, municipal health centres, private pharmacies, and private health care

providers and companies. For the English language version we get a royalty from sales by Wiley-Blackwell.

For the translated versions, local publishers pay a royalty for the translations.

The development of the EBMeDS contents and service has been initially funded by the Finnish Medical

Society Duodecim, by the Centre for Pharmacotherapy Development

under the Ministry of Social Affairs and Health), and by the Finnish Funding Agency for Technology and

Innovation (http://www.tekes.fi/en)

At present, the funding is based on license fees paid by electronic health records vendors who implement

the EBMeDS application and charge license fees from their customers.

The authors of EBM Guidelines and EBMeDS receive rewards or salary directly from Duodecim Medical

Publications Ltd., or work voluntarily without reward as members of the Society. Pharmaceutical companies

or any other companies or organizations do not pay for the authors for producing EBM Guidelines or

EBMeDS contents.

The Current Care Guidelines are funded by the Finnish Ministry of Social Affairs and Health via the National

Institute for Health and Welfare (https://www.thl.fi/en/web/thlfi-en) . and the Finnish Medical Society

Duodecim.

6.2 Transparent about the funding mechanisms for the guidance: Information about the funding of Current Care guidelines is publicly available on Current Care website. Grants for guideline panel members are provided by the Finnish Medical Society, and grant decisions are available in the minutes. The sources for funding of EBM Guidelines and EBMeDS are described in paragraph 6.1 of this document which will be publicly available on EBM Guidelines website. Duodecim Medical Publications Ltd. is 100% owned by the Finnish Medical Society Duodecim, the scientific society of Finnish physicians. 6.3 Recording and stating any potential conflicts of interest of individuals involved in developing the recommendation: The potential conflicts of interests of the members of the Editorial Teams are stated clearly in the product websites. The potential conflicts of interests of the specialist authors of EBM Guidelines have been collected during the updating round in 2016 and will be published in association of author information of each topic in January 2017.

Conflicts of interests policy of Current Care guidelines

Page 22: Methodology Handbook for EBM Guidelines, its Evidence ... tracking... · videos, 14 calculators or interactive forms, and 12 Quick Reference topics. EBMeDS contains 1349 decision

The Current Care guideline is developed in a transparent process, and the evidence behind the recommendations is explicitly reported in the evidence summaries. Therefore, minor conflicts of interests do not constitute a reason to be excluded from taking part in decisions. According to international principles of transparency, all guideline authors are obliged to disclose financial and other significant conflicts of interest for the three previous years that may be relevant to the contents of the guidelines, using the The International Committee of Medical Journal Editors (ICMJE) form. These disclosures are been published with the online versions of Current Care guidelines since 2001. The potential conflicts of interests of the editors are published on the home page of Current Care and EBM Guidelines (Finnish version).

The disclosure of conflicts of interests should be discussed in the first meeting of the guideline authoring group. Request for reporting conflicts of interests is sent to the authors before the guideline is finalized. The disclosures are edited in a uniform format, and the guideline cannot be published before all members have reported their conflicts. At scheduled updates, the conflicts are reported again if the updates are important and more than one year has elapsed since the latest conflicts were published.

Exclusion of a person from authoring a guideline should be considered if the person has a leading position in a medical company, or if the person has a significant financial interest in the topic of the guideline. If the financial interest is confined to a specific part of the guideline, the person can be excluded from participating in the authoring of that part. Also partial conflicts of interest are publicly reported. The potential conflicts are assessed by the panel, and if any uncertainty occurs, in collaboration with the Chief Editor.

A person that has a significant financial interest in the topic of the guideline cannot serve as the chair of the guideline panel. E.g. research funding from a pharmaceutical or health technology company, ownership of shares of a company in the degree that it is constitutes a significant part of the income, or a situation where the recommendations in the guideline influence the person’s income are considered significant conflicts. The Chief Editor Current Care guidelines makes the decision on the significance of the conflict, after hearing the person with the conflict, the methodologist of the group (which is a member of the Editorial Team) and the chairs of the Networking Board and President of the Society.

A member of the guideline panel cannot take part in the assessment of a study where he or she is a coauthor.

1.

Example of filled a COI reporting form of EBM Guidelines

(M.N., author of the article Urinary incontinence in the elderly (ebm00479)

Page 23: Methodology Handbook for EBM Guidelines, its Evidence ... tracking... · videos, 14 calculators or interactive forms, and 12 Quick Reference topics. EBMeDS contains 1349 decision

6.4 Taking account of any potential for bias in the conclusions or recommendations of the guidance: The processes described in this document ensure that any bias is considered and minimized. Potential biases are discussed in the Editorial Team meetings of EBM Guidelines, and peer reviewers are requested specifically to look for and to highlight potential sources of bias.