Meet the PRESS, and other secrets of systematic searchers

CADTH SYMPOSIUM BREAKFAST SESSIONTUESDAY, APRIL 12, 2016

2

Meet the PRESS, and other secrets of systematic searchers: an update on literature search essentials for finding the evidence

Amanda Hodgson, CADTHDr. Jessie McGowan, University of OttawaMelissa Severn, CADTH Danielle Rabb, CADTH

Disclosure• Funded by federal, provincial, and territorial ministries of

health.

• Application fees for three programs:

• CADTH Common Drug Review (CDR)

• CADTH pan-Canadian Oncology Drug Review (pCODR)

• CADTH Scientific Advice

• The presenters declare that there are no conflicts of interest.

3

4

Agenda

• Introduction (Amanda Hodgson)

• PRESS Update (Dr. McGowan)

• Filters Update (Melissa Severn)

• Grey Literature Update (Danielle Rabb)

5

Literature search standards - resources

https://www.cadth.ca/resources/finding-evidence

6

Literature search standards - resourcesCochrane Handbook – Chapter 6: searching for studies (2011)http://handbook.cochrane.org/

Institute of Medicine. Finding what works for healthcare: standards for systematic reviews. Section 3. (2011)

http://www.nationalacademies.org/hmd/Reports/2011/Finding-What-Works-in-Health-Care-Standards-for-Systematic-Reviews.aspx

7

Literature search standards - resourcesSummarized Research in Information Retrieval for HTA

SuRE Infohttp://www.htai.org/vortal/?q=sure-info

8

Literature search standards - resourcesListservs of interest:

Canmedlib (CHLA)Expertsearch (MLA)Cochrane (Information Retrieval Methods group)HTAi (Information Resources Group)

9

Literature search standards - Groups of InterestHealth Technology Assessment International Information

Resources Group (HTAi - IRG)http://www.htai.org/index.php?id=555

Cochrane Information Retrieval Methods Group (IRMG) http://methods.cochrane.org/irmg/

Canadian Health Libraries Association (CHLA)http://www.chla-absc.ca/

What is PRESS?Peer Review of Electronic Search Strategies

The PRESS 2015 Guideline is an includes two documents:• PRESS Guideline Explanation & Elaboration (E&E)• PRESS Guideline Statement

Report is found at: https://www.cadth.ca/sites/default/files/pdfCP0015_PRESS_Update_Report_2016.pdf

Why do we need PRESS?• In preparing systematic review (SR) and health technology

assessment (HTA) reports, researchers aim to provide comprehensive, unbiased, and high-quality distillations of relevant evidence.

• One step in assuring the integrity of the searching process is the peer review of the work of the librarian or information specialist who is seeking the evidence for the researchers.

Searching and PRESS• PRESS focuses on the quality of the Boolean search and

is only one aspect of a comprehensive search for SRs and HTAs.

• Other important aspects include the search plan, search validation, and search reporting.

Searching and PRESS• The Boolean search is typically the core element in the

search plan, identifies the majority of relevant evidence and is the most productive method of identifying relevant studies.

• Peer review of the search strategy is one quality assurance step and provides a subjective validation.• Other validation approaches include reaching data

saturation, testing the search of the main database against eligible studies and testing using a related article search and simple search protocol.

Research components Systematic Review• The SR search focused on research and reports on electronic search

strategies within a health science context.• 6,608 records were identified; 39

Survey: a total of 174 surveys (117 were valid and 108 were complete).

Consensus Forum • The objective was to create recommendations related to the existing

PRESS checklist items based on updated SR evidence and survey findings.

• The outcome was that, of the seven existing PRESS elements, six were retained and one (“Search strategy adaptations”) will no longer be a PRESS element.

PRESS 2015 ImplementationFour components:1. PRESS 2015 evidence-based checklist 2. PRESS 2015 recommendations for librarian practice3. PRESS 2015 assessment form4. PRESS 2015 implementation strategies

PRESS 2015 evidence-based checklist

PRESS 2015 evidence-based checklist

SearchFilters

What is a Search Filter?• Pre-defined• Pre-tested• Pre-saved• Also referred to as hedges, clinical queries, optimal search strategies or methodological search strategies

Why Use Search Filters?• Reduce number of results • Extract higher level evidence• Extract a specific concept• Remove off-topic results• Save time and money

Typical Search Strategy

INDICATION TERMS

INTERVENTION TERMS

SEARCH FILTER(S)

Types of Search Filters–Specific methodology

• RCTs, systematic reviews

–Clinical queries • diagnosis, prognosis, adverse events

–Population specific • humans, pediatrics

• InterTASC Search Filter Resource http://www.york.ac.uk/inst/crd/intertasc/

• CADTH Filters: Strings Attached https://www.cadth.ca/resources/finding-evidence/strings-attached-cadths-database-search-filters

Methodology: RCTs1. Randomized Controlled Trial.pt.2. Randomized Controlled Trials as Topic/3. Randomized Controlled Trial/4. Randomization/5. Double-Blind Method/6. Single-Blind Method/7. Placebos/8. (random* or sham or placebo*).ti,ab9. ((singl* or doubl*) adj (blind* or dumm* or mask*)).ti,ab10. ((tripl* or trebl*) adj (blind* or dumm* or mask*)).ti,ab11. or/1-11

Clinical Query: Prognosis1. Incidence/2. Mortality/3. Follow up studies/4. Prognos*.ti,ab5. Predict*.ti,ab6. Course*.ti,ab7. or/1-6

26

Population Specific: Human Filter ((Animals[MESH] OR Animal Experimentation[MESH] OR "Models, Animal"[ MESH] OR Vertebrates[MESH]) NOT (Humans[MESH] OR Human experimentation[MESH]))

27

Challenges to Filter Development• Author related

• Titles • Abstracts

• Indexer related• Subject headings• Publication type tags

• Database related• Translation

• Testing

28

Gold Standard Testing• A gold standard is a set of

relevant records against which a filter’s performance can be assessed

• Developed through a range of techniques

• Gold standard sets can range in size

29

Harbour J, et al. Reporting methodological search filter performance comparisons: a literature review. Health Info Libr J. 2014;31(3):176-194

Sensitivity/Recall & Precision• Sensitivity (or Recall): proportion of relevant records

retrieved by the filter• Precision: proportion of relevant records in the total

retrieved records

30http://www.creighton.edu/fileadmin/user/HSL/docs/ref/Searching_-_Recall_Precision.pdf

Performance of Search Filter in Ovid MEDLINE - Example

https://www.cadth.ca/sites/default/files/pdf/H0490_Search_Filters_for_Economic_Evaluations_mg_e.pdfDevelopment and Testing of Search Filters to Identify Economic Evaluations in MEDLINE and EMBASE, Oct 2009, pg. 15

32

NEW!

https://www.cadth.ca/resources/finding-evidence/strings-attached-cadths-database-search-filters

33

Diving into the Grey: Searching for grey literature

What is grey literature?

Grey Literature is a field in library and Information science that deals with the production, distribution, and access to multiple document types produced on all levels of government, academics, business, and organization in electronic and print formats not controlled by commercial publishing i.e. where publishing is not the primary activity of the producing body.

Source: GreyNet.org. The Grey Literature Network Service http://www.greynet.org/

34

Trends in grey literature

• PISA Declaration on Policy Development for Grey Literature Resources http://www.greynet.org/greyforumseries/policydevelopment.html

• Teaching grey literature• Publishing of Big Data• Metadata standards• Access points and access repositories (pan-Canadian HTA

database)

35

36

Grey Literature - Groups of Interest

37

What is grey literature?

38

Clinical Practice Guidelines

39

Clinical Practice Guidelines

40

Clinical Practice Guidelines

41

Regulatory approvals

42

Regulatory approvals

43

Clinical Trials

44

Clinical Trials

45

Advisories and Warnings

46

Advisories and Warnings

47

Advisories and Warnings

48

What is grey literature?

Clinical Practice GuidelinesPosition StatementsRegulatory approvalsSafety information & adverse event reportsRecalls & alertsClinical trial data (CSRs, abstracts, posters)Supplemental information to published literature

Canadian Producers

49

50

International Producers

Benefits

Value of grey literature for systematic reviews• Unpublished trials• Conference abstracts• Guidelines

Can reduce publication biasCurrentUsually freeUnique contentInformation on non-mainstream topics

51

Challenges

Large volume of material

Inconsistent access points

Time-consuming to find

Not always available online

Quality of evidence varies

Bibliographic information may be missing

Inconsistent language, no subject indexing

52

Before you begin…

How much time do you have?

What type of information is required?

How are you incorporating your grey lit search?

How well do you know the sites you are searching?

Do you have a well defined PICO?

53

The CADTH Experience

Reasons for searching grey literature:• To see other HTAs• To find the best evidence available• Adhere to standards for systematic reviews – the

exhaustive search for trials• Help our researchers understand the topic area

54

The CADTH Experience

The Practical Stuff• Grey literature searching is built into overall search time• Customized checklists (Grey Matters)• Results are sent separately from database results (word

document)• We do not differentiate between commercial and non-

commercial sources for our clients

55

Grey Matters

56

Grey Matters checklist: CADTH tool for finding grey literature information

Grey MattersCategories:DatabasesNational & international HTA organizationsClinical practice guidelinesDrug class reviewsHealth economicsSafety/advisoriesClinical trial listingRegulatory approvalsDrug formulariesStatistics/prevalenceOther

57

Grey Matters Light

Grey Matters Light checklist: Top 14 Resources for Evidenced Based Medicine Searching

Categories:• HTA organizations• Safety/advisory resources• Clinical practice guidelines

58

Pan-Canadian HTA database Pan-Canadian HTA database

59

Pan-Canadian HTA databaseThe HTA Database Canadian search interface lets you:

• Automatically limit search results to HTAs produced in Canada or internationally

• Search in English or French• Limit your search by province• Easily organize your search by:

• Topic (using medical subject headings)• Title• Author• Agency (CADTH, NICE, INESSS, etc.)

Pan-Canadian HTA database

The pan-Canadian Health Technology Collaborative invites other HTA producers in Canada to contribute to the database. For information about how to get involved, please contact CdnHTA@cadth.ca

62

@CADTH_ACMTS linkedin.com/company/cadth

slideshare.net/CADTH-ACMTS youtube.com/CADTHACMTS

cadth.ca/photoblog requests@cadth.ca

Connect With Us

63

Subscribe

Sign up at www.cadth.ca/subscribe to get updates sent directly to your inbox.

CADTH E-AlertsCalls for stakeholder feedback and patient group input, plus other time-sensitive announcements.

New at CADTHMonthly newsletter including a summary of new reports plus corporate news, announcements of upcoming events, and more.

CADTH Symposium and EventsUpdates about our flagship annual Symposium, workshops, webinars and other events.

64