Expert searching: what are we missing?Sarah Hayman, Research Fellow (Evidence)

HLA PD Development Day10 July 2012

Introduction and overview

Where I work: CareSearchWhat I will discuss today:

– overview of search in systematic reviews– analysis of sources– terms, testing, filters– checking your retrievals– Grey literature

Importance of the searchComprehensivenessHealth professionals use them in providing care

Review question and PICOPopulation (participants, patients)InterventionsComparisonsOutcomes

Example: Does physical rehabilitation benefit older people in long-term care?

P - older people in long-term careI - physical rehabilitationC - no intervention (or alternative intervention)O - improved physical function

Search in systematic reviews

Types of systematic review3 Cochrane types: Intervention reviews, Diagnostic test accuracy reviews,

Methodology reviewsOthers often mentioned: Effectiveness studies, Qualitative studies, Economic

evaluations, Meta-analyses

Record-keeping- for standards compliance- for transparency and reproducibility- for iterative development of your searches- for validation of your searchesRecord (as you go, from the start)

• which databases were searched and when• the list of search terms used and the combinations in which they were applied• the search results

Search in systematic reviews (ctd.)

Identify all electronic database sources for your reviewMedline, EMBASE, CENTRAL, PubMedSubject-specific databases (e.g. CINAHL, PEDro, AgeLine, PsycINFO)Grey literature databases

Understand the sourcesPublished search strategies in systematic reviewsHelp sections for each databasePapers published in library journalsHealth librarians’ discussion listsHealth librarians’ blogs and wikis (eg Laika’s MedLibLog and KSS)

Analysing bibliographic database sources

Identify synonyms, including different terms in different countries Check history and relationships of MeSH terms Consult with experts Develop a term identification set of references Use textual analysis tools if necessary (e.g. Concordance,

Writewords))

Analysing terminology

What can this approach offer?

It is:• evidence-based• iterative

We examine terminology in records that we know are relevant

We can identify and test which terms are likely to retrieve the most results

We can analyse where terms are occurring in the records and how indexing has been done

We can develop search strategies to use for free-text searching in non-indexed sources and sources indexed without MeSH headings

We can adjust searches for greater sensitivity or specificity

Note: free online alternatives exist (e.g. Writewords)

Using search filtersDefinition

Search filters are predetermined literature search strategies, developed using an explicit methodology and tested using a gold standard test comparison study design(http://www.flinders.edu.au/clinical-change/research/search-filters.cfm )

Types:Methodological and subject-based

Further reading:http://www.york.ac.uk/inst/crd/intertasc/https://sites.google.com/site/healthliteraturesearchers/tools/search-filter http://pubmedsearches.blogspot.com.au/http://www.ncbi.nlm.nih.gov/pubmed/clinicalhttp://www.caresearch.com.au/caresearch/tabid/371/Default.aspx

Analysing your results and validating your search

Use analysis to:1. Improve your search for this particular review2. Provide evidence of the validity of your search for this and future reviews

Consult with your subject experts Analyse “missed” items Consider an evaluation instrument such as PRESS (Peer

Review of Electronic Search Strategies)see suggestions at: https://sites.google.com/site/healthliteraturesearchers/tools/search-strategy-peer-review-tools

Not formally published Not accessible through databases such as Medline Can include:

– Conference papers– Government reports– Theses and dissertations

Can be an important source of studies for inclusion; prevent publication bias in intervention studies

May need many different strategies to identify

Grey literature

Grey Literature: some sources

Grey Matters: A Practical Search Tool for Evidence-Based Medicine

(general)http://libguides.newcastle.edu.au/content.php?pid=259537&sid=2201671

(excellent set of advice and links on grey literature)Google scholarProQuest Dissertations & Theses Database

Other sources and methods Handsearching Fulltext journal searching Table of contents services Reference lists in other reviews Guidelines Trials registers Web searching Contacting study authors, experts, manufacturers, and other

organisations Using a project Internet site to canvas for studies

The Cochrane handbook has excellent detailed and well set-out advice about these and more. http://www.cochrane.org/handbook/62-sources-search

Summary and key points

Collaborate with the clinical experts at every stage Do research: investigate other systematic review reports, Cochrane

guidelines and standards for systematic review searching Analyse:

– Your sources– Your search terms– Your search strategy– Your search results

Adjust your searching in response to the analysis: test and redo if necessary

Be thorough and transparent: document and record your steps as you go

Consider using:– Gold standard set– Textual analysis tools

Further readingFehrmann P, Thomas J. Comprehensive computer searches and reporting in systematic reviews. Research Synthesis Methods. 2011;2(1):15-32.Hausner E, Waffenschmidt S, Kaiser T, Simon M. Routine development of objectively derived search strategies. Syst Rev. 2012;1(1):19.Lasserre K. Expert searching in health librarianship: a literature review to identify international issues and Australian concerns. Health Info Libr J. 2012 Mar;29(1):3-1McGowan J, Sampson M. Systematic reviews need systematic searchers. J Med Libr Assoc. 2005 Jan;93(1):74-80.Niederstadt C, Droste S. Reporting and presenting information retrieval processes: the need for optimizing common practice in health technology assessment. International Journal of Technology Assessment in Health Care. 2010;26(04):450-7.Ogilvie D, Hamilton V, Egan M, Petticrew M. Systematic reviews of health effects of social interventions: 1. Finding the evidence: how far should you go? Journal of Epidemiology and Community Health. 2005 September 1, 2005;59(9):804-8.Papaioannou D, Sutton A, Carroll C, Booth A, Wong R. Literature searching for social science systematic reviews: consideration of a range of search techniques. Health Info Libr J. 2010 Jun;27(2):114-22.Sampson M, McGowan J, Cogo E, Grimshaw J, Moher D, Lefebvre C. An evidence-based practice guideline for the peer review of electronic search strategies. J Clin Epidemiol. 2009 Sep;62(9):944-52.Sampson M, Tetzlaff J, Urquhart C. Precision of healthcare systematic review searches in a cross-sectional sample. Research Synthesis Methods. 2011;2(2):119-25.

Cochrane Handbook for Systematic Reviews of Interventions

Systematic reviews: CRD’s guidance for undertaking reviews in health care

http://www.diigo.com/user/sarahhayman/systematicsearching

Thank you!

CareSearch would like to thank the many people

who contribute their time and expertise to the project, including

members of the National Advisory Group and the Knowledge

Network Management Group.

CareSearch is funded by the Australian Government

Department of Health and Ageing.

www.caresearch.com.au