How to do Meta AnalysisArindam BasuAssociate Director,Fogarty International Training ProgramKolkata, IndiaFebruary, 2005arin.basu@gmail.comphone: 919830153666

What is meta analysis?Quantitative approach for systematically combining results of previous research to arrive at conclusions about the body of research.

What does it mean?Quantitative : numbersSystematic : methodicalcombining: putting togetherprevious research: what's already doneconclusions: new knowledge

The popularity of meta analysesNumber of Meta Analysis publications are steadily increasing since 1993. We graphed the counts of journal articles included meta analysis as publication type from Pubmed, from years 1993 through 2004

Chart1

690

797

900

1065

1216

1354

1558

1763

1979

2297

2394

publications

Year of Publications

Number of Publications

Sheet1

yearpublications

93-94690

94-95797

95-96900

96-971065

97-981216

98-991354

99-001558

2000-11763

2001-21979

2002-32297

2003-42394

Sheet2

Sheet3

Four Steps of Meta AnalysisIdentify your studiesDetermine eligibility of studiesInclusion: which ones to keepExclusion: which ones to throw outAbstract Data from the studiesAnalyze data in the studies statistically

Identify your studiesBe methodical: plan firstList of popular databases to searchPubmed/MedlineEmbaseCochrane Review/Trials RegisterOther strategies you may adoptHand search (go to the library...)Personal references, and emailsweb, eg. Google (http://scholar.google.com)

How to Search for literatureFormulate your question appropriatelyIf you are searching pubmedUse Medical Subject Headings (MeSH) [1]Lookup word in text word, abstract, title [2]Combine [1] with [2] using boolean logicSet up proper filtersFor Others, use text word, abstractHuh?!, what is boolean logic??

Boolean Logic: ANDthis is AND zone, covering common area between two ellipses

Boolean Logic ORthis is boolean OR, covering the two ellipses

Example: Research IssueLet's say we want to know whether streptokinase is protective for death from acute myocardial infarction. How should we set up a search strategy? We will search pubmed only

The Searchstreptokinase[text word] OR acute myocardial infarction[text word] produces ALL articles that contain EITHER streptokinase OR acute myocardial infarction anywhere in the text inclusive, manystreptokinase [text word] AND acute myocardial infarction [text word] will capture only those subsets that have BOTH streptokinase AND acute myocardial infarction anywhere in the text restrictive, fewNext, we shall look at the PUBMED Screen

Choose your DATABASE hereRemember to choose both PUBMED, and MESH for formulating search. Choose PUBMED CENTRAL for free articles!

Keep some, throw out others Cannot include all studiesKeep the ones with high levels of evidencegood qualitycheck with QUOROM guidelinesUsually, MA done with RCTs Case series, and case reports definitely outSelection problems are major problemsread the article I sentMA = Meta Analysis; RCT = Randomized Controlled Trial

Plan of Action

ARE THE STUDIES ELIGIBLE FOR MA (STEP I)?DISCARDYESNOENTER INTO A SPECIFIED FORMATABSTRACT THE DATA

How to Abstract Data: GuidelinesCreate a spreadsheet (Excel, or OpenOffice Calc)For each study, create the following columns:name of the studyname of the author, year published number of participants who received interventionnumber of participants who were in control armnumber who developed outcomes in interventionnumber who developed outcomes in control armLets do that to our streptokinase myocardial infarction study, next

Spreadsheet Data for Strepto StudyWe created seven columnstrial: trial identity codetrialname: name of trialyear: year of the studypop1: study populationdeaths1: deaths in studypop0: control populationdeaths0: deaths in controlWe got like 22 studies to do our meta analysis, after all

Analyze Data StatisticallyCombine data to arrive at a summary, 3 measuresEffect Size (Odds Ratio)Variance with 95% Confidence IntervalTest of heterogeneityTwo GraphsForest PlotFunnel PlotExamine why the studies are heterogeneous, if they areUse Statistical Packages, several choices

Lets see what we got for streptokinase versus deaths from AMI

Summary Estimates for Strepto StudyMantel Haenszel OR=0.77

95% Confidence Interval[0.72, 0.83]

Test of Heterogeneity:Chi-square (df=21) = 31.5P-Value = 0.07The pooled Odds Ratio shows that those receiving streptokinase at AMI are about 77% at risk of death (23% less likely to die)That in 95 out of 100 such meta analyses, the pooled Odds Ratio would lie between 0.72 and 0.83, indicating a statistically significant protective effectThat these studies were not significantly heterogeneous

Forest PlotThe dotted line passes across null, or 1.0The Risk Estimate of each study is lined up on each side of the dotted line, with 95% CI spread as the lineThe diamond in the below is the summary estimateThe two ends of the diamond indicate 95% CIThey call it a forest plot so that you dont miss the wood for the trees! The size of the black square box indicates weight of the study

Funnel Plot: what and how to readPlots the effect size against the sample size of the studyTo study a funnel plot, look at its LOWER LEFT corner, thats where negative or null studies are locatedIf EMPTY, this indicates PUBLICATION BIASNote that here, the plot fits in a funnel, and that the left corner is not all that empty, but we cannot rule out publication bias

Issues in meta analysisChoosing a modelFixed effects model or random effects?Bias in meta analysispoor quality of trialspublication biasQuality control in meta analysisQUOROM guidelinesStatistical Software for meta analysis

Fixed Effects or Random Effects Model?conduct if it is reasonable to assume underlying Rx effect is SAME for all studiesPooling: Mantel Haenszel ORTest: test of heterogeneityIf significant, go for random effects modelshort 95% CI for summarysmaller summary estimateOR=0.77 [0.72,0.83]Conduct if test of heterogeneity is significant (shows heterogeneity)Assume that TRUE log odds ratio comes from a normal distributionMethod: DerSimonian Lairs method (DSL) of calculating Odds RatioOR=0.78 [0.69,0.88]Fixed Effects ModelRandom Effects Model

Bias in MetaanalysisPoor Quality of TrialsTo avoid them, learn more at CONSORT statement[http://www.consort-statement.org]Publication Biasstudy showing beneficial effects of new treatment more likely to be published than one showing no effectnegative trials assumed to contribute less; never show up in the literature baseuse several approaches to avoid thisUse Funnel Plots to examine the influence of publication bias

Quality Control in MA:QUOROM Table Detailed Guidelines A Good Checklist Use it for reporting Meta Analysis Systematic reviews

Statistical Software for Meta AnalysisHuge Checklist[http://faculty.ucmerced.edu/wshadish/]Free Software:EpiMeta: from Epi InfoRevman: from Cochrane Collaborationmeta package in R for statistical computingNon-freemeta module in STATA

Summarizing...Defined meta analysis quantitative research synthesisOutlined basic stepsInformation retrievalData AbstractionData AnalysisModel Selection: Fixed Effects or Random effectsOutlined some issues and listed softwareFeel free to shoot questions at arin.basu@gmail.com