Meta-analysis

Overview

Definition A meta-analysis statistically combines the results of

several studies that address a shared research hypotheses.

A study collects data from individual subjects (such as 100 subjects = 100 “data points”)

A meta-analysis collects data from individual studies

(such as 100 studies = 100 “data points”)

Step1

Defines your hypothesis

e.g., Does “authoritarianism” predict verdict choice?

relationship between X (authoritarianism) and Y (verdict)

Step 2

Locate studies

e.g., Sannito & Arnolds (1982)

McGowen & King (1982)

Boehm (1968)

etc.

Step 3

Find “effect size” for each study

e.g., convert data (means, p-value, etc) to “r”

Step 4

Average the “effect sizes” together

e.g., weight by sample size, then take mean

Step 5

If you want, you can analyze “moderators”

e.g., does the “average effect size” get smaller or bigger based upon factors like if the studies have actual jurors versus college students, or if the studies employ written summaries versus actual trials, etc.

Why do a meta-analysis?

Easy Steps are simple, there is software to calculate everything

Cost-effective Since you have already read a bunch of articles to write a

paper, not much more work to synthesize them together

Best type of article Most highly cited type of article. Advantages of both qualitative and quantitative research Truly answers research questions within the literature

(compared to single studies which can’t truly generalize)

Step1 (again)

Defines your hypothesis Typically the hypothesis is relationship between X and Y Best to find studies with diversity (e.g., findings that vary

in size, involve multiple IVs or DVs or stimuli, etc) Best to find studies where controversies or

inconsistencies exist (which the meta-analysis can resolve)

From practical point of view, ideally choose a topic with enough primary studies (20-40) but not too many studies (100+) because hardest part is finding the studies (step 2)

Step 2 (again)

Locate studies A meta-analysis is only informative if it adequately

summarizes the existing literature Techniques - database searches, ancestry

approach, descendancy approach, hand searching, invisible college

Doesn’t have to be comprehensive (fail-safe n) but needs to be close to comprehensive

Step 3 (again)

Find “effect size” for each study For every test that we have covered in this class,

you can convert all of them to “r” or “d” Download “es_calculator.zip” from

http://mason.gmu.edu/~dwilsonb/ma.html In an excel file, have a separate row for each study

in which you input information such as sample size, effect size, codings from moderators, etc.

Example - http://www.lyonsmorris.com/lyons/metaAnalysis/index.cfm

“r” and “d”

The r family Correlation Coefficient - The "r" family includes all types of

correlation coefficients (e.g., r, phi, rho, etc) and Johnson & Eagly, 2000 suggest using r when the studies composing the meta analysis primarily report the correlation between variables.

The d family Standardized Difference - The "d" family includes Cohen's d (unweighted) and Hedges g (weighted), and Johnson & Eagly, 2000 suggest using d when the studies composing the meta-analysis primarily report ANOVAs and t-tests comparisons between groups.

Step 4 (again)

Average the “effect sizes” together First, weight them by sample size Second, sum them together Third, divide by sum of total sample size.

Macros for SPSS can be downloaded from: http://mason.gmu.edu/~dwilsonb/ma.html

Step 5 (again)

If you want, you can analyze “moderators” First, need to ascertain “homogeneity” which tells

you if variance exists in average effect size Can test categorical moderators (categories like

college student versus actual juror) similar to ANOVA

Can test continuous moderators (such as length of stimulus) similar to regression

Use macros downloaded from http://mason.gmu.edu/~dwilsonb/ma.html

For more information…

PsychWiki - http://www.psychwiki.com/wiki/Meta-analysis

Some cool things about meta-analyses…