factorial between-ps anova i: omnibus testsuqwloui1/stats/3010 for post... · 2018. 8. 2. · 3 5...

1

1

psyc3010 lecture 2

factorial between-Ps ANOVA I:omnibus tests

last lecture: introduction to factorial designsnext lecture: factorial between-Ps ANOVA II:

(effect sizes and follow-up tests)

22

general announcementsgeneral announcements

•• tutorials start this weektutorials start this weekattendance strongly recommended!attendance strongly recommended!

•• list of private tutors available on Blackboardlist of private tutors available on Blackboard

•• revisiting upcoming assessment :revisiting upcoming assessment :quiz 1quiz 1 on ANOVA in late Augon ANOVA in late Augassignment 1assignment 1 on ANOVA in early Septon ANOVA in early Sept

2

3

more about quiz 1

• multiple-choice and problem-solving questions

• assesses material on between-subjects ANOVA covered in Lectures 1, 2, 3, and 4

• Practice Questions and Answers posted onBlackboard next week

• Practice Quiz available on Blackboard in Week 4 (to get used to format)

ICEBREAKER!

44

last lecture this lecturein the last lecture, we introduced the concept of factorial designs, focusing on key terminology and concepts– factors / independent variables, dependent variables– crossed designs: A x B– cell means, marginal means, grand means– main effects, interaction effects, simple effects– how one factor qualifies or moderates the effect of

another factor– ordinal and disordinal interactions

in this lecture, we learn more about factorial between-participants ANOVA, focusing on the logic and computation of omnibus tests

3

55

todaytodayconceptual underpinnings of ANOVA

relationship between hypotheses, variance, graphical representations of data, and formulae(one-way and two-way analyses) research questions and hypothesesbuilding up to the F ratiothe F teststructural modelassumptions

conducting 2-way factorial ANOVAdata table and key termsresults in source/summary tablegraphing results

effect sizeswhat they are and why they are importantthree estimates: η2 , ω2 , partial η2

66

anova: conceptual underpinnings anova: conceptual underpinnings

like most statistical procedures we use, anova is like most statistical procedures we use, anova is all about partitioning varianceall about partitioning variancewe want to see if variation due to our we want to see if variation due to our experimental manipulations or groups of interest experimental manipulations or groups of interest is proportionally greater than the rest of the is proportionally greater than the rest of the variance variance (i.e., that is (i.e., that is notnot due to any manipulations etc)due to any manipulations etc)do participants’ scores (on some DV) differ from do participants’ scores (on some DV) differ from one another because they are in different groups one another because they are in different groups of our study, more so than they differ randomly of our study, more so than they differ randomly and due to unmeasured influences?and due to unmeasured influences?

4

77

'There are three kinds of lies: lies, damned 'There are three kinds of lies: lies, damned lies, and statistics.' lies, and statistics.' -- DisraeliDisraeli

88

review of review of oneone--way ANOVAway ANOVA

5

99

remember variance?remember variance?variance = dispersion or spread of scores around a point of central tendency – the meanthere is always some variability in a dataset: every score does not sit on the meanerror variance = can’t be explained (random or due to unmeasured influences)treatment variance = systematic differences due to our IV (e.g., experimental manipulation)

xxx x

xx x

0 100TEST SCORES

xx xx

x xx x xx

x xxxx

x

xx

xx

xxxxx x

x xx

x xx

x

x xxx x

xx x

0 100TEST SCORES

xx xx

x xx x xx

x xxxx

x

xx

xx

xxxxx x

x xx

x xx

x

x

1010

sources of variance in 1sources of variance in 1--way ANOVAway ANOVA

between-groups variance = systematic variance due to membership in different groups / treatments

within-groups variance = error variance (due to random chance or unmeasured influences)

Variance due to treatmentVariance due to treatment ErrorErrorbetweenbetween--groups variancegroups variance withinwithin--groups variancegroups variance

total variationtotal variation

6

1111


1111μ.

μ2μ1

1111

within-groups variance =distribution of individual DV scores around the group mean

between-groups variance =distribution of group means around the grand mean

μ1 = population mean for group 1μ2 = population mean for group 2μ. = population grand mean

1212

hypotheses in 1hypotheses in 1--way ANOVAway ANOVAdifferences between 2 meansdifferences between 2 means

HH00: : μμ11 = = μμ22–– null hypothesis: no differences between treatment meansnull hypothesis: no differences between treatment means

HH11:: μμ11 ≠ μμ22–– alternative hypothesis: difference between treatment meansalternative hypothesis: difference between treatment means

Or another way of putting it: Or another way of putting it: HH00: : μμjj = = μμ. . HH11:: μμjj ≠ μμ.. for at least one groupfor at least one group

μj = population mean of group j

μ. = grand mean

7

1313

hypotheses in 1hypotheses in 1--way ANOVAway ANOVAdifferences among 3+ meansdifferences among 3+ means

HH00: : μμ11 = = μμ2 2 = = μμ3 3 = = = = μμjj–– null hypothesis: no differences between any treatment meansnull hypothesis: no differences between any treatment means–– Could also write, Could also write, HH00: : μμjj = = μμ..

HH11:: null hypothesis is falsenull hypothesis is false–– alternative hypothesis: at least one difference between alternative hypothesis: at least one difference between

treatment meanstreatment means–– Could also write, Could also write, HH11: : μμjj ≠ ≠ μμ. . for at least one jfor at least one j

μj = population mean of group j

μ. = grand mean

1414


1414μ.

μ2μ1

1414

What line is hypothesized to be zero in our null hypothesis?

8

1515

16μ. μ2μ1

what happens in 1what happens in 1--way ANOVAway ANOVA

•• given that there is always given that there is always withinwithin--group “error” variance group “error” variance in the in the sample, we want to know whether there is more sample, we want to know whether there is more betweenbetween--groups groups variation than expected based on this error variancevariation than expected based on this error variance

•• want to be sure that differences between groups are not due want to be sure that differences between groups are not due simply to simply to errorerror

Ok, but it’s never really zero, right? Ok, but it’s never really zero, right? we compare the ratio of betweenwe compare the ratio of between--groups variance to groups variance to withinwithin--groups variancegroups variance

μ. μ2μ1

9

1717

Variance due to treatmentVariance due to treatment ErrorErrorBetweenBetween--groups variancegroups variance WithinWithin--groups variancegroups variance

Total VariationTotal Variationunivariate anova

n∑(X bar j – X bar dot)2 = people per group x sum of squared differences between group means and grand mean = estimate of between groups variability

∑ − 2. )( XXn j

∑(X ij – X bar j)2 = sum of squared differences between individual scores and group mean = estimate of within groups variability

∑ − 2)( jij XX

1818

the test statistic is the Fthe test statistic is the F--ratioratio

F = MStreat/MSerror

where where MSMStreattreat = index of variability among treatment means = index of variability among treatment means (SS(SSTRTR//dfdfTRTR) or () or (SSSSjj//dfdfjj))

and and MSMSerrorerror = index of variability among participants within a = index of variability among participants within a cell, i.e. pooled withincell, i.e. pooled within--cell variance (cell variance (SSSSErrorError//dfdfErrorError))

= average of s= average of s22 from each sample, a good estimate from each sample, a good estimate of of σσee

22 (population variance)(population variance)

–– if if MSMStreattreat is a good estimate of is a good estimate of error variance, error variance, F F = = MSMStreattreat//MSMSerrorerror = 1= 1–– if if MSMStreattreat is more than just error variance, is more than just error variance, F F = = MSMStreattreat//MSMSerrorerror > 1> 1–– larger values of F indicate that H0 is probably wronglarger values of F indicate that H0 is probably wrong

logic of univariate (onelogic of univariate (one--way) anovaway) anova

10

1919

1. Estimate of between1. Estimate of between--groups variabilitygroups variability2. Estimate of within2. Estimate of within--groups variabilitygroups variability3. Weight each variability estimate by # of observations 3. Weight each variability estimate by # of observations used to generate the estimate (“degrees of freedom”)used to generate the estimate (“degrees of freedom”)Compare ratioCompare ratio

[[n∑(X bar j – X bar dot)2] / (j -1) ][[∑(X ij – X bar j)2 ] / [ j (n-1)] ]

When the F ratio is > 1, the treatment effect (variability between groups) is bigger than the “error” variability (variability within groups). Or more specifically:

The sum of the squared differences between the group means and the grand mean x the number of people in each group, divided by the number of groups minus 1, is bigger than

the sum of the squared differences between the observations and the group means, weighted by the number of observations in each group minus 1 x the # of groups

So what is ANOVA ?So what is ANOVA ?

2020

building up to the building up to the FF ratioratio

if you want to know more about the underlying math, take a look online at the extra materials for Lecture 2

for a refresher on DFs:* Field (3rd ed) pg. 37* Field (2nd ed) pg. 319

11

2121

2222

partitioning variance in 1partitioning variance in 1--way ANOVAway ANOVA

errortreatment

12

2323

structural model of 1structural model of 1--way ANOVAway ANOVA

XXijij == μμ.. ++ ττjj ++ eeijij

for i cases and j treatments

Xij, any DV score is a combination of:

μ. the grand mean

τj the effect of the j-th treatment (μj - μ.)eij error for i person in j-th treatment

2424

XXijij = = μμ. + . + ττjj + + eeijij

μ. μ2μ1Xi2


13

2525

Derivation for oneDerivation for one--way ANOVA: way ANOVA: expected mean squaresexpected mean squares

an expected value of a statistic is defined as the ‘longan expected value of a statistic is defined as the ‘long--range range average’ of a sampling statisticaverage’ of a sampling statisticour our expected mean squaresexpected mean squares –– are:are:–– EE(MS(MSerrorerror) ) σσee

22

•• i.e., the long term average of the variances i.e., the long term average of the variances withinwithin each sample (Seach sample (S22) ) would be the population variance would be the population variance σσee

22

–– EE(MS(MStreattreat) ) σσee2 2 + n+ nσσττ

22

•• where where σσττ22 is the long term average of the variance is the long term average of the variance between sample between sample

meansmeans and n is the number of observations in each groupand n is the number of observations in each group•• i.e., the long term average of the variances i.e., the long term average of the variances withinwithin each sample each sample

PLUSPLUS any variance any variance betweenbetween each sampleeach sample•• Basically Basically -- if group means don’t vary then if group means don’t vary then nnσσττ

22 = 0, = 0, and so then and so then EE(MS(MStreattreat) ) = = σσee

2 2 + 0 = + 0 = σσee2 = 2 = EE(MS(MSerrorerror) ) = = σσee

22

See e.g., Howell (2007) p. 303See e.g., Howell (2007) p. 303

2626

twotwo--way factorial way factorial ANOVAANOVA

14

2727

research questionsresearch questionsoneone--way ANOVA:way ANOVA:-- is there a treatment effect (is there between group variability)?is there a treatment effect (is there between group variability)?

twotwo--way ANOVA:way ANOVA:–– is there a main effect of Factor A? is there a main effect of Factor A?

Is there variability between the levels of A, averaging over the Is there variability between the levels of A, averaging over the other factor? Do the A group means differ from each other? other factor? Do the A group means differ from each other? Do the marginal means of A differ from the grand mean?Do the marginal means of A differ from the grand mean?

–– is there a main effect of Factor B? is there a main effect of Factor B? Is there variability between the levels of B, averaging over the Is there variability between the levels of B, averaging over the other factor? Do the B group means differ from each other? other factor? Do the B group means differ from each other? Do the marginal means of B differ from the grand mean?Do the marginal means of B differ from the grand mean?

–– is there an A x B interaction? is there an A x B interaction? Does the simple effect of A change for different B groups? Does Does the simple effect of A change for different B groups? Does the simple effect of B change for different A groups? Does the the simple effect of B change for different A groups? Does the simple effect change across the levels of the other factor? simple effect change across the levels of the other factor? Do the cell means differ from the grand mean more than would Do the cell means differ from the grand mean more than would be expected given the effects of A and B?be expected given the effects of A and B?

2828

Variance due to treatmentVariance due to treatment ErrorErrorbetweenbetween--groups variancegroups variance withinwithin--groups variancegroups variance

total variationtotal variation

variance due to factor A

variance due to factor B

variance due to A X B

1-way ANOVA

2-way ANOVA

15

sources of variance: 2 x 2 ANOVA

μ.

good notes, little study

good notes, much study

bad notes, little study

bad notes, much study

Factor 1 = note quality (bad or good)Factor 2 = study time (little or much)

DV = exam score

4 cells

29

30

sources of variance: main effects of note quality & study time

μ.

Μgood notesΜbad notes





ΜmuchΜlittle

marginal means to assess main effects

16

31

μ.





simple effect of study time for bad notes (compares cell means for S at N1)

simple effect of study time for good notes (compares cell means for S at N2)

the interaction effect will be significant if the simple effects of one factor differ at the various levels of the other factor

sources of variance: simple effects of study time

32

μ.





simple effect of note quality for little study (compares cell means for N at S1)

simple effect of note quality for much study (compares cell means for N at S2)

The interaction effect will be significant if the simple effects of one factor differ at the various levels of the other factor

sources of variance: simple effects of note quality

17

3333

degrees of freedom summarydegrees of freedom summarydf total = N – 1

df factor = # of levels of the factor – 1

dfB = b – 1dfA = a – 1

df interaction = product of df for factors in the interaction

dfBA = (b – 1) x (a –1)

dferror = total # of observations – # of treatments= N – ba

= df for each cell x # of cells= (n – 1) ba

3434

Variance due to treatmentVariance due to treatment ErrorErrorbetweenbetween--groups dfgroups df

= ab = ab -- 11

withinwithin--groups groups dfdf

= N = N –– abab = = abab (n (n -- 1)1)

total df = N total df = N -- 11

df for factor A

= a – 1

df for factor B

= b - 1

df for A X B

= (a - 1) x (b - 1)2-way ANOVA

1-way ANOVA

18

3535

partitioning variancepartitioning variancein 2in 2--way ANOVAway ANOVA

main effect of Amain effect of BAxB interactionerror

3636

19

3737

main effects main effects (shown for an IV with 3 levels)–– HH00:: μμ11 = = μμ22 = = μμ33

•• no differences no differences among means across levels of the among means across levels of the factorfactor

–– HH11:: null is falsenull is false

interaction interaction (one possibility for a 2 x 3 design)(one possibility for a 2 x 3 design)–– HH00:: μμ1111 -- μμ2121 = = μμ12 12 -- μμ2222 = = μμ1313 -- μμ2323

•• if there are differences between particular factor if there are differences between particular factor means, they are means, they are constantconstant at each level of the at each level of the other factor (hence the parallel lines)other factor (hence the parallel lines)

•• the ‘difference of the differences’ is zerothe ‘difference of the differences’ is zero–– HH11:: null is falsenull is false

hypothesis testing in 2hypothesis testing in 2--way ANOVAway ANOVA

μμ1111 = mean of the following group:1st level of Factor A and 1st level of Factor B

3838

interaction interaction (other possibility for a 2 x 3 design)(other possibility for a 2 x 3 design)–– Or HOr H00:: μμ1k1k -- μμ.. = .. = μμ2k 2k -- μμ.. ..

•• differences between the means of factor B and the grand differences between the means of factor B and the grand mean are the same for the 2 levels of factor A (because B mean are the same for the 2 levels of factor A (because B has 3 means, can’t just put up pairs)has 3 means, can’t just put up pairs)

–– HH11:: μμ1k1k -- μμ.. ≠ .. ≠ μμ2k 2k -- μμ.. ..

hypothesis testing in 2hypothesis testing in 2--way ANOVAway ANOVA

20

3939

1. In 21. In 2--way design, estimate betweenway design, estimate between--groups variabilitygroups variability–– Due to main effect of first factorDue to main effect of first factor–– Due to main effect of second factorDue to main effect of second factor–– Due to interaction of two factorsDue to interaction of two factors

2. Estimate within2. Estimate within--groups variabilitygroups variability3. Weight each variability estimate by # of 3. Weight each variability estimate by # of observations used to generate the observations used to generate the estimate (“degrees of freedom”)estimate (“degrees of freedom”)Compare Compare ratios ofratios of–– (1) between(1) between--groups groups variability among levels of variability among levels of

A to A to error; (2) B error; (2) B to to error; error; and and (3) (3) ABcellsABcells(adjusted for main effects) to error(adjusted for main effects) to error

So what is factorial ANOVA ?So what is factorial ANOVA ?

oneone--way ANOVAway ANOVA

F = MStreat / MSerror

as for 1as for 1--way ANOVA, way ANOVA, MSMSerrorerror = pooled variance (average s= pooled variance (average s22) )

but now we have a separate but now we have a separate MSMStreattreat for each effect:for each effect:1) 1) MSMStreattreat for effect of factor A = for effect of factor A = MSMSAA (1st main effect)(1st main effect)2) 2) MSMStreattreat for effect of factor B = for effect of factor B = MSMSBB (2nd main effect)(2nd main effect)3) 3) MSMStreattreat for effect of factor AB = for effect of factor AB = MSMSABAB ((intxintx effect)effect)

4040

the the FF test in 2test in 2--way ANOVAway ANOVA

a ratio of the systematic variance of EACH EFFECT (i.e. of your experimental manipulations or treatments) to the unsystematic variance

21

2 way ANOVA2 way ANOVA

three omnibus teststhree omnibus testsMain effect of AMain effect of AMain effect of BMain effect of BInteractionInteraction

4141

4242

XXijkijk == μμ. . ++ ααjj ++ ββkk ++ αβαβjkjk ++ eeijkijk

for i cases, factor A with j treatments, factor B with k treatments, and the AxB interaction with jk treatments:

Xijk, any DV score is a combination of:μ. the grand meanαα j the effect of the j-th treatment of factor A (μAj - μ.)ββkk the effect of the k-th treatment of factor B (μBk - μ.)αβαβjkjk the effect of differences in factor A treatments at different levels of factor B treatments (μ. - μAj - μBk + μjk)eijk error for i person in j-th and k-th treatments


22

4343

assumptions of ANOVAassumptions of ANOVApopulation– treatment populations are normally distributed

(assumption of normality)– treatment populations have the same variance

(assumption of homogeneity of variance)sample– samples are independent – no two measures are drawn from the

same participant • c.f. repeated-measures ANOVA (more on that later in semester)

– each sample obtained by independent random sampling – within any particular sample, no choosing of respondents on any kind of systematic basis

– each sample has at least 2 observations and equal ndata (DV scores)– measured using a continuous scale (interval or ratio)– mathematical operations (calculations for means, variance, etc)

do not make sense for other kinds of scales

4444

23

4545

conducting a 2conducting a 2--waywaybetweenbetween--subjects subjects factorial ANOVAfactorial ANOVA

4646

application of betweenapplication of between--subjects subjects factorial ANOVAfactorial ANOVA

a psychological study of creativity in complex socioa psychological study of creativity in complex socio--chemical environments (Field, 2000)chemical environments (Field, 2000)

2 factors:2 factors:–– three groups of participants go to the pub and have:three groups of participants go to the pub and have:

•• no beerno beer, or , or 2 pints2 pints, or , or 4 pints4 pints–– half of the participants are half of the participants are distracted distracted and half are and half are

not distracted not distracted (controls)(controls)hence, a hence, a 2 x 3 between2 x 3 between--subjects factorial designsubjects factorial designDV:DV: creativitycreativity– unbiased 3rd parties rate the quality of limericks made

up by each of our participants

24

4747

application of betweenapplication of between--subjects subjects factorial ANOVAfactorial ANOVA

research questions:research questions:is there a is there a main effectmain effect of alcohol consumptionof alcohol consumption??–– does the quality of limerick you make up depend upon does the quality of limerick you make up depend upon

how many pints of beer you have had?how many pints of beer you have had?

is there a is there a main effectmain effect of distractionof distraction??–– does the quality of limerick you make up depend on does the quality of limerick you make up depend on

whether you were distracted or not?whether you were distracted or not?

is there a is there a consumption x distraction interactionconsumption x distraction interaction??–– does the effect of distraction upon creativity depend does the effect of distraction upon creativity depend

upon consumption? (Or, does the effect of upon consumption? (Or, does the effect of consumption upon creativity depend upon consumption upon creativity depend upon distraction?)distraction?)

4848

0 2 4

50 45 3055 60 3080 85 3065 65 55

Distraction 70 70 3575 70 2075 80 4565 60 40

Cell Totals 535 535 285 1355

65 70 5570 65 6560 60 70

Controls 60 70 5560 65 5555 60 6060 60 5055 50 50

Cell Totals 485 500 460 1445

Marginal Totals (A ) 1020 1035 745 2800

Distraction Marginal Totals (B)

Alcohol Consumption (pints)a combination of IV levels, e.g., 0 pints and distracted, is

called a cell

in most between-subjects factorial

designs there are nobservations per cell

the number of cells multiplied by n gives

you N, the total number of

observations

(abn = N)

25

4949

0 2 4

50 45 3055 60 3080 85 3065 65 55

Distraction 70 70 3575 70 2075 80 4565 60 40

Cell Totals 535 535 285 1355Cell Means 66.88 66.88 35.63 56.46

65 70 5570 65 6560 60 70

Controls 60 70 5560 65 5555 60 6060 60 5055 50 50

Cell Totals 485 500 460 1445Cell Means 60.63 62.50 57.50 60.21

Marginal Totals (A ) 1020 1035 745 2800

Means 63.75 64.69 46.56 58.33

Distraction Marginal Totals (B)

Alcohol Consumption (pints)cell means are

calculated – these are the average of the

n observations in each cell

marginal means for each level of each

factor are also calculated –

these are group means averaged over the

levels of the other factor

the grand mean is the average of all N

observations

Means

Summary Table

Source df SS MS F sig

A (cons) 2 3332.3 1666.15 20.07 .000

B (dist) 1 168.75 168.75 2.03 .161

AB 2 1978.12 989.06 11.91 .000

Error 42 3487.5 83.02

Total 47 8966.7

50

review of source / summary table

p value

MS = SS / df F = MStreat / MSerror

if you want to know more about how the SS values were obtained from the data, look at the extra materials for Lecture 2

26

51

Summary Table


A (cons) 2 3332.3 1666.15 20.07 .000

B (dist) 1 168.75 168.75 2.03 .161

AB 2 1978.12 989.06 11.91 .000

Error 42 3487.5 83.02

Total 47 8966.7

the results of this ANOVA show:

a significant main effect of pints consumed

no main effect of distraction

a significant cons. x dist. interaction

5252

0

10

20

30

40

50

60

70

80

0 2 4

Alcohol consumed (pints)

rate

d cr

eativ

ity

DistractionControls

no main effect of distractionno main effect of distraction

27

5353

0

10

20

30

40

50

60

70

80

0 2 4


rate

d cr

eativ

ity

DistractionControls

main effect of alcohol consumedmain effect of alcohol consumed

5454

0

10

20

30

40

50

60

70

80

0 2 4


rate

d cr

eativ

ity

DistractionControls

disordinaldisordinal interactioninteraction

28

5555

effect sizeseffect sizes

5656

are significance tests so useful?researchers have uncritically focused on researchers have uncritically focused on pp values...values...

problems with significance testing as a way of problems with significance testing as a way of determining the importance of findings:determining the importance of findings:

–– use of an arbitrary acceptance criterion (use of an arbitrary acceptance criterion (αα) results in a ) results in a binary outcome (significant or nonbinary outcome (significant or non--significant)significant)

–– no information about the practical significance of findingsno information about the practical significance of findings

–– a large a large pp--value (nonvalue (non--significant) will eventually slip under significant) will eventually slip under the acceptance criterion as the sample size increasesthe acceptance criterion as the sample size increases

thethe magnitude of experimental effectmagnitude of experimental effect, or , or effect sizeeffect size, has been proposed as an , has been proposed as an accompaniment (if not an outright replacement) accompaniment (if not an outright replacement) to significance testing to significance testing

29

5757

magnitude of experimental effectsmagnitude of experimental effects

the effect size gives you another way of assessing the effect size gives you another way of assessing the reliability of the result the reliability of the result in terms of variancein terms of variance

can compare size of effects within a factorial design: can compare size of effects within a factorial design: how much variance explained by factor 1, factor 2, how much variance explained by factor 1, factor 2, their interaction, etc.their interaction, etc.

differentiating effect sizes (Cohen, 1973):differentiating effect sizes (Cohen, 1973):0.2 = small0.2 = small0.5 = medium0.5 = medium0.8 = large0.8 = large

5858

etaeta--squared squared ηη22

ηη22 = = SSSSeffecteffect

SSSStotaltotaldescribes the describes the proportion of proportion of variancevariance in the sample’ssample’sDVDV scoresscores that is accounted that is accounted for by the effectfor by the effectconsidered a considered a biasedbiased estimate estimate of the true magnitude of the of the true magnitude of the effect in the population (tends effect in the population (tends to be larger than in reality)to be larger than in reality)most commonly reported most commonly reported effect size measure because effect size measure because easily interpretable (like Reasily interpretable (like R2))

omegaomega--squared squared ϖϖ22

ϖϖ2 2 = = SSSSeffecteffect –– ((dfdfeffecteffect))MSMSerrorerror

SSSStotaltotal + + MSMSerrorerror

describes the describes the proportion of proportion of variancevariance in thein the population’spopulation’sDV scores that is accounted DV scores that is accounted for by the effectfor by the effecta a less biasedless biased estimate of the estimate of the effect sizeeffect sizea more a more conservativeconservative estimate estimate (smaller)(smaller)

2 main approaches to estimating 2 main approaches to estimating effect sizes in ANOVAeffect sizes in ANOVA

difference between the two estimates depends on sample sizesample size and error varianceerror variance

30

5959

Summary Table – from Slide 41


C (cons) 2 3332.3 1666.15 20.07 .000

D (distr) 1 168.75 168.75 2.03 .161

C x D 2 1978.12 989.06 11.91 .000

Error 42 3487.5 83.02

Total 47 8966.7

etaeta--squared:squared:ηη22 = = SSSSeffecteffect

SSSStotaltotal

ConsumptionConsumption: : = 3332.3 / 8966.7= 3332.3 / 8966.7= .37 (37% var)= .37 (37% var)

number of pints consumednumber of pints consumedexplains 37% of variance in explains 37% of variance in DV scores (creativity ratings)DV scores (creativity ratings)

distractiondistraction explains 2% of explains 2% of variance in DV scoresvariance in DV scores

interactioninteraction (distraction X number (distraction X number of pints consumed) explains of pints consumed) explains 22% in DV scores22% in DV scores

DistractionDistraction: : = 168.75 / 8966.7= 168.75 / 8966.7= .02 (2% var)= .02 (2% var)C x DC x D: : = 1978.12 / 8966.7= 1978.12 / 8966.7= .22 (22% var)= .22 (22% var)

6060

Summary Table – from Slide 41


C (cons) 2 3332.3 1666.15 20.07 .000

D (distr) 1 168.75 168.75 2.03 .161

C x D 2 1978.12 989.06 11.91 .000

Error 42 3487.5 83.02

Total 47 8966.7

omegaomega--squared:squared:ϖϖ 2 2 = = SSSSeffect effect –– (df(dfeffecteffect)MS)MSerrorerror

SSSStotaltotal + MS+ MSerrorerror

produces very similar but smaller produces very similar but smaller (more conservative)(more conservative) estimatesestimates

ConsumptionConsumption= [3332.3 = [3332.3 -- 2(83.02)] / (8966.7 + 83.02)2(83.02)] / (8966.7 + 83.02)= = .34 (.34 (34% 34% var)var) [compared to 37%][compared to 37%]

DistractionDistraction= [168.75 = [168.75 -- 1(83.02)] / (8966.7 + 83.02)1(83.02)] / (8966.7 + 83.02)= .01 (= .01 (1%1% var) var) [compared to 2%][compared to 2%]

C x DC x D= [1978.12 = [1978.12 –– 2(83.02)] / (8966.7 +83.02)2(83.02)] / (8966.7 +83.02)= .20 (= .20 (20%20% var) var) [compared to 22%][compared to 22%]

31

6161

what’s what’s partialpartial eta squared?eta squared?compare means command in SPSScompare means command in SPSS: : asking for an ANOVA with effect size asking for an ANOVA with effect size gives you gives you eta squaredeta squared –– proportion of proportion of total variance total variance accounted for by the effectaccounted for by the effect

total

effect2

SSSS

=η

erroreffect

effectp22

SSSSSS)(partial

+=ηη

residual variance = variance left over to be explained (i.e., not accounted for by any other IV in the model)

UNIANOVA, MANOVA, or GLMUNIANOVA, MANOVA, or GLM: : asking for effect size gives you asking for effect size gives you partial eta squaredpartial eta squared – proportion proportion of of residual varianceresidual variance accounted accounted for by the effectfor by the effect

total variance = SS for all effects + SSerror

6262

limitations of partial η2

1. in factorial ANOVA, [error + effect] is less than [total], so partial η2 is more liberal or inflated

SSSSeffect1effect1 = 4= 4SSSSeffect2effect2 = 4= 4SSSSintxintx = 4= 4SSSSerrorerror = 4= 4

SSSStotaltotal = 16= 16

ηη22 = 4 / 16 = = 4 / 16 = .25

partial η2 = 4 / (4 + 4) = 4 / 8 = .50

effect size for factor 1:

total

effect2

SSSS

=ηerroreffect

effectp22

SSSSSS)(partial

+=ηη

32

6363

limitations of partial limitations of partial ηη22

2. in factorial ANOVA, η2 adds up to a maximum of 100%, but partial η2 can add to > 100%

SSeffect1 = 4SSeffect2 = 4SSintx = 4SSerror = 4

SStotal = 16

η2 partial partial ηη22

effect 1 .25 .5effect 2 .25 .5

interaction .25 .5error .25 .5sum 1.001.00 2.00 (!)2.00 (!)

(calculations as inprevious slide)

hard to make meaningful comparisons with partial η2

6464

Tests of Between-Subjects Effects

Dependent Variable: I am dissatisfied with my performance recently

54.963a 14 3.926 1.822 .066 .367250.920 1 250.920 116.420 .000 .726

.070 1 .070 .032 .858 .00144.031 9 4.892 2.270 .035 .317

9.326 4 2.331 1.082 .377 .090

94.834 44 2.155821.000 59149.797 58

SourceCorrected ModelInterceptgendernumgpaestimategendernum *gpaestimateErrorTotalCorrected Total

Type III Sumof Squares df Mean Square F Sig.

Partial EtaSquared

R Squared = .367 (Adjusted R Squared = .165)a.

example with SPSS output

•• partial partial ηη22 : interaction between the two IVs accounts for 9% of the residual variance

•• ηη22 : interaction only accounts for 6% of the total variance6% of the total variance(9.326 / 149.797)

ω2 < η2 < partial η2

33

6565

summarymain effects and interactions are main effects and interactions are omnibusomnibus teststests–– result from preliminary partitioning of varianceresult from preliminary partitioning of variance–– look for any difference (anywhere) between the meanslook for any difference (anywhere) between the means

statistical significance is not the bestatistical significance is not the be--allall--andand--endend--all all –– useful to estimate the size of effects useful to estimate the size of effects –– gives us more gives us more

information than statistical significanceinformation than statistical significance–– two approaches two approaches –– etaeta--squaredsquared (biased, but common) (biased, but common)

and and omegaomega--squaredsquared (less biased, but uncommon)(less biased, but uncommon)–– partial etapartial eta--squaredsquared is an even more commonly reported is an even more commonly reported

effect size measure (output by SPSS) which is the effect size measure (output by SPSS) which is the portion of residual variance the effects account forportion of residual variance the effects account for

6666

next week: Effect sizes, simple effects and follow-upsReadings:

between-Ps ANOVA I: omnibus tests (this lecture)Field (3rd ed): Chapter 12 (pp 421-430), Chapter 3Field (2nd ed): Chapter 10 (pp 389-397), Chapter 2

between-Ps ANOVA II: follow-up tests (next lecture)Field (3rd ed): Chapter 12 (pp 430-449 and pp 454-456)Field (2nd ed): Chapter 10 (pp 397-420 and pp 425-426)Howell (all eds): Chapter 13 (pp 413-445)

factorial between-ps anova i: omnibus testsuqwloui1/stats/3010 for post... · 2018. 8. 2. · 3 5...

Documents