license information this material is distributed under an attribution‐noncommercial‐sharealike...

1

License information

• This material is distributed under an Attribution‐NonCommercial ShareAlike 3.0 Unported Creative Commons ‐License (CC BY NC SA ), the full details of which may be found ‐ ‐online here:http://creativecommons.org/licenses/by‐nc sa‐ /3.0/ .

• You may re use, edit, or redistribute the content provided that ‐the original source is cited, it is for noncommercial purposes, and provided it is distributed under a similar license.

• If you update or modify this material, please consider reposting your version to the OPOSSEM site. Please also send a copy to the original author: Michelle Dion ([email protected]).

CC BY NC SA‐ ‐

http://creativecommons.org/licenses/by%E2%80%90nc%E2%80%90sa/3.0/





mailto:[email protected]

Comparing means & proportions across different samples

Types of pairs of samples

Independent random samples• Choice of one sample does

not depend on another• Examples

– Men and women– Democracies and non-

democracies

Dependent samples• Natural matching between

samples• One group a two points in

time– Students on both the first and

last day of class– 50 states at two points in

time

CC BY NC SA‐ ‐ 3

4

Overview

1. Comparing means and proportions in pairs of independent samples

A. Examples of comparing meansB. Examples of comparing proportions

2. Comparing means in dependent samplesA. Examples

CC BY NC SA‐ ‐

5

Comparing values in two independent samples

CC BY NC SA‐ ‐

6

Step 1: Are you comparing a quantitative or qualitative variable?

Quantitative• Comparing means across 2

independent samples• Comparing μ1 and μ2

• Examples– Mean height in inches for

men and women, μmen and μwomen

– Mean adult literacy rate in democracies and non-democracies, μdem and μnon-dem

Qualitative• Comparing proportions across

2 independent samples• Comparing P1 and P2

• Examples– Proportions of men and women

with post-secondary education, Pmen and Pwomen

– Proportions of democracies and non-democracies that have ratified the Kyoto Protocol, Pdem and Pnon-dem

CC BY NC SA‐ ‐

7

Step 2: Set up your null and research hypotheses

MeansH0: μ1 = μ2

Ha: μ1 ≠ μ2

ProportionsH0: P1 = P2

Ha: P1 ≠ P2

CC BY NC SA‐ ‐

8

Step 3: Do you have large samples?

Means

• Comparing µ1 and µ2

• n1, n2 ≥ 20

• Both n1 and n2 must be equal or greater than 20

Proportions

• Comparing P1 and P2

• More than 5 observations in each category for each sample

CC BY NC SA‐ ‐

9

Step 4: Calculate the appropriate test statistic

Means• Large samples

• Small samples

Option 1

Option 2

Proportions• Large samples

• Small samples

Use Fisher’s Exact test

CC BY NC SA‐ ‐

2

22

1

2112

12

12

ˆ where,ˆ n

snsyy

z yyyy

:ways 2of 1 in calculated be can

freedomof degrees the and ˆ where

,ˆ

12

12

12

yy

yy

yyt

df for formula complex withˆ 2

22

1

21

12 ns

ns

yy

2 df with

, 1

n1

2

)()(ˆ

21

2121

222

211

12

nn

nnn

yyyy iiyy

)11

)(1( where ,21

1212

12nn

pppp

z pppp

10

Step 5: Interpret p value & conclusion

Means• Small p-values reject the

null hypothesis of no difference in means

Proportions• Small p-values reject the

null hypothesis of no difference in proportions

CC BY NC SA‐ ‐

11

Examples

CC BY NC SA‐ ‐

12

Do Muslims and non-Muslims differ in their opinion of the U.S.?

• Pew Survey of individuals in 22 countries in Spring 2009

• Question: Please tell me if you have a very favorable (1), somewhat favorable (2), somewhat unfavorable (3) or very unfavorable (4) opinion of the United States?

• Larger numbers mean less favorable opinions of the U.S.

CC BY NC SA‐ ‐

13

Do Muslims and non-Muslims differ in their opinion of the U.S.?

• Comparison of mean attitudes of Muslims & non-MuslimsH0: μMuslim = μnon-Muslim

Ha: μMuslim ≠ μnon-Muslim

CC BY NC SA‐ ‐

Muslim non-Muslim0

1

2

3

4

Unfavorable opinion of the U.S.

14

Do Muslims and non-Muslims differ in their opinion of U.S.?

• Calculate large sample test statistic

• Muslims

• Non-Muslims

CC BY NC SA‐ ‐

17431832.

17.2

lim

lim

lim

Musnon

Musnon

Musnon

nsy

7186032.102.3

lim

lim

lim

Mus

Mus

Mus

nsy

?

ˆ where,ˆ

2

22

1

21

12

2

22

1

2112

12

12

z

ns

ns

yyz

ns

nsyy

z yyyy

15


• Calculate large sample test statistic

• Muslims

• Non-Muslims

CC BY NC SA‐ ‐

17431832.

17.2

lim

lim

lim

Musnon

Musnon

Musnon

nsy

7186032.102.3

lim

lim

lim

Mus

Mus

Mus

nsy

006.62014.85.

17431832.

7186032.1

17.202.322

2

22

1

21

12

z

z

z

ns

ns

yyz


• If z = 62.006, what is the associated p-value?• Where is this z score on the z distribution?


Image source: http://upload.wikimedia.org/wikipedia/commons/b/bb/Normal_distribution_and_scales.gif

17


• Is there a statistically significant difference between Muslims and non-Muslims in their mean opinions of the U.S. around the world?

• How do we interpret the P-value associated with this statistical test?

CC BY NC SA‐ ‐

18

Describing the difference between Muslim & non-Muslim opinions of U.S.

• What is the average difference?

• Constructing a confidence interval around the difference

• What is the 99% confidence interval?• Interpretation?

CC BY NC SA‐ ‐

85.limlim MusnonMus yy

)014(.85.

)()(

)ˆ()(

2

22

1

21

limlim

limlim limlim

z

ns

ns

zyy

zyy

MusnonMus

yyMusnonMus MusnonMus

19

Does colonial heritage explain corruption?

• Do countries colonized by the British really have lower levels of corruption than those colonized by the Spanish?

CC BY NC SA‐ ‐

Image source: www.bit.ly/9yNr94

20


• Mean transparency score0 = “highly corrupt”10 = “highly clean”

• Comparing meansH0: μBritish = μSpanish

Ha: μBritish > μSpanish

• Small samples

• British colonies

• Spanish colonies

CC BY NC SA‐ ‐

52732.1940.3

British

British

British

nsy

19

379.1

358.3

Spanish

Spanish

Spanish

n

s

y

21


• Calculate t-score

Option 1

Option 2

• Use software to calculate

• British colonies

• Spanish colonies

CC BY NC SA‐ ‐

52732.1940.3

British

British

British

nsy

19

379.1

358.3

Spanish

Spanish

Spanish

n

s

y

:ways 2of 1 in calculated be can

freedomof degrees the and ˆ where

,ˆ

12

12

12

yy

yy

yyt


22

1

21

12 ns

ns

yy

2 df with

, 1

n1

2

)()(ˆ

21

2121

222

211

12

nn

nnn

yyyy iiyy

23


CC BY NC SA‐ ‐

24


CC BY NC SA‐ ‐

Group Statistics

British and Spanish

colonies N Mean Std. Deviation

Std. Error

Mean

Corruption Perceptions

Index

Colonized by Spanish 19 3.3579 1.37894 .31635

Colonized by British 52 3.9404 1.73208 .24020

Independent Samples Test

Levene's Test for

Equality of

Variances t-test for Equality of Means

F Sig. t df

Sig. (2-

tailed)

Mean

Difference

Std. Error

Difference

95% Confidence

Interval of the

Difference

Lower Upper

Corruption

Perceptions Index

Equal variances

assumed

2.735 .103 -1.319 69 .192 -.58249 .44159 -1.46343 .29845

Equal variances

not assumed

-1.466 40.040 .150 -.58249 .39720 -1.38524 .22027

25


CC BY NC SA‐ ‐

Group Statistics

British and Spanish


Std. Error

Mean


Index




Levene's Test for

Equality of


F Sig. t df

Sig. (2-

tailed)

Mean

Difference

Std. Error

Difference

95% Confidence

Interval of the

Difference

Lower Upper

Corruption

Perceptions Index

Equal variances

assumed

2.735 .103 -1.319 69 .192 -.58249 .44159 -1.46343 .29845

Equal variances

not assumed

-1.466 40.040 .150 -.58249 .39720 -1.38524 .22027

26


CC BY NC SA‐ ‐


Levene's Test for

Equality of


F Sig. t df

Sig. (2-

tailed)

Mean

Difference

Std. Error

Difference

95% Confidence

Interval of the

Difference

Lower Upper

Corruption

Perceptions Index

Equal variances

assumed

2.735 .103 -1.319 69 .192 -.58249 .44159 -1.46343 .29845

Equal variances

not assumed

-1.466 40.040 .150 -.58249 .39720 -1.38524 .22027


22

1

21

12 ns

ns

yy

2 df with

, 1

n1

2

)()(ˆ

21

2121

222

211

12

nn

nnn

yyyy iiyy

Option 2:

Option 1:

27


CC BY NC SA‐ ‐

Group Statistics

British and Spanish


Std. Error

Mean


Index




Levene's Test for

Equality of


F Sig. t df

Sig. (2-

tailed)

Mean

Difference

Std. Error

Difference

95% Confidence

Interval of the

Difference

Lower Upper

Corruption

Perceptions Index

Equal variances

assumed

2.735 .103 -1.319 69 .192 -.58249 .44159 -1.46343 .29845

Equal variances

not assumed

-1.466 40.040 .150 -.58249 .39720 -1.38524 .22027

28


CC BY NC SA‐ ‐

Group Statistics

British and Spanish


Std. Error

Mean


Index




Levene's Test for

Equality of


F Sig. t df

Sig. (2-

tailed)

Mean

Difference

Std. Error

Difference

95% Confidence

Interval of the

Difference

Lower Upper

Corruption

Perceptions Index

Equal variances

assumed

2.735 .103 -1.319 69 .192 -.58249 .44159 -1.46343 .29845

Equal variances

not assumed

-1.466 40.040 .150 -.58249 .39720 -1.38524 .22027

29


• Are countries colonized by the British more transparent (less corrupt) than those colonized by the Spanish? H0: μBritish = μSpanish

Ha: μBritish > μSpanish

• P-value? One or two-tailed test? • Interpretation and conclusion?

CC BY NC SA‐ ‐

30

Summary of tests of means in two independent samples

• Comparing quantitative variable across two categories– A quantitative dependent variable with a nominal

independent variable with two outcomes (binary)• Large versus small samples

– Z tests versus T tests– Remember: As small samples grow, T test becomes the

same as the Z test• Software will report small sample tests (only)• Can you think of other potential means in pairs of

independent samples you might want to compare?

CC BY NC SA‐ ‐

31

More examples

CC BY NC SA‐ ‐

32

Do the U.S.’s neighbors differ in their opinion of the U.S.?

• Canada and Mexico are important allies and neighbors of the U.S.

• Use the same 2009 Pew survey• Are the attitudes toward the U.S. different in

Canada and Mexico?H0: PMexico = PCanada

Ha: PMexico ≠ PCanada

CC BY NC SA‐ ‐

33


• Canada and Mexico are important allies and neighbors of the U.S.

• Use the same 2009 Pew survey

• Are favorable attitudes toward the U.S. different in Canada and Mexico?H0: PMexico = PCanada


• Mexico

• Canada

CC BY NC SA‐ ‐

719708.

210 eUnfavorabl509 Favorable

Canada

Canada

np

953718.


Mexico

Mexico

np

34


• Are favorable attitudes toward the U.S. different in Canada and Mexico?H0: PMexico = PCanada


• Are these large samples?• Test statistic

Need to calculate p

• Mexico

• Canada

CC BY NC SA‐ ‐

)11

)(1( where ,21

1212

12nn

pppp

z pppp

953718.


Mexico

Mexico

np

719708.


Canada

Canada

np

35


• Mexico

• Canada

CC BY NC SA‐ ‐

719708.


Canada

Canada

np

953718.


Mexico

Mexico

np

)11

)(1( where ,21

1212

12nn

pppp

z pppp

• Test statistic

• What is p? Combined favorable proportionCombined favorable = 685+509= 1194Combined n = 953+719=1672Combined p = 1194 /1672 = 0.714

36


• Mexico

• Canada

CC BY NC SA‐ ‐

719708.


Canada

Canada

np

953718.


Mexico

Mexico

np)

11)(1(

)11

)(1( where ,

21

12

21

1212

12

nnpp

ppz

nnpp

ppz pp

pp

• Test statistic

37

• Test statistic

• P-value? One-tailed or two-tailed testp-value = 0.6542

• Interpretation and conclusion?


• Mexico

• Canada

CC BY NC SA‐ ‐

719708.


Canada

Canada

np

953718.


Mexico

Mexico

np

-0.4480.022

0.01-

)719

1953

1)(714.1(714.

718.708.

)11

)(1(

)11

)(1( where ,

21

12

21

1212

12

z

z

z

nnpp

ppz

nnpp

ppz pp

pp


• Are favorable attitudes toward the U.S. different in Canada and Mexico?

• Which has a more favorable opinion?• Is the difference statistically significant?



• Construct a 95% confidence interval around the difference

– Note different standard error from test statistics. Why would it be different?

– Interpretation?


2

22

1

1112

)1()1(n

ppn

ppzpp

40


• How do you test the same hypothesis with SPSS?– Hypotheses the same– SPSS does not provide the exact same test, but

does have several alternatives

CC BY NC SA‐ ‐

41


CC BY NC SA‐ ‐

42


CC BY NC SA‐ ‐

43


CC BY NC SA‐ ‐

Canada (Mexico=0) dummy variable * Favorable view of U.S. (recode of Q11A.) Crosstabulation

Count

Favorable view of U.S. (recode of Q11A.)

Total

Somewhat unfavorable

or very unfavorable

Somewhat favorable or

very favorable

Canada (Mexico=0) dummy variable Mexico 268 685 953

Canada 210 509 719

Total 478 1194 1672

Chi-Square Tests

Value df

Asymp. Sig. (2-

sided)

Exact Sig. (2-

sided)

Exact Sig. (1-

sided) Point Probability

Pearson Chi-Square .237a 1 .627 .662 .333

Continuity Correctionb .186 1 .666

Likelihood Ratio .236 1 .627 .662 .333

Fisher's Exact Test .662 .333

Linear-by-Linear Association .236c 1 .627 .662 .333 .039

N of Valid Cases 1672

a. 0 cells (.0%) have expected count less than 5. The minimum expected count is 205.55.

b. Computed only for a 2x2 table

c. The standardized statistic is -.486.

44

What if you want to compare proportions in small samples?

• Assumptions for small sample– At least one outcome has fewer than 5

observations• Hypotheses the same• Use Fisher’s exact test on the computer• Interpretation of P-values and conclusions will

be the same

CC BY NC SA‐ ‐

45


CC BY NC SA‐ ‐

Canada (Mexico=0) dummy variable * Favorable view of U.S. (recode of Q11A.) Crosstabulation

Count

Favorable view of U.S. (recode of Q11A.)

Total

Somewhat unfavorable

or very unfavorable

Somewhat favorable or

very favorable

Canada (Mexico=0) dummy variable Mexico 268 685 953

Canada 210 509 719

Total 478 1194 1672

Chi-Square Tests

Value df

Asymp. Sig. (2-

sided)

Exact Sig. (2-

sided)

Exact Sig. (1-

sided) Point Probability

Pearson Chi-Square .237a 1 .627 .662 .333

Continuity Correctionb .186 1 .666

Likelihood Ratio .236 1 .627 .662 .333

Fisher's Exact Test .662 .333

Linear-by-Linear Association .236c 1 .627 .662 .333 .039

N of Valid Cases 1672

a. 0 cells (.0%) have expected count less than 5. The minimum expected count is 205.55.

b. Computed only for a 2x2 table

c. The standardized statistic is -.486.

46

Summary of tests of proportions in two independent samples

• Comparing qualitative variable across two categories– A qualitative dependent variable with two outcomes (binary)

and a nominal independent variable with two outcomes (binary)• Large sample

– Easy to calculate with minimal information from contingency table

– Easy to interpret, including confidence intervals• Small sample

– Use Fisher’s Exact Test• Can you think of other potential proportions in pairs of

independent samples you might want to compare?

CC BY NC SA‐ ‐

Comparing means & proportions across dependent samples

48

What about dependent samples?

• Dependent samples, by definition, have matched pairs– Same sample at two points in time (most

common)– Sometimes experiments are designed to “match”

different observations as “pairs”• Medical studies where treatment and placebo groups

are match according to secondary characteristics• Can you think of how that might work in a political

science experiment? What would you “match” on?

CC BY NC SA‐ ‐

49

Comparing means in 2 dependent samples—Paired-means test

• Two strategies1. Create a new variable that measures the

difference between the two values and treat it like a single sample T-test

2. Use statistical software to calculate the paired-means test

• Results should be identical– Why?

CC BY NC SA‐ ‐

50

Comparing means in 2 dependent samples—Paired-means test

• First strategy1. Create a new variable: d = y2 – y1

2. Conduct a regular test of the significance of a mean

CC BY NC SA‐ ‐

)( :CI

0

for deviation standard for mean sample

ns

td

ns

dt

dsdd

d

d

d

Examples of paired-means tests


52

Did the “3rd wave of democratization” increase democracy in the world?

• The Third Wave of Democratization (Huntington 1991)

• Was the average level of democracy throughout the world higher in 2000 than it was in 1975?H0: μ1975 = μ2000

Ha: μ1975 < μ2000

CC BY NC SA‐ ‐


• First strategy1. Create new variable

In SPSS, Transform Compute Variable


12 yyd



2. Do single sample test of a mean


12 yyd





12 yyd





12 yyd

One-Sample Statistics

N Mean Std. Deviation Std. Error Mean

polity20001975 129 5.2093 6.69267 .58926

One-Sample Test

Test Value = 0

t df Sig. (2-tailed)

Mean

Difference

95% Confidence Interval of the

Difference

Lower Upper

polity20001975 8.840 128 .000 5.20930 4.0434 6.3752




3. Interpretation and conclusion?

– One or two-tailed test?


12 yyd

One-Sample Test

Test Value = 0

t df

Sig. (2-

tailed)

Mean

Difference

95% Confidence Interval of

the Difference

Lower Upper

polity20001975 8.840 128 .000 5.20930 4.0434 6.3752


• Second strategy1. Use statistical software

to do a paired-means test






Paired Samples Statistics

Mean N

Std.

Deviation

Std.

Error

Mean

Pair

1

p_polity2.2000: Revised

Combined Polity Score

3.01 129 6.627 .583



-2.20 129 7.485 .659

Paired Samples Correlations

N Correlation Sig.

Pair 1 p_polity2.2000: Revised

Combined Polity Score &



129 .556 .000

Paired Samples Test

Paired Differences

t df

Sig. (2-

tailed)Mean

Std.

Deviation

Std.

Error

Mean

95% Confidence Interval

of the Difference

Lower Upper

Pair 1 p_polity2.2000: Revised Combined Polity Score -

p_polity2.1975: Revised Combined Polity Score

5.209 6.693 .589 4.043 6.375 8.840 128 .000





Paired Samples Test

Paired Differences

t df

Sig. (2-

tailed)Mean

Std.

Deviation

Std.

Error

Mean


of the Difference

Lower Upper



5.209 6.693 .589 4.043 6.375 8.840 128 .000

One-Sample Test

Test Value = 0

t df

Sig. (2-

tailed)

Mean

Difference

95% Confidence Interval of

the Difference

Lower Upper

polity20001975 8.840 128 .000 5.20930 4.0434 6.3752




2. Interpretation and conclusion?


Paired Samples Test

Paired Differences

t df

Sig. (2-

tailed)Mean

Std.

Deviation

Std.

Error

Mean


of the Difference

Lower Upper



5.209 6.693 .589 4.043 6.375 8.840 128 .000

63

Summary of paired-means tests (in dependent samples)

• Comparing quantitative variable across two matched samples – Differences between matched pairs for a quantitative

dependent variable (mean)• Two strategies

– Create new variable that captures the difference between the match pair

– Use statistical software to calculate the paired-means test– Essentially equivalent

• Can you think of other potential paired-means tests you might want to do?

CC BY NC SA‐ ‐

license information this material is distributed under an attribution‐noncommercial‐sharealike...

Documents

cc byncsa1 slide

cc byncsa12 slide

cc byncsa7 slide

cc byncsa18 slide

cc byncsa22 slide

cc byncsa23 slide

cc byncsa17 slide

proportions cc byncsa10