Apr 20, 2023

Chapter 17Chapter 17Comparing Two ProportionsComparing Two Proportions

In Chapter 17:

• 17.1 Data

• 17.2 Proportion Difference (Risk Difference)

• 17.3 Hypothesis Test

• 17.4 Proportion Ratio (Risk Ratio)

• 17.5 Systematic Sources of Error

• 17.6 Power and Sample Size

§17.1 Data• Two independent groups• Binary response• Epidemiologic jargon: Group 1 = “exposed group”

and Group 2 = “nonexposed group”• Count “successes” in each group and convert to

proportions

Sample Proportions

1

11ˆ

n

ap

2

22ˆ

n

ap

Proportion in the exposed group:

Proportion in the nonexposed group:

§17.2 Proportion Difference (Risk Difference)

The risk difference is the absolute difference in incidence proportions in the groups.

21

21

ˆˆ :estimatorpoint DifferenceRisk

:parameter DifferenceRisk

pp

pp

In large samples, the sampling distribution of the risk difference is approximately Normal

Confidence Interval, Risk Difference

Plus-four confidence interval method for a difference in proportions. This method is accurate in samples as small as 5 per group.

~

~~ and ,2~ ,1~

on based ~

~~

~

~~ where

)~~ (

2

22

1

11~

~~121 212

i

iiii

p

pp

n

apnnxa

n

qp

n

qpSE

SEzpp

95% CI for p1 – p2, ExampleWHI data a1 = 751, n1 = 8503, a2 = 623, n2 = 8102

2.0% to0.3% fromor

)01976,.00302(.

)004271.96.1()07700.08839(.)~~ (

.00471 8104

92300.07700.

8508

91161.8839.

07700.~ ,8104~ 624~

08839.8508

752~ ,8508~ 752~

212~~121

~

222

111

pp

p

SEzpp

SE

pna

pna

95% CI for p1 – p2, ExampleThe plus-four method is similar to Wilson’s score method. Here’s output from from WinPepi > Compare2.exe > Program B showing results from the traditional large-sample method and Wilson score CI for the illustrative example.

§17.3 Hypothesis Test

• We test the proportions for a significant (“nonrandom”) difference

• Two methods are covered in this chapter– z test (large sample) – Fisher’s exact procedure (small

samples)• A third method called the chi-square test is

covered in the next chapter

z TestA. Hypotheses. H0: p1 = p2 against Ha:p1 ≠ p2

[One-sided: Ha: p1 > p2 or Ha: p1 < p2]

B. Test statistic.

C. P-value. Convert zstat to P-value [Table B or F]

combined groupsboth ns,observatio total

combined groupsboth successes, of no.

where11

ˆˆ

21

21stat

p

nnqp

ppz

conclude: highly significant

z Test: Notes

• z statistic dissection– numerator is observed difference– denominator is standard error when p1 = p2

• A continuity correction can be optionally applied (p. 382)

• This z test is equivalent to the chi-square test of association (Chapter 18)

• In small samples (fewer than 5 successes expected in either group), avoid the z test and use the exact Fisher or Mid-P procedure

Fisher’s Exact Test (2-by-2)

Successes Failures Total

Group 1 a1 b1 n1

Group 2 a2 b2 n2

Total m1 m2 N

Before conducting Fisher’s test, data are rearranged to form a 2-by-2 table :

1

11ˆ

n

ap

2

22ˆ

n

ap Recall that and

WHI Data, 2-by-2 Format

+ − Total

Estrogen + 751 7755 8506

Estrogen − 623 7479 8102

Total 1374 15234 16608

088.0623

751ˆ1 p 077.0

8102

623ˆ 2 p

011.0077.0088.0ˆˆ 21 pp

Fisher’s Exact Test, Procedure

A. Hypotheses. H0: p1 = p2 vs. Ha: p1 ≠ p2

[one sided Ha: p1 > p2 or Ha: p1 < p2]

B. Test statistic. Observed counts in 2-by-2 table

C. P-value. Use computer program (e.g., WinPepi > Compare2.exe > Program B). The mathematical basis of the test is described on pp. 386–7.

Fisher’s Exact Test, Example

The incidence of colonic necrosis in an exposed group is 2 of 117. The incidence in a non-exposed group is 0 of 862. Is this difference statistically significant?

A. H0: p1 = p2 against Ha: p1 ≠ p2

B. Data. + − Total

Group 1 2 115 117

Group 2 0 862 862

Total 2 977 979

C. P = 0.014 (WinPepi output shown here). The evidence against H0 is “significant.”

§17.4 Proportion (Risk) Ratio• Let RR refer to an risk ratio or prevalence ratios

Interpretation • The RR is a risk multiplier, e.g., an RR of 2

suggests that the exposure doubles risk

• When p1 = p2 , RR = 1. This is the “baseline RR,” indicating no association.

2

1

ˆ

ˆˆp

pRR

RR Example, WHI Data+ − Total

Estrogen + 751 7755 8506

Estrogen − 623 7479 8102

15.11483.107689.0

08829.0ˆ

07689.08102

623ˆ;08829.0

8506

751ˆ 21

RR

pp

The indicates a positive association; specifically, 15% higher risk (in relative terms) with exposure.

(1– α)100% CI for the RR

Note natural log scale of sampling distribution

RRSEzRR

eˆln

21

ˆln

2211

1111ˆln

wherenanaRR

SE

90% CI for RR, WHI Example+ − Total

Estrogen + 751 7755 8506

Estrogen − 623 7479 8102

)25.1 ,05.1(

1.645 use confidence 90%For

051920.0

1382.0)1483.1ln(ˆln 1483.1ˆ

2236.0,0528.0

0854.01382.0)051920.0)(645.1(1382.0

81021

6231

85061

7511

ˆln

e

ee

z

SE

RRRR

RR

CI for RR, Computerized Results

+ − Total

Estrogen + 751 7755 8506

Estrogen − 623 7479 8102

Output from WinPepi > Compare2.exe > Program B.

See prior slide for hand calculations

§17.5 Systematic Error(Advanced Topic)

• In observational studies, systematic errors are often more important than random sampling error

• Three types of systematic error are considered:

– Confounding

– Information bias

– Selection bias

Confounding• Confounding = the mixing together of the effects

of the explanatory variable with the effects of “lurking” variables. Consider this example:

• The WHI estrogen experiment found increased morbidity and mortality in estrogen users

• Earlier, non-experimental studies found the opposite: lower morbidity and mortality in users

• Plausible explanation: In non-experimental studies, estrogen users (self-selected) were more likely to have “lurking” lifestyles factors that contributed to better health, i.e., confounding

Information Bias• Information bias is due to the mismeasurement

or misclassification of variables in the study. • Misclassification may be nondifferential (occurs

to the same extent in the groups) or differential (one groups experiences a greater degree of misclassification than the other)

• Nondifferential misclassification tends to bias results toward the null (or have no effect).

• Differential misclassification can bias results in either direction.

Nondifferential & Differential Misclassification - Examples

Selection Bias

• Selection bias ≡ systematic error related to the manner in which study participants are selected for study

• Example. If we shoot an arrow into the broad side of a barn and later draw a bull’s-eye where it had landed, have we really identified anything worth noting?

17.6 Power and Sample SizePower and sample sizes analysis for comparing proportions requires us to understand relationships between these factors:

r ≡ sample size allocation ratio n1 / n2

1−β ≡ power (type II error)

α ≡ significance level (type I error)

p1 ≡ expected proportion, group 1

p2 ≡ expected proportion in group 2, or some measure of effect size, such as the expected RR

17.6 Power and Sample SizeBecause of the complexity of calculations (pp. 396 – 402), use software…

Here’s WinPepi’s Compare2 Sample size menu.