British Society for Population Studies, 2008 Annual Conference University of Manchester

10-12 September 2008

An Approach to Measuring Differences between Rates that is not Affected by the

Overall Prevalence of an Outcome

James P. ScanlanAttorney at Law

Washington, DC, USAjps@jpscanlan.com

Subjects 1. The problem with standard binary measures of differences between rates (relative differences, absolute differences, odds ratios): that all exhibit patterns of correlation with overall prevalence (i.e., among other things, they tend to change as overall prevalence changes) (BSPS 2006, 2007)

2. A plausible alternative approach that avoids the problem with standard measures: a measure that does not change as overall prevalence changes

References

Measuring Health Disparities page (especially the Solutions tab) and Scanlan’s Rule page on jpscanlan.com

Can We Actually Measure Health Disparities? (Chance 2006) (A12)

Race and Mortality (Society 2000) (A10)

The Misinterpretation of Health Inequalities in the United Kingdom (BSPS 2006) (B6)

BSPS 2007 (B11)

Patterns by Which Binary Measures Tend to Change as the Overall Prevalence of an Outcome Changes – Scanlan’s Rules

SR1: The rarer an outcome, the greater tends to be the relative difference in rates of experiencing it and the smaller tends to be the relative difference in rates of avoiding it (IR1 in BSPS 2006; see Bauld, Day, Judge, 2008)

SR2: As an outcome changes in overall prevalence, the odds ratio tends to change in the same direction of the larger of the two relative differences and the absolute difference tends to change in the same direction as the smaller of the two relative differences ─ where numerators are reversed on the two risk ratios (see Semantic Issues tab on Scanlan’s Rule page)

Fig 1. Ratio of (1) Advantaged Group (AG) Success Rate to Disadvantaged Group (DG) Success Rate at Various Cutoffs Defined by AG Success Rate

1

2

3

4

5

1 3 5 10 20 30 40 50 60 70 80 90 95 97 99

Cutoffs Defined by AG Success Rate

Ra

tio

s (1) AG Succ Rate/DG SuccRate

Fig 2. Ratios of (1) AG Success Rate to DG Success Rate and (2) DG Fail Rate to AG Fail Rate

1

2

3

4

5

1 3 5 10 20 30 40 50 60 70 80 90 95 97 99

Cutoffs Defined by AG Success Rate

Ra

tio

s

(1) AG Succ Rate/DG SuccRate(2) DG Fail Rate/AG FailRate

Fig 3. Ratios of (1) AG Success Rate to DG Success Rate, (2) DG Fail Rate to AG Fail Rate, and (3) DG Fail Odds to AG Fails Odds

1

2

3

4

5

1 3 5 10 20 30 40 50 60 70 80 90 95 97 99

Cutoffs Defined by AG Success Rate

Rat

ios

(1) AG Succ Rate/DG SuccRate(2) Ratio DG Fail Rate/AGFail Rate(3) DG Fail Odds/AG FailOdds

Fig 4. Ratios of (1) AG Success Rate to DG Success Rate, (2) DG Fail Rate to AG Fail Rate, and (3) DG Fail Odds to AG Fails Odds; and Absolute Diff Between Rates

0

1

2

3

4

5

1 3 5 10 20 30 40 50 60 70 80 90 95 97 99

Ra

tio

s

(1) AG Succ Rate/DG SuccRate

(2) Ratio DG Fail Rate/AG FailRate

(3) DG Fail Odds/AG FailOdds

0

10

20

1 3 5 10 20 30 40 50 60 70 80 90 95 97 99

Cutoffs Defined by AG Success Rate

Pe

rce

nta

ge

Po

ints

Absolute DifferenceBetween Rates

Table 1 Illustration of the Problem and Intimation of the Solution (in terms of a favorable outcome increasing in overall prevalence)

Period Yr 1 Yr 2 Yr 3 Yr 4AG Rate 40% 58% 76% 94%DG Rate 23% 39% 58% 85%

Measures of Difference (Blue=decrease; Red=increase)Ratio 1 1.74 1.45 1.31 1.11 Ratio 2 1.28 1.49 1.75 2.50Odds Ratio 2.23 2.16 2.29 2.76Absol Diff .17 .19 .18 .09 EES (z) .50 .50 .50 .50

Estimated Effect Size (EES)

Difference between means of hypothesized underlying distributions of risks of experiencing an outcome, in terms of percentage of a standard deviation, assuming normality of the distributions

Table 2 Simplified Illustration of the Solution (in terms of a favorable outcome increasing in overall prevalence)Period Yr 1 Yr 2 AG Rate 40% 58% DG Rate 23% 40%

Measures of Difference Change DirectionRatio 1 1.74 1.43 IncreaseRatio 2 1.28 1.45 DecreaseOdds Ratio 2.23 2.07 DecreaseAbsol Diff .17 .18 IncreaseEES (z) .50 .47 Decrease

Table 3 Illustration of Meaning of Various Ratios at Different Prevalence Levels

Ratio DGFailRate AGFailRate EES

1.2 60.0% 50.0% 0.26

1.2 18.4% 15.4% 0.12

1.5 75.0% 50.0% 0.68

1.5 45.0% 30.0% 0.39

2.0 40.0% 20.0% 0.59

2.0 20.0% 10.0% 0.44

2.0 1.0% 0.5% 0.24

2.5 24.2% 9.7% 0.60

2.5 7.4% 2.9% 0.44

3.0 44.0% 14.7% 0.90

3.0 14.4% 4.8% 0.60

3.0 2.7% 0.9% 0.44

Caveat – Compositional Differences Differences in the proportions groups being

analyzed comprise of the total population in settings being compared undermine otherwise valid measurement approaches

See Heller et al. (BMJ 2002), Boström and Rosén (SJPH 2003), D43

Solution seems always to compare same proportions (e.g., top vs. bottom quintile), which may often limit analyses to income

Less of an issue for US racial comparisons

Table 4 Illustration Based on Morita et. al. (Pediatrics 2008) Data on Black and White Hepatitis B Vaccination Rates Pre and Post School-Entry Vaccination Requirement (see D52)

Period Grade Year WhRt BlRtFav

RatioAdv

Ratio AbsDf EES

PreRequ 5

(Y1)1996 8% 3% 2.67 1.05 0.05 0.47

PostRequ 5

(Y2)1997 46% 33% 1.39 1.24 0.13 0.34

PreRequ 9

(Y1)1996 46% 32% 1.44 1.26 0.14 0.37

PostRequ 9

(Y2)1997 89% 84% 1.06 1.45 0.05 0.24

Table 5 Illustration of UK Changes Over Time from Table 4.13 of The Widening Gap (rates per 100,000)

Cohort Year Class I Class VMortRatio

SurvivalRatio AbsDf EES

55-64 1921 2247 3061 1.36 1.008397 814 0.14

55-64 1931 2237 2535 1.13 1.003058 298 0.06

55-64 1951 2257 2523 1.12 1.002729 266 0.05

55-64 1961 1699 2912 1.71 1.012494 1213 0.25

55-64 1971 1736 2755 1.59 1.010479 1019 0.21

55-64 1981 1267 2728 2.15 1.015020 1461 0.32

55-64 1991 953 2484 2.61 1.015700 1531 0.39

Table 6 Illustration of UK Differences across Age Groups from Table 4.13 of The Widening Gap

Year Cohort Class I Class VMort

Ratio Survival

Ratio AbsDf EES

1991 25-34 39 187 4.8 1.001483 148 0.47

1991 35-44 101 382 3.8 1.002821 281 0.42

1991 45-54 306 916 3.0 1.006156 610 0.39

1991 55-64 953 2484 2.6 1.015700 1531 0.39

Table 7 Illustration of Comparisons as to Different Conditions from Lawlor (AJPH 2006) (Aberdeen 1950 birth cohort) (rates are per 10,000) (see D28)

Cond Class I Class VAdv

RatioFav

Ratio AbsDf EES

CHD 8.30 20.50 2.5 1.001223 12.2 0.28

Stroke 2.30 7.80 3.4 1.000550 5.5 0.34

Table 8 Illustration of Age Group Comparisons in Whitehall Studies from Marang-van de Mheen (JECH 2001) (rates are per 1,000)

Age HGMR LGMR MortRatio SurvRatio AbsDf EES

55-59 6.80 13.90 2.05 1.0072001 7.1 0.27

60-64 11.30 19.90 1.76 1.0087746 8.6 0.22

65-69 17.50 28.10 1.61 1.0109065 10.6 0.20

70-74 30.90 47.50 1.54 1.0174278 16.6 0.20

75-79 50.60 70.00 1.38 1.0208602 19.4 0.16

80-84 78.30 107.60 1.38 1.0328328 29.3 0.19

85-89 144.30 181.60 1.26 1.0455767 37.3 0.16

Table 9 Illustration Based on Boström and Rosén (SJPH 2003) Data on Mortality by Occupation in Seven European Countries (see D43 caveat)

Country EES 1980-84 EES 1990-94

Denmark 0.14 0.13

England and Wales 0.11 0.15

Finland 0.16 0.23

Ireland 0.10 0.19

Norway 0.12 0.16

Spain 0.12 0.23

Sweden 0.14 0.17

Problems with the Solution Always practical issues (we do not really know

the shape of the underlying distributions)

Sometimes fundamental issues (e.g., where we know distributions are not normal because they are truncated portions of larger distributions, see D43); cf. BSPS 2007, Fig. 6

Absolute minimum issue (D43,B6)

Conclusion If we are mindful of the problems, the approach

provides a framework for cautiously appraising the size of differences between rates.

Regardless of problems, the approach is superior to reliance on standard binary measures of differences between rates without regard to the way those measures tend to be correlated with the overall prevalence of an outcome.

Formula to derive EES?

Example of 8 of 76,960 rows in table downloadable from jpscanlan.com

EES AG Fail DG Fail

0.60 0.50399 0.72907

0.60 0.5 0.72575

0.60 0.5 0.72241

0.60 0.49601 0.71905

0.30 0.50399 0.62172

0.30 0.5 0.61791

0.30 0.5 0.6141

0.30 0.49601 0.61026

Supp Table 1 Illustrations Based on Escarce and McGuire (AJPH 2004) Data on White and Black Coronary Procedure Rates, 1986 and 1997 (see D48)

Proc Year Wh Rt Bl RtFav

RatioAdv

Ratio AbsDf EES

Angrm(Y1)

1986 0.86% 0.43% 1.99 1.05 0.04 0.25

Angrm(Y2)

1997 2.28% 1.61% 1.42 1.09 0.07 0.14

Angpls(Y1)

1986 0.10% 0.03% 3.09 1.01 0.01 0.32

Angpls(Y2)

1997 0.26% 0.16% 1.61 1.01 0.01 0.15

ArtByp(Y1)

1986 0.31% 0.08% 3.78 1.02 0.02 0.41

ArtByp(Y2)

1997 0.59% 0.26% 2.25 1.03 0.03 0.27

Supp Table 2: Illustration Based on Hetemaa et al. (JECH 2003) Data on Finnish Revascularization Rates, 1988 and 1996, by Income Group (see D21, D58)

Gender YearAG

RevRtLowIncRevRt

FavRatio

AdvRatio AbsDf EES

M(Y1)

1988 17.9% 8.3% 2.16 1.12 .096 0.48

M(Y2)

1996 41.2% 25.4% 1.63 1.27 .159 0.44

F(Y1)

1988 10.0% 3.7% 2.70 1.07 .063 0.51

F(Y2)

1996 30.8% 17.1% 1.80 1.20 .137 0.45

Supp Table 3: Illustration Based on Laaksonen et al. (JECH 2008) Data on Mortality Rates of Finnish Men by Owner or Renter Status (see follow-up on D43)

Age OwnMort RentMort AdvRatio FavRatio EES

40–44 1.46% 4.26% 2.91 1.03 0.46

45–49 2.46% 6.04% 2.45 1.04 0.42

50–54 3.68% 9.68% 2.63 1.07 0.49

55–59 5.62% 13.09% 2.33 1.09 0.47

60–64 8.88% 19.89% 2.24 1.14 0.5

65–69 14.33% 29.38% 2.05 1.21 0.53

70–74 24.62% 41.85% 1.70 1.30 0.48

75–79 36.55% 57.75% 1.58 1.50 0.56

Supp Table 4 Illustration from Valkonen et al. (JEPH 2000) Based on All Cause Mortality in Finland for Three Time Periods

Gender Period AGMort DGMort AdvRatio FavRatio EES

M 1981-85 0.64% 0.96% 1.50 1.00321 0.15

M 1986-90 0.53% 0.93% 1.75 1.00404 0.21

M 1991-95 0.46% 0.85% 1.86 1.00395 0.22

F 1981-85 0.29% 0.35% 1.21 1.00060 0.07

F 1986-90 0.26% 0.36% 1.36 1.00095 0.11

F 1991-95 0.24% 0.35% 1.48 1.00113 0.14

f