regression discontinuity design william shadish university of california, merced

Regression Discontinuity Design

William Shadish

University of California, Merced

Regression Discontinuity Design

• Units are assigned to conditions based on a cutoff score on a measured covariate,

• For example, communities that exceed a certain cutoff on arrests for drunk driving for young drivers per 100,000 receive treatment, and communities below that cutoff are in the comparison condition.

• The effect is measured as the discontinuity between treatment and control regression lines at the cutoff (it is not the group mean difference).

Advantages

• When properly implemented and analyzed, RD yields an unbiased estimate of treatment effect (see Rubin, 1977).

• Communities are assigned to treatment based on their need for treatment, consistent with how many policies are implemented.

Disadvantages

• Statistical power is considerably less than a randomized experiment of the same size. Careful attention to power is crucial.

• Effects are unbiased only if the functional form of the relationship between the assignment variable and the outcome variable is correctly modeled, including: – Nonlinear Relationships– Interactions

Citations to Med/PH Examples

Cullen, K.W., Koehly, L.M., Anderson, C., Baranowski, T., Prokhorov, A., Basen-Engquist, K., Wetter, D., & Hergenroeder, A. (1999). Gender differences in chronic disease risk behaviors through the transition out of high school. American Journal of Preventive Medicine, 17, 1-7.

Finkelstein, M.O., Levin, B., & Robbins, H. (1996a). Clinical and prophylactic trials with assured new treatment for those at greater risk: I. A design proposal. American Journal of Public Health, 86, 691-695.

Finkelstein, M.O., Levin, B., & Robbins, H. (1996b). Clinical and prophylactic trials with assured new treatment for those at greater risk: II. Examples. American Journal of Public Health, 86, 696-705.

4

4.1

4.2

4.3

4.4

4.5

4.6

4.7

4.8

4.9

Income Level

Vis

its

wit

h p

hys

icia

n p

er p

erso

n p

er y

ear

Under$3,000

$3,000-$4,999

$5,000-$6,999

$7,000-$9,999

$10,000-$14,999

$15,000or more

Improvements to the Design

• Modeling of functional form is improved if it can be observed prior to implementation of treatment (e.g., if archival data is used).

• Using all the standard methods to improve power (e.g., add covariates).

• Combining randomized and nonrandomized designs

Using Regression Discontinuity as a Design Element

• For those who are cut out of the experiment based on quantitative eligibility, continue to measure their outcome, and they can be added to the design to increase power.

• For those falling below a cutoff on a measure of outcome, or of receipt of treatment, give a booster and reanalyze that part of the data as an RDD.

Summary

• Of the designs being considered for this intervention, RD is the only one that yields an unbiased estimate.

• RD can be used with both archival data and original data.

• But there is question about whether it can be implemented with sufficient power in this case.