An Illustration of How to Use Regression-Discontinuity Designs in Medical Settings: An Alternative to True

Experimental Design in Testing Intervention and Treatment Effectiveness

David Newman, Ph.D.Florida Atlantic University Christine E. Lynn College of Nursing

Isadore Newman, Ph.D.Florida International University, Adjunct Prof. College of Medicine

Ethical Concerns with Experimental Research

• True Experimental Design and Randomized Drug Trials– A recurring control group is difficult– Hospitals tend to implement protocols as a whole

unit or at least on specific floors eliminating the possibility of a control group

– If the suspected TX is better and is most likely going to save lives we are obligated to give our patients the best TX available.

What is Regression Discontinuity Analysis (RD)

• Powerful methodological option to experimental design when one is not feasible

• Traditional RD tests slope difference between control and TX groups

• Alternative RD Approach examines changes in individuals over time. – This can be used to enhance evidence based

practice and intervention in clinical settings.

RD Traditional Vs Alternative

• RD tests pre and post slope differences between the group regression lines at the intervention point for statistical significance.

eXbZbbY ePost Pr210where YPost = post measuresZ = group assignment (0 = control; 1 TX)XPre = pre measures b’s = estimated sample regression weights e = residual error

Traditional Alternative

Baseline After TX

Pain

Traditional RD Steps

• Step 1: Create precut score = Pre- Xpre

(Orthogonal) can also use theoretical cut score

• Step 2: Create interaction term group*precut.

• Step 3: Test: Post = Intercept + group + interact + precut

• Step 4: Test: Post = intercept + group + precut

• Step 5: Compare separate group results

Alternative RD Approach Steps

• Step 1: Create Person Vectors (1 if score belongs to personi, 0 otherwise)

• Step 2: Create Time by TX (Before and after) interaction

ID PV_1 PV_2 PV_3 PV_i1 1 0 0 01 1 0 0 01 1 0 0 02 0 1 0 02 0 1 0 02 0 1 0 03 0 0 1 03 0 0 1 03 0 0 1 0i 0 0 0 1i 0 0 0 1i 0 0 0 1

Alternative RD Approach Steps (Cntd)

• Step 3: Test before and after TX slope differences Restricted (R2

1): Y = acUc + acXc + Zb1(P1) +…Zbn(Pn) + e1

Full (R22): Y= a01U1 + a1X1 + a02 U2 + a2X2 + Zb1(P1) +…Zbn(Pn) + e1

Ho: a1 = a2 = ac (common slope)

• Step 4: Test before and after TX Intercept differences

Restricted (R21): Y = a0U1 + aX1 + a2X2 +Zb1(P1)+…Zbn(Pn) + e2

Full (R22): Y= a01U1 + a1X1 + a02 U2 + a2X2 + Zb1(P1)+…Zbn(Pn) + e1

Ho: a01 = a02 = a0 (common intercept)

• Step 5: Compare separate group results

Descriptive Results

TX Time N Mean SD

Pre TX1 25 8.04 0.902 25 7.72 1.173 25 7.16 0.85

After TX4 25 5.40 0.655 25 3.60 0.586 25 2.20 0.76

• From the descriptive statistics and the linear graph with projected slopes it appears that the treatment did have a large impact on pain reduction.

• But were there significant changes over time? And was there a significant immediate impact?

Results TX Effect on Slope

• This indicates that there was a significant decrease in patients change in pain over time after the initiation of the treatment.

R R2 Std Error R2Change Fchange(df1,df2) p

Restricted 0.93 0.87 0.88 0.87 32.96(25,124) <0.001

Full 0.96 0.92 0.70 0.05 36.01(2,122) <0.001

Results TX Effect on Slopes

R R2 Std Error R2Change Fchange(df1,df2) p

Restricted 0.96 0.92 0.90 0.92 50.97(26,123) <0.001

Full 0.96 0.92 0.90 0.00 3.991(1,122) 0.048

• This indicates that there was a significant decrease in patients’ pain after the initiation of the treatment (Intercept).

Advantages of Using The Alternative RD

• With the Person Vectors we can:– Using the partial regression weight and the significance values in

the coefficients table, we can ID the individuals that had significant TX reactions and those that did not.

– Then code those people as successful or not ( 1, 0)

– See if demographics predict the likelihood of TX success.

– Modify TX so that it best fits the patient • Variables from Theories, • Focus Groups• Qualitative analysis or Q-Factor Analysis

Future Methodology Development

• Additional Analysis that can be used with RD– If the criterion variable was the amount of time it took the patient

to reduce pain to his or her subjective acceptable level, then Survival Analysis can be used.

– HLM is another method I personally use to test RD. However this method does not provide the coefficient table with the Person Vectors and limits further examination of the TX effects.

– GEE is akin to HLM but does not rely on the parametric assumptions of the data. Therefore, this technique should be used if the data does not meet the assumptions required for HLM.

• With ALL of these, linear and nonlinear trends can be analyzed.

References

• Bottenberg, R.A. and Ward, J.H. (1963). Applied Multiple Linear Regression. Lackland Air Force Base, San Antonio, TX: Aerospace Medical Division, No. AD413128.

• Newman, I. (2012). Designing General Linear Models to Test Research Hypotheses. University Press of America: Lanham, MD. • • Schumacker, R.E. ( 2007). Regression Discontinuity: Examining Model

Misspecification. • Multiple Linear Regression Viewpoints, 33(2), 6-10.• • Trochim, William M. K. (1984). Research Design for Program Evaluation, the

Regression Discontinuity Approach. Sage Publications: Beverly Hills, CA.

Contact Information

David Newman, PhD

Assistant Professor, Statistician

Christine E. Lynn College of NursingFlorida Atlantic University777 Glades Rd.

Boca Raton, FL 33431-2048Office Phone: 561-297-2670dnewma14@fau.edu

Isadore Newman, PhD

Visiting Scholar, Office of Research and Graduate Studies, College of Education, Florida International University

Adjunct Professor, Dept. of Human and Molecular Genetics, Herbert Wertheimer College of Medicine, Florida International University

11200 Tamiami Trail, ZEB 310,Miami, FL 33199Office (305) 348-2975

newmani@fiu.edu