external validity

External Validity

Are results obtained

here relevant here

Are results obtained from

These people Relevant for these people

In general. Sometimes.

In this class we are going to try and see whether results obtained in test scores in California, are valid in Massatucetts (or vice versa).

Obtaining the data

The data is online. You can load data into stata directly from the internet using the ‘use’ command. E.g.use http://fmwww.bc.edu/ec-p/data/stockwatson/caschool, clear

(a) Is it the same test?

Californiasum testscr

Variable | Obs Mean Std. Dev. Min Max-------------+-------------------------------------------------------- testscr | 420 654.1565 19.05335 605.55 706.75

Massachusettssum totsc4

Variable | Obs Mean Std. Dev. Min Max-------------+-------------------------------------------------------- totsc4 | 220 709.8273 15.12647 658 740

(b) Do scores have a similar relationship with income?

graph twoway scatter testscr avginc, name(cal, replace)graph twoway scatter totsc4 percap, name(mass, replace)graph combine cal mass, cols(1)

(c)

Normalisation

To compare the effect of class size on test scores we need to have a consistent measure of test scores. One way we can get closer to this is to normalise the scores by generating

nts= (ts-mean(ts))/s.d.(ts)Now both sets of test scores will be on the same scale and so are more comparable. What are we implicitly assuming when we make this comparison?

d) How similar are the effects of class size? External Validity?

We can’t test whether the coefficients are statistically different.

Unless… Suppose we run a pooled regression using data from California and Massachusetts (ignoring the other controls)

nsti = b0 + b1(massdum) + b2 (classsize) + b3(massdum x classsize) + e

Source | SS df MS Number of obs = 640-------------+------------------------------ F( 3, 636) = 12.72 Model | 36.1088041 3 12.036268 Prob > F = 0.0000 Residual | 601.891166 636 .946369758 R-squared = 0.0566-------------+------------------------------ Adj R-squared = 0.0521 Total | 637.99997 639 .998435008 Root MSE = .97282

------------------------------------------------------------------------------ tscore | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- tchratio | -.1196539 .0251215 -4.76 0.000 -.168985 -.0703228 massdum | -.3804042 .7076664 -0.54 0.591 -1.770049 1.009241 tch_massdum | .0060908 .0382728 0.16 0.874 -.0690655 .0812471 _cons | 2.350054 .495675 4.74 0.000 1.376696 3.323411------------------------------------------------------------------------------

e) How similar are the effects of log income? External Validity?