old least squares (ols) · 5/3/2011 · 3/3/2014 1 3 march 2014 1 cds m phil econometrics...

3/3/2014

1

3 March 2014 1

CDS M Phil Econometrics

Vijayamohanan Pillai N

CDS M Phil Econometrics Vijayamohan

Vijayamohan: CDS MPhil: Econometrics

3 March 2014 Vijayamohan: CDS MPhil: Econometrics

2

Old Least Squares Old Least Squares (OLS)(OLS)

CDS M Phil Econometrics Vijayamohan

3 March 2014 Vijayamohan: CDS MPhil: Econometric s 3

Multiple Regression Analysis

• y = β0 + β1x1 + β2x2 + . . . βkxk + u


4


Define column vectors of n observations on yand K – 1 variables.

iikiii uxxxfy += ),...,,( 32

iikkiii uxxxy +++++= ββββ ...33221

This can be expressed as

i

k

jijji uxy +=∑

=1

β

General form of the multiple linear regression mode l:


5

iikk3i32i21i ux...xxy +β++β+β+β=i = 1, …, n

1k1k1331221111 ux...xxxy +β++β+β+β=

2k2k2332222112 ux...xxxy +β++β+β+β=

3k3k3333223113 ux...xxxy +β++β+β+β=

……………………….

nnkk3n32n21n1n ux...xxxy +β++β+β+β=



6

+

β

ββ

=

=

n

2

1

k

2

1

nk2n1n

k22221

k11211

n

2

1

u

...

u

u

...

x...xx

............

x...xx

x...xx

y

...

y

y

y

= Xβ + u

x1 is a column of ones:

[ ] [ ]TT1n11 11xx LL =

K – 1 variables and 1 constant


3/3/2014

2

3 March 2014Vijayamohan: CDS MPhil: Econometrics 7

The Classical Assumptions

A1. Linearity

Linearity of the parameters (and disturbance)Linearity of the parameters (and disturbance)

Sometimes models which appear to be nonSometimes models which appear to be non--linear linear can be estimated using the leastcan be estimated using the least--squares procedure . squares procedure.

∏=

ββ=n

2k

uk expxexpy k1

u)xln()yln(n

2kkk1 ∑

=+β+β=


8


A2. E(u) = 0

The disturbance term has a zero expectation

=

=

0

0

0

u

u

u

E)u(E

n

2

1

MM


9


A3. Nonstochastic Regressors

X is a non-stochastic n x k matrix.

That is, it consists of a set of fixed numbers.


10


What does ‘Nonstochastic’ Mean?

Assumption A3 comes in two forms:

a weak version and a strong version.

• Strong Version: The explanatory variables

should be non - stochastic.

• Weak Version: The explanatory variables are random but distributed independently of the error term.


11


What does ‘Nonstochastic’ Mean?

The term ‘nonstochastic’ essentially means

that the variables are determined “outside”

the context of the regression

(this is another reason why we use the term

independent when referring to explanatory

variables)


12

The Classical Assumptions: Homoscedasticity

Variance

3/3/2014

3


13


A4. Spherical Disturbances

n2T I)uu(E)u(Var σ==

=

)u(E)uu(E)uu(E

)uu(E)u(E)uu(E

)uu(E)uu(E)u(E

)uu(E

2n2n1n

n22212

n12121

T

L

MOMM

L

L

σ

σσ

=

2

2

2

00

00

00

L

MOMM

L

L

n2Iσ=


14

The Classical Assumptions: Spherical disturbances

n2T I)uu(E)u(Var σ==

=

)u(E)uu(E)uu(E

)uu(E)u(E)uu(E

)uu(E)uu(E)u(E

)uu(E

2n2n1n

n22212

n12121

T

L

MOMM

L

L

σ

σσ

=

2

2

2

00

00

00

L

MOMM

L

L

n2Iσ=

Spherical disturbances ⇒⇒

(i)

and

(ii)

22ii )u(E)u(Var σ=≡ n,...,1i =∀ homoskedasticity

0)u,u(E)u,u(Cov jiji =≡ ji ≠∀ No autocorrelation


15

..

x1 x2

Homoscedasticity

E(y|x) = β0 + β1x

y


x

f(y|x)


16

.

xx1 x2

f(y|x)

x3

.. E(y|x) = β0 + β1x


Heteroscedasticity


17


A5. Identifiability

No exact linear relationships among the variables (no perfect multicollinearity).

Specifically,

X is n x K with rank K. (X has full (column) rank)

⇒ the columns of X are linearly independent.

NonNon--singular Xsingular X


18


A5. Identifiability

No exact linear relationships among the variables (no perfect multicollinearity).

Implicit within this assumption are the requirements of more observations than variables (micronumerosity) and sufficient variability in the values of the regressors.

3/3/2014

4


19


A6. Normality

Final assumption: the disturbances are normally distributed.

useful for the purposes of statistical inference

but not necessary for analysing the properties of the estimators.

),0(~ 2nN Iu σ


20

The Classical / Old / Ordinary Least squares (OLS)

Sample counterpart of the k-variable regression mod el:

uˆXy +β=

Where and are the sample counterparts of β and u.

OLS aims to minimise the difference between

an observed value of yi and its predicted value.

⇒ the error be the least.

β u

yyu −=β= ˆXy


21


Specifically, the problem is to find an estimator that minimises the error sum of squares:

)yy()yy(uu −′−=′

yXˆ2ˆXXˆyy ′β′−β′β′+′=


22

A necessary condition for a minimum is that the fir st-order conditions equal zero.


0yX2ˆXX2ˆ

uu=′−β′=

β∂′∂

Therefore, rearrange to give the normal equation

yXˆX)X( ′=β′

yXX)X(ˆ 1 ′′=β −


23


yXX)X(ˆ 1 ′′=β −

For a 2-variable model,

0ar(X)V ; )X(Var

)y,X(Covˆ >=β


24


XTX is invertible provided X has full rank.

(Why?)

XX2ˆˆ

uu2

′=β′∂β∂′∂

XTX is positive definite for a minimum.

3/3/2014

5


25

Statistical Properties of Least Squares Estimator

Mean of β

uXX)X(β

yXX)X(ˆ

1

1

′′+=

′′=β−

−

β)(E =β Property of unbiasedness

[key assumptions: E(u) = 0 and non-stochastic regressors]


26


Variance of β [ ])βˆ)(β(E)(Var ′−β−β=β

Noting that uXX)X(ˆ 1 ′′=β−β −

then])XX(XuuX)XX[(E)(Var 11 −− ′′′′=β

[key assumptions: 22ii )u(E)u(Var σ=≡

0)u,u(E)u,u(Cov jiji =≡and

for all i, i≠j.

12 )XX()(Var −′σ=β


27


Variance of β

12 )XX()(Var −′σ=β

In the scalar case∑

σ=β2i

2

2x

)(Var

MinimumVariance (best)

property


28


Unbiased Estimator of σ2

For statistical inference we require an estimate of

Variance of

kn

u

sˆ

n

1i

2i

22

−==σ∑=

and therefore, σ2.

An unbiased estimator of σ2 is:

β

12 )XX()(Var −′σ=β

Its square root = the standard error of the regression


29

( )

( )

(ESS) squares of sum error the is u

(RSS) squares of sum regression )(explained the is yy

(TSS) squares of sum total the is yy

2i

2

i

2

i

∑

∑

∑

−

−

OLS: Goodness of Fit

( ) ( ) ∑∑∑ +−=− 2i

2i

2i uyyyy

What proportion of the total variation of y is accounted for by the variation in X?

In terms of sums of squares

TSS = RSS + ESS


30

OLS: Measure of Goodness of Fit

Coefficient of determination: TSS

RSSR2 =

or TSS

ESS1R2 −=

• R2 never decreases when a new X variable is added to the model

– This can be a disadvantage when comparing models

• What is the net effect of adding a new variable?

3/3/2014

6


31

• What is the net effect of adding a new

variable?

– We lose a degree of freedom when a new X

variable is added

– Did the new X variable add enough

explanatory power to offset the loss of one

degree of freedom?



32


Adjusted RAdjusted R22

• Shows the proportion of variation in Y explained by all X variables adjusted for the number of X variables used

(where n = sample size, k = number of parameters)

)1n/(TSS

)kn/(ESS1R2

adj −−

−=

−−−−=

kn

1n)R1(1 2


33


Adjusted RAdjusted R22

(where n = sample size, k = number of parameters)

– Penalize excessive use of unimportant independent variables

– Smaller than r2

– Useful in comparing among models

)1n/(TSS

)kn/(ESS1R2

adj −−−=

−−−−=

kn

1n)R1(1 2


34


The relationship between R 2 and 2R

222 Rkn

1n

kn

k1)R1(

kn

1n1R

−−

+−−

=−−−

−=

Two other measures of fit are the Schwartz criterion (SC)

nlnn

k

n

ESSlnSC +=

and the Akaike information criterion (AIC)

n

k2

n

ESSlnAIC +=


OLS: Statistical Inference

we know that the expected value is

β)(E =β

And we know that a suitable estimate of the variance of

the slope parameter is

12 )XX()(Var −′σ=β

12 )XX(s)(Var −′=β

kn

u

sˆ

n

1i

2i

22

−==σ∑=


36


In order to make meaningful inference the variable must be normally distributed.

One of the assumptions introduced above was:

)I,0(N~u n2σ

This means that u has a multivariate normal distribution.

(If it is not, then n must be large – i.e. becomes normally distributed by virtue of the central limit theorem.

3/3/2014

7


37


This implies that the sampling error ( - β) is related to u as follows:

β

uXX)X(

βu)(XβXX)X(

βyXX)X(βˆ

1

1

1

′′⇒

−+′′⇒

−′′=−β

−

−

−

which implies that the sampling error is also multivariate normally distributed:

))XX(,0(N~)β( 12 −′σ−β


38


In hypothesis testing we are concerned about testing whether a particular finding is compatible with some stated hypothesis or not.

A common hypothesis test is formulated as

kk0 β:H β=

kk1 :H β≠β


39


Using our earlier result, if we were looking at the kth diagonal element of (X TX)-1 we obtain

)))XX((,0(N~)( kk12

kk−′σβ−β

where kk1))XX(( −′ is the (k,k) element of (XTX)-1

If we define the ratio z k by dividing

by its standard deviation

kk12

kkk

))XX((

ˆz

−′σ

β−β=

where )1,0(N~zk

kˆ β−β


40


If we do not know σ2, it is natural to replace it with the estimate s2.

This generates the t-ratio

)(SE

ˆ

))XX((s

ˆt

k

kk

kk1T2

kkk β

β−β=β−β=−

where SE( k) denotes the standard error of k. This is distributed as t c

α/2;(n-k) .β β


41


An Example: Investment Function

Theory of the behaviour of investors:They care only about real interest rates.

Real I = f(real gdp, an interest rate (the 90-day T-bill rate), inflation (change in the log of CPI), real disposable personal income, trend)


42



Hypothesis:

Equal increases in interest rates and rate of

inflation would have no independent effect on

investment.

So the Ho : β2 + β3 = 0.The test statistic is:

)ˆ(SE

0ˆˆt

32

32

β+β−β+β

=

3/3/2014

8


43



SE(.) = [0.003192 + 0.002342 +2(–3.718 x 10–6 )]1/2

= 0.002866

845.1002866.0

00331.00086.0t −=+−=

95% critical value: t(203 – 5) = 1.96


44


Testing Linear Restrictions

As well as testing restrictions on individual regression coefficients we may wish to test linear combinations of them. For example

1:H 320 =β+β

430 :H β=β

(c)

(b)

(a)

0...:H k4320 =β==β=β=β


45


Testing Linear Restrictions: The F Test

The t-distribution can be used to test a single null hypothesis

If we want to conduct a “joint” test then we can no longer use the t distribution

For example, suppose we want to test whether all the explanatory variables in the model are significantly different from zero


46



F-Test of Entire Equation (“Testing the Joint Significance of the Explanatory Variables”)

H0: β2 = β3=…= βk= 0 Equivalently: H 0: R2 = 0

H1: H0 not true (at least one of the β is nonzero)

Cannot say: “if the coefficients are individuallyinsignificant this means they must be jointly

insignificant”


47



F-Test of Entire Equation (“Testing the Joint Significance of the Explanatory Variables”)

H0: β2 = β3=…= βk= 0 Equivalently: H 0: R2 = 0

H1: H0 not true (at least one of the β is nonzero)

kn

ESS1k

RSS

F

−

−=

kn

R11k

R

F 2

2

−−

−=or


48


Testing Linear Restrictions: Multiple Restriction F Test

This time suppose we wish to test whether a subset of regression coefficients are zero. In this case

[ ]

uˆXˆX

uˆ

ˆXXy

3322

3

232

+β+β⇒

+

ββ

= H0: β3= 0

H1: H0 not true

3/3/2014

9


49



[ ]

uˆXˆX

uˆ

ˆXXy

3322

3

232

+β+β⇒

+

ββ

= H0: β3= 0

H1: H0 not true

To find the change in the fit of a multiple regress ion when an additional variable x 3 is added to a model that already contains K – 1 variables:

J = 1 linear restriction


50



Thus the unrestricted regression is:

[ ]

uˆXˆXy

uˆ

ˆXXy

3322

3

232

+β+β=

+

ββ=

H0: β3= 0

H1: H0 not true

[ ]

*22

*2

32

uˆXy

u0

ˆXXy

+β=

+

β=And the restricted regression:


51


The residuals from the restricted regression:

22*ˆXyu β−=

whereas the residuals from the unrestricted regress ion:

3322ˆXˆXyu β−β−=

Essentially we wish to see whether the reduction in the ESS is 'large enough' to suggest that X 3 is significant.



52



The test statistic is

)kn/(uu

J/)uuuu(F

T

T*

T*

−−= J = 1; K= 2

)kn/()R1(

J/)RR(F

2

2*

2

−−−=or

Residuals from the restricted regression: 22*ˆXyu β−=

Residuals from the unrestricted regression:

3322ˆXˆXyu β−β−=


53


An Example: Production Function



54


• Cobb-Douglas:

• Translog:

• Translog function relaxes the CD assumption of a unitary elasticity of substitution

• CD obtained by the restriction:

uKlnLlnyln 21 +β+β+α=

Unrestricted model: K = 6

uKlnLln)K(ln2

1)L(ln

2

1KlnLlnyln 5

24

2321 +β+β+β+β+β+α=

0543 =β=β=β

J = 3

3/3/2014

10

3 March 2014 Vijayamohan: CDS MPhil: Econometrics55

Estimated production Functions


56


CD or Translog?

The F-statistic for the hypothesis of CD model:

Ho: CD model is appropriate

Critical value: F(3,21) = 3.07

)kn/(uu

J/)uuuu(F

T

T*

T*

−−=

768.121/67993.0

3/)67993.085163.0(F =

−=

Conclusion?

3 March 2014 Vijayamohan: CDS MPhil: Econometrics57


CD function: Constant Returns to Scale?

1:Ho 32 =β+β

22

t1157.0)00961.0(200728.001586.0

)13757.0603.0(F ==

−+−+=

• Hypothesis of constant returns to scale:

• Equivalent to a restriction that the two coefficients of CD function sum to unity

•F-test with J = 1 and K = 3

• Critical value: F(1,24) = 4.26 Conclusion?3 March 2014 Vijayamohan: CDS MPhil:

Econometrics58


CD function: Constant Returns to Scale?

• Hypothesis of constant returns to scale:

• Can have t-test:

• Critical value: ‘rule of thumb’

1:Ho 32 =β+β

3402.0)]00961.0(200728.001586.0[

)13757.0603.0(t

2/1−=

−+−+=

)1ˆ(SE

1ˆˆt

32

32

−β+β−β+β

=

Conclusion?

= – (F1/2)


59

Test of Structural Break: The Chow test

• To test structural change in time series data


60

5010

015

020

025

030

0S

avin

gs

1000 2000 3000 4000 5000Income


Savings and personal disposable income (billions of dollars) US, 1970 – 1995.

3/3/2014

11


61

5010

015

020

0S

avin

gs

500 1000 1500 2000 2500Income

5010

015

020

0S

avin

gs

500 1000 1500 2000 2500Income

1970 – 1981


150

200

250

300

Sav

ing

s

2000 3000 4000 5000 6000Income

150

200

250

300

Sav

ing

s

2000 3000 4000 5000 6000Income

1982 – 1995

Point of structural break: 1982 (assumed)

5010

015

020

025

030

0S

avin

gs

1000 2000 3000 4000 5000Income

5010

015

020

025

030

0S

avin

gs

1000 2000 3000 4000 5000Income


62


Steps

1. Run the two sub-period regressions and the full-period regressionFull-period: St = a0 + b0Yt n = 26 (n1 + n2)Sub-period 1: St = a1 + b1Yt n1 = 12 Sub-period 2: St = a2 + b2Yt n2 = 14

2. The pooled regression = restricted regression, obtained under the restrictions that a1 = a2 (= a0) and b1 = b2 (= b0).

3. Get the restricted ESS from it (ESSR): )uu( **′


63


1. Run the two sub-period regressions and the full-period regressionFull-period: St = a0 + b0Yt n = 26 (n1 + n2)Sub-period 1: St = a1 + b1Yt n1 = 12 Sub-period 2: St = a2 + b2Yt n2 = 14

4. The sub-period regressions = unrestricted regressions.

5. Get the unrestricted ESS from them: : ESSUR = ESS1 + ESS2.

)uu( ′

Steps


64


Sub-period 1: St = a1 + b1Yt n1 = 12 Sub-period 2: St = a2 + b2Yt n2 = 14

4. The sub-period regressions = unrestricted regressions.

5. Get the unrestricted ESS from them: : ESSUR = ESS1 + ESS2.

)uu( ′

)k2n/(uu

k/)uuuu(F

T

T*

T*

−−

= k = j; n – 2k = (n1 – k) + (n2 – k)

Steps



Pooled Regression Result: 1970 – 1995

)uu( **′Restricted ESS: = 23248.3


Sub-period 1: 1970 – 1981


Unrestricted ESS1 = 1785.033

3/3/2014

12


Sub-period 2: 1982 – 1995


)uu( ′

Unrestricted ESS2 = 10005.221

Unrestricted ESS = ESS1 + ESS2 = 1785.033 + 10005.221 = 11790.254


68


)k2n/(uu

k/)uuuu(F

T

T*

T*

−−

=

69.1022/254.11790

2/)254.117903.23248(F =

−=

k = j; n – 2k = (n1 – k) + (n2 – k)

Critical value: F(2,22) at 1% α = 5.72

H0 = a1 = a2 = a0 and b1 = b2 = b0: Parameter stability

Conclusion?

old least squares (ols) · 5/3/2011 · 3/3/2014 1 3 march 2014 1 cds m phil econometrics...

Documents