Multiple Regression

Fundamentals

Basic Interpretations

Statistical Models

• E(Y) is a conditional mean, a ‘regression’

• A ‘linear’ regression is:

• Then usually we have:

• And the other assumptions about the errors

1122110)( KK XXXYE

211110 )( VarXXY KK

The fitted values

• Where the residual sum of squares

• Is made as small as possible (least squares)

11110ˆ

KK XbXbbY

2)ˆ( YYRSS

Analysis of Variance• Source SS df MS• Regression ESS K-1• Residual RSS n-K MSE

• Total TSS n-1

• The main purpose of such a display is to present the MSE

• The ‘Omnibus F test’ is rarely used as it tests:

• This null hypothesis is rarely of scientific interest• (It is given in most regression output. So what!)

0: 1210 KH

Interpretation

• The meaning of the ‘coefficients’ is different for every model.

• Be careful! We tend to use the same symbols to conceptualize the models but the coefficients can mean very different things EVEN when they are coefficients for the same variables

Water consumption example

• Y is water81• is income• is water80• We write:

• And:

• But any one coefficient is interpreted in light of the others in the model.

• See Hamilton for the details

1X

2X

110)( XYE

22110)( XXYE

Notice that:

• In the second model,

• But in the first model,

• This looks complicated, but it is central to understanding and interpreting

)())1(( 22110221101 XXXX

))(())1(( 1101101 XX

For example, if a household has

• Then the second model says that the expected water consumption for this household is:

• If another household has:

• Then:

• The difference in expected water consumption is:

1040 21 XandX

210 1040)( YE1041 21 XandX

210 1041)( YE

1

But!

• This is true only if the previous water consumption was the SAME in the 2 households

• This addition part to the statement is only required with the second model, but not with the first simpler model that did not involve previous water consumption