appendix 2a. simple regression and multiple regression
DESCRIPTION
By Cheng Few Lee Joseph Finnerty John Lee Alice C Lee Donald Wort. Appendix 2A. Simple regression and multiple regression. Appendix 2A. Simple regression and multiple regression. 2. A.1 INTRODUCTION 2. A.2 SIMPLE REGRESSION Variance of Multiple Regression. - PowerPoint PPT PresentationTRANSCRIPT
Appendix 2A. Simple regression and multiple
regression
ByCheng Few LeeJoseph Finnerty
John LeeAlice C Lee
Donald Wort
Appendix 2A. Simple regression and multiple regression
2. A.1 INTRODUCTION
2. A.2 SIMPLE REGRESSION
Variance of
Multiple Regressionb
2
Appendix 2A. Simple regression and multiple regression
(2.A.1a)
(2.A.1b)
(2.A.2a)
(2.A.2b)
1t t tY a bX
1log logt t tY a b X
1
1 1 1 2cov , 2cov , 2cov , ,
t t t
t t t t t t
Var Y Var a bX
Var a Var bX Var a bX a bX
21t t tVar Y b Var X Var
3
Appendix 2A. Simple regression and multiple regression
(2.A.3)
(2.A.4)
(2.A.5a)
(2.A.5b)
2
21
Variation explained by the explanatory variable
Total variation in the dependent variable
t
t
R
b Var X
Var Y
2 2
11 1
ˆˆ ˆn n
t t t tt t
ESS Y Y Y a bX
11
( ) ˆˆ2 0n
t tt
ESSY a bX
a
1 11
( ) ˆˆ2 0n
t t tt
ESSX Y a bX
b
4
Appendix 2A. Simple regression and multiple regression
(2.A.6a)
(2.A.6b)
11 1
ˆˆn n
t tt t
an b X Y
5
Appendix 2A. Simple regression and multiple regression
(2.A.7)
(2.A.7a)
1
1 1 1 11 1 1 1 1
2 21 11
1 11
21 1
1 1
( ) ( )ˆ
( )
n
tt
n n n n n
t t t t t t tt t t t t
n nn
t ttt tt
n n
t tt t
n Y
X X Y n X Y X Yb
n X Xn X
X X
1
1
[ , ]ˆ[ ]
t t
t
Cov X Yb
Var X
6
Appendix 2A. Simple regression and multiple regression
(2.A.8)
(2.A.8a)
12
2 1 1 11 1 1 1 1 1
2 211 1
1 121 1
2 21 1 1 1 1 1
1 1 1 1 1 1
( )( ) ( )( )ˆ
( )
( )[ ( ) ( ) ] ( )[ ( ) ( )( )]
t n n n n
t t t t tt t t t t t t
n nt
t tt t
t t
n n n n n n
t t t t t t t tt t t t t t t
Y XY X X X YX Y X
an X n X XX X
Y n n X X X n n X Y X X
1
2 21 1
1 1( )
n
n n
t tt t
n X X
ˆa Y Xb
7
Variance of
Equation (2.A.7a) implies that:
(2.A.7b)
Where
b
1121 111
( )ˆ n nt t
t tnt ttt
x yb W y
x
1 1
11 2
11
t t
t t
tt n
tt
x X X
y Y Y
xW
x
8
Variance of
(2.A.7c)
(2.A.9)
b
1 1 11 1
ˆ n n
t t t tt t
b W bx W
2
21 1 1
1 1
21 1 1
1 1
21 1 1
1 1
ˆ ˆ( ) ( )
( )
[( 1) ]
( ) , since 1.
n n
t t t tt t
n n
t t t tt t
n n
t t t tt t
Var b E b b
E W bX W b
E W x b W
E W W x
2 20 1 0 1 1 2 1 2
ˆ( ) [( ) 2( ) ( ) ]Var b E W W W W
9
Variance of b
2 20 1 1 2
2 2 2 20 1 1 2
ˆ( ) ( ) ( )
( ) ( )
Var b E W E W
W E W E
2 21
1
221
1
ˆ( ) ( )
n
t tt
n
tt
Var b W E
W
2112
1 2 221 1 11 1
1
( )
nntt
t n nt t tt t
xW
x x
10
Variance of b
(2.A.10)
(2.A.11)
(2.A.12)
2
211
ˆ( )n
tt
Var bx
2112
211
ˆ( )n
ttn
tt
xVar a
n x
22
11
ˆˆ( , )n
tt
XCov a b
x
11
Multiple Regression
(2.A.13a)
The error sum of squares can be defined as:
Where
1, 1 2, 1t t t tY a bX cX
2 2ˆˆ ( )t t tESS Y Y
1, 1 2, 1ˆˆ ˆ ˆt t t tY a bX cX
12
Multiple Regression
(2.A.14a)
(2.A.14b)
(2.A.14c)
1, 1 2, 10 or t t t
ESSY na b X c X
a
21, 1 1, 1 1, 1 1, 1 2, 10 or t t t t t t
ESSX Y a X b X c X X
b
22, 1 2, 1 1, 1 2, 1 2, 10 or t t t t t t
ESSX Y a X b X X c X
c
13
Multiple Regression
0 = na + b(0) + c(0),
(2.A.15a)
(2.A.15b)
(2.A.15c)
21, 1 1, 1 1, 1 2, 1(0)t t t t tx y a b x c x x
22, 1 1, 1 2, 1 2, 1(0)t t t t tx x a b x x c x
14
Multiple Regression
(2.A.16a)
(2.A.16b)
(2.A.17)
21,21,1
21,2
21,1
1,21,11,22
1,21,1
)())((
)(ˆ
tttt
ttttttt
xxxx
xxyxxyxb
22, 1 1, 1 1, 1 1, 1 2, 1
2 2 21, 1 2, 1 1, 1 2, 1
( )ˆ
( )( ) ( )t t t t t t t
t t t t
x y x x y x xc
x x x x
1 2ˆˆˆ ˆa Y bX cX
15
Multiple Regression
(2.A.13b)
(2.A.18)
(2.A.19)
1, 1 2, 10.2837 0.7564 0.2990 (0.4323) (0.3288) (0.2240)
t t tY X X
ˆ 1.7071(0.7564)(1.8448) (0.2990)(1.6904 ) 0.2837.a
ˆ ˆ( ) ( ) ( )t t t t t tY Y Y Y Y Y
1, 1 2, 1ˆˆ ˆ ˆt t tY a bX cX
16
Multiple Regression
(2.A.20)
where
TSS = Total sum of squares;
ESS = Residual sum of squares; and
RSS = Regression sum of squares.
2 2 2ˆ ˆ( ) ( ) ( ) , TSS ESS RSS
t t t t t tY Y Y Y Y Y
17
Multiple Regression
(2.A.21)
(2.A.22)
Where
and k = the number of independent variables.
2 22
2 2
ˆ ˆ( )RSS1
TSS ( ) ( )t t t
t t t t
Y YR
Y Y Y Y
2
2 ˆ1
( )t
t
RVar Y
22 ˆ
ˆ( ) ttVar
n k
2( )
( )1
tt
Y YVar Y
n
18
Multiple Regression
(2.A.23)
where F(k-1, n-k) represents F-statistic with
k- 1 and n- k degrees of freedom.
2 2 11 (1 )
nR R
n k
2
2( 1, )
1 1
R n kF k n k
R k
19
Appendix 2B.
Instrumental variables and two-stage least squares
ByCheng Few LeeJoseph Finnerty
John LeeAlice C Lee
Donald Wort
Appendix 2B. Instrumental variables and two-stage least squares
2. B.1 ERRORS-IN-VARIABLE PROBLEM2. B.2 INSTRUMENTAL VARIABLES2. B.3 TWO-STAGE, LEAST-SQUARE
21
2. B.1 ERRORS-IN-VARIABLE PROBLEM
(2.B.1)
(2.B.2)
(2.B.3)
ttmjjtj RBAR ,,
*, ,m t m t tR R V
* 2 2, ,( ) ( )m t m t t m VVar R Var R V
22
2. B.1 ERRORS-IN-VARIABLE PROBLEM
(2.B.4)
(2.B.5)
ttmjjtj RBAR *,
*,
*, , ,
*, ,
,
2 2,
( , ) ( , )ˆ( ) ( ) ( )
( , , ) ( , )
( ) ( ) 1 /
m t jt m t t j j m t tj
m t m t t
j m t m t t t j
m t t V M
Cov R R Cov R V B RB
Var R Var R Var V
B Cov R R Cov V B
Var R Var V
23
2. B.2 INSTRUMENTAL VARIABLES
(2.B.6)
(2.B.7)
(2.B.8a)
(2.B.8b)
( , ) ( , ) ( , )j j mCov R Z B Cov R Z Cov Z
*
( , ) ( , )ˆ( , ) ( , )
j jj
m m
Cov R Z Cov R ZB
Cov R Z Cov R Z
12101 EYAAY
2121102 EZBYBBY
24
2. B.2 INSTRUMENTAL VARIABLES
(2.B.9a)
(2.B.9b)
(2.B.10a)
(2.B.10b)
1222101 EZAYAAY
2121102 EZBYBBY
1 0 1 2 2 2 3 3 1Y A AY A Z A Z E Y
2121102 EZBYBBY
25
2.B.3 TWO-STAGE, LEAST-SQUARE
(2.B.11a)
(2.B.11b)
(2.B.10′a)
(2.B.10′b)
133221101 EZCZCZCCY
233221102 EZDZDZDDY
133222101ˆ EZAZAYAAY
2121102ˆ EZBYBBY
26
2.B.3 TWO-STAGE, LEAST-SQUARE
(2.B.12a)
(2.B.12b)
21 1 10.2399 0.8198 1.9004 , 0.3449,
(0.1012) (0.2802) (1.245)Y Z Z R
22 1 20.0746 0.1133 0.7849 , 0.4240,
(0.0195) (0.0541) (0.2405)Y Z Z R
27