1 multivariate linear regression models shyh-kang jeng department of electrical engineering/...
TRANSCRIPT
![Page 1: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/1.jpg)
11
Multivariate Linear Multivariate Linear Regression ModelsRegression Models
Shyh-Kang JengShyh-Kang JengDepartment of Electrical Engineering/Department of Electrical Engineering/Graduate Institute of Communication/Graduate Institute of Communication/
Graduate Institute of Networking and MultiGraduate Institute of Networking and Multimediamedia
![Page 2: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/2.jpg)
22
Regression AnalysisRegression Analysis
A statistical methodology A statistical methodology – For predicting value of one or more For predicting value of one or more
response (dependent) variablesresponse (dependent) variables– Predict from a collection of predictor Predict from a collection of predictor
(independent) variable values (independent) variable values
![Page 3: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/3.jpg)
33
Example 7.1 Fitting a Straight LineExample 7.1 Fitting a Straight Line
Observed dataObserved data
Linear regression modelLinear regression model
zz11 00 11 22 33 44
yy 11 44 33 88 99
110)( responseMean zYE
![Page 4: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/4.jpg)
44
Example 7.1 Fitting a Straight LineExample 7.1 Fitting a Straight Line
0 321 4 5
z
y
2
4
6
8
10
0
![Page 5: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/5.jpg)
55
Classical Linear Regression ModelClassical Linear Regression Model
1,
1
1
1
0
2
1
1
0
21
22221
11211
2
1
110
j
nrnrnn
r
r
n
rr
z
zzz
zzz
zzz
Y
Y
Y
zzY
εβZY
![Page 6: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/6.jpg)
66
Classical Linear Regression ModelClassical Linear Regression Model
Iεεε
ε2
2
)'()Cov(
0)(
,0),Cov(
)Var(
0)(
E
E
kj
E
kj
j
j
![Page 7: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/7.jpg)
77
Example 7.1Example 7.1
43210
11111'
98341'
,,
1
1
1
,
5
2
1
1
0
51
21
11
5
2
1
Z
y
εβZY
εZβY
z
z
z
Y
Y
Y
![Page 8: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/8.jpg)
88
Examples 6.6 & 6.7Examples 6.6 & 6.7
level 1% at the rejected is 0:
27.13)01.0(5.195/10
2/78
/SS
)1/(SS
53-3)2(3 d.f.,10SS
213 d.f. ,78SS
128SS,216SS
011
11
121
222
33
444
444
44
444
213
20
969
3210
5,2
H
Fgn
gF
res
tr
res
tr
meanobs
![Page 9: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/9.jpg)
99
Example 7.2 One-Way ANOVAExample 7.2 One-Way ANOVA
11100000
00011000
00000111
11111111
'
21320969'
otherwise0
population from isn observatio theif1
,,,
,,
3322110
3322110
333222111
Z
Y
jz
zzzY
eXeXeX
j
jjjjj
jjjjjj
![Page 10: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/10.jpg)
1010
Method of Least SquaresMethod of Least Squares
βZyyyyβZyε
bβ
ZbyZby
b
b
b
ˆˆ fitted,ˆˆˆ
ˆˆˆˆ residuals
)(minargˆ
'
)(
minimize toas so Selects
110
1
2110
jrrjjj
n
jjrrjj
zzyε
S
zbzbbyS
![Page 11: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/11.jpg)
1111
Result 7.1Result 7.1
βZyyHIyyyyεε
yZyZεyεZ
yHIyyε
ZZZZHHyβZy
yZZZβZ
ˆ''ˆ'ˆ'ˆ
ˆ'',0ˆ'ˆ,0ˆ'
ˆˆ
')'(,ˆˆ
')'(ˆ,1rank full has 1
1
nr
![Page 12: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/12.jpg)
1212
Proof of Result 7.1Proof of Result 7.1
)(minargˆ
0)(')''('
)'''(''ˆ
ˆ'ˆ2
ˆ''ˆˆ'ˆ
')(
ˆˆˆˆ
''ˆ
1
1
1
bβ
ZZyZZZZZIy
ZZZZZIyZβZy
bβZβZy
bβZZbββZyβZy
ZbyZbyb
bβZβZyZbβZβZyZby
yZZZβ
bS
S
![Page 13: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/13.jpg)
1313
Proof of Result 7.1Proof of Result 7.1
yyyyyZZZZIy
yZZZZZZZZZZZZIy
yZZZZIZZZZIy
yyyyεε
εZβεy
yZZZZIZ
βZyZyyZεZ
ˆ'')''('
)''''''2('
)'')(''('
)ˆ()'ˆ(ˆ'ˆ
0ˆ''ˆˆ'ˆ
0)''('
ˆ')ˆ('ˆ'
1
111
11
1
![Page 14: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/14.jpg)
1414
Example 7.1 Fitting a Straight LineExample 7.1 Fitting a Straight Line
Observed dataObserved data
Linear regression modelLinear regression model
zz11 00 11 22 33 44
yy 11 44 33 88 99
110)( responseMean zYE
![Page 15: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/15.jpg)
1515
Example 7.3Example 7.3
6ˆ'ˆ equations of sum residual
'01210ˆˆ
'97531ˆˆ,21ˆ
2
1''
ˆ
ˆˆ,
70
25'
1.02.0
2.06.0',
3010
105'
1
1
0
1
εε
yyε
βZy
yZZZβyZ
ZZZZ
zy
![Page 16: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/16.jpg)
1616
Coefficient of DeterminationCoefficient of Determination
n
jj
n
jj
n
jj
n
jj
n
jj
n
jj
n
jj
n
jj
n
jj
n
jj
yy
yy
yyR
yyyy
ynyn
yyyy
1
2
1
2
1
2
1
2
2
1
2
1
2
1
2
22
111
ˆˆ
1
ˆˆ
ˆ'ˆˆˆ'ˆ'
ˆˆˆˆ'00ˆ'
ˆ'ˆˆ'ˆ)ˆˆ()'ˆˆ()ˆˆ()'ˆˆ('
0ˆ'ˆ
εεyyyy
ε1εZ
εεyyεyεyyyyyyyyy
εy
![Page 17: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/17.jpg)
1717
Geometry of Least SquaresGeometry of Least Squares
plane modelˆ,plane modelon ˆˆ),(minargˆ
)()'()(
plane) modelin (vector - vector)observed(
1
1
1
)( 2
1
1
21
11
10
εβZybβ
ZbyZbyb
Zby
εZβY
ZβY
bS
S
z
z
z
z
z
z
E
nr
r
r
r
n
![Page 18: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/18.jpg)
1818
Geometry of Least SquaresGeometry of Least Squares
![Page 19: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/19.jpg)
1919
Projection MatrixProjection Matrix
βZyZZZZyqqyqq
qqq
y
qqZeZeZZZZ
eeZeZeqqZeq
eeeeeeZZ
eeeeeeZZ
ˆ''
is ,,,
by dconstructe plane model on the of projection
'''
'
111'
'
11
1
'1
1
'
121
1
1
'1
1
'11
'2/12/1'2/12/1'2/1
'
1
'22
2
'11
1
1
'1
'222
'111
r
iii
r
iii
r
r
iii
r
iiii
ikkkikikikikiiii
rrr
rrr
![Page 20: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/20.jpg)
2020
Result 7.2 Result 7.2
eduncorrelat are ˆ and ˆ
)(,1
)('
)1(
ˆ'ˆ
1ˆ'ˆ
''ˆCov,0ˆ
)(ˆˆ
')ˆCov(,ˆ
''ˆ,
222
2
212
12
1
εβ
YHIYεε
εε
HIZZZZIεε
YHIβZYε
ZZβββ
YZZZβεZβY
sErnrn
s
rnE
E
E
![Page 21: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/21.jpg)
2121
Proof of Result 7.2Proof of Result 7.2
''
''')Cov('')ˆCov(
0)('')ˆ(
'')Cov('')ˆ(Cov
)('')ˆ(
''
''''ˆ
'')(''''ˆ
12
11
1
1211
1
1
11
111
ZZZZI
ZZZZIεZZZZIε
εZZZZIε
ZZZZZεZZZβ
βεZZZββ
εZZZZI
εZβZZZZIYZZZZIε
εZZZβεZβZZZYZZZβ
εZβY
EE
EE
![Page 22: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/22.jpg)
2222
Proof of Result 7.2Proof of Result 7.2
)1(
''tr''tr
)'(''tr'ˆˆ
'''tr
'''tr'''
''''''ˆˆ
0''''
'')'(''ˆˆ)ˆ,ˆCov(
2
12212
1
1
11
11
112
11
rn
tr
EE
EE
ZZZZIZZZZI
εεZZZZIεε
εεZZZZI
εZZZZIεεZZZZIε
εZZZZIZZZZIεεε
ZZZZIZZZ
ZZZZIεεZZZεββεβ
![Page 23: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/23.jpg)
2323
Result 7.3Result 7.3Gauss Least Square TheoremGauss Least Square Theorem
βc
Ya'
βc
βcβcβcβc
IεεεZβY
'for unbiased arethat
form theofestimator all among
variancepossiblesmallest thehas ' of
estimatoran as ˆˆˆˆ'
)Cov(,0)(,
2211
1100
2
nn
rr
YaYaYa
E
![Page 24: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/24.jpg)
2424
Proof of Result 7.3Proof of Result 7.3
(BLUE) ˆ'*'' when minimum is
**'*'*
**'**
')'Var(''Var'Var
'*',*''''ˆ'
')ˆ'(
'''')''()'(
,' ofestimator unbiasedan as 'For
2
2
2
1
βcYaYa
aaaaaa
aaaaaa
aIaεaεaZβaYa
cZaYaYZZZcβc
βcβc
cZaβcZβaεaZβaYa
βcYa
E
EE
![Page 25: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/25.jpg)
2525
Result 7.4Result 7.4
21
22
22
121
1
2
:ˆ'ˆˆ
ofestimator likelihood maximum:ˆ
ˆ oft independen
',:ˆ
''ˆestimator squaresleast the
as same theis ofestimator likelihood maximul
),(:,
rn
r
n
n
N
N
εε
ZβYε
ZZββ
YZZZβ
β
I0εεZβY
![Page 26: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/26.jpg)
2626
Proof of Result 7.4Proof of Result 7.4
oft independen,''ˆ
)()'(minarg,maxarg
, fixedFor
2
1
2
1
2
1,
1
2
2
2/)()'(2/
2/'2/
2/
1
2
2
222
yZZZβ
ZβyZβyβ
β
ββ
ZβyZβy
εε
L
e
eeL
nn
nn
n
j
j
![Page 27: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/27.jpg)
2727
Proof of Result 7.4Proof of Result 7.4
n
n
L
εε
βZyβZy
β
ˆ'ˆ
'ˆ'ˆ
),ˆ(maxargˆ 22
2
![Page 28: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/28.jpg)
2828
Proof of Result 4.11Proof of Result 4.11
SxxxxΣ
ΣΣμ
xμ
μxΣμxxxxxΣ
Σμ
xxxxΣ
n
n
n
eL
n
L
n
jjj
nnp
n
jjj
n
jjj
1'
1ˆ
2
1),ˆ(
ˆ
'2
1'tr
2
1
:),( ofExponent
1
'tr
2/2/
1
1
1
1
1
![Page 29: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/29.jpg)
2929
Proof of Result 7.4Proof of Result 7.4
''|'
|'
')Cov(
ˆ
Cov
''
''
ˆ
1
1
2
1
1
ZZZZI0
0ZZ
AεA
ε
β
Aεαε
ZZZZI
ZZZ
0
β
ε
β
![Page 30: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/30.jpg)
3030
Proof of Result 7.4Proof of Result 7.4
'11
'22
'11
1
1121
211
1
2
121
1
''
0,1
''tr
1''tr
1,0
''''
''
rnrn
nrnrnrn
n
rn
eeeeeeZZZZI
ZZZZI
ZZZZI
ee
ZZZZIZZZZI
eeZZZZI
![Page 31: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/31.jpg)
3131
Proof of Result 7.4Proof of Result 7.4
2
122
12
22
1
1
1
'12
2
2''
1
'1
'2
'1
1
2
1
:
''''ˆ'ˆˆ
),0(t independen :
)Cov(),Cov(
)Cov(,:
rnrn
rn
iii
i
ikkiki
rn
rnrn
VVV
n
NV
VV
N
V
V
V
εeeεεZZZZIεεε
eεe
V0ε
e
e
e
V
![Page 32: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/32.jpg)
3232
22 Distribution Distribution
)2,12/on with distributi (Gamma
0,0
0,)2/(2
1)(
(d.f.) freedom of degrees :,
)1,0(:);,(:
,),,(:),,(:
2
22/12/22/2
2
1
2
2
2222
2111
2
n
enf
x
NX
ZNX
NXNX
n
nn
i i
ii
i
iii
![Page 33: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/33.jpg)
3333
Result 7.5Result 7.5
i
i
rnrii
rnr
n
s
Fr
Fsr
N
ˆ toingcorrespond
' ofelement diagonal:)ˆ(Var
)()1()ˆ(Varˆ
for intervals confidence )%-(1100 usSimultaneo
)1(ˆ''ˆ
for region confidence )%-(1100
),0(:,
12^
1,1
^
i
1,12
2
ZZ
ββZZββ
β
IεεZβY
![Page 34: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/34.jpg)
3434
Proof of Result 7.5Proof of Result 7.5
)()1('ˆ''ˆ
region Confidence
:)1/(1
)1/('
:1
:'),,0(:
')ˆCov(')Cov(
0)(,ˆ'
1,12
1,12
21
22
21
221
22/12/1
2/1
rnr
rnr
rn
rr
Fsr--
Frnsrn
r
srn
N
E
VVββZZββ
VV
VVIV
IZZβZZV
VββZZV
![Page 35: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/35.jpg)
3535
Example 7.4 (Real Estate Data)Example 7.4 (Real Estate Data)20 homes in a Milwaukee, Wisconsin, 20 homes in a Milwaukee, Wisconsin, neighborhoodneighborhood
Regression modelRegression model
dollars) of (thousands valueassessed:
feet) squared of (hundereds size dwelling total:
dollars) of (thousands price selling:
2
1
22110
z
z
Y
zzY jjj
![Page 36: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/36.jpg)
3636
Example 7.4Example 7.4
647.0,556.0:for interval confidence %95
285.0110.2045.0)ˆ(Var025.0ˆ
834.0,045.0634.2967.30ˆ
285.00067.0,785.00512.0,88.71523.5
473.3,
045.0
634.2
967.30
''ˆ
0067.00172.01463.0
0512.02544.0
1523.5
'
2
2
^
172
22
)285.0(1
)785.0()88.7(
1
1
t
Rzzy
sss
syZZZβ
ZZ
![Page 37: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/37.jpg)
3737
Result 7.6Result 7.6
yZZZβ
βZyβZyβZyβZy
ZZ
εβZβZεβ
βZZεZβY
I0εβ
ZZ
β
''ˆ
)ˆ()'ˆ()ˆ()'ˆ(
)(SS)(SS
),(:,
)()/()(SS)(SS
if 0: rejects test ratio Likelihood
11
11)1(
)1(1)1(1
1
)2(2)1(1)2(
)1(21
221
')2(
1,21
)2(0
resres
nrqq
rnqrresres
N
Fs
qr
H
![Page 38: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/38.jpg)
3838
Effect of RankEffect of Rank
In situations where In situations where ZZ is not of full rank, is not of full rank, rrank(ank(ZZ)) replaces replaces r+1r+1 and and rank(rank(ZZ11)) replaces replaces q+1q+1 in Result 7.6 in Result 7.6
![Page 39: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/39.jpg)
3939
Proof of Result 7.6Proof of Result 7.6
)ˆ()'ˆ(ˆ and ''ˆat occurs
maximum the where,ˆ2
1),(max
and ,0:Under
)ˆ()'ˆ(ˆ and ''ˆat occurswhich
ˆ2
1
2
1max),(max
)1(1)1(1211
111(1)
2/
12/
2)1(
,
)1(1)2(0
21
2/2/
2/)()'(2/
,
2
,
2(1)
2
22
βZyβZyyZZZβ
β
εβZYβ
βZyβZyyZZZβ
β
β
ZβyZβy
ββ
n
nn
n
nn
nn
eL
H
e
eL
![Page 40: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/40.jpg)
4040
Proof of Result 7.6Proof of Result 7.6
1,2
122
12
122
21
2
221
22210
2/
2
221
2/
2
21
2
,
2)1(
,
)2(0
:,:ˆ,:ˆ
)/()(SS)(SS
)1/(ˆ)/(ˆˆ
or ˆ/ˆˆ largefor reject toequivalent is
ˆ
ˆˆ1
ˆ
ˆ
),(max
),(max
of valuessmallfor 0:Reject
2
2(1)
rnqrqnrn
resres
nn
FFnn
Fs
qr
rnn
qrn
H
L
L
H
ZZ
β
β
β
β
β
![Page 41: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/41.jpg)
4141
Wishart DistributionWishart Distribution
)'|'(:')|(:
)|(:
)|(:),|(:
:Properties
definite positive:
21
2
)|(
2121
2211
1
2/)1(4/)1(2/)1(
2/tr2/)2(
1
21
21
1
CΣΣCACCACΣAA
ΣAAAA
ΣAAΣAA
A
Σ
AΣA
AΣ
mm
mm
mm
p
i
nppnp
pn
n
WW
W
WW
in
ew
![Page 42: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/42.jpg)
4242
Generalization of Result 7.6Generalization of Result 7.6
2211
12
1,2
11
0
:ˆ)''('ˆ
and )')'(,(:ˆ since
)( ˆ)''('ˆ
if levelat :Reject
matrix )1()(:
rn
qr
rnqr
NC
Fqrs
H
rqr
βCCZZCβC
CZZCCββ
βCCZZCβC
0Cβ
C
![Page 43: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/43.jpg)
4343
Example 7.5 (Service Ratings Example 7.5 (Service Ratings Data)Data)
![Page 44: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/44.jpg)
4444
Example 7.5: Design MatrixExample 7.5: Design Matrix
![Page 45: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/45.jpg)
4545
Example 7.5 Example 7.5
tsignifican isgender hebut that t
effect,location no is thererify that further vecan We
level eappropriatany for ant insignific :89.0
12/)(SS
2/)(SS)(SS)46/()(SS)(SS
0:
14418)rank(,1.3419)(SS
of columnssix first :
12)rank(,4.2977)(SS,6)rank(
'
12
1
3231222112110
11
1
323122211211213210
Z
ZZZZ
ZZ
ZZ
ZZZ
β
res
resresresres
res
res
sF
H
n
n
![Page 46: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/46.jpg)
4646
Result 7.7Result 7.7
20
1'01
'0
00
20
1'0
'0
00'0
'00011000
00001'0
2
'2/ˆ
is | of interval congidence )%1(100
'ˆVar
variance.minimumwith
| ofestimator unbiased theis ˆ
|
at response:,1
),0(:,
st
YE
YE
zzYE
Yzz
N
rn
rr
r
n
zZZzβz
z
zZZzβz
zβz
βzz
zz
IεεZβY
![Page 47: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/47.jpg)
4747
Proof of Result 7.7Proof of Result 7.7
1
01'
02
'0
'0
22
01'
02'
0'0
20
1'0
'0
'0
21
22
121
20
1'00
'00
'0
'0
i'0
:'
ˆ
/
'/ˆ
)',(:ˆ
)1/(:/ oft independen iswhich
)',(:ˆ
')ˆCov(ˆVar
7.3Result by varianceminimum
with the ofestimator unbiased theis ˆ
s' ofn combinatiolinear a is
rn
rn
r
tss
N
rns
N
zZZz
βzβzzZZzβzβz
zZZzβzβz
ZZββ
zZZzzβzβz
βzβz
βz
![Page 48: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/48.jpg)
4848
Result 7.8Result 7.8
01'
02
1'0
0
01'
02'
00
'00
220
0'00
'12/ˆ
:for interval prediction )%1(100
'1)ˆ Var(
ˆ: ofpredictor unbiased
,ˆ , oft independen ),0(:
zZZzβz
zZZzβz
βz
βε
βz
st
Y
Y
Y
sN
Y
rn
![Page 49: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/49.jpg)
4949
Proof of Result 7.8Proof of Result 7.8
1
01'
02
'00
21
22
01'
02'
00
01'
02
'00
'00
'00
'00
'00
'00
'0
'00
:'1
)ˆ(
:)1(
)'1,0(:)ˆ(
'1
))ˆ(Var()Var()ˆVar(
0))ˆ(()()ˆ(
)ˆ(ˆˆerror Forecast
rn
rn
ts
Y
srn
NY
Y
EEYE
Y
zZZz
βz
zZZzβz
zZZz
ββzβz
ββzβz
ββzβzβzβz
![Page 50: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/50.jpg)
5050
Example 7.6 (Computer Data)Example 7.6 (Computer Data)
![Page 51: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/51.jpg)
5151
Example 7.6Example 7.6
155.86) (148.08,or '1)025.0(ˆ
at interval prediction %95
153.94) (150.00,or ')025.0(ˆ
timeCPUmean for the interval confidence %95
776.2)025.0(,71.0'
97.1515.7*42.0130*08.142.8ˆ,204.1
01440.000107.008831.0
00052.006411.0
17969.8
'
42.008.142.8ˆ,5.71301
01'
04'0
0
01'
04'0
401'
0
'0
1
21'0
zZZzβz
z
zZZzβz
zZZz
βz
ZZ
z
st
st
ts
s
zzy
![Page 52: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/52.jpg)
5252
Adequacy of the ModelAdequacy of the Model
),0(: of estimatean isˆ
')'(ˆ
ˆˆˆˆ
ˆˆˆˆ
ˆˆˆˆ
2
1
110
2211022
1111011
N
zzy
zzy
zzy
jj
nrrnnn
rr
rr
yHIyZZZZIε
![Page 53: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/53.jpg)
5353
Residual PlotsResidual Plots
![Page 54: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/54.jpg)
5454
Q-QQ-Q Plots and Histograms Plots and Histograms
Used to detect the presence of Used to detect the presence of unusual observations or severe unusual observations or severe departures from normality that may departures from normality that may require special attention in the require special attention in the analysisanalysis
If If nn is large, minor departures from is large, minor departures from normality will not greatly affect normality will not greatly affect inferences about inferences about
![Page 55: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/55.jpg)
5555
Test of Independence of TimeTest of Independence of Time
)1(2
ˆ
ˆˆ
TestWatson -Durbin
ˆ
ˆˆ
ationautocorrelfirst thefrom dconstructeTest
1
1
2
2
21
1
2
21
1
r
r
n
jj
n
jjj
n
jj
n
jjj
![Page 56: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/56.jpg)
5656
Example 7.7: Residual PlotExample 7.7: Residual Plot
![Page 57: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/57.jpg)
5757
LeverageLeverage
““Outliers” in either the response or Outliers” in either the response or explanatory variables may have a explanatory variables may have a considerable effect on the analysis considerable effect on the analysis and determine the fitand determine the fit
Leverage for simple linear Leverage for simple linear regression with one explanatory regression with one explanatory variable variable zz
n
r
zz
zz
nh n
jj
jjj
1 average,
1
1
2
2
![Page 58: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/58.jpg)
5858
Mallow’s Mallow’s CCpp Statistic Statistic
Select variables from all possible Select variables from all possible combinationscombinations
)2(model fullfor varianceresidual
interceptan including ,parameters with
modelssubset for squares of sum residual
pnp
C p
![Page 59: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/59.jpg)
5959
Usage of Mallow’s Usage of Mallow’s CCpp Statistic Statistic
![Page 60: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/60.jpg)
6060
Stepwise RegressionStepwise Regression1. The predictor variable that 1. The predictor variable that explains the largest significant explains the largest significant proportion of the variation in proportion of the variation in YY is the is the first variable to enterfirst variable to enter2. The next to enter is the one that 2. The next to enter is the one that makes the highest contribution to makes the highest contribution to the regression sum of squares. Use the regression sum of squares. Use Result 7.6 to determine the Result 7.6 to determine the significance (significance (FF-test) -test)
![Page 61: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/61.jpg)
6161
Stepwise RegressionStepwise Regression3. Once a new variable is included, 3. Once a new variable is included, the individual contributions to the the individual contributions to the regression sum of squares of the regression sum of squares of the other variables already in the other variables already in the equation are checked using equation are checked using FF-tests. -tests. If the If the FF-statistic is small, the variable -statistic is small, the variable is deletedis deleted4. Steps 2 and 3 are repeated until 4. Steps 2 and 3 are repeated until all possible additions are non-all possible additions are non-significant and all possible deletions significant and all possible deletions are significantare significant
![Page 62: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/62.jpg)
6262
Treatment of ColinearityTreatment of ColinearityIf If ZZ is not of full rank, is not of full rank, Z’ZZ’Z does not have an inv does not have an inverse erse Colinear ColinearNot likely to have exact colinearityNot likely to have exact colinearityPossible to have a linear combination of coluPossible to have a linear combination of columns of mns of ZZ that are nearly 0 that are nearly 0Can be overcome somewhat byCan be overcome somewhat by– Delete one of a pair of predictor variables that are sDelete one of a pair of predictor variables that are s
trongly correlatedtrongly correlated– Relate the response Relate the response YY to the principal components to the principal components
of the predictor variablesof the predictor variables
![Page 63: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/63.jpg)
6363
Bias Caused by a Bias Caused by a Misspecified ModelMisspecified Model
)1(
)2(211
11)1(
)2(2)1(111
1111
11)1(
11
11)1(
)2(2)1(1)2(
)1(21
21
ofestimator biased
''
'')('')ˆ(
''ˆ
β
βZZZZβ
βZβZZZZYZZZβ
YZZZβ
εβZβZεβ
βZZY
ZZZ
EE
![Page 64: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/64.jpg)
6464
Example 7.3Example 7.3Observed dataObserved data
Regression modelRegression model
2112022
1111011
zY
zY
zz11 00 11 22 33 44
yy11 11 44 33 88 99
yy22 -1-1 -1-1 22 33 22
![Page 65: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/65.jpg)
6565
Multivariate Multiple RegressionMultivariate Multiple Regression
nrnn
r
r
jmjjjjmjjj
m
mrrmmmm
rr
rr
zzz
zzz
zzz
YYY
E
zzY
zzY
zzY
10
22120
11110
21'
21'
21
110
22112022
11111011
,
)Var(,0)(,'
Z
εY
Σεεε
![Page 66: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/66.jpg)
6666
Multivariate Multiple RegressionMultivariate Multiple Regression
)4()3()2()1(
21
11211
00201
)()2()1(
21
22221
11211
βββββ
YYYY
rmrr
m
m
m
nmnn
m
m
YYY
YYY
YYY
![Page 67: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/67.jpg)
6767
Multivariate Multiple RegressionMultivariate Multiple Regression
ik
ikkii
n
m
nmnn
m
m
mkiE
Σ
Iεεε
εZβY
ε
ε
ε
εεεε
,,2,1,,,Cov,0)( )()()(
'
'2
'1
)()2()1(
21
22221
11211
![Page 68: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/68.jpg)
6868
Multivariate Multiple RegressionMultivariate Multiple Regression
)()()()()1()1()()(
)()()1()1()1()1()1()1(
)()2()1(1
)()2()1(1
)()2()1(
)(1
)(
)()()()(
''
''
'
,''ˆ
''ˆˆˆˆ
''ˆ
,,2,1,)Cov(,
mmmmmm
mm
m
mm
ii
iiiiii mi
ZbYZbYZbYZbY
ZbYZbYZbYZbY
ZBYZBY
bbbBYZZZβ
YYYZZZββββ
YZZZβ
IεεZβY
![Page 69: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/69.jpg)
6969
Multivariate Multiple RegressionMultivariate Multiple Regression
ˆby minimized
also is ' variancedGeneralize
ˆby minimized is 'tr
' minimizes ˆ)()()()()()(
βB
ZBYZBY
βBZBYZBY
ZbYZbYβb
iiiiii
![Page 70: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/70.jpg)
7070
Multivariate Multiple RegressionMultivariate Multiple Regression
βZZβYYYYYYεε
εεYYεYεYYY
0YZZZZIZβεY
0YZZZZIZεZ
YZZZZIYYε
YZZZZβZY
ˆ''ˆ'ˆ'ˆ'ˆ'ˆ
ˆ'ˆˆ'ˆˆˆ'ˆˆ'
''''ˆˆ'ˆ
'''ˆ'
''ˆˆ:Residuals
''ˆˆ : valuesPredicted
1
1
1
1
![Page 71: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/71.jpg)
7171
Example 7.8Example 7.8
2211
)2()1(1
)2()1(
)2(1
)2(
)1(1
)1(
1
1ˆ,21ˆ
''12
11ˆˆˆ
'11''ˆ
'21''ˆ
1.02.0
2.06.0',
43210
11111'
zyzy
yyZZZβββ
yZZZβ
yZZZβ
ZZZ
![Page 72: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/72.jpg)
7272
Example 7.8Example 7.8
verifiedis ˆ'ˆˆ'ˆ'
42
26ˆ'ˆ,
1545
45165ˆ'ˆ,1943
43171'
00
00ˆ'ˆ'11110
01210ˆˆ
39
27
15
03
11
12
11
41
31
21
11
01
ˆˆ
εεYYYY
εεYYYY
YεYYε
βZY
![Page 73: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/73.jpg)
7373
Result 7.9Result 7.9
eduncorrelat are ˆ and ˆ
1
ˆ'ˆ,0ˆ
)1(ˆˆ,ˆ
ˆˆˆˆˆ
'ˆ,ˆCov
ˆor ˆ
)('
)()(
)()2()1(
1)()(
)()(
βε
Σεε
ε
εε0ε
βZYεεεε
ZZββ
ββββ
rnEE
rnEE
EE
ikkii
m
ikki
ii
![Page 74: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/74.jpg)
7474
Proof of Result 7.9Proof of Result 7.9
11'
)()(1
)()()()()()(
)()()(
)(1
)(1
)()()(
)(1
)()(1
)()(
')()()()()()(
''''
'ˆˆˆ,ˆCov
ˆ,ˆ
''
''ˆˆ
''''ˆ
,0,
ZZZZZεεZZZ
ββββββ
0εββ
εZZZZI
YZZZZIYYε
εZZZβYZZZββ
IεεεεZβY
ikki
kkiiki
iii
i
iiii
iiiii
iiiiiiii
E
E
EE
EE
![Page 75: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/75.jpg)
7575
Proof of Result 7.9Proof of Result 7.9
Σεε
ZZZZI
IZZZZI
εZZZZIεεε
UUAAUUAUU
AU
1
ˆˆ
)1(')'(tr
')'(tr
')'(ˆˆ
)'(tr'tr'
matrix fixed: vector,random :
)('
)(
1
1
)(1'
)()('
)(
rnE
rn
EE
EEE
ki
ikik
ik
kiki
![Page 76: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/76.jpg)
7676
Proof of Result 7.9Proof of Result 7.9
0ZZZZZZ
ZZZZIIZZZ
ZZZZIεεZZZ
ZZZZIεεZZZ
εβ
''''
''''
''''
''''
ˆ,ˆCov
11
11
1')()(
1
1')()(
1
)()(
ik
ik
ki
ki
ki
E
E
![Page 77: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/77.jpg)
7777
Forecast ErrorForecast Error
0)ˆ(
ˆˆˆ
,0)(
oft independen
'
'ˆˆ'ˆˆ
)ˆ(,ˆˆ
,1
)('00
)()('00)(
'0)(
'0)(
'00)(
'00
000
00201'0
01'
0
0)()()()('00)()()()(
'0
'0
'0)(
'0)(
'0)(
'0
0)('00001
'0
ii
iiiiiiiii
ikkii
m
ik
kkiikkii
iii
iiir
YE
YY
EE
EE
EEE
Yzz
βz
ββzβzβzβzβz
εε
zZZz
zββββzzββββz
βzβzβzβzβz
βzz
![Page 78: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/78.jpg)
7878
Forecast ErrorForecast Error
0)ˆ(,'ˆ
'1
'ˆ)ˆ(
'ˆˆ
ˆˆ
ˆˆ
0)()()(1
)()(
01'
0
0)()(00)()('0
0)()()()('000
)()('00)()(
'00
)('00)(
'00
kiiiii
ik
kkikii
kkiiki
kkkiii
kkii
E
EE
EE
E
YYE
ββεZ'ZZββ
zZZz
zββββz
zββββz
ββzββz
βzβz
![Page 79: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/79.jpg)
7979
Result 7.10Result 7.10
)( as ddistribute is ˆ and
ˆ'ˆ1ˆ'ˆ
1ˆby given
ofestimator likelihood maximum theoft independen is ˆ
'ˆ,ˆCov
ˆon with distributi normal:ˆ
ofestimator likelihood maximum theis ˆ
ondistributi normal:,
1,
1)()(
ΣΣ
βZYβZYεεΣ
Σβ
ZZββ
βββ
ββ
εεZβY
rnp
ikki
Wn
nn
E
![Page 80: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/80.jpg)
8080
Result 7.11Result 7.11
2)(
1
11
0
)1(1)1(11,1
1,
)2(0
))(()2(
))1(()1(
~ˆ
ˆln2/)1(1 large, For
ˆˆˆ
ˆln
ˆ
ˆlnln
of valueslargefor Reject
ˆ'ˆˆ,:ˆˆ
oft independen :ˆ'ˆˆ
:,
qrm
qrp
rnp
mqr
mq
qrmrnn
nn
nnnn
H
nWn
Wn
H
Σ
Σ
ΣΣΣ
Σ
Σ
Σ
βZYβZYΣΣΣΣ
ΣβZYβZYΣ
0β
β
β
β
![Page 81: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/81.jpg)
8181
Example 7.9Example 7.9
0
2)(
1
111
111
)2(0
)2(
1
reject not do ,49.9)05.0(
28.3ˆˆˆ
ˆln)1(1
31)rank(,51)rank(
,05.0,:
parametersn interactiogender locatin:
12.36616.246
16.24676.441ˆˆ
95.205072.1021
72.102139.2977ˆ
index.quality -service more one plus data 7.5 Example
11H
nn
nqrmrn
qr
H
n
n
qrm
ΣΣΣ
Σ
ZZ
0β
β
ΣΣ
Σ
![Page 82: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/82.jpg)
8282
Other Multivariate Test StatisticsOther Multivariate Test Statistics
1
1
1
1
1
1
1
121
1
1 root greatest sRoy'
trceLawlay tra-Hotelling
1
1)(tr tracesPillai'
1
1 lambda sWilk'
,min , of seigenvalue:
ˆˆ,ˆ
s
ii
s
i i
s
i i
s qrps
nn
HE
EHH
HE
E
HE
ΣΣHΣE
![Page 83: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/83.jpg)
8383
Predictions from RegressionsPredictions from Regressions
)()1(
'
''ˆ1
ˆ'''ˆ
:'for ellipsoid confidence )%-100(1
'
''ˆ
1
ˆ'
'
''ˆ
as ddistributetly independen:ˆ
',':'ˆ7.10Result
,01'
0
00
1
00
0
01'
0
00
1
01'
0
002
1
01'
000
mrnm
rn
m
Fmrn
rnm
rn
n
rn
nT
Wn
N
zZZz
zβzβΣ
zβzβ
zβ
zZZz
zβzβΣ
zZZz
zβzβ
ΣΣ
ΣzZZzzβzβ
![Page 84: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/84.jpg)
8484
Predictions from RegressionsPredictions from Regressions
Σ
zZZz
βz
βz
ˆ ofelement diagonalth the:ˆ
ˆ1
')()1(
ˆ
:)(for
intervals confidence ussimultaneo )%1(100
01'
0,
)('0
)('0
i
rn
nF
mrn
rnm
YE
ii
iimrnm
i
ii
![Page 85: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/85.jpg)
8585
Predictions from RegressionsPredictions from Regressions
iimrnmi
i
mrnm
m
rn
nF
mrn
rnm
Y
Fmrn
rnm
rn
n
n
N
ˆ1
)'1()()1(ˆ
:for intervals prediction ussimultaneo )%1(100
)()1(
)'1(
'ˆˆ1
''ˆ
:for ellipsoid prediction )%1(100
ˆ oftly independen
))'1(,0(:'ˆ'ˆ
,'
01'
0,)('0
0
,01'
0
00
1
00
0
01'
00000
000
zZZzβz
zZZz
zβYΣzβY
Y
Σ
ΣzZZzεzββzβY
εzβY
![Page 86: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/86.jpg)
8686
Example 7.10Example 7.10
13.1330.5
30.580.5
ˆ'ˆˆ'ˆ
ˆ'ˆˆ'ˆˆ
17.349ˆ,97.151ˆ
4208.142.8ˆ 7.6, Example From
34725.0','67.525.214,14ˆ
812.1,67.525.214.14ˆ :equation Fitted
capacity I/Odisk : data 7.6 Example
)2()2()2()2()1()1()2()2(
)2()2()1()1()1()1()1()1(
)2('0)1(
'0
)1(
0'0)2(
212
2
βZyβZyβZyβZy
βZyβZyβZyβZyΣ
βzβz
β
zZZzβ
n
szzy
Y
![Page 87: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/87.jpg)
8787
Example 7.10Example 7.10
34725.1'1
with34725.0' replace :ellipse prediction 95%
)05.0(3
4234725.0
17.349
97.151
13.1330.5
30.580.517.349,97.151
:'for ellipse confidence 95%
2,2,7
17.349
97.151ˆ
ˆ
ˆ
ˆ'ˆ
01'
0
01'
0
3,2
)2('0
)1('0
1
)2('0)1(
'0
0
)2('0
)1('0
0')2(
')1(
0
zZZz
zZZz
βz
βzβzβz
zβ
βz
βzz
β
βzβ
F
mrn
![Page 88: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/88.jpg)
8888
Example 7.10Example 7.10
![Page 89: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/89.jpg)
8989
Linear RegressionLinear Regression
20
0
022110
'
21
21
' error squaremean
' error prediction
' predictor linear
,
:matrix covariance and mean
normaly necessarilnot :),,,,(
variablesrandom:,,,,
Zb
Zb
Zb
Σσ
σΣ
μμ
Σμ
ZZZ
Z
bYE
bY
bZbZbZbb
zzzyf
ZZZY
rr
Y
YYY
Z
Y
r
r
![Page 90: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/90.jpg)
9090
Result 7.12Result 7.12
YY
YY
YY
b
YYYY
YY
bYY
Y
YE
Y
ZZZZZZ
b
ZZZZ
ZZZZ
σΣσβΣβ
ZbZβ
Zβ
σΣσZβ
μβσΣ
Zβ
1'
0,
0
0
1'20
01
0
0
'
',Corrmax',Corr
n with correlatio
maximum havingpredictor linear theis ' Also,
)'(
',
response theof predictorslinear all
among squaremean minimum has 'predictor Linear
0
![Page 91: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/91.jpg)
9191
Proof of Result 7.12Proof of Result 7.12
ZZZZZZZZ
ZZZZ
ZZZZZZZZZ
ZZZZ
Z
ZZ
ZZ
ZZ
μσΣμbβσΣb
σΣσ
μbσΣbΣσΣb
σbμbbΣb
μZb
μbμZb
μbμZb
Zb
μbμbZbZb
'',at minimized
''
'2''
'2
''
''
'
''''
10
1
1'
20
11
20
20
22
20
20
00
YYYY
YY
YYYYY
YYYY
Y
YY
YY
YY
b
b
b
YE
bEYE
bYE
bYE
bb
![Page 92: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/92.jpg)
9292
Proof of Result 7.12Proof of Result 7.12
βΣββΣΣσβσσΣσ
βσΣb
σΣσZb
σΣbσΣbσb
bΣb
σbZb
bZb
ZZZZZZZZZZZZ
ZZZ
ZZZZ
ZZZZZZZ
ZZ
Z
Z
'
for equality with
,'Corr
''
inequality Schwartz-Cauchy Extended
'
','Corr
','Cov
1''1'
1
1'2
0
1'2
2
20
0
YYYY
Y
YY
YY
YYY
YY
Y
Y
Yb
Yb
Yb
![Page 93: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/93.jpg)
9393
Population Multiple Population Multiple Correlation CoefficientCorrelation Coefficient
error no with predicted becan :1
in power predictive no:0
1
:forecast to' usingin error squareMean
:iondeterminat oft coefficien Population
:tcoefficienn correlatio multiple Population
2)(
2)(
2)(
1'
0
2)(
1'
)(
Y
Yβ
Y
Y
YYYYYYY
Y
YY
YYY
Z
Z
ZZZZZ
Z
ZZZZZ
Z
σΣσ
Zβ
σΣσ
![Page 94: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/94.jpg)
9494
Example 7.11Example 7.11
548.010
3
7 error squaremean
23' :predictorlinear best
3','21
23|1
37|1
11|10
|
|
,
0
2
5
1'
)(
1'
210
01
'
YY
YYY
YYYY
YY
Y
YYYY
ZZ
ZZZZZ
ZZZZ
ZZZZ
ZZZ
Z
Z
σΣσ
σΣσ
Zβ
μβσΣβ
Σσ
σ
Σ
μ
μ
![Page 95: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/95.jpg)
9595
Linear Predictors and NormalityLinear Predictors and Normality
error. squaremean smallest with the
predicts stillit ss,Neverthele linear. benot need
| normal,not is population When the
function) regression(linear
')(|
),(
: with ofon distributi lConditiona
),(:'
01'
1'1'
121
Y
YE
YE
N
Y
NZZZY
YY
YYYYYY
rr
z
zβμzΣσz
σΣσμzΣσ
zZ
Σμ
zZZZ
ZZZZzZZZ
![Page 96: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/96.jpg)
9696
Result 7.13Result 7.13
YYYYYY
Y
YY
Y
YYY
r
sn
n
YE
Y
YY
sY
NY
ZZZZZ
ZZZ
ZZZZZZ
ZZZ
Z
sSs
Zβ
ZzSszβ
ZβZSssSβ
Ss
sS
Zμ
ΣμZ
1'
20
1'0
1'0
1
'
1
1ˆ
' ofestimator likelihood maximum
)('ˆˆ
estimator likelihood maximum
'ˆˆ,ˆ
tscoefficien theofestimator likelihood maximum
,ˆ
),(: and ofon distributiJoint
![Page 97: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/97.jpg)
9797
Proof of Result 7.13Proof of Result 7.13
1
'ˆˆ
1
1
:for estimator Unbiased
and for 1ˆ and ˆ estimators
likelihood maximum theofon substitutiby conclusion get the to
error squaremean
'
,,'
and ,likelihood maximum ofproperty invariance Apply the
1
2
01'
1'
1'0
110
rn
Y
srn
n
n
n
n
jjj
YYYY
YY
YYYYYY
YY
YYY
Zβ
sSs
ΣμSΣμ
σΣσ
μzΣσzβ
σΣβμσΣ
ZZZZ
Z
ZZZZZ
ZZZZ
ZZZZZZZ
![Page 98: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/98.jpg)
9898
Invariance PropertyInvariance Property
)Var( of MLE1
ˆ
ˆ of MLE
ˆˆ'ˆ' of MLE
:Examples
)( ofestimator likelihood maximum:)ˆ(
ofestimator likelihood maximum:ˆ
1
2
11
i
n
jijiii
iiii
XXXn
hh
μΣμμΣμ
![Page 99: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/99.jpg)
9999
Example 7.12Example 7.12
657.13034.28|983.35
034.28200.377|763.418
983.35763.418|913.467
|
|
,
547.3
24.130
44.150
ˆ
,,on nsobservatio 7 data, 7.6 Example
'
21
ZZZ
Z
Ss
s
S
z
μ
Y
YYYs
y
ZZYn
![Page 100: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/100.jpg)
100100
Example 7.12Example 7.12
894.01
error squaremean theof estimate likelihood maximum
42.008.142.8'ˆˆ
function regression estimated
421.8'ˆˆ,420.0
079.1ˆ
1'
210
01
YYYY
Y
sn
n-
zz
y
ZZZZ
ZZZ
sSs
zβ
zβsSβ
![Page 101: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/101.jpg)
101101
Prediction of Several VariablesPrediction of Several Variables
tscoefficien regression ofmatrix :
of prediction for theerror squaremean theminimizes
|
example,For s.regression univariate of composed
|
: and of regression temultivaria
|
|
,),,(:
1
1
)()(
)()(
)1(
)1(
)1(
)1(
11
1ZZYZ
Z1
ZZZ
Z1
ZZYZY
ZZZY
YZYY
Z
Y
ΣΣβ
μzΣΣz
μzΣΣμzY
ZY
ΣΣ
ΣΣ
Σ
μ
μ
μΣμ
Z
Y
Y
YE
m
E
N
YY
rrmr
rmmm
r
m
rm
r
m
![Page 102: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/102.jpg)
102102
Result 7.14Result 7.14
ZY
1ZZYZYYZYY
1ZZYZ
ZY1
ZZYZYYZYY
Z1
ZZYZ
SSSSΣ
ZzSSYzββ
ΣΣΣΣΣβZβYβZβY
μzΣΣβzβZY
ΣμZY
n
n
E
N
Y
rm
1ˆ
ˆˆ
estimators likelihood maximum
'
: and of regression
),(: and
0
00
0
![Page 103: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/103.jpg)
103103
Example 7.13Example 7.13
657.13034.28|558.140983.35
034.28200.377|976.1008763.418
558.140976.1008|491.3072556.1148
983.35763.418|556.1148913.467
547.3
24.130
79.327
44.150
ˆ
7.10. and 7.6 Example of Data
S
z
y
μ
![Page 104: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/104.jpg)
104104
Example 7.13Example 7.13
205.2893.0
893.0894.01
of estimate likelihood maximum
547.3665.524.130254.2
547.3420.024.130079.1
79.327
44.150
ˆˆ
21
21
0
ZY1
ZZYZYY
ZYY
1ZZYZ
SSSS
Σ
)z(zSSyzββ
n
n
zz
zz
![Page 105: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/105.jpg)
105105
Partial Correlation CoefficientPartial Correlation Coefficient
ZZ
ZZ
ZZ
ZZ
ZY1
ZZYZYYZYY
Z1
ZZZZ1
ZZZ
Z
ΣΣΣΣΣ
μZΣΣμZΣΣ
2211
21
21
2211
21
21
2211
tcoefficienn correlatio partial Sample
: of effects thegeliminatin
, and between t coefficienn correlatio Partial
matrix covarianceError
,
:errors ofPair
21
21
YYYY
YYYY
YYYY
YYYY
YYYY
ss
sr
YY
YY
![Page 106: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/106.jpg)
106106
Example 7.14Example 7.14
responsesboth on of effects thegeliminatinafter
reducedsharply been has and between n Correlatio
96.0
64.0572.2043.1
042.1
572.2042.1
042.1043.1
data 7.13 Example
21
21
2211
21
21
Z
SSSS
ZZ
ZZ
ZY1
ZZYZYY
YY
r
ss
sr
YY
YYYY
YYYY
![Page 107: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/107.jpg)
107107
Mean Corrected Form of the Mean Corrected Form of the Regression ModelRegression Model
2'
22'
2'
2
2'
'22
11
2121
1111
111*
110
'''
0,|
1
1
1
matrixdesign correctedmean
)()(
ccccc
ccc
cc
rnrn
rr
rr
c
jrjrrj
jjrrjj
n
zzzz
zzzz
zzzz
zzzz
zzY
ZZ0
0
ZZ1Z
Z111ZZ
1ZZ1Z
![Page 108: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/108.jpg)
108108
Mean Corrected Form of the Mean Corrected Form of the Regression ModelRegression Model
21
2'
2
2
21'
*
**
1
2'
22'
*
'2
1
2'
2'
2
1
2'
2
'1'*
'/
)ˆCov()ˆ,ˆCov(
)ˆ,ˆCov()ˆVar(
'ˆˆˆ
''/1
ˆ
ˆ
cc
cc
cc
c
cccc
cccccc
ccc
c
n
yy
yn
ZZ0
0
ZZββ
β
zzZZZyzzβ
yZZZyZ
y1
ZZ0
0
yZZZβ
![Page 109: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/109.jpg)
109109
Mean Corrected Form for Mean Corrected Form for Multivariate Multiple RegressionsMultivariate Multiple Regressions
ii
ii
ii
zzii
zzii
zziji
iccc
i
sn
sn
snzz
y
i
)1(ˆ~̂
)1(~
)1(/
ablesinput vari edstandardiz
ˆ
:responseth the
for t vectorscoefficien theof estimates squareleast
)('
2
1
2'
2
)(
(i)
yZZZβ
![Page 110: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/110.jpg)
110110
Relating the FormulationsRelating the Formulations
1'1'
1
2'
22
1
2'
22
2222
1'1
2'
22'
0*
1
2'
22'
*
1'0
)1()1(
''
''''
'ˆ'ˆ,ˆˆ
'ˆˆˆ
:form correctedmean
'ˆˆ :7.13Result
ZZZZZZ
ZZZ
ZZZ
SsSs
ZZZ1yZZZy
Z1yZ1Z1yZy
βSsZZZyβ
zzZZZyzzβ
zzSszβ
YY
cccccc
cccc
Ycccc
cccc
Y
nn
y
yyy
y
yy
y
![Page 111: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/111.jpg)
111111
Example 7.15Example 7.15
Example 7.6, classical linear Example 7.6, classical linear regression modelregression model
Example 7.12, joint normal Example 7.12, joint normal distribution, best predictor as the distribution, best predictor as the conditional meanconditional mean
Both approaches yielded the same Both approaches yielded the same predictor of predictor of YY11
21 42.008.142.8ˆ zzy
![Page 112: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/112.jpg)
112112
Remarks on Both FormulationRemarks on Both Formulation
Conceptually differentConceptually different
Classical modelClassical model– Input variables are set by experimenterInput variables are set by experimenter– Optimal among linear predictorsOptimal among linear predictors
Conditional mean modelConditional mean model– Predictor values are random variables Predictor values are random variables
observed with the response valuesobserved with the response values– Optimal among all choices of predictorsOptimal among all choices of predictors
![Page 113: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/113.jpg)
113113
Example 7.16 Natural Gas DataExample 7.16 Natural Gas Data
![Page 114: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/114.jpg)
114114
Example 7.16 : First ModelExample 7.16 : First Model
52.0ˆ
ˆˆ
)ˆ( ation autocorrel lag1
But,
intercept except the t,significan are tscoefficien All
952.0
Weekend15.857 - Windspeed1.315
DHDLag 1.405 DHD 5.874 1.858Sendout
1
2
21
1
2
n
jj
n
jjj
r
R
![Page 115: 1 Multivariate Linear Regression Models Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed25503460f94be1086/html5/thumbnails/115.jpg)
115115
Example 7.16 : Second ModelExample 7.16 : Second Model
negligible all are residuals theof nscorrelatio auto
89.228ˆ
240.0470.0
Weekend10.109 - Windspeed1.207
DHDLag 1.426 DHD 5.810 2.130 Sendout
as model fitted aget toSASApply
noise siveautoregresan with errorst independen theReplace
2
71
7771
jjjj
jjjj
NNN
NNN