Download - Linear Regression Handout
-
8/6/2019 Linear Regression Handout
1/17
Linear Regression - Topics
Basics of Linear Regression
Variation in Linear Regression
Linear Regression Analysis Goodness of Fit
Standard Error terms for Linear Regression
Hypothesis testing
-
8/6/2019 Linear Regression Handout
2/17
Regression - Types
-
8/6/2019 Linear Regression Handout
3/17
Linear Regression
A statistical technique that uses a single,independent variable (X) to estimate a single
dependent variable (Y).
Based on the equation for a line:
Y = b + mX
-
8/6/2019 Linear Regression Handout
4/17
Linear Regression - Model
iI
X
Y
Y XFF!Yi
Xi
?(the actual value ofYi)
-
8/6/2019 Linear Regression Handout
5/17
Linear Regression - Model
ii iY F F I
Regression Coefficients for a . . .
Population
SampleY = b0 + b1Xi + e
Y = b0 + b1Xi
-
8/6/2019 Linear Regression Handout
6/17
ANOVA D - Variati
SST
SSE
SST
SST = SST + SSE
3-hour 1-day 10-week
70 61 82
77 75 88
76 67 90
80 63 96
84 66 92
78 68 98
SST is a measure f the t talvariati f bservati s. Ameasure f the differe ces ibservati s.
Due t treatme ts.
and m/unexplained.
-
8/6/2019 Linear Regression Handout
7/17
Linear Regression - Variation
X Y
Temperature Sales
63 1.52
70 1.68
73 1.875 2.05
80 2.36
82 2.25
85 2.68
88 2.9
90 3.14
91 3.06
92 3.24
75 1.92
98 3.4100 3.28
92 3.17
87 2.83
84 2.58
88 2.86
80 2.26
82 2.14
76 1.98
Ice Cream ExampleIce Cream Sales
0
0.5
1
1.5
2
2.5
3
3.5
4
0 20 40 60 80 100 120
Y
Y= 2.53
-
8/6/2019 Linear Regression Handout
8/17
Linear egressi n - Variati n
X Y
Temperature Sales
63 1 52
70 1 68
73 1 875 2 05
80 2 36
82 2 25
85 2 68
88 2 9
90 3 14
91 3 06
92 3 24
75 1 92
98 3 4
100 3 28
92 3 17
87 2 83
84 2 58
88 2 86
80 2 26
82 2 14
76 1 98
Ice Cream Example Ice eam Sa es
0
0 5
1
1 5
2
2 5
3
3 5
4
0 20 40 60 80 100 120
0 1Y b b X ! ! Sample
Regression Line
-
8/6/2019 Linear Regression Handout
9/17
Linear egressi n - Variati n
SST
SSE
SS
SST = SS + SSE
ue t regressi n.
and m/unexplained.
-
8/6/2019 Linear Regression Handout
10/17
Linear Regression - Variation
Xi
Y
X
Y
SST= (Yi-Y)2
SSE=(Yi-Yi)2
SSR= (Yi-Y)2
_
_
_
-
8/6/2019 Linear Regression Handout
11/17
Determining the RegressionLine/Model
Use Excel (or any other popular statisticalsoftware)
Select Tools, Data Analysis, Regression Provide the X range
Provide the Y range
Output the analysis to a new sheet
Manual Calculations
X Y
T
m
r r
S
-
8/6/2019 Linear Regression Handout
12/17
Determining the RegressionLine/Model using Excel
SUMMARY OUTPUT
Regression Statistics
Multiple R 0.969534312
R Square 0.939996782
Adjusted R Square 0.936838718
Standard Error 0.1461076
Observations 21
ANOVA
df SS MS F
Regression 1 6.35405596 6.354056 297.6496823
Residual 19 0.405601183 0.021347
Total 20 6.759657143
Coefficients Standard Error t Stat P-value
Intercept -2.534985905 0.295223266 -8.58667 5.7673E-08
X Variable 1 0.060727986 0.003519947 17.25253 4.58812E-13
-
8/6/2019 Linear Regression Handout
13/17
Determining the RegressionLine/Model Manual Calculations
SST= (Yi-Y)2SSE=(Yi-Yi)
2 SSR= (Yi-Y)2
__
SSx =(Xi- X )2_
SSy =(Yi- Y)2
_
SSxy =(Xi- X )(Xi- Y )_ _
b1=SSxy/SSx
b0 = Y b1X_ _
MSE = SSE / dfMSR = SSR / df
R2 = SSR/SST
YX
SSES
n-2!
t-test = b1/ Sb1
-
8/6/2019 Linear Regression Handout
14/17
Measures of Model Goodness
1. R2 Coefficient ofDetermination
2. F-test > F-crit or p-value less than alpha
3. Standard Error
4. t-test
-
8/6/2019 Linear Regression Handout
15/17
Hypothesis testing for
Testing to see if the linear relationshipbetween X and Y is significant at the
population level. t-test
Follow the 5-step process
H0
:
HA:
t-crit, alpha or alpha/2, n-2 df
F
-
8/6/2019 Linear Regression Handout
16/17
Standard Error Terms in LinearRegression
Se (standard error of the estimate)A measure of variation around the regression line
If the Se is small
Standard deviation Of the Errors
Sb1 (standard error of the the sampling distribution of b1)
Standard deviation of the slopes
A measure of the variation of the slopes from differentsamplesIf the Sb1 is smallour b1 estimate is probably very accurate
Estimates of
b1
b1
b1
-
8/6/2019 Linear Regression Handout
17/17
Linear Regression Example
Petfood, Estimate Sales based on Shelf Space
Two sets of samples, 12 observations each
Perform a Regression Analysis on both sets ofdata
Space Sales
5 1.
5 2.2
5 1.
10 1.10 2.
10 2.
15 2.
15 2.
15 2.
20 2.
20 2.
20 3.1
Space Sales
5 1.
5 1.
5 1.
10 210 2.
10 2.
15 3.5
15 3.2
15 3.3
20 4.2
20 4.
20 4.5
Sample1 Sample2