chapter 20 linear regression. what if… we believe that an important relation between two measures...
TRANSCRIPT
Chapter 20
Linear Regression
What if…We believe that an important relation
between two measures exists?For example, we ask 5 people about
their salary and education levelFor each observation we have two
measures, and those two measures came from the same person
What would we “predict”? Does more education mean more salary? Does more salary mean more education? Does more education mean less salary? Does more salary mean less education? Are salary and education related?
RegressionDescriptive vs. Inferential Bivariate data - measurements on two
variables for each observation– Heights (X) and weights (Y)– IQ (X) and SAT(Y) scores – Years of educ. (X) and Annual salary (Y)– Number of Policemen (X) and Number of
crimes (Y) in US cities
Regression
How are the two sets of scores related?
Using a scatterplot we can “look” at the relationship
Constructed by plotting each of the bivariate observations (X, Y)
9 11 13 15 17 19 210
10
20
30
40
50
60
70
Yrs of Education
An
nu
al
Sala
ry (
in 1
00
0's
)
Regression
Which one’s X and which one’s Y?
That’s up to you, but… Generally, the X
variable is thought of as the “predictor” variable
We try to predict a Y score given an X score
9 11 13 15 17 19 210
10
20
30
40
50
60
70
Yrs of Education
An
nu
al
Sala
ry (
in 1
00
0's
)
Regression
If the scores seem to “line up,” we call this a “linear relationship”
9 11 13 15 17 19 210
10
20
30
40
50
60
70
Yrs of Education
An
nu
al
Sala
ry (
in 1
00
0's
)
Interpreting Scatterplots
If the following relations hold:
low x - high ymid x - mid yhigh x - low y,
“A negative linear relationship”
9 11 13 15 17 19 210
10
20
30
40
50
60
70
Yrs of Education
An
nu
al
Sala
ry (
in 1
00
0's
)
Interpreting Scatterplots
If the following relations hold:
low x - low ymid x - mid yhigh x - high
y,
“A positive linear relationship”
2 4 6 80
1
2
3
4
5
6
7
8
9
10
Police per 1000 citizensN
um
be
r o
f C
rim
es (
10
00
s)
Interpreting Scatterplots
However, there also can be “no relation” also
2 4 6 899
100
101
102
103
104
105
106
107
Shoe Size
IQ
Interpreting Scatterplots
Curvelinear
50 55 60 65 70 75100
105
110
115
120
125
130
135
HeightW
eig
ht
Measuring Linear RelationshipsThe first measure of a linear
relationship (not in the book) is COVARIANCE (sXY)
Or
SPXY is known as the “Sum of Products” or the sum of the products of the deviations of X and Y from their means
Easy Calculation
Covariance
Interpretation:– positive = positive linear relationship– negative = negative linear relationship– zero = no relationship
Magnitude (strength of the relationship)?– Uninterpretable– for example, a large covariance does not
necessarily mean strong relationship
But, we can use covariance Which line best fits our
data? Do we just draw one
that looks good? No, we can use
something called “least squares regression” to find the equation of the best-fit line (“Best-fit linear regression”)
9 11 13 15 17 19 2120
25
30
35
40
45
50
55
60
65
Yrs of EducationA
nn
ual
Sala
ry (
in 1
00
0's
)
Linear Equations
Yi = mXi + bm = slopeb = y-intercept
Finding the Slope
Or…
Finding the y-intercept (b)After finding the slope (m), find b using:
Least Squares Criterion
The best line has the property of least squares
The sum of the squared deviations of the points from the line are a minimum
9 11 13 15 17 19 2120
25
30
35
40
45
50
55
60
65
Yrs of EducationA
nn
ual
Sala
ry (
in 1
00
0's
)
What’s the “least” again?What are we trying to minimize?
– The best fit line will be described by the function Yi = mXi + b
– Thus, for any Xi, we can estimate a corresponding Yi value
– Problem: for some Xi’s we already have Yi’s
– So, let’s call the estimated value
(“Y-sub-I-hat”), to differentiate it from the “real” Yi
Least Squares Criterion
For example, when
Xi = 15we would estimate that = 44,000
But, we have a “real” Yi value corresponding to Xi =15 (35,000)
9 11 13 15 17 19 2120
25
30
35
40
45
50
55
60
65
Yrs of Education
An
nu
al
Sala
ry (
in 1
00
0's
)When Xi = 15
Our estimatedY value is44,000
A “real”Y valueof 35,000
iY
Minimize this…
For every Xi, we have the a value Yi, and an estimate of Yi ( )
Consider the quantity:– Which is the deviation of the real score from the
estimated score, for any give Xi value The sum of these deviations will be zero
• But, by squaring those deviations and summing,
• We want the line that makes the above quantity the minimum (the least squares criterion)
• This is also called the sums of squares error or SSE (how much do our estimates “err” from our real values?)
How accurate are our Estimates?Two ways to measure how “good” our
estimates are:– Standard Error of the Estimate– Coefficient of Determination (not covered
in our book, yet)
Standard Error of the Estimate
but, this term is very hard to interpret. (Hurrah, there are better ways to measure the goodness of the fit!)
Coefficient of Determination
cd = r2
Now You:ID INCOME NUMDRK
2001 1 1
2002 6 2
2003 5 8
2004 4 1
2005 6 3
Practice:ID INCOME NUMDRK
XY
2001 1 1
2002 6 2
2003 5 8
2004 4 1
2005 6 3
Σ
n
M
SS(X)
Practice:ID INCOME NUMDRK
XY
2001 1 1 1
2002 6 2 12
2003 5 8 40
2004 4 1 4
2005 6 3 18
Σ 22 15 75
n 5 5
M 4.4 3
SS(X) 17.2 34
Practice:ID INCOME NUMDRK
XY
2001 1 1 1
2002 6 2 12
2003 5 8 40
2004 4 1 4
2005 6 3 18
Σ 22 15 75
n 5 5
M 4.4 3
SS(X) 17.2 34
0 1 2 3 4 5 6 70
1
2
3
4
5
6
7
8
9
f(x) = 0.523255813953 x + 0.697674418605