Wednesday, October 6

Correlation and Linear Regression

RDG

MATH

SCI

CIV

CONCPT

20

40

60

80

20 40 60 80

40

60

80

40 60 80

20

40

60

80

20 40 60 80

20

40

60

80

20 40 60 80

-2

0

2

-2 0 2

Explain to a non-statistician what it means to say “reading and math scores are correlated r=.69 in this population”.

.69

Explain to a non-statistician what it means to say “reading and math scores are correlated r=.69 in this population”.

• Using either the reading or math score, you can predict the other value by how much sd it is from its mean. Or r=.069 means both values are 0.69 sd from their means.

Explain to a non-statistician what it means to say “reading and math scores are correlated r=.69 in this population”.

• There is a strong(ish) positive linear relationship between scores on a reading test and scores on a math test in this population. Thus, we have a fair amount of predictive power when it comes to guessing a math score for a particular individual if we have that individual’s reading score. Specifically, one standard deviation unit is equal to 0.69 standard deviation units.”

Explain to a non-statistician what it means to say “reading and math scores are correlated r=.69 in this population”.

• .69 is a valuable number to help guess someone’s reading score given their math score. .69 suggests that the math score can help us guess more closely what the actual reading score is. With a 1.00 r value, we could guess with certainty that the reading score=the math score. With a 0 r value, we would guess with uncertainty that the reading score was the mean of reading scores. In this case it would not be helpful to know the math score. However, .69 suggests that the reading score helps us determine the math score.

Explain to a non-statistician what it means to say “reading and math scores are correlated r=.69 in this population”.

• When reading and math scores are correlated at r=.69, this means that the relationship between the subjects is quite strong. If you imagine all of these math scores and reading scores plotted against each other, a PERFECT correlation would be a straight line. Conversely, zero correlation would have data points all over with no trend at all. Thus, if r=.69, this is quite close to 1, meaning that the relationship between the math and reading scores has a high correlation.

Explain to a non-statistician what it means to say “reading and math scores are correlated r=.69 in this population”.

• A correlation of .69 between reading and math scores means that, on average, as a student’s reading score increases, so do his or her math scores and vice versa. The strength of the relationship is moderately strong, so the value of a math score is a moderately strong predictor for the reading score.

Explain to a non-statistician what it means to say “reading and math scores are correlated r=.69 in this population”.

• The scores of reading and math are correlated, which means that you can take one score and predict more or less what the other may be. The correlation between the two is .69, which means that if you have figured out the average score, and know how far a student is away from the average on his reading test, you can calculate how far from the average he will be on his math. …

You will not leave the room until…

• you have understood that a correlation is a systematic quantitative expression of the proportion of explained and unexplained co-variation of two variables.

You will not leave the room until…

• you have understood that a correlation is a systematic quantitative expression of the proportion of explained and unexplained co-variation of two variables … and you love knowing this fact!

zy = zx

When X and Y are perfectly correlated

We can say that zx perfectly predicts zy

zy’ = zx

Or

zy = zx

^

When they are imperfectly correlated, i.e., rxy ≠ 1 or -1

zy’ = rxyzx

r is the slope of the predicted line, with a zero-intercept of z’y=0

-2-1

01

2

-2 -1 0 1 2z_meas

z_est Fitted values

Oh no, hands again…

24

68

10

5 6 7 8 9Measured

Estimated Fitted values

What about in raw original units?

When we want to express the prediction in terms of raw units:

zy’ = rxyzx

Y’ = bYXX + aYX

24

68

10

5 6 7 8 9Measured

Estimated Fitted values

Let’s try it by eyeball…

When we want to express the prediction in terms of raw units:

zy’ = rxyzx

Y’ = bYXX + aYX

bYX = rYX (σy / σx)

aYX = Y - bYXX

_ _

Explained and unexplained variance

SStotal = SSexplained + SSunexplained

SStotal = SSexplained + SSunexplained

N N N

-2-1

01

2

-2 -1 0 1 2z_meas

z_est Fitted values

Oh no, hands again…

Explained and unexplained variance

r2XY = 1 -

σ2Y’ [ =unexplained]

σ2Y [ =total]

=

σ2Y - σ2

Y’

σ2Y

r2 is the proportion explained variance to the total variance.