stat 430/510: lecture 17jpiette/stat430-su10/lecture17.pdfadmin. stuff covariance correlation...

Admin. Stuff Covariance Correlation Conditional Expectation To Do

STAT 430/510: Lecture 17

James Piette

June 28, 2010

Admin. Stuff Covariance Correlation Conditional Expectation To Do

Updates

HW4 is due today.HW2 grades are (finally) up, as are the HW3 solutions. I’llget around to grading that soon.Discuss HW5.

Admin. Stuff Covariance Correlation Conditional Expectation To Do

Formalization

Def: The covariance between two r.v.’s X and Y is definedas

Cov(X ,Y ) = E [(X − E(X ))(Y − E(Y ))]

An alternative form of covariance is

Cov(X ,Y ) = E [XY ]− E [X ]E [Y ]

Remembering back, Cov comes up when we look at X andY not independent and . . .

Var(X + Y ) = Var(X ) + Var(Y ) + 2Cov(X ,Y )

Admin. Stuff Covariance Correlation Conditional Expectation To Do

Properties

Cov(X ,Y ) = Cov(Y ,X ).Cov(X ,X ) = Var(X ).Cov(aX ,bY ) = ab · Cov(X ,Y ).Cov(

∑i Xi ,

∑j Yj) =

∑i∑

j Cov(Xi ,Yj).

Var(∑n

i=1 Xi) =∑n

i=1 Var(Xi) + 2∑

i<j Cov(Xi ,Xj).If Xi are pairwise independent, then

Var(n∑

i=1

Xi) =n∑

i=1

Var(Xi)

Admin. Stuff Covariance Correlation Conditional Expectation To Do

Example 1

Let

X =

1 w.p. 1/30 w.p. 1/3−1 w.p. 1/3

and

Y =

{1 if X = 00 if X 6= 0

Question: What is XY and E [XY ]?Solution: XY = 0, because Y is 0 if X is not and Y is not0 if X is.Thus, the E [XY ] = 0.Question: Are X and Y independent?Solution: No, even though Cov(X ,Y ) = 0.

Admin. Stuff Covariance Correlation Conditional Expectation To Do

Example 2

Think back to example 2c, in this chapter.Let X1, . . . ,Xn be i.i.d r.v.’s having expected values µ andvariance σ2.Let X̄ =

∑ni=1

Xin be the sample mean.

Let the quantities Xi − X̄ be called deviations.These are all the differences between the individual dataand the sample mean.

The r.v. S2 =∑n

i=1(Xi−X̄)2

n−1 is called the sample variance.

Question: What is Var(X̄ )?

Admin. Stuff Covariance Correlation Conditional Expectation To Do

Example 2 (cont.)

Solution:

Var(X̄ ) = Var

(n∑

i=1

Xi

n

)

=

(1n

)2

Var

(n∑

i=1

Xi

)

=

(1n

)2 n∑i=1

Var(Xi) by independence

=σ2

n

Question: E [S2]?

Admin. Stuff Covariance Correlation Conditional Expectation To Do

Example 2 (cont.)

Solution: Start off by multiplying by (n − 1) to eliminatethe constant. Then . . .

(n − 1)S2 =n∑

i=1

(Xi − X̄ )2

=n∑

i=1

(Xi − µ+ µ− X̄ )2 (adding 0)

=n∑

i=1

(Xi − µ)2 +n∑

i=1

(X̄ − µ)2 − 2(X̄ − µ)n∑

i=1

(Xi − µ)

=n∑

i=1

(Xi − µ)2 + n(X̄ − µ)2 − 2(X̄ − µ)n(X̄ − µ)

=n∑

i=1

(Xi − µ)2 − n(X̄ − µ)2

Admin. Stuff Covariance Correlation Conditional Expectation To Do

Example 2 (cont.)

Now, take the expectation of that:

(n − 1)E [S2] =n∑

i=1

E [(Xi − µ)2]− nE [(X̄ − µ)2]

= nσ2 − nVar(X̄ )

= (n − 1)σ2

Thus, E [S2] = σ2, which is what we would want whenestimating the variance of some data.

Admin. Stuff Covariance Correlation Conditional Expectation To Do

Example 3

Let X = X1 + . . .+ Xn, where X1, . . . ,Xn are i.i.d Bernoullitrials with prob. of success p. Then, X is . . .Binomial with parameters (n,p). We’ve talked about thevariance of a Binomial r.v. before; this is how we can proveit:

Var(X ) = Var(n∑

i=1

Xi)

=n∑

i=1

Var(Xi)

Admin. Stuff Covariance Correlation Conditional Expectation To Do

Example 3 (cont.)

Note that:

Var(Xi) = E [X 2i ]− (E [Xi ])

2

= E [Xi ]− (E [Xi ])2

= p − p2

Then,Var(X ) = np(1− p)

Admin. Stuff Covariance Correlation Conditional Expectation To Do

Formalization

Def: The correlation of two r.v.’s X and Y , denoted byρ(X ,Y ), is defined as

ρ(X ,Y ) =Cov(X ,Y )√

Var(X )Var(Y )

Note that−1 ≤ ρ(X ,Y ) ≤ 1

If ρ(X ,Y ) = 0, then X and Y are said to be uncorrelated.X and Y are uncorrelated if and only if

E [XY ] = E [X ]E [Y ]

Correlation indicates the strength of a linear relationshipbetween two variables.

Admin. Stuff Covariance Correlation Conditional Expectation To Do

Properties

Covariance depends on the unit of measurement, thus,making it difficult to interpret a computed value.Correlation is scale independent:

ρ is not affected by a linear change in the units ofmeasurement (e.g. pound← kilo).If b and d are both positive or both negative, thenρ(a + bX , c + dY ) = ρ(X ,Y ).

When |ρ(X ,Y )| = 1, then Y = a + bX for some a,b.If X and Y are independent, then ρ = 0.However, ρ = 0 does not imply independence between Xand Y , as was seen in the previous example.

Admin. Stuff Covariance Correlation Conditional Expectation To Do

Example 4

Let

IA =

{1 if A occurs0 otherwise

, IB =

{1 if B occurs0 otherwise

Question: What is Cov(IA, IB)?

Admin. Stuff Covariance Correlation Conditional Expectation To Do

Example 4 (cont.)

Solution: We know that:

E [IA] = P(A)

E [IB] = P(B)

E [IAIB] = P(AB)

Then,

Cov(IA, IB) = P(AB)− P(A)P(B)

= P(B)[P(A|B)− P(A)]

The indicator variables for A and B are either positivelycorrelated, uncorrelated, or negatively correlated,depending on how B affects the prob. of A occurring.

Admin. Stuff Covariance Correlation Conditional Expectation To Do

Example 5

Let X be the number of 1’s and Y the number of 2’s thatoccur in n rolls of a fair die.Question: What is Corr(X ,Y )?Solution: X and Y and be talked about as the sum of . . .

Xi =

{1 roll i lands on 10 otherwise

, Yi =

{1 roll i lands on 20 otherwise

Admin. Stuff Covariance Correlation Conditional Expectation To Do

Example 5 (cont.)

Then, the covariance between Xi and Yi is . . .

Cov(Xi ,Yi) = E [XiYj ]− E [Xi ]E [Yj ]

=

{− 1

36 i = j1

36 −136 = 0 i 6= j

So, the covariance must be

Cov(∑

i

Xi ,∑

j

Yj) =∑

i

∑j

Cov(Xi ,Yj)

= − n36

Admin. Stuff Covariance Correlation Conditional Expectation To Do

Example 5 (cont.)

To finish this off, we need the variance of X and Y :

Var(X ) = Var(∑

i

Xi)

=∑

i

Var(Xi)

= n16

56

= n5

36= Var(Y )

So, the correlation must be:

Corr(X ,Y ) =Cov(X ,Y )√

Var(X )Var(Y )

=−n/36

(5n)/36= −1

5

Admin. Stuff Covariance Correlation Conditional Expectation To Do

Formalization

Def: If X and Y are jointly discrete r.v.’s, then theconditional expectation of X given Y = y , for all valuesof y s.t. pY (y) > 0, is defined as

E [X |Y = y ] =∑

x

x · pX |Y (x |y)

Def: If X and Y are jointly continuous r.v.’s, then theconditional expectation of X given Y = y , provided thatfY (y) > 0, is defined as

E [X |Y = y ] =

∫ ∞−∞

x · fX |Y (x |y)dx

Admin. Stuff Covariance Correlation Conditional Expectation To Do

Properties

Proposition 7.5.1:

E [X ] = E [E [X |Y ]]

That is, for a discrete r.v.,

E [X ] =∑

y

E [X |Y = y ]P(Y = y)

And, for a continuous r.v.,

E [X ] =

∫ ∞−∞

E [X |Y = y ]fY (y)dy

Law of Total Variance:

Var(X ) = Var(E(X |Y )) + E(Var(X |Y ))

Admin. Stuff Covariance Correlation Conditional Expectation To Do

Example 6

A miner is trapped in a mine containing 3 doors. The firstdoor leads to a tunnel that will take him to safety after 3hours of travel. The second door leads to a tunnel that willreturn him to the mine after 5 hours. The third door leadsto a tunnel that will return him to the mine after 7 hours.Question: If we assume that the miner is at all timesequally likely to choose any one of the doors, what is theexpected length of time until he reaches safety?

Admin. Stuff Covariance Correlation Conditional Expectation To Do

Example 6 (cont.)

Solution: Let X denote the amount of time (in hours) untilthe miner reaches safety and let Y denote the door heinitially chooses. So, E [X ] is . . .

E [X ] = E [X |Y = 1]P(Y = 1) + E [X |Y = 2]P(Y = 2)

+E [X |Y = 3]P(Y = 3)

=13

(E [X |Y = 1] + E [X |Y = 2] + E [X |Y = 3])

What are each of those conditional expectations?

E [X |Y = 1] = 3E [X |Y = 2] = 5 + E [X ]

E [X |Y = 3] = 7 + E [X ]

Admin. Stuff Covariance Correlation Conditional Expectation To Do

Example 6 (cont.)

Thus, plugging this back in, we get . . .

E [X ] =13

(3 + 5 + E [X ] + 7 + E [X ])

⇒ 13

E [X ] = 5

or E [X ] = 15

Admin. Stuff Covariance Correlation Conditional Expectation To Do

Now, covered section 7.4 and are on 7.5.Putting up HW5 after class.