Expectation Continued: TailSum, Coupon Collector, and

Functions of RVs

CS 70, Summer 2019

Lecture 20, 7/29/19

1 / 26

Last Time...

I Expectation describes the weightedaverage of a RV.

I For more complicated RVs, use linearity

Today:

I Proof of linearity of expectation

I The tail sum formula

I Expectations of Geometric and Poisson

I Expectation of a function of an RV

2 / 26

Sanity CheckLet X be a RV that takes on values in A.Let Y be a RV that takes on values in B.Let c 2 R be a constant.

Both c · X and X + Y are also RVs!

3 / 26

Xty

Values a,

a Values :{ atb,

AEA,bEB}

Probe : probs :

IPCC X = ca ]=p[X=a ] ipfxty - Atb ]=P[X=a,7=b ]

Proof of Linearity of Expectation IRecall linearity of expectation:

E[X1 + . . .+ Xn] = E[X1] + . . .+ E[Xn]

For constant c , E[cXi ] = c · E[Xi ]

First, we show E[cXi ] = c · E[Xi ]:

4 / 26

Xi values in A.

Efc Xi ) = I ¢a) IPCX = a ]A E A

= c ÷ ,

a . IP EX - a ] = c IE [ X i ]-

EC Xi ]

Proof of Linearity of Expectation IINext, we show E[X + Y ] = E[X ] + E[Y ].

Two variables to n variables?

5 / 26

X has values MA, y has values in B

.

Efxty ]= I ( at b) p[X=a , Y - b ]AEA ,

BEB

rewrite" 4*94 . b)ate,£⇒b

. PCx=aY=b ]

AFAFEB -#

of= ¥aaq⇒pCx=a ,

't b) tfepbff.PK/--a.Y--b]

EE 7¥ ¥=b]BEBAEA

= qfaa.ipcx-ay-qf.gg b - PLY - b ]

TEXT TEEInduction .

The Tail Sum FormulaLet X be a RV with values in {0, 1, 2, . . . , n}.We use “tail” to describe P[X � i ].

What doesP1i=1 P[X � i ] look like?

Small example: X only takes values {0, 1, 2}:

6 / 26

Anon - neg

integers

team

€7PEX Z i ] = PEX 21 ] t PEX 22 ]/ \

=

p Cx - I] t PIX= 2) t¥ EX = 2 ]

Or PEX +1. IPCX -

- I ] -12 . PCX -

- 2 ]

EF

The Tail Sum FormulaThe tail sum formula states that:

E[X ] =1X

i=1

P[X � i ]

Proof: Let pi = P[X = i ].

7 / 26

x Ronningers=

€,9P[ x Z i ] = PEX Z IT t P EX Z 2) t PEX 233?

=

( p , t¥pzt. . . ) t (¥713t Pat . . . ) !

+ I p ,t Pg t

. . . )O ' P

I . p , t 2 ' p a t 3 . Pz t. - -

= E EXT

Expectation of a Geometric ILet X ⇠ Geometric(p).

P[X � i ] =

Apply the tail sum formula:

8 / 26

i - I

H - p )

÷,

-

pfxzi ] = It ( I -

p ) th - pit. . .

-Geometric series

firstEsma =L,

r -- H - P )

-

- Fr -

- Ifpi

Expectation of a Geometric IIUse memorylessness: the fact that thegeometric RV “resets” after each trial.

Two Cases:

9 / 26

X - aeomcp)

ft art trials failedP first trial .succeedfall 1st trial .

"

win in2day .

" " win in1€E[X] days

"

" reset,

"

day 2 looks identicalto day I .

ECX ] =p (1) t ( I - p ) f ITEM ]Solve

for ECX ].

PIECX I ' HttpEfxtipt

Expectation of a Geometric IIILastly, an intuitive but non-rigorous idea.

Let Xi be an indicator variable for success in asingle trial. Recall trials are i.i.d.

Xi ⇠

E[X1 + X2 + . . .+ Xk ] =

10 / 26

X - Geom Cp)

Ber ( p )use linearity of expectation

Efx , It ECX.IT . ..tECXn ]

= KEEX , ]= Kp

Need K = pt in order for ECX ,t

. . .

t Xx ) = I-

# of successes .

Coupon Collector I(Note 19.) I’m out collecting trading cards.

There are n types total. I get a random tradingcard every time I buy a cereal box.

What is the expected number of boxes I need tobuy in order to get all n trading cards?

High level picture:

11 / 26

Time T , Tz -13

=Lxz Xz X4

get get get1St 2nd 3rd

card ! card ! card .

Coupon Collector II

Let Xi =

What is the dist. of X1?

What is the dist. of X2?

What is the dist. of X3?

In general, what is the dist. of Xi?

12 / 26

time / " # boxes"

between Ci-Dth card

and the

ithGeom CL)X ,

=L always .

Xz ~ Geom ( ht )

Xz ~ Geom ( MT )

Xi - Geom ( nifty

Coupon Collector IIILet X =

X =

E[X ] =

13 / 26

total # boxes to get all n cards

X ,t Xzt . -

t XnIE [ Xitxzt

. . . txn ] linearity= Efx , ] HECK ) t

. - TECXN] d

Xi n Geom I n-ci ,

It IT t # t. . . thy

%= nEi e- ? ?

Aside: (Partial) Harmonic SeriesHarmonic Series:

P1k=1

1

k

Approximation forPnk=1

1

k in terms of n?

14 / 26

Diverges

E.it#fIdx=lnx/7--lnn-In1/"

↳x inn .

⇒ coupon collector : E EXT F N log N.

¥¥IE¥. EE

BreakA Bad Harmonic Series Joke...

A countably infinite number of mathematicians

walk into a bar. The first one orders a pint of

beer, the second one orders a half pint, the third

one orders a third of a pint, the fourth one orders

a fourth of a pint, and so on.

The bartender says ...

15 / 26

Expectation of a Poisson IRecall the Poisson distribution: values 0, 1, 2, . . . ,

P[X = i ] =�i

i !e��

We can use the definition to find E[X ]!

16 / 26

X - poi CX )

ECXI -

- Eto if¥1e-

a ) touswrwiesfor

⇒

te.

=

= Het -

- DD

Expectation of a Poisson IIOptional but intuitive / non-rigorous approach:

Think of a Poisson(�) as a Bin(n, �n ) distribution,taken as n !1.

Let X ⇠ Bin(n, �n ).

X =

17 / 26

* I 71¥

Rest of Today: Functions of RVs!Recall X from Lecture 19:

X =

8><

>:

1 wp 0.4

1

2wp 0.25

�12

wp 0.35

Refresh your memory: What is X 2?

18 / 26

XI ftWp . 0.4

-4 Wp 0.25

¥ Wp 0.35

= { ¥WP 0.4

Wp 0.6

Example: Functions of RVs

X 2 =

(1 wp 0.4

1

4wp 0.6

What is E[X 2]?

What is E[3X 2 � 5]?

19 / 26

Efx 21 -

- I . PC XIII t 4 PIX '-

- I ]= I .0.4t t . 0.6=0.55

linearity of exp .

IE [3×2] - ECS ) 3 EHF- 5

310.55 ) - 5

In General: Functions of RVs

Let X be a RV with values in A.Distribution of f (X ):

E[f (X )] =

20 / 26

f Cx ) = {f !!)

wppxf-ajac.CAHa ) pfX=a ]

Square of a Bernoulli

Let X ⇠ Bernoulli(p).Write out the distribution of X .

What is X 2? E[X 2]?

21 / 26

X -

- { f wppWp tp

X'

-- { to IfIp ECM =p .

Product of RVs

Let X be a RV with values in A.Let Y be a RV with values in B.

XY is also a RV! What is its distribution?(Use the joint distribution!)

22 / 26

Exercise

Product of Two Bernoullis

Let X ⇠ Bernoulli(p1), and Y ⇠ Bernoulli(p2).X and Y are independent.

What is the distribution of XY ?

What is E[XY ]?

23 / 26

Exercise

Square of a Binomial I

Let X ⇠ Bin(n, p).Decompose into Xi ⇠ Bernoulli(p).

X =

E[X ] =

24 / 26

X , t X zt

. . .

t X n

IE [ X ,t X at . . .

t X n ]

= Efx . I t IEC X a It . . .t Ef X n ]

= n p .

Square of a Binomial IIRecall, E[X 2i ] = 1, and E[XiXj ] = p2.

25 / 26

PECXZ ]=Ef( X ,tXzt .

.

.tt/n)2J=tEfCXi2tXz2t...tXn4t(XiXztXiXst . . . ))- -

square terms cross terms

n of them MIN - 1) of

= Ef square terms ] TEAMS terms ] them .I ::*::*: " "

:i÷i :*.

ECXIZ ] - P Efx ,Xz3=PZ> = nptnln - 1) P2

SummaryToday:

I Proof of linearity of expectation: did not use

independence, but did use joint distribution

I Tail sum for non-negative int.-valued RVs!

I Coupon Collector: break problem down into a

sum of geometrics.

I Expectation of a function of an RV: canapply definition and linearity of expectation

(after expanding) as well!!

26 / 26