14: conditional distributions

14: Conditional DistributionsDavid VarodayanFebruary 7, 2020Adapted from slides by Lisa Yan

1

Sum of independent random variables

2

Review

𝑋𝑋~Bin 𝑛𝑛1,𝑝𝑝 ,𝑌𝑌~Bin(𝑛𝑛2,𝑝𝑝) 𝑋𝑋 + 𝑌𝑌 ~Bin(𝑛𝑛1 + 𝑛𝑛2,𝑝𝑝)𝑋𝑋,𝑌𝑌 independent

𝑋𝑋~Poi 𝜆𝜆1 ,𝑌𝑌~Poi 𝜆𝜆2𝑋𝑋,𝑌𝑌 independent

𝑋𝑋 + 𝑌𝑌 ~Poi(𝜆𝜆1 + 𝜆𝜆2)

𝑋𝑋~𝒩𝒩 𝜇𝜇1,𝜎𝜎12𝑌𝑌~𝒩𝒩 𝜇𝜇2,𝜎𝜎22𝑋𝑋,𝑌𝑌 independent

𝑋𝑋 + 𝑌𝑌 ~𝒩𝒩(𝜇𝜇1 + 𝜇𝜇2,𝜎𝜎12 + 𝜎𝜎22)

Note: these also hold in the general case (≥ 2 variables)

Quick questionLet 𝐴𝐴~Bin 30, 0.01 and 𝐵𝐵~Bin 50, 0.02 be independent RVs. Can we use sum of independent Poisson RVs to approximate 𝑃𝑃 𝐴𝐴 + 𝐵𝐵 = 2 ?

3

Review

Quick questionLet 𝐴𝐴~Bin 30, 0.01 and 𝐵𝐵~Bin 50, 0.02 be independent RVs. Can we use sum of independent Poisson RVs to approximate 𝑃𝑃 𝐴𝐴 + 𝐵𝐵 = 2 ?

4

Review

𝐴𝐴 ≈ 𝑋𝑋~Poi 𝜆𝜆1 = 30 ⋅ 0.01 = 0.3𝐵𝐵 ≈ 𝑌𝑌~Poi 𝜆𝜆2 = 50 ⋅ 0.02 = 1 𝑃𝑃 𝐴𝐴 + 𝐵𝐵 = 2 ≈ 𝑃𝑃 𝑋𝑋 + 𝑌𝑌 = 2

Sol 1: Approximate as sum of Poissons

𝑋𝑋 + 𝑌𝑌~Poi 𝜆𝜆 = 𝜆𝜆1 + 𝜆𝜆2 = 1.3

=𝜆𝜆2

2!𝑒𝑒−𝜆𝜆

Sol 2: No approximation

𝑃𝑃 𝐴𝐴 + 𝐵𝐵 = 2 = �𝑘𝑘=0

2

𝑃𝑃 𝐴𝐴 = 𝑘𝑘 𝑃𝑃 𝐵𝐵 = 2 − 𝑘𝑘

≈ 0.2302

= �𝑘𝑘=0

230𝑘𝑘 0.01𝑘𝑘 0.99 30−𝑘𝑘 50

2 − 𝑘𝑘 0.022−𝑘𝑘0.9848+𝑘𝑘 ≈ 0.2327

Today’s plan

Sum of two Uniform independent RVs

Expectation of sum of two RVs

Discrete conditional distributions

Continuous conditional distributions

5

Ratio of probabilities interlude

Convolution

6

the convolutionof 𝑓𝑓𝑋𝑋 and 𝑓𝑓𝑌𝑌

For independent continuous random variables 𝑋𝑋 and 𝑌𝑌:

𝑓𝑓𝑋𝑋+𝑌𝑌 𝛼𝛼 = �−∞

∞𝑓𝑓𝑋𝑋 𝑘𝑘 𝑓𝑓𝑌𝑌 𝛼𝛼 − 𝑘𝑘 𝑑𝑑𝑘𝑘

the convolutionof 𝑝𝑝𝑋𝑋 and 𝑝𝑝𝑌𝑌

Recall for independent discrete random variables 𝑋𝑋 and 𝑌𝑌:

𝑃𝑃 𝑋𝑋 + 𝑌𝑌 = 𝑛𝑛 = �𝑘𝑘

𝑃𝑃 𝑋𝑋 = 𝑘𝑘 𝑃𝑃 𝑌𝑌 = 𝑛𝑛 − 𝑘𝑘

Sum of independent UniformsLet 𝑋𝑋~Uni 0,1 and 𝑌𝑌~Uni 0,1 be independent random variables.What is the distribution of 𝑋𝑋 + 𝑌𝑌, 𝑓𝑓𝑋𝑋+𝑌𝑌?

7


∞𝑓𝑓𝑋𝑋 𝑥𝑥 𝑓𝑓𝑌𝑌 𝛼𝛼 − 𝑥𝑥 𝑑𝑑𝑥𝑥

𝑋𝑋 and 𝑌𝑌independent+ continuous


∞𝑓𝑓𝑋𝑋 𝑘𝑘 𝑓𝑓𝑌𝑌 𝛼𝛼 − 𝑘𝑘 𝑑𝑑𝑘𝑘

𝑓𝑓𝑋𝑋 𝑘𝑘 𝑓𝑓𝑌𝑌 𝛼𝛼 − 𝑘𝑘 = 1 when:

𝑓𝑓𝑋𝑋+𝑌𝑌 𝛼𝛼 = �𝑎𝑎 0 ≤ 𝑎𝑎 ≤ 1

2 − 𝑎𝑎 1 ≤ 𝑎𝑎 ≤ 20 otherwise

0

1/2

𝛼𝛼

𝑓𝑓 𝑋𝑋+𝑌𝑌𝛼𝛼

1/2 1 3/2 2

1

00 ≤ 𝑘𝑘 ≤ 1 and0 ≤ 𝛼𝛼 − 𝑘𝑘 ≤ 1 ⇒ 𝛼𝛼 − 1 ≤ 𝑘𝑘 ≤ 𝛼𝛼

The precise integrationbounds on 𝑘𝑘 depend on 𝛼𝛼.

Today’s plan





8

Properties of Expectation, extended to two RVs1. Linearity:

𝐸𝐸 𝑎𝑎𝑋𝑋 + 𝑏𝑏𝑌𝑌 + 𝑐𝑐 = 𝑎𝑎𝐸𝐸 𝑋𝑋 + 𝑏𝑏𝐸𝐸 𝑌𝑌 + 𝑐𝑐

2. Expectation of a sum = sum of expectation:

𝐸𝐸 𝑋𝑋 + 𝑌𝑌 = 𝐸𝐸 𝑋𝑋 + 𝐸𝐸 𝑌𝑌

3. Unconscious statistician:

9

(we’ve seen this; we’ll prove this next)

𝐸𝐸 𝑔𝑔 𝑋𝑋,𝑌𝑌 = �𝑥𝑥

�𝑦𝑦

𝑔𝑔 𝑥𝑥,𝑦𝑦 𝑝𝑝𝑋𝑋,𝑌𝑌(𝑥𝑥,𝑦𝑦)

𝐸𝐸 𝑔𝑔 𝑋𝑋,𝑌𝑌 = �−∞

∞�−∞

∞𝑔𝑔 𝑥𝑥,𝑦𝑦 𝑓𝑓𝑋𝑋,𝑌𝑌 𝑥𝑥,𝑦𝑦 𝑑𝑑𝑥𝑥 𝑑𝑑𝑦𝑦

Proof of expectation of a sum of RVs

10

Even if the joint distribution is unknown, you can calculate the expectation of sum as sum of expectations.

Example: 𝐸𝐸 ∑𝑖𝑖=1𝑛𝑛 𝑋𝑋𝑖𝑖 = ∑𝑖𝑖=1𝑛𝑛 𝐸𝐸 𝑋𝑋𝑖𝑖 despite dependent trials 𝑋𝑋𝑖𝑖

𝐸𝐸 𝑋𝑋 + 𝑌𝑌 = 𝐸𝐸 𝑋𝑋 + 𝐸𝐸 𝑌𝑌

�𝑥𝑥

�𝑦𝑦

𝑥𝑥 + 𝑦𝑦 𝑝𝑝𝑋𝑋,𝑌𝑌 𝑥𝑥,𝑦𝑦 LOTUS,𝑔𝑔 𝑋𝑋,𝑌𝑌 = 𝑋𝑋 + 𝑌𝑌

= �𝑥𝑥

�𝑦𝑦

𝑥𝑥𝑝𝑝𝑋𝑋,𝑌𝑌 𝑥𝑥,𝑦𝑦 + �𝑥𝑥

�𝑦𝑦

𝑦𝑦𝑝𝑝𝑋𝑋,𝑌𝑌 𝑥𝑥,𝑦𝑦

Linearity of summations(cont. case: linearity of integrals)

= �𝑥𝑥

𝑥𝑥�𝑦𝑦

𝑝𝑝𝑋𝑋,𝑌𝑌 𝑥𝑥,𝑦𝑦 + �𝑦𝑦

𝑦𝑦�𝑥𝑥

𝑝𝑝𝑋𝑋,𝑌𝑌 𝑥𝑥,𝑦𝑦

Marginal PMFs for 𝑋𝑋 and 𝑌𝑌= �𝑥𝑥

𝑥𝑥𝑝𝑝𝑋𝑋 𝑥𝑥 + �𝑦𝑦

𝑦𝑦𝑝𝑝𝑌𝑌 𝑦𝑦

= 𝐸𝐸 𝑋𝑋 + 𝐸𝐸[𝑌𝑌]

𝐸𝐸 𝑋𝑋 + 𝑌𝑌 =

Expectations of common RVs

11

𝑋𝑋~Bin(𝑛𝑛,𝑝𝑝) 𝐸𝐸 𝑋𝑋 = 𝑛𝑛𝑝𝑝

𝐸𝐸 𝑋𝑋 = 𝐸𝐸 �𝑖𝑖=1

𝑛𝑛

𝑋𝑋𝑖𝑖 = �𝑖𝑖=1

𝑛𝑛

𝐸𝐸 𝑋𝑋𝑖𝑖 = �𝑖𝑖=1

𝑛𝑛

𝑝𝑝 = 𝑛𝑛𝑝𝑝𝑋𝑋 = �𝑖𝑖=1

𝑛𝑛

𝑋𝑋𝑖𝑖Let 𝑋𝑋𝑖𝑖 = 𝑖𝑖th trial is heads𝑋𝑋𝑖𝑖~Ber 𝑝𝑝 ,𝐸𝐸 𝑋𝑋𝑖𝑖 = 𝑝𝑝

Review


12


𝑌𝑌~NegBin(𝑟𝑟, 𝑝𝑝) 𝐸𝐸 𝑌𝑌 = 𝑟𝑟𝑝𝑝

𝑋𝑋 = �𝑖𝑖=1

𝑛𝑛

𝑋𝑋𝑖𝑖 𝐸𝐸 𝑋𝑋 = 𝐸𝐸 �𝑖𝑖=1

𝑛𝑛


𝑛𝑛


𝑛𝑛

𝑝𝑝 = 𝑛𝑛𝑝𝑝Let 𝑋𝑋𝑖𝑖 = 𝑖𝑖th trial is heads𝑋𝑋𝑖𝑖~Ber 𝑝𝑝 ,𝐸𝐸 𝑋𝑋𝑖𝑖 = 𝑝𝑝

How should we define 𝑌𝑌𝑖𝑖? A. 𝑌𝑌𝑖𝑖 = 𝑖𝑖th trial is heads. 𝑌𝑌𝑖𝑖~Ber 𝑝𝑝 , 𝑖𝑖 = 1, … ,𝑛𝑛B. 𝑌𝑌𝑖𝑖 = # trials to get 𝑖𝑖th success (after 𝑖𝑖 − 1 th success)

𝑌𝑌𝑖𝑖~Geo 𝑝𝑝 , 𝑖𝑖 = 1, … , 𝑟𝑟C. 𝑌𝑌𝑖𝑖 = # successes in 𝑛𝑛 trials

𝑌𝑌𝑖𝑖~Bin 𝑛𝑛,𝑝𝑝 , 𝑖𝑖 = 1, … , 𝑟𝑟, we look for 𝑃𝑃 𝑌𝑌𝑖𝑖 = 1

𝑌𝑌 = �𝑖𝑖=1

?

𝑌𝑌𝑖𝑖

Suppose:


13


𝑌𝑌~NegBin(𝑟𝑟, 𝑝𝑝) 𝐸𝐸 𝑌𝑌 = 𝑟𝑟𝑝𝑝

𝑋𝑋 = �𝑖𝑖=1

𝑛𝑛

𝑋𝑋𝑖𝑖 𝐸𝐸 𝑋𝑋 = 𝐸𝐸 �𝑖𝑖=1

𝑛𝑛


𝑛𝑛


𝑛𝑛

𝑝𝑝 = 𝑛𝑛𝑝𝑝Let 𝑋𝑋𝑖𝑖 = 𝑖𝑖th trial is heads𝑋𝑋𝑖𝑖~Ber 𝑝𝑝 ,𝐸𝐸 𝑋𝑋𝑖𝑖 = 𝑝𝑝

𝑌𝑌 = �𝑖𝑖=1

𝑟𝑟

𝑌𝑌𝑖𝑖

Let 𝑌𝑌𝑖𝑖 = # trials to get 𝑖𝑖thsuccess (after𝑖𝑖 − 1 th success)

𝑌𝑌𝑖𝑖~Geo 𝑝𝑝 ,𝐸𝐸 𝑌𝑌𝑖𝑖 = 1𝑝𝑝

𝐸𝐸 𝑌𝑌 = 𝐸𝐸 �𝑖𝑖=1

𝑟𝑟

𝑌𝑌𝑖𝑖 = �𝑖𝑖=1

𝑟𝑟

𝐸𝐸 𝑌𝑌𝑖𝑖 = �𝑖𝑖=1

𝑟𝑟1𝑝𝑝

=𝑟𝑟𝑝𝑝

Midterm exam

When: Monday, February 10, 7:00pm-9:00pmWhere: Cubberley Auditorium

Not permitted: book/computer/calculatorPermitted: Three 8.5”x11” double-sided sheets of notes

Covers: Up to (and including) week 4 + Lecture Notes 11Practice: http://web.stanford.edu/class/cs109/exams/midterm.htmlReview session: Tomorrow Saturday, 3-5pm, STLC 111

Announcements

14

not recorded; materials will be posted though

http://web.stanford.edu/class/cs109/exams/midterm.html

Today’s plan





15

CS109 roadmap

16

Multiple events:

𝑃𝑃 𝐸𝐸 𝐹𝐹 =𝑃𝑃 𝐸𝐸𝐹𝐹𝑃𝑃 𝐹𝐹

conditional probability

𝑃𝑃 𝐸𝐸𝐹𝐹 = 𝑃𝑃 𝐸𝐸 𝑃𝑃(𝐹𝐹)

independence

Joint (Multivariate) distributions

joint PMF/PDF

𝑝𝑝𝑋𝑋,𝑌𝑌 𝑥𝑥,𝑦𝑦𝑓𝑓𝑋𝑋,𝑌𝑌 𝑥𝑥,𝑦𝑦

intersection

𝑃𝑃 𝐸𝐸 ∩ 𝐹𝐹= 𝑃𝑃 𝐸𝐸𝐹𝐹

conditional distributions?

today

independentRVs

sum of independent RVs

Discrete conditional distributionsRecall the definition of the conditional probability of event 𝐸𝐸 given event 𝐹𝐹:

𝑃𝑃 𝐸𝐸 𝐹𝐹 =𝑃𝑃 𝐸𝐸𝐹𝐹𝑃𝑃 𝐹𝐹

For discrete random variables 𝑋𝑋 and 𝑌𝑌, the conditional PMF of 𝑋𝑋 given 𝑌𝑌 is

𝑃𝑃 𝑋𝑋 = 𝑥𝑥 𝑌𝑌 = 𝑦𝑦 =𝑃𝑃 𝑋𝑋 = 𝑥𝑥,𝑌𝑌 = 𝑦𝑦

𝑃𝑃 𝑌𝑌 = 𝑦𝑦

𝑝𝑝𝑋𝑋|𝑌𝑌 𝑥𝑥|𝑦𝑦 =𝑝𝑝𝑋𝑋,𝑌𝑌 𝑥𝑥,𝑦𝑦𝑝𝑝𝑌𝑌 𝑦𝑦

17

Different notation,same idea:

Quick checkNumber or function?

1. 𝑃𝑃 𝑋𝑋 = 2 𝑌𝑌 = 5

2. 𝑃𝑃 𝑋𝑋 = 𝑥𝑥 𝑌𝑌 = 5

3. 𝑃𝑃 𝑋𝑋 = 2 𝑌𝑌 = 𝑦𝑦

4. 𝑃𝑃 𝑋𝑋 = 𝑥𝑥 𝑌𝑌 = 𝑦𝑦

18

True or false?

5.

6.

7. �𝑥𝑥

�𝑦𝑦

𝑃𝑃 𝑋𝑋 = 𝑥𝑥|𝑌𝑌 = 𝑦𝑦 𝑃𝑃 𝑌𝑌 = 𝑦𝑦 = 1

�𝑦𝑦

𝑃𝑃 𝑋𝑋 = 2|𝑌𝑌 = 𝑦𝑦 = 1

�𝑥𝑥

𝑃𝑃 𝑋𝑋 = 𝑥𝑥|𝑌𝑌 = 5 = 1



Discrete probabilities of CS109Each student responds with

(major 𝑋𝑋, year 𝑌𝑌, bool excited about midterm 𝑀𝑀):

19

CS𝑋𝑋 = 1

SymSys/MCS/EE𝑋𝑋 = 2

Other Eng/Sci/Math𝑋𝑋 = 3

Hum/SocSci/

Ling𝑋𝑋 = 4

Double major𝑋𝑋 = 5

Undec.𝑋𝑋 = 6

𝑌𝑌 = 1 .006 .000 .000 .000 .000 .000𝑌𝑌 = 2 .155 .069 .034 .006 .023 .029𝑌𝑌 = 3 .092 .063 .023 .006 .006 .000𝑌𝑌 = 4 .017 .029 .011 .006 .000 .000𝑌𝑌 ≥ 5 .029 .006 .011 .006 .000 .000𝑌𝑌 = 1 .000 .000 .000 .000 .000 .000𝑌𝑌 = 2 .126 .040 .017 .017 .000 .017𝑌𝑌 = 3 .046 .040 .006 .011 .000 .006𝑌𝑌 = 4 .006 .006 .000 .000 .000 .000𝑌𝑌 ≥ 5 .006 .000 .017 .011 .000 .000

𝑀𝑀=

0𝑀𝑀

=1

𝑃𝑃 𝑌𝑌 ≥ 5,𝑋𝑋 = 1 ,𝑀𝑀 = 1

Joint PMF of 𝑋𝑋,𝑌𝑌,𝑀𝑀

CS𝑋𝑋 = 1


Other Eng/Sci/

Math𝑋𝑋 = 3

Hum/SocSci/

Ling𝑋𝑋 = 4


Undec.𝑋𝑋 = 6

𝑀𝑀 = 0 .299 .167 .080 .023 .029 .029𝑀𝑀 = 1 .184 .086 .040 .040 .000 .023

𝑃𝑃 𝑋𝑋 = 1 ,𝑀𝑀 = 1 = 0.18

Joint PMF of 𝑋𝑋,𝑀𝑀

Discrete probabilities of CS109The below tables are probability tables for the conditional PMFs𝑃𝑃 𝑀𝑀 = 𝑚𝑚|𝑋𝑋 = 𝑥𝑥 and 𝑃𝑃 𝑋𝑋 = 𝑥𝑥|𝑀𝑀 = 𝑚𝑚 .

1. Which table is which?2. Fill in the missing probability.

20

CS𝑋𝑋 = 1


Other Eng/Sci/

Math𝑋𝑋 = 3

Hum/SocSci/

Ling𝑋𝑋 = 4


Undec.𝑋𝑋 = 6

𝑀𝑀 = 0 .299 .167 .080 .023 .029 .029𝑀𝑀 = 1 .184 .086 .040 .040 .000 .023

𝑃𝑃 𝑋𝑋 = 1 ,𝑀𝑀 = 1 = 0.18

Joint PMF of 𝑋𝑋,𝑀𝑀

CS𝑋𝑋 = 1


Other Eng/Sci/

Math𝑋𝑋 = 3

Hum/SocSci/

Ling𝑋𝑋 = 4


Undec.𝑋𝑋 = 6

𝑀𝑀 = 0 .619 .659 .667 .364 1.000 .556𝑀𝑀 = 1 .381 .341 .333 .636 .000 .444

CS𝑋𝑋 = 1


Other Eng/Sci/

Math𝑋𝑋 = 3

Hum/SocSci/

Ling𝑋𝑋 = 4


Undec.𝑋𝑋 = 6

𝑀𝑀 = 0 .477 .266 .128 .037 .046 .046𝑀𝑀 = 1 .231 .108 .108 .000 .062



Be clear about which probabilities should sum to one.

Today’s plan





21


Relative probabilities of continuous random variablesLet 𝑋𝑋 = time to finish problem set 2.Suppose 𝑋𝑋~𝒩𝒩 10,2 .How much more likely are you to complete in 10 hours than 5 hours?

22

𝑃𝑃 𝑋𝑋 = 10𝑃𝑃 𝑋𝑋 = 5

=

A. 00

= undefined

B. 𝑓𝑓 10𝑓𝑓 5

Relative probabilities of continuous random variables

23

Ratios of PDFs are meaningful

𝑃𝑃 𝑋𝑋 = 𝑎𝑎 = 𝑃𝑃 𝑎𝑎 −𝜀𝜀2≤ 𝑋𝑋 ≤ 𝑎𝑎 +

𝜀𝜀2

𝑃𝑃 𝑋𝑋 = 10𝑃𝑃 𝑋𝑋 = 5

=𝑓𝑓 10𝑓𝑓 5

𝑃𝑃 𝑋𝑋 = 𝑎𝑎𝑃𝑃 𝑋𝑋 = 𝑏𝑏

=𝜀𝜀𝑓𝑓 𝑎𝑎𝜀𝜀𝑓𝑓 𝑏𝑏

=𝑓𝑓 𝑎𝑎𝑓𝑓 𝑏𝑏Therefore

= �𝑎𝑎−𝜀𝜀2

𝑎𝑎+𝜀𝜀2𝑓𝑓 𝑥𝑥 𝑑𝑑𝑥𝑥 ≈ 𝜀𝜀𝑓𝑓(𝑎𝑎)

=

1𝜎𝜎 2𝜋𝜋

𝑒𝑒−10 − 𝜇𝜇 2

2𝜎𝜎2

1𝜎𝜎 2𝜋𝜋

𝑒𝑒−5 − 𝜇𝜇 2

2𝜎𝜎2

=𝑒𝑒−

10 −10 2

2⋅2

𝑒𝑒−5 − 10 2

2⋅2

=𝑒𝑒0

𝑒𝑒−254

= 518

Let 𝑋𝑋 = time to finish problem set 2.Suppose 𝑋𝑋~𝒩𝒩 10,2 .How much more likely are you to complete in 10 hours than 5 hours?

Today’s plan





24


For continuous RVs 𝑋𝑋 and 𝑌𝑌, the conditional PDF of 𝑋𝑋 given 𝑌𝑌 is

𝑓𝑓𝑋𝑋|𝑌𝑌 𝑥𝑥|𝑦𝑦 =𝑓𝑓𝑋𝑋,𝑌𝑌 𝑥𝑥,𝑦𝑦𝑓𝑓𝑌𝑌 𝑦𝑦

Intuition:

Note that conditional PDF 𝑓𝑓𝑋𝑋|𝑌𝑌 is a true density:


25


𝑃𝑃 𝑌𝑌 = 𝑦𝑦𝑓𝑓𝑋𝑋|𝑌𝑌 𝑥𝑥 𝑦𝑦 𝜀𝜀𝑋𝑋 =

𝑓𝑓𝑋𝑋,𝑌𝑌 𝑥𝑥,𝑦𝑦 𝜀𝜀𝑥𝑥𝜀𝜀𝑦𝑦𝑓𝑓𝑌𝑌 𝑦𝑦 𝜀𝜀𝑦𝑦

�−∞

∞𝑓𝑓𝑥𝑥 𝑥𝑥|𝑦𝑦 𝑑𝑑𝑥𝑥 = �

−∞

∞ 𝑓𝑓𝑋𝑋,𝑌𝑌 𝑥𝑥,𝑦𝑦𝑓𝑓𝑌𝑌 𝑦𝑦

𝑑𝑑𝑥𝑥 =∫−∞∞ 𝑓𝑓𝑋𝑋,𝑌𝑌 𝑥𝑥,𝑦𝑦 𝑑𝑑𝑥𝑥

𝑓𝑓𝑌𝑌 𝑦𝑦=𝑓𝑓𝑌𝑌 𝑦𝑦𝑓𝑓𝑌𝑌 𝑦𝑦

= 1

Tracking in 2-D space?

27

You want to knowthe 2-D location ofan object.

Your satellite pinggives you a noisy 1-Dmeasurement of thedistance of the objectfrom the satellite (0,0).

Tracking in 2-D space• You have a prior belief about the 2-D location of an object, 𝑋𝑋,𝑌𝑌 .• You observe a noisy distance measurement, 𝐷𝐷 = 4.• What is your updated (posterior) belief of the 2-D location of the object after

observing the measurement?

28

posteriorbelief

likelihood(of evidence)

priorbelief

normalization constant

Recall Bayes terminology:

𝑓𝑓𝑋𝑋,𝑌𝑌|𝐷𝐷 𝑥𝑥, 𝑦𝑦|𝑑𝑑 =𝑓𝑓𝐷𝐷|𝑋𝑋,𝑌𝑌 𝑑𝑑|𝑥𝑥, 𝑦𝑦 𝑓𝑓𝑋𝑋,𝑌𝑌 𝑥𝑥, 𝑦𝑦

𝑓𝑓𝐷𝐷 𝑑𝑑

Top-down view



29

Let 𝑋𝑋,𝑌𝑌 = object’s 2-D location.(your satellite is at (0,0)

Suppose the prior distribution is asymmetric bivariate normal distribution:

𝑥𝑥

𝑦𝑦

𝑓𝑓 𝑋𝑋,𝑌𝑌𝑥𝑥,𝑦𝑦

3-D view

𝑓𝑓𝑋𝑋,𝑌𝑌 𝑥𝑥,𝑦𝑦 =1

2𝜋𝜋22𝑒𝑒−

𝑥𝑥−3 2+ 𝑦𝑦−3 2

2 22

normalizing constant

= 𝐾𝐾1 ⋅ 𝑒𝑒− 𝑥𝑥−3 2+ 𝑦𝑦−3 2

8



30

Let 𝐷𝐷 = distance from the satellite (radially).Suppose you knew your actual position: 𝑥𝑥,𝑦𝑦 .• 𝐷𝐷 is still noisy! Suppose noise is unit variance: 𝜎𝜎2 = 1• On average, 𝐷𝐷 is your actual position: 𝜇𝜇 = 𝑥𝑥2 + 𝑦𝑦2

If you knew your actual location 𝑥𝑥,𝑦𝑦 , you could say how likely

a measurement 𝐷𝐷 = 4 is!!

Tracking in 2-D space

31

prob

abili

ty d

ensi

ty 𝜇𝜇 = 𝑥𝑥2 + 𝑦𝑦2

𝜎𝜎2 = 1

Distance measurement of a ping is normal with respect to the true location.



32

If noise is normal: 𝐷𝐷|𝑋𝑋,𝑌𝑌~𝑁𝑁 𝜇𝜇 = 𝑥𝑥2 + 𝑦𝑦2 ,𝜎𝜎2 = 1

Distance measurement of a ping is normal with respect to the true location.


a measurement 𝐷𝐷 = 4 is!!

prob

abili

ty d

ensi

ty 𝜇𝜇 = 𝑥𝑥2 + 𝑦𝑦2

𝜎𝜎2 = 1



33


a measurement 𝐷𝐷 = 4 is!!𝐷𝐷|𝑋𝑋,𝑌𝑌~𝒩𝒩 𝜇𝜇 = 𝑥𝑥2 + 𝑦𝑦2 ,𝜎𝜎2 = 1

𝑓𝑓𝐷𝐷|𝑋𝑋,𝑌𝑌 𝐷𝐷 = 𝑑𝑑|𝑋𝑋 = 𝑥𝑥,𝑌𝑌 = 𝑦𝑦 =1

𝜎𝜎 2𝜋𝜋𝑒𝑒− 𝑑𝑑−𝜇𝜇 2

2𝜎𝜎2

=12𝜋𝜋

𝑒𝑒− 𝑑𝑑− 𝑥𝑥2+𝑦𝑦2

2

2 = 𝑒𝑒− 𝑑𝑑− 𝑥𝑥2+𝑦𝑦2

2

2

normalizing constant

𝐾𝐾2 ⋅

Tracking in 2-D spaceWhat is your updated (posterior) belief of the 2-D location of the object after observing the measurement?

35

𝑓𝑓𝑋𝑋,𝑌𝑌|𝐷𝐷 𝑋𝑋 = 𝑥𝑥,𝑌𝑌 = 𝑦𝑦|𝐷𝐷 = 4 =𝑓𝑓𝐷𝐷|𝑋𝑋,𝑌𝑌 𝐷𝐷 = 4|𝑋𝑋 = 𝑥𝑥,𝑌𝑌 = 𝑦𝑦 𝑓𝑓𝑋𝑋,𝑌𝑌 𝑥𝑥,𝑦𝑦

𝑓𝑓(𝐷𝐷 = 4)Bayes’Theorem

=𝐾𝐾2 ⋅ 𝑒𝑒

−4− 𝑥𝑥2+𝑦𝑦2

2

2 ⋅ 𝐾𝐾1 ⋅ 𝑒𝑒−

𝑥𝑥−3 2+ 𝑦𝑦−3 2

8

𝑓𝑓(𝐷𝐷 = 4)

likelihood of 𝐷𝐷 = 4 prior belief

=𝐾𝐾3 ⋅ 𝑒𝑒

−4− 𝑥𝑥2+𝑦𝑦2

2

2 +𝑥𝑥−3 2+ 𝑦𝑦−3 2

8

𝑓𝑓(𝐷𝐷 = 4)

= 𝐾𝐾4 ⋅ 𝑒𝑒−

4− 𝑥𝑥2+𝑦𝑦22

2 + 𝑥𝑥−3 2+ 𝑦𝑦−3 2

8

Tracking in 2-D space: Posterior belief

36

𝑓𝑓𝑋𝑋,𝑌𝑌 𝑥𝑥,𝑦𝑦 = 𝐾𝐾1 ⋅ 𝑒𝑒− 𝑥𝑥−3 2+ 𝑦𝑦−3 2

8

Prior belief Posterior beliefTop-down view

𝑦𝑦

3-D view

𝑥𝑥

𝑦𝑦

𝑥𝑥

Top-down view 3-D view

𝑓𝑓𝑋𝑋,𝑌𝑌|𝐷𝐷 𝑥𝑥,𝑦𝑦|4 =

𝐾𝐾4⋅ 𝑒𝑒−

4− 𝑥𝑥2+𝑦𝑦22

2 +𝑥𝑥−3 2+ 𝑦𝑦−3 2

8

14: conditional distributions

Documents