6 hhkyrd - home | computer science at ubcschmidtm/courses/340-f17/t3.pdf · pp2`pb2r rl bp2"...

28

Upload: others

Post on 22-Jan-2021

0 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: 6 HHkyRd - Home | Computer Science at UBCschmidtm/Courses/340-F17/T3.pdf · Pp2`pB2r RL Bp2" v2b*H bbB}2` kLQM@S ` K2i`B+JQ/2Hb.2}MBiBQMb ELL jh` BMBM;-h2biBM;- M/o HB/ iBQMa2i kfR9
Page 2: 6 HHkyRd - Home | Computer Science at UBCschmidtm/Courses/340-F17/T3.pdf · Pp2`pB2r RL Bp2" v2b*H bbB}2` kLQM@S ` K2i`B+JQ/2Hb.2}MBiBQMb ELL jh` BMBM;-h2biBM;- M/o HB/ iBQMa2i kfR9
Page 3: 6 HHkyRd - Home | Computer Science at UBCschmidtm/Courses/340-F17/T3.pdf · Pp2`pB2r RL Bp2" v2b*H bbB}2` kLQM@S ` K2i`B+JQ/2Hb.2}MBiBQMb ELL jh` BMBM;-h2biBM;- M/o HB/ iBQMa2i kfR9

yi i i y

xi i i X

xij j i

d i

Page 4: 6 HHkyRd - Home | Computer Science at UBCschmidtm/Courses/340-F17/T3.pdf · Pp2`pB2r RL Bp2" v2b*H bbB}2` kLQM@S ` K2i`B+JQ/2Hb.2}MBiBQMb ELL jh` BMBM;-h2biBM;- M/o HB/ iBQMa2i kfR9

yi i i y

xi i i X

xij j i

d i

Page 5: 6 HHkyRd - Home | Computer Science at UBCschmidtm/Courses/340-F17/T3.pdf · Pp2`pB2r RL Bp2" v2b*H bbB}2` kLQM@S ` K2i`B+JQ/2Hb.2}MBiBQMb ELL jh` BMBM;-h2biBM;- M/o HB/ iBQMa2i kfR9

yi

p(yi|xi) ∝ p(xi|yi)p(yi)

we want to compare P(y=c|x_i) for different values of cand choose the maximum value
Page 6: 6 HHkyRd - Home | Computer Science at UBCschmidtm/Courses/340-F17/T3.pdf · Pp2`pB2r RL Bp2" v2b*H bbB}2` kLQM@S ` K2i`B+JQ/2Hb.2}MBiBQMb ELL jh` BMBM;-h2biBM;- M/o HB/ iBQMa2i kfR9

yi

p(yi|xi) ∝ p(xi|yi)p(yi)

Page 7: 6 HHkyRd - Home | Computer Science at UBCschmidtm/Courses/340-F17/T3.pdf · Pp2`pB2r RL Bp2" v2b*H bbB}2` kLQM@S ` K2i`B+JQ/2Hb.2}MBiBQMb ELL jh` BMBM;-h2biBM;- M/o HB/ iBQMa2i kfR9

p(xi|yi)p(yi) = p(xi, yi) = p(xi1, ..., xid, yi)

p(xi1, ..., xid, yi) = p(xi1|xi2, ..., xid, yi)p(xi2, ..., xid, yi)

= ...

= p(xi1|xi2, ..., xid, yi)p(xi2|xi3, ..., xid, yi) p(xid|yi)p(yi)

xi yi

p(xij |xij+1, ..., xid, yi) = p(xij |yi)

p(yi, xi) ∝ p(yi)d∏

j=1

p(xij |yi)

P(xi1,xi2,…,yi) = P(xi1|xi2,xi3,…yi) P(xi2,xi3,…,yi)
Page 8: 6 HHkyRd - Home | Computer Science at UBCschmidtm/Courses/340-F17/T3.pdf · Pp2`pB2r RL Bp2" v2b*H bbB}2` kLQM@S ` K2i`B+JQ/2Hb.2}MBiBQMb ELL jh` BMBM;-h2biBM;- M/o HB/ iBQMa2i kfR9

p(xi|yi)p(yi) = p(xi, yi) = p(xi1, ..., xid, yi)

p(xi1, ..., xid, yi) = p(xi1|xi2, ..., xid, yi)p(xi2, ..., xid, yi)

= ...

= p(xi1|xi2, ..., xid, yi)p(xi2|xi3, ..., xid, yi) p(xid|yi)p(yi)

xi yi

p(xij |xij+1, ..., xid, yi) = p(xij |yi)

p(yi, xi) ∝ p(yi)d∏

j=1

p(xij |yi)

P(xi1|yi)
P(xi2|yi)
P(xid|yi)
P(yi)
These are our parameters
Page 9: 6 HHkyRd - Home | Computer Science at UBCschmidtm/Courses/340-F17/T3.pdf · Pp2`pB2r RL Bp2" v2b*H bbB}2` kLQM@S ` K2i`B+JQ/2Hb.2}MBiBQMb ELL jh` BMBM;-h2biBM;- M/o HB/ iBQMa2i kfR9

p(xi|yi)p(yi) = p(xi, yi) = p(xi1, ..., xid, yi)

p(xi1, ..., xid, yi) = p(xi1|xi2, ..., xid, yi)p(xi2, ..., xid, yi)

= ...

= p(xi1|xi2, ..., xid, yi)p(xi2|xi3, ..., xid, yi) p(xid|yi)p(yi)

xi yi

p(xij |xij+1, ..., xid, yi) = p(xij |yi)

p(yi, xi) ∝ p(yi)d∏

j=1

p(xij |yi)

Page 10: 6 HHkyRd - Home | Computer Science at UBCschmidtm/Courses/340-F17/T3.pdf · Pp2`pB2r RL Bp2" v2b*H bbB}2` kLQM@S ` K2i`B+JQ/2Hb.2}MBiBQMb ELL jh` BMBM;-h2biBM;- M/o HB/ iBQMa2i kfR9

p(xi|yi)p(yi) = p(xi, yi) = p(xi1, ..., xid, yi)

p(xi1, ..., xid, yi) = p(xi1|xi2, ..., xid, yi)p(xi2, ..., xid, yi)

= ...

= p(xi1|xi2, ..., xid, yi)p(xi2|xi3, ..., xid, yi) p(xid|yi)p(yi)

xi yi

p(xij |xij+1, ..., xid, yi) = p(xij |yi)

p(yi, xi) ∝ p(yi)d∏

j=1

p(xij |yi)

our score for a given yi
Page 11: 6 HHkyRd - Home | Computer Science at UBCschmidtm/Courses/340-F17/T3.pdf · Pp2`pB2r RL Bp2" v2b*H bbB}2` kLQM@S ` K2i`B+JQ/2Hb.2}MBiBQMb ELL jh` BMBM;-h2biBM;- M/o HB/ iBQMa2i kfR9
Page 12: 6 HHkyRd - Home | Computer Science at UBCschmidtm/Courses/340-F17/T3.pdf · Pp2`pB2r RL Bp2" v2b*H bbB}2` kLQM@S ` K2i`B+JQ/2Hb.2}MBiBQMb ELL jh` BMBM;-h2biBM;- M/o HB/ iBQMa2i kfR9
We first need to compute our parameters
Prior: P(flu=N)= 3/6 =1/2
conditional: P(head=Y|flu=N)= 1/3
Page 13: 6 HHkyRd - Home | Computer Science at UBCschmidtm/Courses/340-F17/T3.pdf · Pp2`pB2r RL Bp2" v2b*H bbB}2` kLQM@S ` K2i`B+JQ/2Hb.2}MBiBQMb ELL jh` BMBM;-h2biBM;- M/o HB/ iBQMa2i kfR9

p( = N | = Y, = N, = Y ) ∝p( = Y | = N)p( = N | = N)p( =

Y | = N)p( = N) = 13 ∗ 2

3 ∗ 13 ∗ 1

2 = 0.0370

p( = Y | = Y, = N, = Y ) ∝p( = Y | = Y )p( = N | = Y )p( =

Y | = Y )p( = Y ) = 23 ∗ 1

3 ∗ 23 ∗ 1

2 = 0.0741

Page 14: 6 HHkyRd - Home | Computer Science at UBCschmidtm/Courses/340-F17/T3.pdf · Pp2`pB2r RL Bp2" v2b*H bbB}2` kLQM@S ` K2i`B+JQ/2Hb.2}MBiBQMb ELL jh` BMBM;-h2biBM;- M/o HB/ iBQMa2i kfR9

p( = N | = Y, = N, = Y ) ∝p( = Y | = N)p( = N | = N)p( =

Y | = N)p( = N) = 13 ∗ 2

3 ∗ 13 ∗ 1

2 = 0.0370

p( = Y | = Y, = N, = Y ) ∝p( = Y | = Y )p( = N | = Y )p( =

Y | = Y )p( = Y ) = 23 ∗ 1

3 ∗ 23 ∗ 1

2 = 0.0741

Page 15: 6 HHkyRd - Home | Computer Science at UBCschmidtm/Courses/340-F17/T3.pdf · Pp2`pB2r RL Bp2" v2b*H bbB}2` kLQM@S ` K2i`B+JQ/2Hb.2}MBiBQMb ELL jh` BMBM;-h2biBM;- M/o HB/ iBQMa2i kfR9

p( = N | = Y, = N, = Y ) ∝p( = Y | = N)p( = N | = N)p( =

Y | = N)p( = N) = 13 ∗ 2

3 ∗ 13 ∗ 1

2 = 0.0370

p( = Y | = Y, = N, = Y ) ∝p( = Y | = Y )p( = N | = Y )p( =

Y | = Y )p( = Y ) = 23 ∗ 1

3 ∗ 23 ∗ 1

2 = 0.0741

Page 16: 6 HHkyRd - Home | Computer Science at UBCschmidtm/Courses/340-F17/T3.pdf · Pp2`pB2r RL Bp2" v2b*H bbB}2` kLQM@S ` K2i`B+JQ/2Hb.2}MBiBQMb ELL jh` BMBM;-h2biBM;- M/o HB/ iBQMa2i kfR9

p( = N | = Y, = N, = Y ) ∝p( = Y | = N)p( = N | = N)p( =

Y | = N)p( = N) = 13 ∗ 2

3 ∗ 13 ∗ 1

2 = 0.0370

p( = Y | = Y, = N, = Y ) ∝p( = Y | = Y )p( = N | = Y )p( =

Y | = Y )p( = Y ) = 23 ∗ 1

3 ∗ 23 ∗ 1

2 = 0.0741

Page 17: 6 HHkyRd - Home | Computer Science at UBCschmidtm/Courses/340-F17/T3.pdf · Pp2`pB2r RL Bp2" v2b*H bbB}2` kLQM@S ` K2i`B+JQ/2Hb.2}MBiBQMb ELL jh` BMBM;-h2biBM;- M/o HB/ iBQMa2i kfR9
Page 18: 6 HHkyRd - Home | Computer Science at UBCschmidtm/Courses/340-F17/T3.pdf · Pp2`pB2r RL Bp2" v2b*H bbB}2` kLQM@S ` K2i`B+JQ/2Hb.2}MBiBQMb ELL jh` BMBM;-h2biBM;- M/o HB/ iBQMa2i kfR9

p(F |G) = 1 1%

p(F | ∼ G) = 0.01

p(G|F ) =?

F

G

∼ G

Page 19: 6 HHkyRd - Home | Computer Science at UBCschmidtm/Courses/340-F17/T3.pdf · Pp2`pB2r RL Bp2" v2b*H bbB}2` kLQM@S ` K2i`B+JQ/2Hb.2}MBiBQMb ELL jh` BMBM;-h2biBM;- M/o HB/ iBQMa2i kfR9

p(G|F ) =p(F |G)p(G)

p(F )=

p(F |G)p(G)

p(F |G)p(G) + p(F | ∼ G)p(∼ G)

p(G) =? →

Page 20: 6 HHkyRd - Home | Computer Science at UBCschmidtm/Courses/340-F17/T3.pdf · Pp2`pB2r RL Bp2" v2b*H bbB}2` kLQM@S ` K2i`B+JQ/2Hb.2}MBiBQMb ELL jh` BMBM;-h2biBM;- M/o HB/ iBQMa2i kfR9

p(G|F ) =p(F |G)p(G)

p(F )=

p(F |G)p(G)

p(F |G)p(G) + p(F | ∼ G)p(∼ G)

p(G) =? →

Page 21: 6 HHkyRd - Home | Computer Science at UBCschmidtm/Courses/340-F17/T3.pdf · Pp2`pB2r RL Bp2" v2b*H bbB}2` kLQM@S ` K2i`B+JQ/2Hb.2}MBiBQMb ELL jh` BMBM;-h2biBM;- M/o HB/ iBQMa2i kfR9

Page 22: 6 HHkyRd - Home | Computer Science at UBCschmidtm/Courses/340-F17/T3.pdf · Pp2`pB2r RL Bp2" v2b*H bbB}2` kLQM@S ` K2i`B+JQ/2Hb.2}MBiBQMb ELL jh` BMBM;-h2biBM;- M/o HB/ iBQMa2i kfR9

Page 23: 6 HHkyRd - Home | Computer Science at UBCschmidtm/Courses/340-F17/T3.pdf · Pp2`pB2r RL Bp2" v2b*H bbB}2` kLQM@S ` K2i`B+JQ/2Hb.2}MBiBQMb ELL jh` BMBM;-h2biBM;- M/o HB/ iBQMa2i kfR9

P (if depth is given)
P (if features are discrete)
Non-p
(the number of trees are fixed, but the depth usually varies with data) Non-p
P (k is given)
Page 24: 6 HHkyRd - Home | Computer Science at UBCschmidtm/Courses/340-F17/T3.pdf · Pp2`pB2r RL Bp2" v2b*H bbB}2` kLQM@S ` K2i`B+JQ/2Hb.2}MBiBQMb ELL jh` BMBM;-h2biBM;- M/o HB/ iBQMa2i kfR9
Page 25: 6 HHkyRd - Home | Computer Science at UBCschmidtm/Courses/340-F17/T3.pdf · Pp2`pB2r RL Bp2" v2b*H bbB}2` kLQM@S ` K2i`B+JQ/2Hb.2}MBiBQMb ELL jh` BMBM;-h2biBM;- M/o HB/ iBQMa2i kfR9
Page 26: 6 HHkyRd - Home | Computer Science at UBCschmidtm/Courses/340-F17/T3.pdf · Pp2`pB2r RL Bp2" v2b*H bbB}2` kLQM@S ` K2i`B+JQ/2Hb.2}MBiBQMb ELL jh` BMBM;-h2biBM;- M/o HB/ iBQMa2i kfR9
Page 27: 6 HHkyRd - Home | Computer Science at UBCschmidtm/Courses/340-F17/T3.pdf · Pp2`pB2r RL Bp2" v2b*H bbB}2` kLQM@S ` K2i`B+JQ/2Hb.2}MBiBQMb ELL jh` BMBM;-h2biBM;- M/o HB/ iBQMa2i kfR9
Page 28: 6 HHkyRd - Home | Computer Science at UBCschmidtm/Courses/340-F17/T3.pdf · Pp2`pB2r RL Bp2" v2b*H bbB}2` kLQM@S ` K2i`B+JQ/2Hb.2}MBiBQMb ELL jh` BMBM;-h2biBM;- M/o HB/ iBQMa2i kfR9