noisy or gate

NOISY OR GATEJavier Turek and Eyal Regev

Several causes sharing a common effect.

Y Z

Ui

e+

e-

X

PolytreesU1 Un

… …

Like a tree, but with multiple parents.

Link matrix Tx|u

Tx|u contains the conditional probabilities:P(X=x | U1=u1,…,Un=un)

1. The table is huge (contains 2n entries).

2. Who can know such information anyway?You cannot expect to find the P(X | U1,…,Un) table.However, you may know how every Ui influences X

separately.

The OR-gateHaven’t done

homeworkCaught

cheating

Failed an exam More likely!

OR

Inhibitors

Failed an exam

Caught cheating

OR

AND

Paid the TA

Haven’t done

homework

AND

Re-doing the

course

InhibitorsU1

X

U2Un

I1

I2In

OR

AND

AND

AND

Inhibitors are independent

Associate probability to an inhibitor.

Noisy OR-GateU1

X

U2Un

I1

I2In

OR

AND

AND

AND

| , :i k iP x u u k i q

Noisy OR-GateU1

X

U2Un

I1

I2In

OR

AND

AND

AND

:u iT i U True

Tu

if

|1 if +

u

u

ii T

ii T

q xP x u

q x

Message Passing – updating X

Y Z

U

e+

e-

X

| , | |Belief x P x e e P e x P x e D x A x

Y ZD x U x U x |x u Xu

A x T C u

YU x

y

XC u

ZU x

u

Message Passing – updating X

Y Z

U

e+

W

e-

X

| , | | ABelief x P x e e P e x P x e D x x

YU x

y

XC u u

ZU x

XC w

Y ZD x U x U x | ,,

x u Xw

Xwu

T C CuA x w

| , | |Belief x P x e e P e x P x e D x A x

|x u Xu

A x T C u

Message Passing – updating U,Y,Z

Y Z

U

e+

e-

X

Y ZD x U x U x |x u Xu

A x T C u

XU u

YC x

ZC x

|xXx

uTU u D x ZY A xC x x U

| ,x u wX Xx w

T CU D x wu

Y Z

U

e+

W

e-

X

XU u

YC x

ZC x

Message Passing – updating W,U,Y,Z

XU w

Y ZD x U x U x | ,,

x u Xw

Xwu

T C CuA x w

ZY A xC x x U ZY A xC x x U

|x u Xu

A x T C u

|xXx

uTU u D x

Y Z

Ui

e+

e-

X

X iU u

YC x

ZC x

Message Passing – many parents

X nU u

|:k

i u ku k i k i

X Xx

xU D T uCu x

ZY A xC x x U

| X ku k

x uT CA x u Y ZD x U x U x

Link matrix

U1 Un

… …

1XU u

Belief Update – Noisy OR-Gate

if

1 if +

u

u

i X ku i T k

i X ku i T k

D x q C u x

Bel xD x q C u x

u u

i X i X ku i T k T

qC u C u

u u

i X i X ku i T k T

qC u C u

1u u

i X i X ku i T k T

qC u C u

Belief Update – Noisy OR-Gate

1 1 if

1 1 1 if +

i X ii

i X ii

D x q C u x

Bel xD x q C u x

1u u

i X i X ku i T k T

qC u C u

1i X i X ii

qC u C u 1 1 i X i

i

q C u

Update messages

Where

The message to the child is the same:

1 if +

1 if

i i i i i

X ii i i

D x q D x q uU u

D x D x u

1 1i k X kk i

q C u

kY

ii

kXC Ay U x x

Example

Windows Vista

No electricity Virus 2K bug

Does not start

Wrong date

Stolen Password Lost data

D1 D2 D3 D4

M1 M2 M3 M4

Example

D1 D2 D3 D4

M1 M2 M3 M4

0.01\0.99 0.1\0.9 0.2\0.8 0.2\0.8

ijq0.8

0.1 0.9

0.2 0.5

0.1 0.7

0.80.2

|W , forhere j iij kP m d d k iq

\ (1 )i ip p

Conjunction query Conjunction query q: find the belief that

several events happen simultaneously.

Applying the chain rule:

Product of m belief updates

Q

i ii I

q X x

1 2 2 1

1 2 1 1 2 1

1 2 2 1

, , , , ,

| | , ,

| , , , ,

m m m

m m

m m m

P q P x x x x x

P x P x x P x x x

P x x x x x

Answering a query In our example:

Computing P(q):

1 1 2 3 4q d m m m m

1 1 1 2 1 1

3 2 1 1

4 3 2 1 1

| | ,

| , ,

| , , ,

P q P d P m d P m m d

P m m m d

P m m m m d

Example Update the belief on M1

Update likelihoods and priors: 1 11 21 1 2| 1 1 1 1 1 1 0.272P m q qd p

1 1p 2p

1 2MU d

4 2MC d

1 2 11 21 111 ,1 0.92,0.2MU d q q q

4 1 12 2 2 2 2, 1 0.338,0.662M M MC d U d p U d p

D1 D2 D3 D4

M1 M2 M3 M4

Example Update the belief on M2

Update likelihoods and priors:

2 2 121 421 41| | 1 1 1 1 1 0.81,m d dP m P m q q p

1 1p

2 4 12 42 12, 0.45,0.9MU d q q q

4 2 24 4 4 4 4, 1 0.111,0.889M M MC d U d p U d p

2 4MU d

4p

4 4MC d

D1 D2 D3 D4

M1 M2 M3 M4

Example Update the belief on M3

Update likelihoods and priors:

3 32 1 1

13 33 3

1| |

1 1 1 1 1 1 0.836

, ,P m P m

q q p

m m d d

1 1p

3 3 13 33 131 ,1 0.98,0.8MU d q q q

4 3 33 3 3 3 3, 1 0.23,0.77M M MC d U d p U d p

3 3MU d

3p 4 3MC d

D1 D2 D3 D4

M1 M2 M3 M4

Example Update the belief on M4

4

3 2 1 14

42,3,4

|

1 1 0

,

.

,

7

,

81i M ii

m m m dP m

q C d

4 3MC d

4 4MC d

4 2MC d

D1 D2 D3 D4

M1 M2 M3 M4

Example And the final solution is…

Example And the final solution is…

1 1 1 2 1 1

3 2 1 1

4 3 2 1 1

3

| | ,

| , ,

| , , ,

0.01 0.272 0.81 0.836 0.781 1.439 10

P q P d P m d P m m d

P m m m d

P m m m m d

Thank You!