session 7 bayesian networks i - dtai...0.7 p(¬b|a)= 0.3 f ? p(¬b|¬a) = 0.9 probability and...

Session 7Bayesian Networks I

Solutions

Conditional Probability

Conditional probabilityProbability of event happening given that other event happened

Normalization

Exercise 1: aNormalize the following table so that it represents the conditional probability P(A|B)

Note that A is a binary variable with domain {1, 2} andB a ternary variable with domain {true, false, maybe}

Solution

P(A|B) B

true false maybe

A1 0.1/(0.1+0.5) = 1/6 0.5/(0.5+0.2) = 5/7 0.3/(0.3+0.3) = 1/2

2 0.5/(0.1+0.5) = 5/6 0.2/(0.5+0.2) = 2/7 0.3/(0.3+0.3) = 1/2

Exercise 1: bDo the same for the following tables.

The variables A,B and C have the domain {true, false}

Solution

P(A,B|C=true) B

true false

Atrue 0.3/1.3 = 3/13 0.5/1.3 = 5/13

false 0.2/1.3 = 2/13 0.3/1.3 = 3/13

N = 0.3 + 0.5 + 0.2 + 0.3 = 1.3

P(A,B|C=false) B

true false

Atrue 0.2/0.6 = 1/3 0.1/0.6 = 1/6

false 0.2/0.6 = 1/3 0.1/0.6 = 1/6

N = 0.2 + 0.1 + 0.2 + 0.1 = 0.6

Independence

Exercise 2

IndependenceP(A,B) = P(A) P(B) must hold

Two methods:

● Naive● Ratio Trick

Independence: Naïve methodNotice that:

Which can be solved to yield:

Analogously: P(A=2, B=maybe) = 0.2

Independence: Naïve methodSimilarly, for the second exercise

Independence: Ratio trickWe can really exploit what it exactly means that A and B are independent of each other. For example: learning something about B shouldn't give us any new information about A. In other words, given that we know one of the values P(A = 1, B) or P(A = 2, B) for a given B, we don't need to know the actual value of B in order to compute the other one. This can happen in the following two cases.

1. The value that we know is 1. This entails that all other values in the table are 0. In our exercise, this doesn't hold.

2. The ratio between P(A = 1; B = b) and P(A = 2; B = b) is equal for all b.

Independence: Ratio trickThis can easily be shown by exploiting the independence relation, as

It is now a piece of cake to compute the missing values

Independence: Ratio trickWe can again use the fact that for different instantiations of a the quotient of the joint probabilities is a constant:

Independence: Ratio trick

P(A,B) B

true false maybe

A1 0.05 0.1 0.15

2 0.05 x 2.33 0.1 x 2.33 0.15 x 2.33

Conditional Independence

Exercise 3

Conditional IndependenceAlso use ratio trick. However, due to conditional independence, ratio trick only works for a given C. This means that we handle the two tables separately.

Conditional Independence

P(A,B,c=true) B

true false

A

x 0.056 0.084

y 0.08 0.012

z 0.016 0.024

P(A,B,c=false) B

true false

A

x 0.072 0.008

y 0.432 0.048

z 0.216 0.024

Probability & Bayes’ Rule

Probability and Bayes’ RuleIn Leuven 80% of all cars are red. You see a car at night that does not appear red to you. You know that you correctly identify a red car in only 70% of the cases, while you can identify a non-red car correctly in 90% of the cases.

Probability and Bayes’ Rule

In Leuven 80% of all cars are red. You see a car at night that does not appear red to you. You know that you correctly identify a red car in only 70% of the cases, while you can identify a non-red car correctly in 90% of the cases.

1. List all conditional and non-conditional probabilities that you can determine directly from the task description.

2. Compute the probability that the car is actually red, when you perceive a car as red in Leuven at night.

Probability and Bayes’ Rule

In Leuven 80% of all cars are red. You see a car at night that does not appear red to you. You know that you correctly identify a red car in only 70% of the cases, while you can identify a non-red car correctly in 90% of the cases.

1. List all conditional and non-conditional probabilities that you can determine directly from the task description.

Probability and Bayes’ Rule

In Leuven 80% of all cars are red. You see a car at night that does not appear red to you. You know that you correctly identify a red car in only 70% of the cases, while you can identify a non-red car correctly in 90% of the cases.

Probability and Bayes’ Rule

In Leuven 80% of all cars are red. You see a car at night that does not appear red to you. You know that you correctly identify a red car in only 70% of the cases, while you can identify a non-red car correctly in 90% of the cases.

A: A car is red

T F

P(A) = 0.8 ?

Probability and Bayes’ Rule

In Leuven 80% of all cars are red. You see a car at night that does not appear red to you. You know that you correctly identify a red car in only 70% of the cases, while you can identify a non-red car correctly in 90% of the cases.

A: A car is red

T F

P(a) = 0.8 P(¬a) = 0.2 Indirect knowledge!

Probability and Bayes’ Rule

In Leuven 80% of all cars are red. You see a car at night that does not appear red to you. You know that you correctly identify a red car in only 70% of the cases, while you can identify a non-red car correctly in 90% of the cases.

Probability and Bayes’ Rule

In Leuven 80% of all cars are red. You see a car at night that does not appear red to you. You know that you correctly identify a red car in only 70% of the cases, while you can identify a non-red car correctly in 90% of the cases.

B: A car is perceived to be red

T F

? ?

Probability and Bayes’ Rule

In Leuven 80% of all cars are red. You see a car at night that does not appear red to you. You know that you correctly identify a red car in only 70% of the cases, while you can identify a non-red car correctly in 90% of the cases.

Probability and Bayes’ Rule

In Leuven 80% of all cars are red. You see a car at night that does not appear red to you. You know that you correctly identify a red car in only 70% of the cases, while you can identify a non-red car correctly in 90% of the cases.

P(B|A) B: A car is perceived to be red

A: A car is red

T F

T P(b|a) = 0.7

?

F ? ?

Probability and Bayes’ Rule

In Leuven 80% of all cars are red. You see a car at night that does not appear red to you. You know that you correctly identify a red car in only 70% of the cases, while you can identify a non-red car correctly in 90% of the cases.

P(B|A) B: A car is perceived to be red

A: A car is red

T F

T P(b|a) = 0.7

P(¬b|a)=0.3

F ? ?

Probability and Bayes’ Rule

In Leuven 80% of all cars are red. You see a car at night that does not appear red to you. You know that you correctly identify a red car in only 70% of the cases, while you can identify a non-red car correctly in 90% of the cases.

P(B|A) B: A car is perceived to be red

A: A car is red

T F

T P(b|a) = 0.7

P(¬b|a)=0.3

F ? P(¬b|¬a) = 0.9

Probability and Bayes’ Rule

In Leuven 80% of all cars are red. You see a car at night that does not appear red to you. You know that you correctly identify a red car in only 70% of the cases, while you can identify a non-red car correctly in 90% of the cases.

P(B|A) B: A car is perceived to be red

A: A car is red

T F

T P(b|a) = 0.7

P(¬b|a)=0.3

F P(b|¬a) = 0.1

P(¬b|¬a) = 0.9

Probability and Bayes’ Rule

In Leuven 80% of all cars are red. You see a car at night that does not appear red to you. You know that you correctly identify a red car in only 70% of the cases, while you can identify a non-red car correctly in 90% of the cases.

2. Compute the probability that the car is actually red, when you perceive a car as red in Leuven at night.

Probability and Bayes’ Rule

2. Compute the probability that the car is actually red, when you perceive a car as red in Leuven at night.

P(B|A) B: A car is perceived to be red

A: A car is red

T F

T P(b|a) = 0.7

P(¬b|a)=0.3

F P(b|¬a) = 0.1

P(¬b|¬a) = 0.9

A: A car is red

T F

P(a) = 0.8 P(¬a) = 0.2

Probability and Bayes’ Rule

2. Compute the probability that the car is actually red, when you perceive a car as red in Leuven at night.

Compute P(a|b)

P(B|A) B: A car is perceived to be red

A: A car is red

T F

T P(b|a) = 0.7

P(¬b|a)=0.3

F P(b|¬a) = 0.1

P(¬b|¬a) = 0.9

A: A car is red

T F

P(a) = 0.8 P(¬a) = 0.2

Probability and Bayes’ Rule

2. Compute the probability that the car is actually red, when you perceive a car as red in Leuven at night.

P(a|b) = P(b|a) · P(a) / P(b)

P(B|A) B: A car is perceived to be red

A: A car is red

T F

T P(b|a) = 0.7

P(¬b|a)=0.3

F P(b|¬a) = 0.1

P(¬b|¬a) = 0.9

A: A car is red

T F

P(a) = 0.8 P(¬a) = 0.2

Probability and Bayes’ Rule

2. Compute the probability that the car is actually red, when you perceive a car as red in Leuven at night.

P(a|b) = P(b|a) · P(a) / P(b)

=

= 0.7·0.8 / (0.7·0.8 + 0.1·0.2) = 0.9655

P(b|a)·P(a)

P(b|a)·P(a) + P(b|¬a)·P(¬a)P(B|A) B: A car is perceived

to be red

A: A car is red

T F

T P(b|a) = 0.7

P(¬b|a)=0.3

F P(b|¬a) = 0.1

P(¬b|¬a) = 0.9

A: A car is red

T F

P(a) = 0.8 P(¬a) = 0.2

Fred’s LISP Dilemma

Fred Lisp Dilemma

The LISP interpreter is unresponsive.

There are only two situations that could cause the LISP interpreter I to stop running: There are either problems with the computer hardware H, or there is a bug in Fred’s code C. Fred is also running an editor E in which he is writing and editing his LISP code;

Fred Lisp Dilemma: Model

I

C H

E

Fred Lisp DilemmaIf the hardware is functioning properly, then the text editor should still be running with a probability of 95%.

When the hardware isn’t functioning properly, the text editor could still be running with a probability of 10%.

The hardware is pretty reliable, and is OK about 99% of the time, whereas Fred’s LISP code is often buggy, say 40% of the time.

In the case of bad hardware and buggy code, the interpreter stops running with a probability of 99%.

In the case of good hardware and buggy code, the interpreter stops running with a probability of 40 %.

In the case of good code and bad hardware, the interpreter stops running with a probability of 95%.

In the case of good code and good hardware, the interpreter stops running with a probability of 10%.

Fred’s Lisp Dilemma: Joint Probability Distributionb) Generally:

Here:

c) Full Joint (no independence information):

24-1 = 15 values needed

d) Utilising network structures (independences between nodes used):

20 +20 + 22 + 21 = 8 values needed

I

C H

E

Fred Lisp Dilemma: Model

I

C H

EC

p(C) bug no_bug

0.4 0.6

H

p(H) good bad

0.99 0.01

C H P(I = responding|C,H)

bug good 0.6

bug bad 0.01

no_bug good 0.9

no_bug bad 0.05

H

p(E|H) good bad

Erunning 0.95 0.1

not_running 0.05 0.9

Calculate probabilityProbability of bug when interpreter stops working:

I

C H

E

Calculate probability(Push summation over E inside: sum = 1)

I

C H

E

Modelling a Bayesian Network

Modelling a Bayesian NetworkIn your local nuclear power station, there is an alarm that senses when a temperature gauge exceeds a given threshold. The gauge measures the temperature of the core. Consider the Boolean variables A (alarm sounds), FA (alarm is faulty), and FG (gauge is faulty) and the multivalued nodes G (gauge reading) and T (actual core temperature).

Modelling a Bayesian NetworkIn your local nuclear power station, there is an alarm that senses when a temperature gauge exceeds a given threshold. The gauge measures the temperature of the core. Consider the Boolean variables A (alarm sounds), FA (alarm is faulty), and FG (gauge is faulty) and the multivalued nodes G (gauge reading) and T (actual core temperature).

1. Draw a Bayesian network for this domain, given that the gauge is more likely to fail when the core temperature gets too high.

Modelling a Bayesian NetworkIn your local nuclear power station, there is an alarm that senses when a temperature gauge exceeds a given threshold. The gauge measures the temperature of the core. The gauge is more likely to fail when the core temperature gets too high.

T

G

Modelling a Bayesian NetworkIn your local nuclear power station, there is an alarm that senses when a temperature gauge exceeds a given threshold. The gauge measures the temperature of the core. The gauge is more likely to fail when the core temperature gets too high.

FGT

G

Modelling a Bayesian NetworkIn your local nuclear power station, there is an alarm that senses when a temperature gauge exceeds a given threshold. The gauge measures the temperature of the core. The gauge is more likely to fail when the core temperature gets too high.

FGT

G FA

A

Modelling a Bayesian NetworkIn your local nuclear power station, there is an alarm that senses when a temperature gauge exceeds a given threshold. The gauge measures the temperature of the core. Consider the Boolean variables A (alarm sounds), FA (alarm is faulty), and FG (gauge is faulty) and the multivalued nodes G (gauge reading) and T (actual core temperature).

2. Suppose there are just two possible actual and measured temperatures, normal and high; the probability that the gauge gives the correct temperature is x when it is working, but y when it is faulty. Give the conditional probability table associated with G.

Modelling a Bayesian NetworkSuppose there are just two possible actual and measured temperatures, normal and high; the probability that the gauge gives the correct temperature is x when it is working, but y when it is faulty. Give the conditional probability table associated with G.

FG = False (correct) FG = True (incorrect)

G T = normal T = high T = normal T = high

G = high

G = normal

Modelling a Bayesian NetworkSuppose there are just two possible actual and measured temperatures, normal and high; the probability that the gauge gives the correct temperature is x when it is working, but y when it is faulty. Give the conditional probability table associated with G.

FG = False (correct) FG = True (incorrect)

G T = normal T = high T = normal T = high

G = high x

G = normal x

Modelling a Bayesian NetworkSuppose there are just two possible actual and measured temperatures, normal and high; the probability that the gauge gives the correct temperature is x when it is working, but y when it is faulty. Give the conditional probability table associated with G.

FG = False (correct) FG = True (incorrect)

G T = normal T = high T = normal T = high

G = high 1 - x x

G = normal x 1 - x

Indirect knowledge!

Modelling a Bayesian NetworkSuppose there are just two possible actual and measured temperatures, normal and high; the probability that the gauge gives the correct temperature is x when it is working, but y when it is faulty. Give the conditional probability table associated with G.

FG = False (correct) FG = True (incorrect)

G T = normal T = high T = normal T = high

G = high 1 - x x 1 - y y

G = normal x 1 - x y 1 - y

Modelling a Bayesian NetworkIn your local nuclear power station, there is an alarm that senses when a temperature gauge exceeds a given threshold. The gauge measures the temperature of the core. Consider the Boolean variables A (alarm sounds), FA (alarm is faulty), and FG (gauge is faulty) and the multivalued nodes G (gauge reading) and T (actual core temperature).

3. Suppose the alarm works correctly unless it is faulty, in which case it never sounds. Give the conditional probability table associated with A.

Modelling a Bayesian NetworkSuppose the alarm works correctly unless it is faulty, in which case it never sounds. Give the conditional probability table associated with A.

FA = False (works) FA = True (faulty)

A G = normal G = high G = normal G = high

A = True (sounds)

A = False (silent)

Modelling a Bayesian NetworkSuppose the alarm works correctly unless it is faulty, in which case it never sounds. Give the conditional probability table associated with A.

FA = False (works) FA = True (faulty)

A G = normal G = high G = normal G = high

A = True (sounds)

0 1

A = False (silent)

1 0

Modelling a Bayesian NetworkSuppose the alarm works correctly unless it is faulty, in which case it never sounds. Give the conditional probability table associated with A.

FA = False (works) FA = True (faulty)

A G = normal G = high G = normal G = high

A = True (sounds)

0 1 0 0

A = False (silent)

1 0 1 1

Modelling a Bayesian NetworkIn your local nuclear power station, there is an alarm that senses when a temperature gauge exceeds a given threshold. The gauge measures the temperature of the core. Consider the Boolean variables A (alarm sounds), FA (alarm is faulty), and FG (gauge is faulty) and the multivalued nodes G (gauge reading) and T (actual core temperature).

4. Suppose the alarm and gauge are working and the alarm sounds. Calculate an expression for the probability that the temperature of the core is too high, in terms of the various conditional probabilities in the network.

Modelling a Bayesian NetworkSuppose the alarm and gauge are working and the alarm sounds. Calculate an expression for the probability that the temperature of the core is too high, in terms of the various conditional probabilities in the network.

FA = False (works), FG = False (works), A = True (sounds)

Modelling a Bayesian NetworkSuppose the alarm and gauge are working and the alarm sounds. Calculate an expression for the probability that the temperature of the core is too high, in terms of the various conditional probabilities in the network.

FA = False (works), FG = False (works), A = True (sounds)

Compute: T = t

Modelling a Bayesian Network

Since the alarm sounds and is not faulty, the gauge reading must be high!

So we can rewrite the problem as follows:

Modelling a Bayesian Network

Applying Bayes’ rule:

Modelling a Bayesian Network

Factorizing the numerator:

Modelling a Bayesian Network

Writing denominator as marginal:

Modelling a Bayesian Network

Factorizing to denominator:

Modelling a Bayesian Network

Factors independent of t before the sum:

Modelling a Bayesian Network

Simplifying:

Modelling a Bayesian Network

Writing out the denominator:

Modelling a Bayesian Network

From the table we know the probabilities:

Modelling a Bayesian Network

Finally, substituting the probabilities: