bayesian belief networks compound bayesian decision theory

Srihari: CSE 555 0

Bayesian Belief NetworksCompound Bayesian Decision

Theory

Srihari: CSE 555 1

Bayesian Belief Networks

• In certain situations statistical properties are not directly expressed by a parameter vector but by causal relationships among variables

Srihari: CSE 555 2

Statistically dependent and independent variables

Three-dimensional distribution which obeys p(x1, x3) = p(x1) p(x3)Thus x1 and x3 are statistically independent but the other feature pairsare not

x1

x2

x3

Srihari: CSE 555 3

Causal relationships

• State of automobile• Temperature of engine• Pressure of brake fluid• Pressure of air in tires• Voltages in the wires

• Oil pressure and air pressure are not causally related

• Engine temperature and oil temperature are

Srihari: CSE 555 4

Parent-Child Relationship

Node X has variable values (x1,x2,….)

Srihari: CSE 555 5

Bayesian Belief Net or Causal Network or Belief Net

Node A has states {a1, a2,…} = a Node B has states {b1, b2,…}= b

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

)|()|()|()|()|()|()|(

)|(

31

232221

131211

acPacPacPacPacPacPacP

acP

Conditional Probability Table

Rows sum to one

Srihari: CSE 555 6

A simple belief net

Srihari: CSE 555 7

Determining a joint probability

P(a3 , b1 , x2 , c3 , d2) = P(a3) P(b1) P(x2|a3,b1) P(c3|x2) P(d2,x2)= 0.25 x 0.6 x 0.4 x 0.5 x 0.4= 0.012

Only X has 2 parents thus only the P(x2|..) has two conditioning variables

Srihari: CSE 555 8

Determining Probability of variables in a Bayes Belief Net

Linear Chain Belief Net

To compute

proceed as above

Srihari: CSE 555 9

Determining probabilities in a net with a loop

Computing the probabilities of variables at H in the network

Belief net with a simple loop

Differs somewhatfrom linear networkbecause of loop

Srihari: CSE 555 10

Evidence

Given the values of some variables (evidence) determine someconfiguration of other variablesDetermine fish came from North Atlantic, given it is springtimeand fish is a light salmon, or P(b1|a2,x1,c1)

Query variable

Evidence

b1=North Atlantica2 = Spring

c1=lightx1=salmon

d = thickness unknown

Srihari: CSE 555 11

Example of Evaluation (Classification)What is the classification when fish is light (c1),

caught in South Atlantic (b2)

and do not know time of year and thickness?

Srihari: CSE 555 12

Evaluation StepsSimilarly,P(x2|c1,b2) = α 0.066

Normalizing,P(x1|c1,b2)=0.63P(x2|c1,b2)=0.37

Given the evidenceclassify as a salmonNote

Srihari: CSE 555 13

Naïve Bayes Rule

When dependency relationships among the features used bya classfier are unknown, assume features are conditionally independent

Srihari: CSE 555 14

Medical Diagnosis Application of Belief Nets

• Uppermost nodes (without parents) • Biological agent such as presence of bacteria or

virus• Intermediate nodes

• Diseases such as emphysema or flu• Lowermost nodes

• Symptoms such as high temperature or coughing• Physician enters measured values in net and

finds most likely disease or cause

Srihari: CSE 555 15

Compound Bayesian Decision Theory and Context