physical fluctuomatics 7th~10th belief propagation
DESCRIPTION
Physical Fluctuomatics 7th~10th Belief propagation. Kazuyuki Tanaka Graduate School of Information Sciences, Tohoku University [email protected] http://www.smapip.is.tohoku.ac.jp/~kazu/. Textbooks. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/1.jpg)
Physics Fluctuomatics (Tohoku University) 1
Physical Fluctuomatics7th~10th Belief propagation
Kazuyuki TanakaGraduate School of Information Sciences, Tohoku University
[email protected]://www.smapip.is.tohoku.ac.jp/~kazu/
![Page 2: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/2.jpg)
Physics Fluctuomatics (Tohoku University) 2
Textbooks
Kazuyuki Tanaka: Introduction of Image Processing by Probabilistic Models, Morikita Publishing Co., Ltd., 2006 (in Japanese) , Chapter 8.Kazuyuki Tanaka: Mathematics of Statistical Inference by Bayesian Network, Corona Publishing Co., Ltd., October 2009 (in Japanese), Chapters 6-9.
![Page 3: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/3.jpg)
Physics Fluctuomatics (Tohoku University) 3
What is an important point in computational complexity?
How should we treat the calculation of the summation over 2N configuration?
FT, FT, FT,
211 2
,,,x x x
NN
xxxf
}}
} ;,,,
F){or Tfor(
F){or Tfor( F){or Tfor(
;0
21
2
1
L
N
xxxfaax
xx
a
N fold loops
If it takes 1 second in the case of N=10, it takes 17 minutes in N=20, 12 days in N=30 and 34 years in N=40.
Markov Chain Monte Carlo MethodBelief Propagation Method This Talk
![Page 4: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/4.jpg)
Physics Fluctuomatics (Tohoku University) 4
Probabilistic Model and Belief Propagation
Probabilistic Information Processing
Probabilistic Models
Bayes Formulas
Belief Propagation
J. Pearl: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference (Morgan Kaufmann, 1988).C. Berrou and A. Glavieux: Near optimum error correcting coding and decoding: Turbo-codes, IEEE Trans. Comm., 44 (1996).
Bayesian Networks
![Page 5: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/5.jpg)
Physics Fluctuomatics (Tohoku University) 5
Mathematical Formulation of Belief Propagation
Similarity of Mathematical Structures between Mean Field Theory and Bepief PropagationY. Kabashima and D. Saad, Belief propagation vs. TAP for decoding corrupted messages, Europhys. Lett. 44 (1998). M. Opper and D. Saad (eds), Advanced Mean Field Methods ---Theory and Practice (MIT Press, 2001).
Generalization of Belief PropagationS. Yedidia, W. T. Freeman and Y. Weiss: Constructing free-energyapproximations and generalized belief propagation algorithms, IEEE Transactions on Information Theory, 51 (2005).
Interpretations of Belief Propagation based on Information GeometryS. Ikeda, T. Tanaka and S. Amari: Stochastic reasoning, free energy, and information geometry, Neural Computation, 16 (2004).
![Page 6: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/6.jpg)
Physics Fluctuomatics (Tohoku University) 6
Generalized Extensions of Belief Propagation based on Cluster Variation Method
Generalized Belief PropagationJ. S. Yedidia, W. T. Freeman and Y. Weiss: Constructing free-energy approximations and generalized belief propagation algorithms, IEEE Transactions on Information Theory, 51 (2005).
Key Technology is the cluster variation method in Statistical PhysicsR. Kikuchi: A theory of cooperative phenomena, Phys. Rev., 81 (1951).T. Morita: Cluster variation method of cooperative phenomena and its generalization I, J. Phys. Soc. Jpn, 12 (1957).
![Page 7: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/7.jpg)
Physics Fluctuomatics (Tohoku University) 7
Belief Propagation in Statistical Physics
In graphical models with tree graphical structures, Bethe approximation is equivalent to Transfer Matrix Method in Statistical Physics and give us exact results for computations of statistical quantities.
In Graphical Models with Cycles, Belief Propagation is equivalent to Bethe approximation or Cluster Variation Method.
Bethe Approximation
Trandfer Matrix Method
(Tree Structures)Belief Propagation
Cluster Variation Method(Kikuchi Approximation)
Generalized Belief Propagation
![Page 8: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/8.jpg)
Physics Fluctuomatics (Tohoku University) 8
Applications of Belief PropagationsImage ProcessingK. Tanaka: Statistical-mechanical approach to image processing (Topical Review), J. Phys. A, 35 (2002).A. S. Willsky: Multiresolution Markov Models for Signal and Image Processing, Proceedings of IEEE, 90 (2002).
Low Density Parity Check CodesY. Kabashima and D. Saad: Statistical mechanics of low-density parity-check codes (Topical Review), J. Phys. A, 37 (2004). S. Ikeda, T. Tanaka and S. Amari: Information geometry of turbo and low-density parity-check codes, IEEE Transactions on Information Theory, 50 (2004).
CDMA Multiuser Detection AlgorithmY. Kabashima: A CDMA multiuser detection algorithm on the basis of belief propagation, J. Phys. A, 36 (2003).T. Tanaka and M. Okada: Approximate Belief propagation, density evolution, and statistical neurodynamics for CDMA multiuser detection, IEEE Transactions on Information Theory, 51 (2005).
Satisfability ProblemO. C. Martin, R. Monasson, R. Zecchina: Statistical mechanics methods and phase transitions in optimization problems, Theoretical Computer Science, 265 (2001).M. Mezard, G. Parisi, R. Zecchina: Analytic and algorithmic solution of random satisfability problems, Science, 297 (2002).
![Page 9: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/9.jpg)
Physics Fluctuomatics (Tohoku University) 9
Strategy of Approximate Algorithm in Probabilistic Information Processing
It is very hard to compute marginal probabilities exactly except some tractable cases.
What is the tractable cases in which marginal probabilities can be computed exactly?Is it possible to use such algorithms for tractable cases to compute marginal probabilities in intractable cases?
FT, FT, FT,
21112 3
,,,x x x
LL
xxxPxP
![Page 10: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/10.jpg)
Physics Fluctuomatics (Tohoku University) 10
Graphical Representations of Tractable Probabilistic Models
A B C D E
),(),(),(),( EDWDCWCBWBAW DECDBCAB
),( BAWAB ),( CBWBC ),( DCWCD ),( EDWDE
A B C D E
),( BAWAB ),( CBWBC ),( DCWCD ),( EDWDE
B C DX X X=
=
![Page 11: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/11.jpg)
Physics Fluctuomatics (Tohoku University) 11
Graphical Representations of Tractable Probabilistic Models
A B C D EA B C D E
A BA B C D E
B C D EX
![Page 12: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/12.jpg)
Physics Fluctuomatics (Tohoku University) 12
Graphical Representations of Tractable Probabilistic Models
A B C D EA B C D E
A BA B C D E
B C D EX
A B
B C D E A
B C D E
![Page 13: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/13.jpg)
Physics Fluctuomatics (Tohoku University) 13
Graphical Representations of Tractable Probabilistic Models
A B C D EA B C D E
A BA B C D E
B C D EX
A B
B C D E A
B C D E
A B
![Page 14: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/14.jpg)
Physics Fluctuomatics (Tohoku University) 14
Graphical Representations of Tractable Probabilistic Models
A B C D EA B C D E
A BA B C D E
B C D EX
A B
B C D E A
B C D E
A B
A B C D EB C D E
![Page 15: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/15.jpg)
Physics Fluctuomatics (Tohoku University) 15
Graphical Representations of Tractable Probabilistic Models
A B C D EB C D E
![Page 16: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/16.jpg)
Physics Fluctuomatics (Tohoku University) 16
Graphical Representations of Tractable Probabilistic Models
B C D E
C D EX
A B C D EB C D E
A B C
![Page 17: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/17.jpg)
Physics Fluctuomatics (Tohoku University) 17
Graphical Representations of Tractable Probabilistic Models
B C D E
C D EX
C D E B
C D E
A B C D EB C D E
A B C
A B C X
![Page 18: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/18.jpg)
Physics Fluctuomatics (Tohoku University) 18
Graphical Representations of Tractable Probabilistic Models
B C D E
C D EX
C D E B
C D E
B C
A B C D EB C D E
A B C
A B C X
![Page 19: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/19.jpg)
Physics Fluctuomatics (Tohoku University) 19
Graphical Representations of Tractable Probabilistic Models
B C D E
C D EX
C D E B
C D E
B C
A B C D EB C D E
A B C
A B C
B C D EC D E
X
![Page 20: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/20.jpg)
Physics Fluctuomatics (Tohoku University) 20
Graphical Representations of Tractable Probabilistic Models
A B C D EA B C D E
![Page 21: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/21.jpg)
Physics Fluctuomatics (Tohoku University) 21
Graphical Representations of Tractable Probabilistic Models
A B C D EB C D E
A B C D EA B C D E
![Page 22: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/22.jpg)
Physics Fluctuomatics (Tohoku University) 22
Graphical Representations of Tractable Probabilistic Models
A B C D EB C D E
B C D EC D E
A B C D EA B C D E
![Page 23: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/23.jpg)
Physics Fluctuomatics (Tohoku University) 23
Graphical Representations of Tractable Probabilistic Models
A B C D EB C D E
B C D EC D E
A B C D EA B C D E
C D ED E
![Page 24: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/24.jpg)
Physics Fluctuomatics (Tohoku University) 24
Graphical Representations of Tractable Probabilistic Models
A B C D EB C D E
B C D EC D E
A B C D EA B C D E
C D ED E
D EE
![Page 25: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/25.jpg)
Physics Fluctuomatics (Tohoku University) 25
Graphical Representations of Tractable Probabilistic Models
A B C E E
),(),(),(),(),( FEWEDWECWCBWCAW EFDECEBCAC
),( BAWAB ),( CBWBC ),( DCWCD ),( EDWDE
C C DX X X=
=
F
),( FEWEF
EX
A
B
EC
D
F
![Page 26: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/26.jpg)
Physics Fluctuomatics (Tohoku University) 26
Graphical Representations of Tractable Probabilistic Models
A
B
ECA B C D E F
D
F
![Page 27: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/27.jpg)
Physics Fluctuomatics (Tohoku University) 27
Graphical Representations of Tractable Probabilistic Models
A
B
ECA B C D E F
D
F
A
B
EC
D
F
B C D E F
A
A
C
A
C
![Page 28: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/28.jpg)
Physics Fluctuomatics (Tohoku University) 28
Graphical Representations of Tractable Probabilistic Models
A
B
ECA B C D E F
D
F
A
B
EC
C D E F
D
FA
B
EC
B C D E F
D
F
B
CBB
C
![Page 29: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/29.jpg)
Physics Fluctuomatics (Tohoku University) 29
Graphical Representations of Tractable Probabilistic Models
A
B
ECA B C D E F
D
F
A
B
EC
C D E F
D
FA
B
EC
B C D E F
D
F
ECD E F
D
F
![Page 30: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/30.jpg)
Physics Fluctuomatics (Tohoku University) 30
Graphical Representations of Tractable Probabilistic Models
A
B
ECA B C D E F
D
F
A
B
EC
C D E F
D
FA
B
EC
B C D E F
D
F
ECD E F
D
F
ECE F
D
F
![Page 31: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/31.jpg)
Physics Fluctuomatics (Tohoku University) 31
Graphical Representations of Tractable Probabilistic Models
A
B
ECA B C D E F
D
F
A
B
EC
C D E F
D
FA
B
EC
B C D E F
D
F
ECD E F
D
F
ECE F
D
F
EF F
![Page 32: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/32.jpg)
Physics Fluctuomatics (Tohoku University) 32
Graphical Representations of Tractable Probabilistic Models
Graphical Representation of Marginal Probability in terms of Messages A
B
ECA B C D F
E}Pr{
D
F
![Page 33: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/33.jpg)
Physics Fluctuomatics (Tohoku University) 33
Graphical Representations of Tractable Probabilistic Models
Graphical Representation of Marginal Probability in terms of Messages A
B
ECA B C D F
E}Pr{
D
F
A
B
ECA B C E
D E
FF
D
=
![Page 34: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/34.jpg)
Physics Fluctuomatics (Tohoku University) 34
Graphical Representations of Tractable Probabilistic Models
Graphical Representation of Marginal Probability in terms of Messages A
B
ECA B C D F
E}Pr{
D
F
A
B
ECA B C E
D E
FF
D
=
= ECE
D E
F
![Page 35: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/35.jpg)
Physics Fluctuomatics (Tohoku University) 35
Graphical Representations of Tractable Probabilistic Models
Graphical Representation of Marginal Probability in terms of Messages A
B
ECA B C D F
E}Pr{
D
F
A
B
ECA B C E
D E
FF
D
=
= ECE
D E
FEC
D
F
=
![Page 36: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/36.jpg)
Physics Fluctuomatics (Tohoku University) 36
Graphical Representations of Tractable Probabilistic Models
Graphical Representation of Marginal Probability in terms of Messages A
B
ECA B D F
EC },Pr{
D
F
A
CA E
D E
FF
D
=
= ECE
D E
F=
ECB
CB
A
C B
CA
B
EC
D
F
![Page 37: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/37.jpg)
Physics Fluctuomatics (Tohoku University) 37
Graphical Representations of Tractable Probabilistic Models
Graphical Representation of Marginal Probability in terms of Messages
},Pr{ ECA
B
EC
D
F
=}Pr{E EC
D
F
=
EC
A
B
ECC
EC
D
F
A
B
EC
D
F
C
C
ECE },Pr{}Pr{
Recursion Formulas for Messages
![Page 38: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/38.jpg)
Physics Fluctuomatics (Tohoku University) 38
Graphical Representations of Tractable Probabilistic Models
Graphical Representation of Marginal Probability in terms of Messages
A
B
ECC
EC
EC
D
FE
E
FEC
D
FE
E
D
EC
D
F
E
EC
A
B
ECC
A
C
A
B
ECC
B
C
A
A
C
A
C B
CBB
C
E
DE
DD
E
FE
FF
A
B
EC
D
FStep 1
Step 2
Step 3
![Page 39: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/39.jpg)
Physics Fluctuomatics (Tohoku University) 39
Graphical Representations of Tractable Probabilistic Models
Graphical Representation of Marginal Probability in terms of Messages
Step 1
Step 2
Step 3
A
B
EC
D
F
A
B
EC
D
F
A
B
EC
D
F},Pr{ EC
A
B
EC
D
F=
}Pr{E EC
D
F=
}Pr{BB
C
=
},Pr{ CAA
B
EC=
![Page 40: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/40.jpg)
Physics Fluctuomatics (Tohoku University) 40
Belief Propagation
ab
1
cd
2
3 4
56
2221}2,1{11
21
,,,,,,,,,,
xWxWxxWxWxWxxP
CBA dcbadcba
a3xc5x
b4xd6x
Probabilistic Models with no Cycles
![Page 41: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/41.jpg)
Physics Fluctuomatics (Tohoku University) 41
Belief Propagation
1
2 21}2,1{ , xxW
b41
1, xWB b
1
a3
1, xWA a
d
26
2, xWC c
c5
2
2, xWD d
2221}2,1{
11
21
,,,,,
,,,,,
xWxWxxWxWxW
xxP
C
BA
dcba
dcba
ab
1
cd
2
3 4
56
Probabilistic Model on Tree Graph
![Page 42: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/42.jpg)
Physics Fluctuomatics (Tohoku University) 42
Probabilistic Model on Tree Graph
ab
1
cd
2
3 4
56
22522621}2,1{114113
2121}2,1{
,
,,,,,,
xMxMxxWxMxM
xxPxxP
dc,b,a,
dcba
a
a 1113 , xWxM A b
b 1114 , xWxM B
c
c 2225 , xWxM C d
D xdWxM 2226 ,
![Page 43: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/43.jpg)
Physics Fluctuomatics (Tohoku University) 43
Belief Propagation
ab
1
cd
2
3 4
56
1
1
11411321}2,1{
1121}2,1{221
,
,,,
x
x a bBA
xMxMxxW
xWxWxxWxM ba
Probabilistic Model on Tree Graph
a
a 1113 , xWxM A
b
b 1114 , xWxM B
![Page 44: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/44.jpg)
Physics Fluctuomatics (Tohoku University) 44
Belief Propagation for Probabilistic Model on Tree Graph
},{
},{ ,1Prji
jiji xxWZ
xX
No Cycles!!
1X
2X 3X
1kX
kX
2kX
3kX
1kX
![Page 45: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/45.jpg)
Physics Fluctuomatics (Tohoku University) 45
Belief Propagation for Probabilistic Model on Square Grid Graph
E: Set of all the links
Eji
jiijL xxWxxxPxP},{
21 ,,,,
![Page 46: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/46.jpg)
Physics Fluctuomatics (Tohoku University) 46
Belief Propagation for Probabilistic Model on Square Grid Graph
![Page 47: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/47.jpg)
Physics Fluctuomatics (Tohoku University) 47
Belief Propagation for Probabilistic Model on Square Grid Graph
![Page 48: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/48.jpg)
Physics Fluctuomatics (Tohoku University) 48
Marginal Probability
1 3 4
,,,,, 432122x x x x
NN
xxxxxPxP
![Page 49: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/49.jpg)
Physics Fluctuomatics (Tohoku University) 49
Marginal Probability
1 3 4x x x xN
2
1 3 4
,,,,, 432122x x x x
NN
xxxxxPxP
![Page 50: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/50.jpg)
Physics Fluctuomatics (Tohoku University) 50
Marginal Probability
1 3 4x x x xN
2 2
1 3 4
,,,,, 432122x x x x
NN
xxxxxPxP
![Page 51: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/51.jpg)
Physics Fluctuomatics (Tohoku University) 51
Marginal Probability
3 4
,,,,,, 432121}2,1{x x x
NN
xxxxxPxxP
![Page 52: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/52.jpg)
Physics Fluctuomatics (Tohoku University) 52
Marginal Probability
3 4x x xN
3 4
,,,,,, 432121}2,1{x x x
NN
xxxxxPxxP
1 2
![Page 53: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/53.jpg)
Physics Fluctuomatics (Tohoku University) 53
Marginal Probability
3 4x x xN
3 4
,,,,,, 432121}2,1{x x x
NN
xxxxxPxxP
1 2 1 2
![Page 54: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/54.jpg)
Physics Fluctuomatics (Tohoku University) 54
Belief Propagation for Probabilistic Model on Square Grid Graph
1
21}2,1{22 ,x
xxPxP
![Page 55: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/55.jpg)
Physics Fluctuomatics (Tohoku University) 55
Belief Propagation for Probabilistic Model on Square Grid Graph
1
21}2,1{22 ,x
xxPxP
14
5
3
2
6
8
7
![Page 56: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/56.jpg)
Physics Fluctuomatics (Tohoku University) 56
Belief Propagation for Probabilistic Model on Square Grid Graph
21 7
6
8
1
21}2,1{22 ,x
xxPxP
14
5
3
2
6
8
7
![Page 57: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/57.jpg)
Physics Fluctuomatics (Tohoku University) 57
Belief Propagation for Probabilistic Model on Square Grid Graph
21 7
6
8
1
21}2,1{22 ,x
xxPxP
Message Update Rule
14
5
3
2
6
8
7
3
2 1
5
41x
12
![Page 58: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/58.jpg)
Physics Fluctuomatics (Tohoku University) 58
Belief Propagation for Probabilistic Model on Square Grid Graph
21
3
4
5
3
2 1
5
413M
14M
15M
1x1
21M2
1 2
1
1151141132112
1151141132112
221 ,
,
z z
z
zMzMzMzzW
zMzMzMxzWxM
MM
Fixed Point Equations for Messages
![Page 59: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/59.jpg)
Physics Fluctuomatics (Tohoku University) 59
Fixed Point Equation and Iterative Method
Fixed Point Equation ** MM
![Page 60: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/60.jpg)
Physics Fluctuomatics (Tohoku University) 60
Fixed Point Equation and Iterative Method
Fixed Point Equation ** MM
Iterative Method
0
xy
)(xy
y
x*M
![Page 61: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/61.jpg)
Physics Fluctuomatics (Tohoku University) 61
Fixed Point Equation and Iterative Method
Fixed Point Equation ** MM
Iterative Method
0M0
xy
)(xy
y
x*M
![Page 62: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/62.jpg)
Physics Fluctuomatics (Tohoku University) 62
Fixed Point Equation and Iterative Method
Fixed Point Equation ** MM
Iterative Method
01 MM
0M
1M
0
xy
)(xy
y
x*M
![Page 63: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/63.jpg)
Physics Fluctuomatics (Tohoku University) 63
Fixed Point Equation and Iterative Method
Fixed Point Equation ** MM
Iterative Method
12
01
MM
MM
0M1M
1M
0
xy
)(xy
y
x*M
![Page 64: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/64.jpg)
Physics Fluctuomatics (Tohoku University) 64
Fixed Point Equation and Iterative Method
Fixed Point Equation ** MM
Iterative Method
12
01
MM
MM
0M1M
1M
0
xy
)(xy
y
x*M
2M
![Page 65: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/65.jpg)
Physics Fluctuomatics (Tohoku University) 65
Fixed Point Equation and Iterative Method
Fixed Point Equation ** MM
Iterative Method
23
12
01
MM
MM
MM
0M1M
1M
0
xy
)(xy
y
x*M
2M
![Page 66: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/66.jpg)
Physics Fluctuomatics (Tohoku University) 66
Belief Propagation for Probabilistic Model on Square Grid Graph
Four Kinds of Update Rule with Three Inputs and One Output
![Page 67: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/67.jpg)
Physics Fluctuomatics (Tohoku University) 67
Interpretation of Belief Propagation based on Information Theory
0ln)(
x
xxx P
QQPQD
x
xx 1)( ,0 QQ
ZQF
ZQQxxWQPQD
QF
Ejijiij
ln][
lnln)(,ln)(]|[
][
},{
xxxxx
0 PQDPQ xx
ZPFQQFQ
ln][1][min
x
x
x Eji
jiij xxWZ},{
,
Eji
jiijV xxWZ
xxxP},{
||21 ,1,,,
Free Energy
Kullback-Leibler Divergence
![Page 68: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/68.jpg)
Physics Fluctuomatics (Tohoku University) 68
Interpretation of Belief Propagation based on Information Theory
ZQFPQD ln
xxxx
xxxx
xxx
xx
xx xx
xx
QQWQ
QQWQ
QQxWQQF
E
E
E
ln)(ln
ln)(ln)(
ln)(ln)(
\
0ln)(
x
xxx P
QQPQD
Free EnergyKL Divergence
E
WZ
P
xx 1
x\x
x
x
)(
)(
Q
Q
![Page 69: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/69.jpg)
Physics Fluctuomatics (Tohoku University) 69
Interpretation of Belief Propagation based on Information Theory
ZQFPQD ln
Ejijjiiijij
Viii
Ejiijij
Ejiijij
QQQQQQ
WQ
WQQF
},{
},{
},{
lnln,ln,
ln
,ln,
ln)(
,ln,
xxx
Bethe Free Energy
Free EnergyKL Divergence
E
WZ
P
xx 1
x\x
xx )()( QQ
ix
ii QxQ\x
x)()(
![Page 70: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/70.jpg)
Physics Fluctuomatics (Tohoku University) 70
Interpretation of Belief Propagation based on Information Theory
FPQDQQ
minargminarg
,iji QQ
ZQQFPQD iji ln,Bethe
ijiQQQ
QQFPQDiji
,minargminarg Bethe,
1,
iji QQ
Ejijjiiijij
Viii
Ejiijijiji
QQQQQQ
QQWQQQF
},{
},{Bethe
lnln,ln,
ln,ln,,
![Page 71: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/71.jpg)
Physics Fluctuomatics (Tohoku University) 71
Interpretation of Belief Propagation based on Information Theory
Ejiijij
Viii
Vi ijijiji
ijiiji
QQFQQL
},{
,
BetheBethe
1,1
,
,,
1, ,,,minarg Bethe,
ijiijiijiQQ
QQQQQQFiji
Lagrange Multipliers to ensure the constraints
![Page 72: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/72.jpg)
Physics Fluctuomatics (Tohoku University) 72
Interpretation of Belief Propagation based on Information Theory
Bijijij
Viii
Vi ijijiji
Ejijjiiijij
Viii
Ejiijij
Ejiijij
Viii
Vi ijijijiijiiji
QQQQ
QQQQQQ
QQWQ
QQQQFQQL
1,1,
lnln,ln,
ln,ln,
1,1
,,,
,
},{
},{
},{
,BetheBethe
0,Bethe
ijiii
QQLxQ
Extremum Condition
0,, Bethe
ijijiij
QQLxxQ
![Page 73: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/73.jpg)
Interpretation of Bethe Approximation (7)
FGfP yxyxyx g ,,,
ikikiiii x
ixQ )(
1||1exp },{,
)()(exp
,,
},{,},{, jjijijii
jiijjiij
aa
xxWxxQ
Extremum Condition 0,Bethe
ijiii
QQLxQ 0,
, Bethe
iji
jiij
QQLxxQ
73Physics Fluctuomatics (Tohoku
University)
ik
iiki
ii xMZ
xQ 1
}{\}{\,1,
ijljjljiij
jikiik
ijjiij xMxxWxM
ZxxQ
)()(exp\
},{, ijik
ikijii xMx
![Page 74: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/74.jpg)
Physics Fluctuomatics (Tohoku University) 74
Interpretation of Belief Propagation based on Information Theory
FGfP yxyxyx g ,,,14 2
5
13M
14M
15M
12M
3
26M14
5
13M
14M
15M
12W3
2
6
27M
8
7
28M
115114
1131121
111
xMxM
xMxMZ
xQ
2282272262112
11511411312
2112
,
1,
xMxMxMxxW
xMxMxMZ
xxQ
Extremum Condition 0,Bethe
ijiii
QQLxQ 0,
, Bethe
iji
jiij
QQLxxQ
![Page 75: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/75.jpg)
Physics Fluctuomatics (Tohoku University) 75
Interpretation of Belief Propagation based on Information Theory
FGfP yxyxyx g ,,,14 2
5
13M
14M
15M
12M
3
26M14
5
13M
14M
15M
12W3
2
6
27M
8
7
28M
,121 QQ
115114
1131121
111
xMxM
xMxMZ
xQ
2282272262112
11511411312
2112
,
1,
xMxMxMxxW
xMxMxMZ
xxQ
1514
1312
21
,
MM
MW
M
Message Update Rule
![Page 76: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/76.jpg)
Physics Fluctuomatics (Tohoku University) 76
Interpretation of Belief Propagation based on Information Theory
15141312
15141312
21 ,
,
MMMW
MMMWM
1
3
4 2
5
13M
14M
15M
21M
14
5
3
2
6
8
7
2a
14 2
5
3
=
Message Passing Rule of Belief Propagation
![Page 77: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/77.jpg)
Physics Fluctuomatics (Tohoku University)
Graphical Representations for Probabilistic Models
Probability distribution with two random variables is assigned to a edge.
),(},Pr{ 21}2,1{2211 xxfxXxX 1 2
)(}Pr{ 1111 xfxX 1 Node
Edge
Probability distribution with one random variable is assigned to a graph with one node.
),,(},,Pr{ 321}3,2,1{332211 xxxfxXxXxX 31
2Hyper-edge
77
![Page 78: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/78.jpg)
Physics Fluctuomatics (Tohoku University)
Hyper-graph
Bayesian Network and Graphical Model
),(),(},,Pr{ 32}3,2{21}2,1{332211 xxfxxfxXxXxX
321
4
),(),(),(},,Pr{
13}1,3{32}3,2{21}2,1{
332211
xxfxxfxxfxXxXxX
Tree
31
2Cycle
More practical probabilistic models are expressed in terms of a product of functions and is assigned to chain, tree, cycle or hyper-graph representation.
1 2 3Chain
),(),(),(},,,Pr{
41}4,1{31}3,1{21}2,1{
44332211
xxfxxfxxfxXxXxXxX
312
45),,(),,(
},,,,Pr{
543}5,4,3{321}3,2,1{
5544332211
xxxfxxxfxXxXxXxXxX
78
![Page 79: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/79.jpg)
Physics Fluctuomatics (Tohoku University) 79
Graphical Representations of Tractable Probabilistic Models
),,(),,(),,(),,( IHEWGFDWEDCWCBAW EHIDFGCDEABC
),,( CBAWABC ),,( EDCWCDE
X=C
D
EA
CB
D
F GX
E H
I
=
A
B C
D
E
F G
H
I
),,( IHEWEHI),,( GEDWDEG
X
![Page 80: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/80.jpg)
H IF GA B
Physics Fluctuomatics (Tohoku University) 80
Graphical Representations of Tractable Probabilistic Models
A B F G H I
EDC },,Pr{
A
B C
D
E
F G
H
I
C
D
E A
B C F G
D
E H
I
C
D
E A
B C F G
D
E H
I
A
B C
D
E
F G
H
I
=
= x x x =
![Page 81: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/81.jpg)
D E H IF GA B
Physics Fluctuomatics (Tohoku University) 81
Graphical Representations of Tractable Probabilistic Models
A B D E F G H I
C}Pr{
A
B C
D
E
F G
H
I
C
D
EA
B C
C
D
EA
B C F G
D
E H
I=
= x =
A
B C
D
E
![Page 82: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/82.jpg)
Physics Fluctuomatics (Tohoku University) 82
Graphical Representations of Tractable Probabilistic Models
},,Pr{ EDCC
D
E A
B C F G
D
E H
Ix x x
C
D
EA
B C x}Pr{C
C
D
E
F G
D
E H
IA
B Cx x xC
D
EA
B C x D E
C
D
E
F G
D
E H
Ix x
C
D
ED E
D E
C
D
E
F G
H
I
D E
EDCC },,Pr{}Pr{
![Page 83: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/83.jpg)
Physics Fluctuomatics (Tohoku University) 83
Belief Propagation on Hypergraph Representations in terms of Cactus Tree
C
D
ED E
C
D
E
F G
H
I
F G
D
E H
I
A
B C
F G
D
E H
I
F G
H I
A B A
B C
A
B C
D
E}Pr{C
A
B C
D
E
F G
H
I
Update Flow of Messages in computing the marginal probability Pr{C}
![Page 84: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/84.jpg)
Physics Fluctuomatics (Tohoku University) 84
Interpretation of Belief Propagationfor Hypergraphs based on Information Theory
or :, VEE
We consider hypergraphs which satisfy
Cactus Tree
HypergraphV: Set of all the nodesE: Set of all the hyperedges
![Page 85: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/85.jpg)
Physics Fluctuomatics (Tohoku University) 85
Interpretation of Belief Propagation based on Information Theory
0ln)(
x
xxx P
QQPQD
x
xx 1)( ,0 QQ
ZQF
ZQQWQPQD
QF
E
ln][
lnln)(ln)(]|[
][
xxxxxx
0 PQDPQ xx
ZPFQQFQ
ln][1][min
x
x
x
xE
WZ
E
V xWZ
xxxP
1,,, ||21
Free Energy
Kullback-Leibler Divergence
![Page 86: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/86.jpg)
Physics Fluctuomatics (Tohoku University) 86
Interpretation of Belief Propagation based on Information Theory
ZQFPQD ln
xxxx
xxxx
xxx
xx
xx xx
xx
QQWQ
QQWQ
QQxWQQF
E
E
E
ln)(ln
ln)(ln)(
ln)(ln)(
\
0ln)(
x
xxx P
QQPQD
Free EnergyKL Divergence
E
WZ
P
xx 1
x\x
x
x
)(
)(
Q
Q
![Page 87: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/87.jpg)
E i xiiii
Viii
E
E
i
xQxQQQ
WQ
QQWQQF
lnln
ln
ln
ln)(ln
x
x
xx
xx
xx
xxxx
Physics Fluctuomatics (Tohoku University) 87
Interpretation of Belief Propagation based on Information Theory
ZQFPQD ln
Free EnergyKL Divergence
E
WZ
P
xx 1
x\x
xx )()( QQ
ix
ii QxQ\x
x)()(
Bethe Free Energy
![Page 88: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/88.jpg)
Physics Fluctuomatics (Tohoku University) 88
Interpretation of Belief Propagation based on Information Theory
FPQDQQ
minargminarg
ix
ii QxQ\
x
x
ZQQFPQD iji ln,Bethe
QQFPQD iQQQ i
,minargminarg Bethe,
1
x
xQxQix
ii
E i xiiii
Viii
Ei
i
xQxQQQ
QQWQQQF
lnln
lnln,Bethe
x
x
xx
xx
![Page 89: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/89.jpg)
Physics Fluctuomatics (Tohoku University) 89
Interpretation of Belief Propagation based on Information Theory
EVi xiii
Vi i xiii
ii
QxQ
QxQ
QQFQQL
i
i
11
,,
\,
BetheBethe
x
x
x
x
1 ,,minarg\
Bethe,
xx
xx QxQQxQQQFiii x
iix
iiiQQ
Lagrange Multipliers to ensure the constraints
![Page 90: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/90.jpg)
Physics Fluctuomatics (Tohoku University) 90
Interpretation of Belief Propagation based on Information Theory
Eij
Vi xiii
Vi i x xiiii
E Vi xii
Viii
xii
E
EVi xiii
Vi i x xiiiiii
QxQQxQx
QQQQ
xQxQWQ
QxQ
QxQxQQFQQL
ii i
i
i
i
i i
11
lnln
lnln
11
,,
\,
\,BetheBethe
xx
x
x
x
x
xx
xx
xx
x
x
0,Bethe
QQLxQ iii
Extremum Condition
0,Bethe
QQLQ ix
![Page 91: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/91.jpg)
Physics Fluctuomatics (Tohoku University) 91
Interpretation of Belief Propagation based on Information Theory
i
iiii
ii xMZ
xQ
\1
j jjjj xMxW
ZxQ
\\
1
Extremum Condition 0,Bethe
QQLxQ iii 0,
, Bethe
QQL
xxQ ijiij
ix
ii QxQ\
x
x
ij jjjj
iiii xMxW
ZZxM
\ \\\
![Page 92: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/92.jpg)
Physics Fluctuomatics (Tohoku University) 92
Summary
Belief Propagation and Message Passing RuleInterpretation of Belief Propagation in the stand point of Information Theory
Future Talks
11th Probabilistic image processing by means of physical models 12th Bayesian network and belief propagation in statistical inference
![Page 93: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/93.jpg)
Physics Fluctuomatics (Tohoku University) 93
Practice 9-1
a b c d
XY ya,b,c,d,xPyxP ,,
a
AXA xaWxM , b
BXB xbWxM ,
c
CYC ycWyM , d
DYD ydWyM ,
ydWycWyxWxbWxaWya,b,c,d,xP DCXYBA ,,,,,,
We consider a probability distribution P(a,b,c,d,x,y) defined by
Show that marginal Probability
is expressed by
yMyMyxWxMxMyxP DCXYXBXAXY 22,,
![Page 94: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/94.jpg)
Physics Fluctuomatics (Tohoku University) 94
Practice 9-2
1151141131121
111 xMxMxMxMZ
xQ
228227226211211511411312
2112 ,1, xMxMxMxxWxMxMxMZ
xxQ
2
228227226211212
1112 ,
x
xMxMxMxxWZZxM
By substituting
2
211211 ,x
xxQxQto , derive the following equation.
![Page 95: Physical Fluctuomatics 7th~10th Belief propagation](https://reader035.vdocument.in/reader035/viewer/2022062323/568162a5550346895dd32287/html5/thumbnails/95.jpg)
Physics Fluctuomatics (Tohoku University) 95
Practice 9-3
Make a program to solve the nonlinear equation x=tanh(Cx) for various values of C. Obtain the solutions for C=0.5, 1.0, 2.0 numerically. Discuss how the iterative procedures converge to the fixed points of the equations in the cases of C=0.5, 1.0, 2.0 by drawing the graphs of y=tanh(Cx) and y=x.
23
12
01
tanhtanhtanh
CxxCxxCxx