kazuyuki tanaka graduate school of information sciences tohoku university, sendai 980-8579, japan
DESCRIPTION
Cluster Variation Method for Correlation Function of Probabilistic Model with Loopy Graphical Structure. Kazuyuki Tanaka Graduate School of Information Sciences Tohoku University, Sendai 980-8579, Japan [email protected]. Introduction. Cluster Variation Method (CVM) - PowerPoint PPT PresentationTRANSCRIPT
1
Cluster Variation Methodfor Correlation Function of Probabilistic Model
with Loopy Graphical Structure
Cluster Variation Methodfor Correlation Function of Probabilistic Model
with Loopy Graphical Structure
Kazuyuki TanakaGraduate School of Information Sciences
Tohoku University, Sendai 980-8579, [email protected]
Kazuyuki TanakaGraduate School of Information Sciences
Tohoku University, Sendai 980-8579, [email protected]
2
Introduction
Cluster Variation Method (CVM)Cluster Variation Method (CVM) Stat. Phys. [R. Kikuchi 1951]Stat. Phys. [R. Kikuchi 1951] NIPS [J. S. Yedidia et al, 2000], [H. J. Kappen et al, 2001]NIPS [J. S. Yedidia et al, 2000], [H. J. Kappen et al, 2001]
Approximate marginal probability in probabilistic modelApproximate marginal probability in probabilistic model
Linear Response Theory (LRT)Linear Response Theory (LRT) MFA + LRT: H. J. Kappen et al 1998], [T. Tanaka 1998]MFA + LRT: H. J. Kappen et al 1998], [T. Tanaka 1998]
Correlation between any pair of nodesCorrelation between any pair of nodes
General CVM Approximate Formula of CorrelationGeneral CVM Approximate Formula of CorrelationGeneral CVM Approximate Formula of CorrelationGeneral CVM Approximate Formula of Correlation
++
3
Linear Response Theory
}.{ nodes ofset a is }|{ iixi ΩΩx
ji
ii xiii
xiiiiii xPxxPxPxPxx
Nodeat Field External respect to with Nodeat Average ofDeviation
)()(~
)()(~
xx
xx
i
j
hjiji h
xxxxx
i
P
0
lim
)(in Covariant x
x
x)(Pxx iix
x)(Pxxxx jiji
)exp()(~1
)(~
ii xhPZ
P xx
4
Final Result
}{}{ sSubcluster of SetClustersBasic of Set C).( )( xx PP for GBP of meansby calculated be canBelief
),( )()()(|| B\CxxxxxxxAxxx
PPP
),( ||)(||},,|{
1 B\CAGC
||lim 1
0ji
h
xxxxx
i
j
hjiji
i
G
1ix
}{ ClustersBasic of SetB
5
Basic Cluster and Subcluster
Example
}8,7,6,5
,4,3,2,1{Ω
1
3
2
4
6 5
87
3 4
6
1
3
5
8
6
2
5
7 6
4
2
1
3
2
4
6 5
87
3
3
6
5
5
4
4
6
6
22}67,568,346
,25,24,13{B
B\CB
BC of sSubcluster
}6,5,4,3,2{
,, , 2423463133
set. orderedpartially a is
C
6
Probabilistic Model Joint Probability Distribution
C
xx
)()(1
)( wZ
P
)()(1)(},|{
CC
1)78()568()346()25()24()13( 1)5()4()3()2( 2)6(
1x
3x
2x
4x
6x 5x
8x7x
13w
67w
24w
25w346w
568w
Example
1ix
7
Cluster Variation Method
Minimization of Free Energy
][minarg)(1 )( PFwZ P
C
x
CVM
)( ][)()(ln)(ln)(][
Cx C
xxx
PFPwPPF
x
xxx )(ln)(ln)(][ PwPPF
x\x
xx )()( PP
1)(),()(][)(minarg
}|{
}|{
xQxQxxQxQF
P
xxxQ CC
C
8
Marginal Distribution in CVM
)exp()(1
)()(}\,|{
,
BCx\x
xxxx
wZ
PP
0)( ,},|{
C
)~
exp()(~1
)exp()(~1
)(~
}\,|{,
Field External
BCx\x
xxxx
wZ
xhPZ
P ii
iih ,,},|{
~)(
C
)(~
)(~
xxxxxx PP
)()(
xxxxxx PP
Present Probabilistic ModelPresent Probabilistic Model
Probabilistic Model with External FieldProbabilistic Model with External Field
9
Linear Response in CVM
},|{,,
~||
)()(~
B\C
xx
A
xxxxx
PP
iih ,,,},|{
)~
()(
C
xxxxA ||
x
},|{
1,, ||
~
B\C
A
,},|{ },|{
1 ||)( iihA
C B\C
x
xx
x
xxx )( P
10
Correlation Function in CVM
),( || B\CxxxxA
),(|| ,}|{
ΩB\CGB\C
ihx ii
),( ||)(||},,|{
1 B\CAGC
),(
||lim 1
0
ΩC
G
ji
jih
xxxxx
i
j
hjiji
i
11
Numerical Experiments
8544.061 xx
1334.083 xx
0890.062 xx
0828.072 xx
8544.061 xx
1402.083 xx
0890.062 xx
0828.072 xx
Cluster Variation Method
Exact
1x
3x
2x
4x
6x 5x
8x7x
13w
67w
24w
25w346w
568w
12
Conclusions
Cluster Variation Method + Linear Response TheoryCluster Variation Method + Linear Response Theory→ → General CVM Approximate Formula for CorrelationGeneral CVM Approximate Formula for Correlation
Other Related Previous Work Other Related Previous Work CVM + LRTCVM + LRT → → General CVM Approximate Formula General CVM Approximate Formula for Fourier Transform of Correlation for Fourier Transform of Correlation of Probabilistic Model on Regular Lattice.of Probabilistic Model on Regular Lattice. [K. Tanaka, T. Horiguchi and T. Morita 1991][K. Tanaka, T. Horiguchi and T. Morita 1991]
||lim 1
0ji
h
xxxxx
i
j
hjiji
i
G
:Result Main
Extension of [H. J. Kappen et al 1998] and [T. Tanaka 1998]Extension of [H. J. Kappen et al 1998] and [T. Tanaka 1998]