belief propagation in large, highly connected graphs for 3d part-based object recognition frank...
DESCRIPTION
Part-Based Object Recognition Probability of a configuration U={u i } – given an image I – is the product of potential functions For part-based object recognition Skeletal graph for tightly coupled parts Occupancy graph ensures no other parts collide in 3D spaceTRANSCRIPT
![Page 1: Belief Propagation in Large, Highly Connected Graphs for 3D Part-Based Object Recognition Frank DiMaio and Jude Shavlik Computer Sciences Department University](https://reader035.vdocument.in/reader035/viewer/2022062302/5a4d1b0e7f8b9ab05998d4f1/html5/thumbnails/1.jpg)
Belief Propagation in Large, Highly Connected
Graphs for 3D Part-Based Object Recognition
Frank DiMaio and Jude ShavlikComputer Sciences Department
University of Wisconsin – MadisonUSA
![Page 2: Belief Propagation in Large, Highly Connected Graphs for 3D Part-Based Object Recognition Frank DiMaio and Jude Shavlik Computer Sciences Department University](https://reader035.vdocument.in/reader035/viewer/2022062302/5a4d1b0e7f8b9ab05998d4f1/html5/thumbnails/2.jpg)
Part-Based Object Recognition
A part-based model describes an object using a pairwise Markov Field(Felzenszwalb et al 2000, Sudderth et al 2004, Isard 2003)
Object described using Undirected part graph G=(V,E) Vertex potential functions Edge potential functions
![Page 3: Belief Propagation in Large, Highly Connected Graphs for 3D Part-Based Object Recognition Frank DiMaio and Jude Shavlik Computer Sciences Department University](https://reader035.vdocument.in/reader035/viewer/2022062302/5a4d1b0e7f8b9ab05998d4f1/html5/thumbnails/3.jpg)
Part-Based Object Recognition
Probability of a configuration U={ui} – given an image I – is the product of potential functions
edges
( )st s ts t
ψ u ,u
vertices
( | )s ss
u I
( | )P U I
For part-based object recognition Skeletal graph for tightly coupled parts Occupancy graph ensures no other parts
collide in 3D space
![Page 4: Belief Propagation in Large, Highly Connected Graphs for 3D Part-Based Object Recognition Frank DiMaio and Jude Shavlik Computer Sciences Department University](https://reader035.vdocument.in/reader035/viewer/2022062302/5a4d1b0e7f8b9ab05998d4f1/html5/thumbnails/4.jpg)
Inferring Part Locations with Belief Propagation (BP) Want to find part configuration maximizing
product of potential functions
Use belief propagation (BP) to approximate marginal distributions
Iterative, message-passing method (Pearl 1988)
A message, mi→j, from part i to part j indicates where i currently expects to find j
![Page 5: Belief Propagation in Large, Highly Connected Graphs for 3D Part-Based Object Recognition Frank DiMaio and Jude Shavlik Computer Sciences Department University](https://reader035.vdocument.in/reader035/viewer/2022062302/5a4d1b0e7f8b9ab05998d4f1/html5/thumbnails/5.jpg)
Belief Propagation Example
b( torso | image)
b( head | image)
b( left arm | image)
b( left leg | image)
b( right arm | image)
b( right leg | image)
![Page 6: Belief Propagation in Large, Highly Connected Graphs for 3D Part-Based Object Recognition Frank DiMaio and Jude Shavlik Computer Sciences Department University](https://reader035.vdocument.in/reader035/viewer/2022062302/5a4d1b0e7f8b9ab05998d4f1/html5/thumbnails/6.jpg)
Belief Propagation Example
m head→torso(torso)
b( torso | image)
b( head | image)
mR.leg→torso
mL.leg→torso
mR.arm→torso
mL.arm→torso
![Page 7: Belief Propagation in Large, Highly Connected Graphs for 3D Part-Based Object Recognition Frank DiMaio and Jude Shavlik Computer Sciences Department University](https://reader035.vdocument.in/reader035/viewer/2022062302/5a4d1b0e7f8b9ab05998d4f1/html5/thumbnails/7.jpg)
Belief Propagation Example
b( torso | image)
b( head | image)
b( left arm | image)
b( left leg | image)
b( right arm | image)
b( right leg | image)
![Page 8: Belief Propagation in Large, Highly Connected Graphs for 3D Part-Based Object Recognition Frank DiMaio and Jude Shavlik Computer Sciences Department University](https://reader035.vdocument.in/reader035/viewer/2022062302/5a4d1b0e7f8b9ab05998d4f1/html5/thumbnails/8.jpg)
What if the Graph has Thousands of Parts? In a graph with N parts and E edges
BP running time and memory requirements O(E) Skeleton graph typically sparse – O(N) edges Occupancy graph fully connected – O(N 2) edges
In very large graphs, O(N 2) runtime intractable
AggBP (our system) approximates O(N 2) occupancy messages using O(N) messages
![Page 9: Belief Propagation in Large, Highly Connected Graphs for 3D Part-Based Object Recognition Frank DiMaio and Jude Shavlik Computer Sciences Department University](https://reader035.vdocument.in/reader035/viewer/2022062302/5a4d1b0e7f8b9ab05998d4f1/html5/thumbnails/9.jpg)
Message Approximation Illustrated
2
3
5 71
6
8
3
6
5
4
441
1 4
bm
442
2 4
bm
448
8 4
bm
4
477 4
bm
4 4b
4 4b
4 4b 4 4b
![Page 10: Belief Propagation in Large, Highly Connected Graphs for 3D Part-Based Object Recognition Frank DiMaio and Jude Shavlik Computer Sciences Department University](https://reader035.vdocument.in/reader035/viewer/2022062302/5a4d1b0e7f8b9ab05998d4f1/html5/thumbnails/10.jpg)
Message Approximation Illustrated
2
3
5 71
6
8
4
Accumulator
![Page 11: Belief Propagation in Large, Highly Connected Graphs for 3D Part-Based Object Recognition Frank DiMaio and Jude Shavlik Computer Sciences Department University](https://reader035.vdocument.in/reader035/viewer/2022062302/5a4d1b0e7f8b9ab05998d4f1/html5/thumbnails/11.jpg)
Experiment I: Density Map InterpretationGLU TYR PHE THR LEU GLN ILE
ARG GLY ARG GLU ARG PHE…
GLY31ALA30
GLN29ALA28
LYS26
LEU25GLU24
LEU23
GLU21
ASN20
LEU19
GLU18ARG17
PHE16
MET15
GLU14PHE13
ARG12
GLU11ARG10
GLY9ARG8
ILE7
GLN6
LEU5
THR4
PHE3
TYR2
GLU1
ALA22
ASP27
![Page 12: Belief Propagation in Large, Highly Connected Graphs for 3D Part-Based Object Recognition Frank DiMaio and Jude Shavlik Computer Sciences Department University](https://reader035.vdocument.in/reader035/viewer/2022062302/5a4d1b0e7f8b9ab05998d4f1/html5/thumbnails/12.jpg)
LoopyBP vs. AggBP:Runtime
Number of Parts
Nor
mal
ized
Run
time
0
5
10
15
20
25
30
15 25 35 45 55 65 75 85 95
AggBP
LoopyBP
![Page 13: Belief Propagation in Large, Highly Connected Graphs for 3D Part-Based Object Recognition Frank DiMaio and Jude Shavlik Computer Sciences Department University](https://reader035.vdocument.in/reader035/viewer/2022062302/5a4d1b0e7f8b9ab05998d4f1/html5/thumbnails/13.jpg)
LoopyBP vs. AggBP: Accuracy
BP iteration
0
2
4
6
8
10
0 5 10 15 20
AggBP
LoopyBP RM
S E
rror
![Page 14: Belief Propagation in Large, Highly Connected Graphs for 3D Part-Based Object Recognition Frank DiMaio and Jude Shavlik Computer Sciences Department University](https://reader035.vdocument.in/reader035/viewer/2022062302/5a4d1b0e7f8b9ab05998d4f1/html5/thumbnails/14.jpg)
Experiment II: Synthetic Graph Generator
increase branching factor
allow spatialoverlap
vary radii
[skeleton graph]
![Page 15: Belief Propagation in Large, Highly Connected Graphs for 3D Part-Based Object Recognition Frank DiMaio and Jude Shavlik Computer Sciences Department University](https://reader035.vdocument.in/reader035/viewer/2022062302/5a4d1b0e7f8b9ab05998d4f1/html5/thumbnails/15.jpg)
LoopyBP vs. AggBP:Accuracy
0 1 2 5
stdev(part size)
RM
S E
rror
0
2
4
6
0 2 4
LoopyBP
AggBP
![Page 16: Belief Propagation in Large, Highly Connected Graphs for 3D Part-Based Object Recognition Frank DiMaio and Jude Shavlik Computer Sciences Department University](https://reader035.vdocument.in/reader035/viewer/2022062302/5a4d1b0e7f8b9ab05998d4f1/html5/thumbnails/16.jpg)
Conclusions AggBP makes belief propagation tractable in
large, highly connected graphs
For part-based modeling, runtime and storage is reduced from O(N 2) to O(N)
AggBP’s solutions on two datasets are as good as standard BP’s in less time
![Page 17: Belief Propagation in Large, Highly Connected Graphs for 3D Part-Based Object Recognition Frank DiMaio and Jude Shavlik Computer Sciences Department University](https://reader035.vdocument.in/reader035/viewer/2022062302/5a4d1b0e7f8b9ab05998d4f1/html5/thumbnails/17.jpg)
Acknowledgements Dr. George Phillips NLM Grant 1R01 LM008796 NLM Grant 1T15 LM007359