rewriting our energy function - artificial...
TRANSCRIPT
Rewriting our energy function
Is a function of labelings and segment models, but our models are functions of the labelings, so we rewrite to be a function of ONLY the labelings.
5 2 3= [0.5 0.2 0.3]
2 4 4= [0.2 0.4 0.4]
Background Pixels Foreground Pixels
Rewriting our energy function
5 2 3= [0.5 0.2 0.3]
2 4 4= [0.2 0.4 0.4]
Background Pixels Foreground Pixels
Rewriting our energy function
5 2 3= [0.5 0.2 0.3]
2 4 4= [0.2 0.4 0.4]
Background Pixels Foreground Pixels
Rewriting our energy function
Rewriting our energy function
Concave. Minimized when pixels in a bucket are all of the same label.
Convex. Minimized when all pixels are split evenly between foreground and background.
Some intuition
When every pixel in its own bucket, then for all buckets:
Then is constant w.r.t labelings.
Bias towards splitting pixels evenly between foreground and background.
Some intuition
When every pixel in its own bucket, then for all buckets:
Then is constant w.r.t labelings.
Bias towards splitting pixels evenly between foreground and background.
Consider other extreme, where all pixels are together in one bucket.
Then and , so, convex and concave parts cancel.
No bias.
Some intuition
If no pixels in a bucket are neighbors, then there is an optimal solution where all pixels in that bucket are assigned together.
So how do we optimize this?
An aside.......... (in paper)
What is st-mincut?
SOURCE
SINK
(Background Pixels)
(Foreground Pixels)
Mapping unary and pairwise potential to mincut
SOURCE
SINK
(Background Pixels)
(Foreground Pixels)
How do we map the cliques onto this?
Mapping clique potential to mincut
Recall intuition about
Mapping clique potential to mincut
Recall intuition about
= +
+
...
Mapping clique potential to mincut
Recall intuition about
=
SOURCE
(Foreground Pixels)
A lot of edges!
Mapping clique potential to mincut
Mapping clique potential to mincut
Mapping clique potential to mincut
Back to the original problem...
Recall: we have figured out how to optimize the Dual Decomposition of our joint optimization problem.
Back to the original problem...
This can perform poorly.
Combine approaches:
is our segmentation from dual decomposition, which gives energy
Compute models and empirically
Do EM by minimizing (not joint)
Final formulationMinimize w.r.t. segmentation labelings:
Solution cannot be worse than just the EM step, or just the joint minimization approximation, if we can find it.
DD works well for small
Experimentation (add weight term)-163 histogram buckets evenly covering RGB space-Neighbors defined as all 8 surrounding pixels
-
-~10 minutes per image
Results: Dual Decomposition Only
Can get global optimum from DD.
Results: DD+EM MethodSolution 1:
Solution 2:
% times better than EM Optimum on
Results
1. Find solution using EM and DD separately, choose best.
2. Run DD+EM with = 0.75
3. Run DD+EM with = 0.5
4. Run DD+EM with = 0.25
5. Run DD+EM with = 0
7.2% error rate on GrabCut dataset.
8.1% error rate on GrabCut with histogram model
Extensions
Reduces error on globally optimal set from 4.1% to 3%
Summary
● Reformulated energy for simultaneous optimization of labeling and color model
● Derived dual decomposition so we could try to find optimal solution
● Optimized dual decomposition in polynomial time
● [Reviewed min-st cut]
● Adjusted algorithm to improve results