rewriting our energy function - artificial...

Rewriting our energy function

Is a function of labelings and segment models, but our models are functions of the labelings, so we rewrite to be a function of ONLY the labelings.

5 2 3= [0.5 0.2 0.3]

2 4 4= [0.2 0.4 0.4]

Background Pixels Foreground Pixels

Rewriting our energy function

5 2 3= [0.5 0.2 0.3]

2 4 4= [0.2 0.4 0.4]

Background Pixels Foreground Pixels

Rewriting our energy function

5 2 3= [0.5 0.2 0.3]

2 4 4= [0.2 0.4 0.4]

Background Pixels Foreground Pixels

Rewriting our energy function

Rewriting our energy function

Concave. Minimized when pixels in a bucket are all of the same label.

Convex. Minimized when all pixels are split evenly between foreground and background.

Some intuition

When every pixel in its own bucket, then for all buckets:

Then is constant w.r.t labelings.

Bias towards splitting pixels evenly between foreground and background.

Some intuition

When every pixel in its own bucket, then for all buckets:

Then is constant w.r.t labelings.

Bias towards splitting pixels evenly between foreground and background.

Consider other extreme, where all pixels are together in one bucket.

Then and , so, convex and concave parts cancel.

No bias.

Some intuition

If no pixels in a bucket are neighbors, then there is an optimal solution where all pixels in that bucket are assigned together.

So how do we optimize this?

An aside.......... (in paper)

What is st-mincut?

SOURCE

SINK

(Background Pixels)

(Foreground Pixels)

Mapping unary and pairwise potential to mincut

SOURCE

SINK

(Background Pixels)

(Foreground Pixels)

How do we map the cliques onto this?

Mapping clique potential to mincut

Recall intuition about

Mapping clique potential to mincut

Recall intuition about

= +

+

...

Mapping clique potential to mincut

Recall intuition about

=

SOURCE

(Foreground Pixels)

A lot of edges!

Mapping clique potential to mincut

Mapping clique potential to mincut

Mapping clique potential to mincut

Back to the original problem...

Recall: we have figured out how to optimize the Dual Decomposition of our joint optimization problem.

Back to the original problem...

This can perform poorly.

Combine approaches:

is our segmentation from dual decomposition, which gives energy

Compute models and empirically

Do EM by minimizing (not joint)

Final formulationMinimize w.r.t. segmentation labelings:

Solution cannot be worse than just the EM step, or just the joint minimization approximation, if we can find it.

DD works well for small

Experimentation (add weight term)-163 histogram buckets evenly covering RGB space-Neighbors defined as all 8 surrounding pixels

-

-~10 minutes per image

Results: Dual Decomposition Only

Can get global optimum from DD.

Results: DD+EM MethodSolution 1:

Solution 2:

% times better than EM Optimum on

Results

1. Find solution using EM and DD separately, choose best.

2. Run DD+EM with = 0.75

3. Run DD+EM with = 0.5

4. Run DD+EM with = 0.25

5. Run DD+EM with = 0

7.2% error rate on GrabCut dataset.

8.1% error rate on GrabCut with histogram model

Extensions

Reduces error on globally optimal set from 4.1% to 3%

Summary

● Reformulated energy for simultaneous optimization of labeling and color model

● Derived dual decomposition so we could try to find optimal solution

● Optimized dual decomposition in polynomial time

● [Reviewed min-st cut]

● Adjusted algorithm to improve results