relaxations and moves for map estimation in mrfs m. pawan kumar stanfordstanford vladimir...
Post on 21-Dec-2015
214 Views
Preview:
TRANSCRIPT
Relaxations and Moves forMAP Estimation in MRFs
M. Pawan Kumar
STANFORD
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
Vladimir Kolmogorov
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Philip Torr
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Daphne Koller
Our Problem
v1 v2 v3 v4
2
5
4
2
6
3
3
7
0
1 1
0
0
2 3
1
1
4 1
0Label l1
Label l2
Random Variables V = {v1, ... ,v4}
Label Set L = {l1, l2}
Labeling f: V L (shown in red)
Our Problem
v1 v2 v3 v4
2
5
4
2
6
3
3
7
0
1 1
0
0
2 3
1
1
4 1
0Label l1
Label l2
Random Variables V = {v1, ... ,v4}
Label Set L = {l1, l2}
Labeling f: V L (shown in red)
Energy of Labeling E(f) = 13 (shown in green)
Our Problem
v1 v2 v3 v4
2
5
4
2
6
3
3
7
0
1 1
0
0
2 3
1
1
4 1
0Label l1
Label l2
Find f* = argminf E(f)
Arbitrary topology, discrete label set, potentials (NP-hard)
Pairwise energy function: unary and pairwise potentials(still NP-hard)
Outline
• Convex Relaxations– Integer Programming Formulation– LP Relaxation– SDP Relaxation– SOCP Relaxation– Comparing Relaxations
• Move Making Algorithms
• Some Interesting Open Problems
Integer Programming Formulation
v1 v2
2
5
4
2
0
1 3
0
Unary Potentials
Unary Potential u = [ 5
Cost of v1 = 1
2
Cost of v1 = 2
; 2 4 ]
Labeling f shown in red
Label l1
Label l2
Label vector x = [ -1
v1 1
1
v1 = 2
; 1 -1 ]T
Recall that the aim is to find the optimal x
Integer Programming Formulation
v1 v2
2
5
4
2
0
1 3
0
Unary Potentials
Labeling f shown in red
Label l1
Label l2
Unary Potential u = [ 5 2 ; 2 4 ]
Label vector x = [ -1 1 ; 1 -1 ]T
Sum of Unary Potentials = 12
∑i ui (1 + xi)
Integer Programming Formulation
v1 v2
2
5
4
2
0
1 3
0
Unary Potentials
Labeling f shown in red
Label l1
Label l2
Unary Potential u = [ 5 2 ; 2 4 ]
0Cost of v1 = 1 and v1 = 1
0
00
0Cost of v1 = 1 and v2 = 1
3
Cost of v1 = 1 and v2 = 21 0
00
0 0
10
3 0
Pairwise Potential P
Integer Programming Formulation
v1 v2
2
5
4
2
0
1 3
0
Pairwise Potentials
Labeling f shown in red
Label l1
Label l2
Pairwise Potential P
0 0
00
0 3
1 0
00
0 0
10
3 0
Sum of Pairwise Potentials14
∑ij Pij (1 + xi)(1+xj)
Integer Programming Formulation
v1 v2
2
5
4
2
0
1 3
0
Pairwise Potentials
Labeling f shown in red
Label l1
Label l2
Sum of Pairwise Potentials14
∑ij Pij (1 + xi +xj + xixj)
14
∑ij Pij (1 + xi + xj + Xij)=
X = x xT Xij = xi xj
Integer Programming Formulation
v1 v2
2
5
4
2
0
1 3
0
Pairwise Potentials
Labeling f shown in red
Label l1
Label l2
Pairwise Potential P
0 0
00
0 3
1 0
00
0 0
10
3 0
Constraints
• Uniqueness Constraint
∑ xi = 2 - |L|i va
• Integer Constraints
xi {-1,1}
X = x xT
Integer Programming Formulation
x* = argmin 12
∑ ui (1 + xi) + 14
∑ Pij (1 + xi + xj + Xij)
∑ xi = 2 - |L|i va
xi {-1,1}
X = x xT
ConvexNon-Convex
Integer Programming Formulation
Outline
• Convex Relaxations– Integer Programming Formulation– LP Relaxation– SDP Relaxation– SOCP Relaxation– Comparing Relaxations
• Move Making Algorithms
• Some Interesting Open Problems
LP Relaxation
x* = argmin 12
∑ ui (1 + xi) + 14
∑ Pij (1 + xi + xj + Xij)
∑ xi = 2 - |L|i va
xi {-1,1}
X = x xT
Retain Convex PartSchlesinger, 1976
Relax Non-ConvexConstraint
LP Relaxation
x* = argmin 12
∑ ui (1 + xi) + 14
∑ Pij (1 + xi + xj + Xij)
∑ xi = 2 - |L|i va
xi [-1,1]
X = x xT
Retain Convex PartSchlesinger, 1976
Relax Non-ConvexConstraint
LP Relaxation
x* = argmin 12
∑ ui (1 + xi) + 14
∑ Pij (1 + xi + xj + Xij)
∑ xi = 2 - |L|i va
xi [-1,1]
X = x xT
Retain Convex PartSchlesinger, 1976
Relax Non-ConvexConstraint
LP Relaxation
x* = argmin 12
∑ ui (1 + xi) + 14
∑ Pij (1 + xi + xj + Xij)
∑ xi = 2 - |L|i va
xi [-1,1],
Retain Convex PartSchlesinger, 1976
Xij [-1,1]
1 + xi + xj + Xij ≥ 0
∑ Xij = (2 - |L|) xij vb
Outline
• Convex Relaxations– Integer Programming Formulation– LP Relaxation– SDP Relaxation– SOCP Relaxation– Comparing Relaxations
• Move Making Algorithms
• Some Interesting Open Problems
SDP Relaxation
x* = argmin 12
∑ ui (1 + xi) + 14
∑ Pij (1 + xi + xj + Xij)
∑ xi = 2 - |L|i va
xi {-1,1}
X = x xT
Retain Convex PartLasserre, 2000
Relax Non-ConvexConstraint
SDP Relaxation
x* = argmin 12
∑ ui (1 + xi) + 14
∑ Pij (1 + xi + xj + Xij)
∑ xi = 2 - |L|i va
xi [-1,1]
X = x xT Relax Non-ConvexConstraint
Lasserre, 2000Retain Convex Part
x1
x2
xn
1
...
1 x1 x2... xn
1 xT
x X
=
Rank = 1
Xii = 1
Positive SemidefiniteConvex
Non-Convex
SDP Relaxation
SDP Relaxation
x* = argmin 12
∑ ui (1 + xi) + 14
∑ Pij (1 + xi + xj + Xij)
∑ xi = 2 - |L|i va
xi [-1,1]
X = x xT Relax Non-ConvexConstraint
Lasserre, 2000Retain Convex Part
SDP Relaxation
x* = argmin 12
∑ ui (1 + xi) + 14
∑ Pij (1 + xi + xj + Xij)
∑ xi = 2 - |L|i va
xi [-1,1]
Xii = 1 X - xxT 0
Accurate Inefficient
Lasserre, 2000Retain Convex Part
PositiveSemidefinite
Outline
• Convex Relaxations– Integer Programming Formulation– LP Relaxation– SDP Relaxation– SOCP Relaxation– Comparing Relaxations
• Move Making Algorithms
• Some Interesting Open Problems
SOCP Relaxation
x* = argmin 12
∑ ui (1 + xi) + 14
∑ Pij (1 + xi + xj + Xij)
∑ xi = 2 - |L|i va
xi [-1,1]
Xii = 1 X - xxT 0
Derive SOCP relaxation from the SDP relaxation
Further Relaxation
SOCP Relaxation
Choose a matrix C1 = UUT 0
Kim and Kojima, 2000
Choose a sub-graph G
Variables xG and XG
(XG - xGxGT)C1 ≥ 0
Choose a matrix C2 = UUT 0
REPEAT
Outline
• Convex Relaxations– Integer Programming Formulation– LP Relaxation– SDP Relaxation– SOCP Relaxation– Comparing Relaxations
• Move Making Algorithms
• Some Interesting Open Problems
Dominating Relaxation
For all MAP Estimation problem (u, P)
A dominates B
A B
≥
Dominating relaxations are better
SOCP Relaxation
Choose a matrix C1 = UUT 0
Kim and Kojima, 2000
Choose a sub-graph G
Variables xG and XG
(XG - xGxGT)C1 ≥ 0
If G is a tree, LP dominates SOCP
Examples
Muramatsu and Suzuki, 2003(MAXCUT)
Ravikumar and Lafferty, 2006 (QP Relaxation)
Kumar, Torr and Zisserman, 2006 (Equivalent SOCP Relaxation)
SOCP Relaxation
Choose a matrix C1 = UUT 0
Kim and Kojima, 2000
Choose a sub-graph G
Variables xG and XG
(XG - xGxGT)C1 ≥ 0
If G is a cycle with non-negative P
SOCP Relaxation
Choose a matrix C1 = UUT 0
Kim and Kojima, 2000
Choose a sub-graph G
Variables xG and XG
(XG - xGxGT)C1 ≥ 0
If G is an even cycle with non-positive P
SOCP Relaxation
Choose a matrix C1 = UUT 0
Kim and Kojima, 2000
Choose a sub-graph G
Variables xG and XG
(XG - xGxGT)C1 ≥ 0
If G is an odd cycle with 1 non-positive P
SOCP Relaxation
What about other cycles?
Dominated by linear cycle inequalities
Cliques?
Dominated by clique inequalities
Kumar, Kolmogorov and Torr, 2007
Outline
• Convex Relaxations
• Move Making Algorithms– State of the Art– Comparison with LP Relaxation– Improved Moves
• Some Interesting Open Problems
MRFs in Vision
va vb
li
lkPab(i,k)Pab(i,k) = wab min{ d(i-k), M }
wab is non-negative
Truncated Linear Truncated Quadratic
d(.) is a semi-metric distanceua(i) ub(k)
Move Making
Search Neighbourhood
Current Solution
Optimal Move
Solution Space
En
erg
y
Slide courtesy of Pushmeet Kohli
Outline
• Convex Relaxations
• Move Making Algorithms– State of the Art– Comparison with LP Relaxation– Improved Moves
• Some Interesting Open Problems
Expansion MoveVariables take label or retain current label
Boykov, Veksler, Zabih 2001Slide courtesy of Pushmeet Kohli
Sky
House
Tree
Ground
Initialize with TreeStatus: Expand GroundExpand HouseExpand Sky
[Boykov, Veksler, Zabih]
Expansion MoveVariables take label or retain current label
Boykov, Veksler, Zabih 2001Slide courtesy of Pushmeet Kohli
Outline
• Convex Relaxations
• Move Making Algorithms– State of the Art– Comparison with LP Relaxation– Improved Moves
• Some Interesting Open Problems
Multiplicative Bounds
LPMove-Making
Potts
Truncated Linear
Truncated Quadratic
Metric Labeling
2 2
2 + √2 2M
O(√M) 2M
O(log h) 2M
Expansion Bounds as bad as ICM Bounds
Outline
• Convex Relaxations
• Move Making Algorithms– State of the Art– Comparison with LP Relaxation– Improved Moves
• Some Interesting Open Problems
Randomized Rounding
0 y’0 y’i y’k y’h = 1
y’i = y0 + y1 + … + yi
Choose an interval of length L’
yi = (1 + xi)/2
Randomized Rounding
0 y’0 y’i y’k y’h = 1
Generate a random number r (0,1]
r
y’i = y0 + y1 + … + yi
yi = (1 + xi)/2
Randomized Rounding
0 y’0 y’i y’k y’h = 1
Assign label next to r (if within the interval)
r
y’i = y0 + y1 + … + yi
yi = (1 + xi)/2
Move Making
va vb
• Initialize the labeling
• Choose interval I of L’ labels
• Each variable can
• Retain old label
• Choose a label from I
• Choose best labeling
Iterate over intervals
Non-submodular move? Submodular overestimation
Truncated Convex Models
Pab(i,k) = wab min{ d(i-k), M }
Truncated Linear Truncated Quadratic
d(.) is convex d(x+1) - 2d(x) + d(x-1) ≥ 0
Move Making
va vb
• Choose interval I of L’ labels
• Each variable can
• Retain old label
• Choose a label from I
• Choose best labeling
Large L’ => Non-submodular
LP Bounds
Kumar and Torr, NIPS 08
Type of Problem Bound
Potts 2
Truncated Linear 2 + √2
Truncated Quadratic O(√M)
Metric Labeling O(log h)
Kumar and Koller, UAI 09
Move Making
Problem 1
Relationship between rounding and move-making?
What happens when n < h ??(Should we even use move-making here??)
What about semi-metric MRFs??
top related