facial reduction for symmetry reduced semidefinite...

Facial reduction for symmetry reduced semidefiniteprograms

Hao Hua , Renata Sotirova and Henry Wolkowiczb

Updated: 2019/08/07

a Tilburg Universityb University of Waterloo

The graph partitioning problem

The graph partitioning problem (GP)

• GP: partition the vertices of a graph into k subsets of given sizes

so that the number of edges between different subsets is minimized

• Partition the graph below into 2 sets of equal size

1 2

34

1 2

34

1 2

34

Input graph objective value 4 objective value 2

2

Symmetry in the graph partitioning problem

• The input graph is ”invariant” under certain permutation of its

vertices

1 2

34

2 3

41

2 1

43

Input graph rotate clock-wise flip horizontally

• How can we exploit the symmetry to attack the problem?

3

Matrix ∗-algebra

Matrix ∗-algebra

• A set M⊆ Cn×n is a matrix ∗-algebra over C if it is closed

under addition, scalar and matrix multiplication, and taking

conjugate transpose, i.e.,

αX + βY ∈M ∀α, β ∈ CX ∗ ∈MXY ∈M,

for all X ,Y ∈M

5

Block diagonalization of Matrix ∗-algebra

• Theorem (Wedderburn 1907) Matrix ∗-algebras containing the

identity matrix have a canonical block-diagonal structure after

some unitary transformation, i.e., there exists a unitary matrix Q

and some integer t such that

Q∗MQ =

M1 0 · · · 0

0 M2...

.... . . 0

0 · · · 0 Mt

,

where each Mi ⊆ Cni×ni is basic

6

An example of block-diagonalization

• Consider the matrix ∗-algebra spanned by

B0 =

1 0 0 0

0 1 0 0

0 0 1 0

0 0 0 1

,B1 =

0 1 0 1

1 0 1 0

0 1 0 1

1 0 1 0

,B2 =

0 0 1 0

0 0 0 1

1 0 0 0

0 1 0 0

,where B1 is the adjacency matrix in the graph partitioning example

• The unitary matrix Q below diagonalizes B0,B1,B2

Q =1

2

1 1 1 1

1 −i −1 i

1 −1 1 −1

1 i −1 −i

7

An example of block-diagonalization

• The matrix ∗-algebra spanned by

B0 =

1 0 0 0

0 1 0 0

0 0 1 0

0 0 0 1

,B1 =

0 1 0 1

1 0 1 0

0 1 0 1

1 0 1 0

,B2 =

0 0 1 0

0 0 0 1

1 0 0 0

0 1 0 0

• The (block)-diagonalized matrices Bi = QTBiQ are

B0 =

1 0 0 0

0 1 0 0

0 0 1 0

0 0 0 1

, B1 =

8 0 0 0

0 0 0 0

0 0 −8 0

0 0 0 0

, B2 =

4 0 0 0

0 −4 0 0

0 0 4 0

0 0 0 −4

8

Symmetry reduction

Semidefinite program (SDP)

• Consider an SDP in standard form

infX{〈A0,X 〉 | 〈Ai ,X 〉 = bi for i = 1, . . . ,m,X ∈ Sn+}, (1)

where Sn+ is the cone of positive semidefinite matrices

Assume the data matrices A0, . . . ,Am and the identity matrix

are contained in a matrix ∗-algebra M.

If SDP (1) has an optimal solution, then it has an optimal

solution in M.

• References:

a) Kanno et al., 2001, de Klerk 2009, etc

b) Gatermann, Parrilo 2004, Vallentin 2009, etc

c) Schrijver 2005, Laurent 2007, etc10

Symmetry reduction for SDP

• Assume B1, . . . ,Bd is a basis of M. There exists an optimal

solution

X =d∑

k=1

xkBk ∈M

• The p.s.d. constraint X ∈ Sn+ can be simplifies as

∑dk=1 xk

block-diagonal︷︸︸︷(QTBkQ) ∈ Sn+ ⇐⇒

B∗1(x) 0 · · · 0

0 B∗2(x)...

.... . . 0

0 · · · 0 B∗t (x)

∈ Sn+

where B∗j (x) is the j-th block

• We have X ∈ Sn+ if and only if B∗j (x) ∈ Snj+ for every j = 1 . . . , t11

An example of symmetry reduction

• An SDP relaxation for the cut minimization problem (Pong et al.

’14)

minX 〈C ,X 〉s.t. A(X ) = b,X ≥ 0

X ∈ Snk+ ,

where n is the number of vertices and k is the number of subsets

in the partition

• Instance can161 with n = 161 vertices and k = 3 partitions

• The size of X ∈ Snk+ is nk = 483, and very difficult to solve /

12


• The feasible solutions X under certain unitary transformation,

i.e., QTXQ, has the following block-diagonal structure

0 100 200 300 400

nz = 27189

0

50

100

150

200

250

300

350

400

450

• The sizes of these 9 blocks are 60, 60, 60, 60, 60, 60, 60, 33, 30

13


• An SDP relaxation for cut minimization problems (Pong et al.

’14) minX 〈C ,X 〉s.t. A(X ) = b,X ≥ 0

X ∈ S483+///////////B∗1 (x) ∈ S60+

...

B∗9 (x) ∈ S30+

• After symmetry reduction,

the sizes of p.s.d. constraints

Original SDP 483

Symmetry reduced 60, 60, 60, 60, 60, 60, 60, 33, 30

Instance can16114

Facial reduction

Facial reduction

• Slater’s condition (strict feasibility) is a constraint qualification

in convex optimization problems

• Without strict feasibility:

- the KKT conditions may not be necessary for the optimality

- strong duality may not hold

- small perturbations may render the problem infeasible

- many solvers might run into numerical errors

• Facial reduction is a regularization technique that can be used

for semidefinite programs that fail strict feasibility (Borwein,

Wolkowicz, ’81)

16

Facial reduction

• Given the SDP in standard form

infX{〈C ,X 〉 | A(X ) = b,X ∈ Sn+} (2)

Then exactly one of the following alternatives holds

1. The SDP (2) is strictly feasible:

A(X ) = b, X ∈ Sn++

2. The auxiliary system is consistent:

0 6= A∗(y) ∈ Sn+ and 〈b, y〉 = 0

• We call A∗(y) an exposing vector

• The feasible region of (2) is contained in A∗(y)⊥ ∩ Sn+17

Facial reduction for the cut minimization problem


’14)

minX 〈C ,X 〉s.t. A(X ) = b,X ≥ 0

X ∈ Snk+where n is the number of vertices and k is the number of subsets

in the partition

18

Facial reduction for the cut minimization problem


’14)minX 〈C ,X 〉s.t. A(X ) = b,X ≥ 0

X ∈ Snk+////////// =⇒ X = VRV T ,R ∈ S(n−1)(k−1)+

where the columns of V span A∗(y)⊥


Original SDP 483

Facially reduced 321


Instance can161

19

Facial reduction for symmetry re-duced SDP

Facial reduction for symmetry reduced SDP

Theorem (H., Sotirov, Wolkowicz) Let W be an exposing

vector of the minimal face of a given SDP instance. Then

1. There exists an exposing vector WG ∈M of the

minimal face of the input SDP instance

2. QTWGQ is an exposing vector of the minimal face of

the symmetry reduced SDP

• In plain words, we know how to do facial reduction for the

symmetry reduced SDP now

21

Facial reduction for the symmetry reduced program


’14)

minX 〈C ,X 〉s.t. A(X ) = b,X ≥ 0

X ∈ Snk+where n is the number of vertices and k is the number of subsets

in the partition

22


• A symmetry reduced SDP relaxation for cut minimization

problems

minX 〈C ,X 〉s.t. A(X ) = b,X ≥ 0

X ∈ S483+///////////B∗1 (x) ∈ S60+

...

B∗9 (x) ∈ S30+

23


• The facially + symmetry reduced SDP relaxation for cut

minimization problems

minX 〈C ,X 〉s.t. A(X ) = b,X ≥ 0

X ∈ S483+///////////B∗1 (x) = V1R1V

T1 and R1 ∈ S40+ ,

...

B∗9 (x) = V9R9VT9 and R9 ∈ S20+ ,

24

Facial reduction for symmetry reduced SDP

• In the cut minimization problem, we obtain


Original SDP 483

Facially reduced 321


Facially + Symmetry 40, 40, 40, 40, 38, 40, 40, 21, 20

Instance can161

• Now lets check if our theory works?

25

Numerical results on the cut minimization problem

• We solve the SDP relaxation from Pong et al. ’14 using interior

point method, and the number of partition k = 3

•

Instance Symmetry Facial+Symmetry

can144

bound 0.3838 0.6233

iteration 35 18

time 32.27s 5.8s

can161

bound 0.4828 0.5485

iteration 24 20

time 375.63s 108.05s

• Without facial reduction, it takes longer time and iteration to get

a weaker bound.26

Summary

Input 1 Input 2 Output

SDP matrix ∗-algebra

symmetry reduced SDP

+ reduced problem size

– numerical issues

SDP exposing vector

facially reduced SDP

+ numerically stable

– symmetry not exploited

symmetryreduced

SDP

exposing vectorin the algebra

facially & symmetry reduced SDP

+ numerically stable

+ reduced problem size

27

facial reduction for symmetry reduced semidefinite...

Documents