Polyhedral OptimizationLecture 3 – Part 2

M. Pawan Kumar

pawan.kumar@ecp.fr

Slides available online http://cvn.ecp.fr/personnel/pawan/

• Operations on Matroids– Truncation– Deletion– Contraction– Duality of Deletion and Contraction

• Maximum Weight Independent Set

• Polytopes

Outline

t-Truncation of Matroid

M = (S, I) M’ = (S, I’)

X ∈I’ if two conditions are satisfied

(i) X ∈I

(ii) |X| ≤ t

M’ is a matroid Proof?

Example – Uniform Matroid

S = {1, 2, 3, 4, 5, 6} k = 3

t = 2

Is {1, 2, 3} independent in M’ ?

NO

t-Truncation denoted by M’

Example – Uniform Matroid

S = {1, 2, 3, 4} k = 3

t = 2

Is {1, 2} independent in M’ ?

YES

t-Truncation denoted by M’

Example – Uniform Matroid

S = {1, 2, 3, 4} k = 3

t = 2

Is {1} independent in M’ ?

YES

t-Truncation denoted by M’

Example – Linear Matroid

1

2

3

4

2

4

6

8

1

1

1

1

2

2

2

2

3

3

3

3

1

2

1

2

2

4

2

4

1

2

1

2

2

4

2

4

t = 3 t-Truncation denoted by M’

Independent in M’ ? NO

Example – Linear Matroid

1

2

3

4

2

4

6

8

1

1

1

1

2

2

2

2

3

3

3

3

1

2

1

2

2

4

2

4

1

2

1

2

2

4

2

4

t = 3 t-Truncation denoted by M’

Independent in M’ ? NO

Example – Linear Matroid

1

2

3

4

2

4

6

8

1

1

1

1

2

2

2

2

3

3

3

3

1

2

1

2

2

4

2

4

1

2

1

2

2

4

2

4

t = 3 t-Truncation denoted by M’

Independent in M’ ? YES

Example – Graphic Matroid

v1

v0

v2

v6

v4

v5

v3

t = 3 t-Truncation denoted by M’

Example – Graphic Matroid

v1

v0

v2

v6

v4

v5

v3

t = 3 t-Truncation denoted by M’

Independent in M’ ? NO

Example – Graphic Matroid

v1

v0

v2

v6

v4

v5

v3

t = 3 t-Truncation denoted by M’

Independent in M’ ? NO

Example – Graphic Matroid

v1

v0

v2

v6

v4

v5

v3

t = 3 t-Truncation denoted by M’

Independent in M’ ? YES

• Operations on Matroids– Truncation– Deletion– Contraction– Duality of Deletion and Contraction

• Maximum Weight Independent Set

• Polytopes

Outline

Deletion

M = (S, I) M\s = (S-s, I’)

X ∈I’ if two conditions are satisfied

(i) X S-s⊆

(ii) X ∈I

M\s is a matroid Proof?

Example – Uniform Matroid

S = {1, 2, 3, 4, 5, 6} k = 3

s = 2

Is {1, 2, 3} independent in M’ ?

NO

Matroid after deletion of s denoted by M’

Example – Uniform Matroid

S = {1, 2, 3, 4, 5, 6} k = 3

s = 2

Is {1, 3, 4, 5} independent in M’ ?

NO

Matroid after deletion of s denoted by M’

Example – Uniform Matroid

S = {1, 2, 3, 4, 5, 6} k = 3

s = 2

Is {1, 3, 4} independent in M’ ?

YES

Matroid after deletion of s denoted by M’

Example – Linear Matroid

1

2

3

4

2

4

6

8

1

1

1

1

2

2

2

2

3

3

3

3

1

2

1

2

2

4

2

4

1

2

1

2

2

4

2

4

s

Matroid after deletion of s denoted by M’

Independent in M’ ? NO

Example – Linear Matroid

1

2

3

4

2

4

6

8

1

1

1

1

2

2

2

2

3

3

3

3

1

2

1

2

2

4

2

4

1

2

1

2

2

4

2

4

s

Matroid after deletion of s denoted by M’

Independent in M’ ? NO

Example – Linear Matroid

1

2

3

4

2

4

6

8

1

1

1

1

2

2

2

2

3

3

3

3

1

2

1

2

2

4

2

4

1

2

1

2

2

4

2

4

s

Matroid after deletion of s denoted by M’

Independent in M’ ? YES

Example – Graphic Matroid

v1

v0

v2

v6

v4

v5

v3

Example – Graphic Matroid

v1

v0

v2

v6

v4

v5

v3

s

Matroid after deletion of s denoted by M’

Example – Graphic Matroid

v1

v0

v2

v6

v4

v5

v3

s

Matroid after deletion of s denoted by M’

Independent in M’ ? NO

Example – Graphic Matroid

v1

v0

v2

v6

v4

v5

v3

s

Matroid after deletion of s denoted by M’

Independent in M’ ? NO

Example – Graphic Matroid

v1

v0

v2

v6

v4

v5

v3

s

Matroid after deletion of s denoted by M’

Independent in M’ ? YES

Effect on Rank Function

M = (S, I) M\s = (S-s, I’)

rM\s(X) = rM(X), for all X S-s⊆

Proof?

• Operations on Matroids– Truncation– Deletion– Contraction– Duality of Deletion and Contraction

• Maximum Weight Independent Set

• Polytopes

Outline

Contraction

M = (S, I) M/s = (S-s, I’)

X ∈I’ if two conditions are satisfied

(i) X S-s⊆

(ii) X + s ∈I

M/s is a matroid Proof?

Assume {s} ∈I

Contraction

M = (S, I) M/s = (S-s, I’)

X ∈I’ if two conditions are satisfied

(i) X S-s⊆

(ii) X ∈I

M/s is the same as M\s when {s} ∉ I

Assume {s} ∉ I

Example – Uniform Matroid

S = {1, 2, 3, 4, 5, 6} k = 3

s = 2

Is {1, 2, 3} independent in M’ ?

NO

Matroid after contraction of s is M’

Example – Uniform Matroid

S = {1, 2, 3, 4, 5, 6} k = 3

s = 2

Is {1, 3, 4} independent in M’ ?

NO

Matroid after contraction of s is M’

Example – Uniform Matroid

S = {1, 2, 3, 4, 5, 6} k = 3

s = 2

Is {1, 4} independent in M’ ?

YES

Matroid after contraction of s is M’

Example – Linear Matroid

1

2

3

4

2

4

6

8

1

1

1

1

2

2

2

2

3

3

3

3

1

2

1

2

2

4

2

4

1

2

1

2

2

4

2

4

s

Matroid after contraction of s denoted by M’

Independent in M’ ? NO

Example – Linear Matroid

1

2

3

4

2

4

6

8

1

1

1

1

2

2

2

2

3

3

3

3

1

2

1

2

2

4

2

4

1

2

1

2

2

4

2

4

s

Matroid after contraction of s denoted by M’

Independent in M’ ? NO

Example – Linear Matroid

1

2

3

4

2

4

6

8

1

1

1

1

2

2

2

2

3

3

3

3

1

2

1

2

2

4

2

4

1

2

1

2

2

4

2

4

s

Matroid after contraction of s denoted by M’

Independent in M’ ? YES

Example – Graphic Matroid

v1

v0

v2

v6

v4

v5

v3

Example – Graphic Matroid

v1

v0

v2

v6

v4

v5

v3

s

Matroid after contraction of s denoted by M’

Example – Graphic Matroid

v1

v0

v2

v6

v4

v5

v3

s

Matroid after contraction of s denoted by M’

Independent in M’ ? NO

Example – Graphic Matroid

v1

v0

v2

v6

v4

v5

v3

s

Matroid after contraction of s denoted by M’

Independent in M’ ? NO

Example – Graphic Matroid

v1

v0

v2

v6

v4

v5

v3

s

Matroid after contraction of s denoted by M’

Independent in M’ ? YES

Effect on Rank Function

M = (S, I) M/s = (S-s, I’)

rM/s(X) = rM(X+s) – rM({s}), for all X S-s⊆

Proof?

• Operations on Matroids– Truncation– Deletion– Contraction– Duality of Deletion and Contraction

• Maximum Weight Independent Set

• Polytopes

Outline

Duality

(M/s)* = M*\s Proof?

If {s} ∉ I, deletion doesn’t affect independence

If {s} ∉ I, contraction doesn’t affect independence

Proof is trivial

Duality

(M/s)* = M*\s Proof? Assume {s} ∈I

X is independent in (M/s)*

⟺ X B⊆ 1 for some base B1 of (M/s)*

⟺ X ∩ B2 = Null for some base B2 of M/s

s X∉

⟺ X ∩ B3 = Null for some base B3 = B2 + s of M

⟺ X B⊆ 4 for some base B4 of M*

Hence proved.

• Operations on Matroids

• Maximum Weight Independent Set

• Polytopes

Outline

Independent Set

Matroid M = (S, I)

X S is independent if X ⊆ I

X S is dependent if X ⊆ ∉ I

Weight of an Independent Set

Matroid M = (S, I)

Weight function w: S → Non-negative Real

Weight of an independent set X

The sum of weight of its elements

Weight of an Independent Set

Matroid M = (S, I)

w(X) = ∑s X∈ w(s)

Weight of an independent set X

Weight function w: S → Non-negative Real

Example (Uniform Matroid)

S = {1,2,…,9} s w(s)

1 10

2 5

3 2

4 1

5 3

6 6

7 12

8 2

9 1

k = 4

X = {1, 3, 5}

Example (Uniform Matroid)

S = {1,2,…,9} s w(s)

1 10

2 5

3 2

4 1

5 3

6 6

7 12

8 2

9 1

k = 4

X = {1, 3, 5}

w(X)?

15

Example (Graphic Matroid)

v1

v0

v2

v6

v4

v5

v3

2 6

2

5

31

3 2

4

S = E

Forest X

Example (Graphic Matroid)

v1

v0

v2

v6

v4

v5

v3

2 6

2

5

31

3 2

4

S = E

Forest X

w(X)?

10

Maximum Weight Independent Set

Matroid M = (S, I)

maxX∈I w(X)

Find an independent set with maximum weight

Weight function w: S → Non-negative Real

Maximum Weight Independent Set

Matroid M = (S, I)

maxX∈I ∑s X∈ w(s)

Find an independent set with maximum weight

Weight function w: S → Non-negative Real

True or False

There exists an optimal solutionthat is a base of the matroid

TRUE

Example (Uniform Matroid)

S = {1,2,…,9} s w(s)

1 10

2 5

3 2

4 1

5 3

6 6

7 12

8 2

9 1

k = 4

Feasible Solutions?

Example (Uniform Matroid)

S = {1,2,…,9} s w(s)

1 10

2 5

3 2

4 1

5 3

6 6

7 12

8 2

9 1

k = 4

Optimal Solution?

Example (Uniform Matroid)

S = {1,2,…,9} s w(s)

1 10

2 5

3 2

4 1

5 3

6 6

7 12

8 2

9 1

k = 4

Example (Graphic Matroid)

v1

v0

v2

v6

v4

v5

v3

2 6

2

5

31

3 2

4

S = E

Feasible Solutions?

Example (Graphic Matroid)

v1

v0

v2

v6

v4

v5

v3

2 6

2

5

31

3 2

4

S = E

Optimal Solution?

Example (Graphic Matroid)

v1

v0

v2

v6

v4

v5

v3

2 6

2

5

31

3 2

4

S = E

Efficient Algorithm?

• Operations on Matroids

• Maximum Weight Independent Set– Greedy Algorithm– Efficiency– Optimality: Necessity– Optimality: Sufficiency– Extensions

• Polytopes

Outline

Greedy Algorithm

Start with an empty set

Repeat

Pick a new element with maximum weight

such that the new set is independent

Until no more elements can be added

Add the element to the set

Greedy Algorithm

X ← ϕ

Repeat

Pick a new element with maximum weight

such that the new set is independent

Until no more elements can be added

Add the element to the set

Greedy Algorithm

X ← ϕ

Repeat

s* = argmaxx S\X∈ w(s)

such that the new set is independent

Until no more elements can be added

Add the element to the set

Greedy Algorithm

X ← ϕ

Repeat

s* = argmaxx S\X∈ w(s)

such that X {s} ∪ ∈ I

Until no more elements can be added

Add the element to the set

Greedy Algorithm

X ← ϕ

Repeat

s* = argmaxx S\X∈ w(s)

such that X {s} ∪ ∈ I

Until no more elements can be added

X ← X {s}∪

Example (Uniform Matroid)

S = {1,2,…,9} s w(s)

1 10

2 5

3 2

4 1

5 3

6 6

7 12

8 2

9 1

k = 4

Example (Uniform Matroid)

S = {1,2,…,9} s w(s)

1 10

2 5

3 2

4 1

5 3

6 6

7 12

8 2

9 1

k = 4

Example (Uniform Matroid)

S = {1,2,…,9} s w(s)

1 10

2 5

3 2

4 1

5 3

6 6

7 12

8 2

9 1

k = 4

Example (Uniform Matroid)

S = {1,2,…,9} s w(s)

1 10

2 5

3 2

4 1

5 3

6 6

7 12

8 2

9 1

k = 4

Example (Uniform Matroid)

S = {1,2,…,9} s w(s)

1 10

2 5

3 2

4 1

5 3

6 6

7 12

8 2

9 1

k = 4

Example (Graphic Matroid)

v1

v0

v2

v6

v4

v5

v3

2 6

2

5

31

3 2

4

S = E

Example (Graphic Matroid)

v1

v0

v2

v6

v4

v5

v3

2 6

2

5

31

3 2

4

S = E

Example (Graphic Matroid)

v1

v0

v2

v6

v4

v5

v3

2 6

2

5

31

3 2

4

S = E

Example (Graphic Matroid)

v1

v0

v2

v6

v4

v5

v3

2 6

2

5

31

3 2

4

S = E

Example (Graphic Matroid)

v1

v0

v2

v6

v4

v5

v3

2 6

2

5

31

3 2

4

S = E

Example (Graphic Matroid)

v1

v0

v2

v6

v4

v5

v3

2 6

2

5

31

3 2

4

S = E

Example (Graphic Matroid)

v1

v0

v2

v6

v4

v5

v3

2 6

2

5

31

3 2

4

S = E

• Operations on Matroids

• Maximum Weight Independent Set– Greedy Algorithm– Efficiency– Optimality: Necessity– Optimality: Sufficiency– Extensions

• Polytopes

Outline

Efficiency of Greedy Algorithm

X ← ϕ

Repeat

s* = argmaxx S\X∈ w(s)

such that X {s} ∪ ∈ I

Until no more elements can be added

X ← X {s}∪

O(|S|)

O(|S|)

Independence Testing Oracle

Efficiency depends on independence testing

Uniform matroid?

Graphic matroid?

Brute-force is not an option

• Operations on Matroids

• Maximum Weight Independent Set– Greedy Algorithm– Efficiency– Optimality: Necessity– Optimality: Sufficiency– Extensions

• Polytopes

Outline

Optimality: Necessity

(S, I) is a matroid

⟹(S, I) admits an optimal greedy algorithm

Proof?

Proof Sketch

Matroid M = (S, I)

Proof by induction

Current solution X

X is part of optimal solution B (base of M)

Proof Sketch

Next step chooses to add s If s B, trivial∈

Otherwise, consider base B’ such that

B’ = (B {s})\{t}∪

w(B’) ≥ w(B) by construction

X {s} B’ ∪ ⊆ ⊆ B {s}∪

• Operations on Matroids

• Maximum Weight Independent Set– Greedy Algorithm– Efficiency– Optimality: Necessity– Optimality: Sufficiency– Extensions

• Polytopes

Outline

Optimality: Sufficiency

(S, I) is a matroid

⟹(S, I) admits an optimal greedy algorithm

Proof?

Proof Sketch

X ∈ I

Proof by contradiction

There should exist an s Y\X, X {s} ∈ ∪ ∈ I

Matroid M = (S, I)

Y ∈ I k = |X| < |Y|

Proof Sketch

w(s) =

k+2, if s X ∈

k+1, if s Y\X ∈

0, otherwise

Greedy solution has weight k(k+2)

w(Y) ≥ (k+1)(k+1) > k(k+2)

Recall k = |X|

• Operations on Matroids

• Maximum Weight Independent Set– Greedy Algorithm– Efficiency– Optimality: Necessity– Optimality: Sufficiency– Extensions

• Polytopes

Outline

Maximum Weight Base

Matroid M = (S, I)

maxX∈B w(X)

Find a base with maximum weight

Does greedy work? YES

Weight function w: S → Real

Minimum Weight Base

Matroid M = (S, I)

minX∈B w(X)

Find a base with minimum weight

Does greedy work? YES

Weight function w: S → Real

Maximum Weight Independent Set

Matroid M = (S, I)

maxX∈I w(X)

Find an independent set with maximum weight

Does greedy work? YES

Weight function w: S → Real

• Operations on Matroids

• Maximum Weight Independent Set

• Polytopes

Outline

Incidence Vector of Set

Matroid M = (S, I) Set X S⊆

Incidence vector vX {0,1}∈ |S|

vX(s) =

1, if s X ∈

0, if s X ∉

Example (Uniform Matroid)

S = {1,2,…,9} s w(s)

1 10

2 5

3 2

4 1

5 3

6 6

7 12

8 2

9 1

k = 4

X = {1, 3, 5}

1

0

1

0

1

0

0

0

0

vX?

Example (Graphic Matroid)

v1

v0

v2

v6

v4

v5

v3

e1 e2

e4

e6

e5e3

e7 e9

e8

S = E

X S⊆

Example (Graphic Matroid)

v1

v0

v2

v6

v4

v5

v3

S = E

X S ⊆

vX?

e1 e2

e4

e6

e5e3

e7 e9

e8

1

0

0

1

0

0

0

1

1

Incidence Vectors of Independent Sets

Convex HullAx ≤ b

Independent Set Polytope

A?

b?

Matroid M = (S, I)

First, a property !!

vX {0,1}∈ |S|, X ∈ I

Maximum Weight Independent Set

w* = maxX∈I w(X) Matroid M = (S, I)

S = {s1,s2,…,sm}, such that w(si) ≥ w(si+1) ≥ 0

Ui = {s1,…si}

X = {si | rM(Ui) > rM(Ui-1)}

Optimal solution Why?

Maximum Weight Independent Set

w* = maxX∈I w(X) Matroid M = (S, I)

w(X)

X = {si | rM(Ui) > rM(Ui-1)}

= ∑s X∈ w(s)

= ∑i w(si)(rM(Ui)-rM(Ui-1))

= ∑i λi rM(Ui)

w = ∑i λi vUiRelationship between w and λ?

Maximum Weight Independent Set

w* = maxX∈I w(X) Matroid M = (S, I)

w(X) = w* = ∑i λi rM(Ui)

such that w = ∑i λi vUi

We need to remember this for what follows

• Operations on Matroids

• Maximum Weight Independent Set

• Polytopes– Independent Set Polytope– Base Polytope– Spanning Set Polytope

Outline

Independent Set Polytope

Convex HullAx ≤ b

Matroid M = (S, I)

vX {0,1}∈ |S|, X ∈ I

x Real∈ |S|

Independent Set Polytope

vX {0,1}∈ |S|, X ∈ I

Matroid M = (S, I)

x Real∈ |S|

xs ≥ 0, for all s S ∈

∑s U∈ xs ≤ rM(U), for all U S ⊆

Necessary conditions

Why?

Why?

Sufficient for integral x Proof?

Independent Set Polytope

vX {0,1}∈ |S|, X ∈ I

Matroid M = (S, I)

x Real∈ |S|

xs ≥ 0, for all s S ∈

∑s U∈ xs ≤ rM(U), for all U S ⊆

Necessary conditions

Why?

Why?

Sufficient for all x Proof?

Independent Set Polytope

maxx wTx

xs ≥ 0, for all s S ∈

∑s U∈ xs ≤ rM(U), for all U S ⊆

Dual?

miny ∑U S⊆ yUrM(U)

yU ≥ 0, for all U S ⊆

∑U S⊆ yUvU ≥ w

Independent Set Polytope

maxx wTx

xs ≥ 0, for all s S ∈

∑s U∈ xs ≤ rM(U), for all U S ⊆

miny ∑U S⊆ yUrM(U)

yU ≥ 0, for all U S ⊆

∑U S⊆ yUvU = w

Greedy algorithm integral solution

x’

Independent Set Polytope

maxx wTx

xs ≥ 0, for all s S ∈

∑s U∈ xs ≤ rM(U), for all U S ⊆

miny ∑U S⊆ yUrM(U)

yU ≥ 0, for all U S ⊆

∑U S⊆ yUvU = w

Solution obtained using the property

x’

y’

Independent Set Polytope

maxx wTx

xs ≥ 0, for all s S ∈

∑s U∈ xs ≤ rM(U), for all U S ⊆

miny ∑U S⊆ yUrM(U)

yU ≥ 0, for all U S ⊆

∑U S⊆ yUvU = w

Relationship between objectives for x’ and y’ ?

x’

y’

Independent Set Polytope

∑s w(s)x’s = ∑U S⊆ y’UrM(U)

Dual upper bounds primal

x’ and y’ must be optimal solutions

For all integer w, we have integer optimal value

Hence provedNegative values in w?

• Operations on Matroids

• Maximum Weight Independent Set

• Polytopes– Independent Set Polytope– Base Polytope– Spanning Set Polytope

Outline

Base Polytope

Convex HullAx ≤ b

Matroid M = (S, I)

vX {0,1}∈ |S|, X ∈ B

x Real∈ |S|

Base Polytope

vX {0,1}∈ |S|, X ∈ B

Matroid M = (S, I)

x Real∈ |S|

xs ≥ 0, for all s S ∈

∑s U∈ xs ≤ rM(U), for all U S ⊆

∑s S∈ xs = rM(S)

• Operations on Matroids

• Maximum Weight Independent Set

• Polytopes– Independent Set Polytope– Base Polytope– Spanning Set Polytope

Outline

Spanning Set Polytope

Convex HullAx ≤ b

Matroid M = (S, I)

vX {0,1}∈ |S|, X ∈ S

x Real∈ |S|

Independent set of M*

S\X?

Spanning Set Polytope

vX {0,1}∈ |S|, X ∈ B

Matroid M = (S, I)

x Real∈ |S|

1-xs ≥ 0, for all s S ∈

∑s U∈ (1-xs) ≤ rM*(U), for all U S ⊆

xs ≥ 0, for all s S ∈

Spanning Set Polytope

vX {0,1}∈ |S|, X ∈ B

Matroid M = (S, I)

x Real∈ |S|

xs ≤ 1, for all s S ∈

∑s U∈ (1-xs) ≤ rM*(U), for all U S ⊆

xs ≥ 0, for all s S ∈

Spanning Set Polytope

vX {0,1}∈ |S|, X ∈ B

Matroid M = (S, I)

x Real∈ |S|

xs ≤ 1, for all s S ∈

∑s U∈ xs ≥ rM(S) - rM(S\U), for all U S ⊆

xs ≥ 0, for all s S ∈

Questions?