linear programmingocw.snu.ac.kr/sites/default/files/note/10130.pdf · 2018-01-30 · • convex...

Linear Programming(4541.554 Introduction to Computer-Aided Design)

School of EECSSeoul National University

Optimization Problems

Optimization Problems• General nonlinear programming problem

– minimize f(x)subject to gi(x) ≥ 0 i=1,...,m

hj(x)=0 j=1,...,px ∈ Rn

• Convex programming problem– f : convex

gi: concavehj: linear

– local optimum = global optimum• Linear programming problem

– f,gi, hj: linear (can be considered as convex or concave)– select a solution from a finite set of possible solutions– Simplex algorithm (1947 by G. B. Dantzig)

• Integer linear programming problem– integer-valued coordinates

continuousvariables,continuousoptimization

discrete variablescombinatorialoptimization

boundary

Definitions

Definitions• Instance of an optimization problem

– given (F, c)where F: domain of feasible points

c: F -> R1 : cost functionfind f ∈ F for which c(f) ≤ c(y) for all y ∈ F-> f is a globally optimal solution

– e.g. instance of Traveling Salesman Problem (vertices and edges are given)

• Optimization problem – a set of instances of an optimization problem– e.g. Traveling Salesman Problem

• Locally optimal solution– c(f) ≤ c(g) for all g ∈ N(f)

where N is a neighborhood defined for each instance– e.g. Nε (f) = {x: x ∈ F and ||x-f|| ≤ ε}

Ff

Definitions

• Convex combination of x, y ∈ Rn is any point of the form:z = λx + (1-λ)y, λ ∈ R1 and 0 ≤ λ ≤ 1

• A set S ⊆ Rn is convex if it contains all convex combinations of pairs of points x, y ∈ S

• Lemma 1The intersection of any number of convex sets is convex.

x

y

a

b

z1-λ

λ

a

b convex set non-convex set

Definitions

• Convex Function– Let S ⊆ Rn be a convex set. Function c: S -> R1 is convex

in S if c(λx + (1-λ)y) ≤ λc(x) + (1-λ)c(y), λ∈R1 and 0 ≤ λ ≤ 1 for all x, y ∈ S

• Lemma 2Let c(x) be a convex function on a convex set S. Thenset St = {x : c(x) ≤ t, x ∈ S} is convex.

ProofFor any x, y ∈ St, λx + (1-λ)y is in S andc(λx + (1-λ)y) ≤ λc(x) + (1-λ)c(y) ≤ λt + (1-λ)t ≤ t => λx + (1-λ)y is in St => St is convex

x y a

c(a)

z

a

c(a)

t

x y

Convex Programming Problem

Convex Programming Problem• Theorem 1

For an instance of an optimization problem (F, c) and neighborhood Nε (x) = {y : y ∈ F and ||x-y|| ≤ ε}, where F ⊆ Rn is a convex set and c is a convex function, a locally optimal point with respect to Nε is also a globally optimal point for any ε > 0.

ProofChoose a λ such thaty = λx + (1-λ)z lies within Nε (x)c(y) = c(λx + (1-λ)z) ≤ λc(x) + (1-λ)c(z)=> c(z) ≥ (c(y) - λ c(x))/(1- λ)

≥ (c(x) - λ c(x))/(1- λ) = c(x)

xyz

any pointin F

x is a local optimum point

Convex Programming Problem

• Convex programming problemminimize f(x)subject to gi(x) ≥ 0 i=1,...,m

hj(x)=0 j=1,...,px ∈ Rn

wheref: convexgi: concavehj: linear

F = {x : gi(x) ≥ 0}= {x : -gi(x) ≤ 0}

=> F is convex by Lemma 1 and Lemma 2=> For (F, f), local optimum = global optimum by theorem 1

convex function

Linear Programming Problem

Linear Programming Problem• General Form of LP

• Canonical FormNjx

NjxMib

Mib

j

j

i

i

∈

∈≥∈≥

∈=

nedunconstrai

0

subject tominimize

x'a

x'a

xc'

i

i

jxib

j

i

∀≥∀≥

0

subject tominimize

x'a

xc'

i

• Standard Form

jxib

j

i

∀≥∀=

0

subject tominimize

x'a

xc'

i

Linear Programming Problem

• Conversion

ablesslack vari ofvector :

variablessurplus ofvector :

0

0nedunconstrai

s0s

bsAxbAx

s0s

bsAxbAx

bAxbAx

bAx

⎩⎨⎧

≥=+

⇒≤

⎩⎨⎧

≥=−

⇒≥

⎪⎪⎩

⎪⎪⎨

⎧

≥

≥

−=

⇒

⎩⎨⎧

−≥−≥

⇒=

−

+

−+

j

j

jjj

j

x

x

xxx

x

Examples

Examples• Diet Problem

day)per required is nutrient ofmgr least (at nutrient each for t requiremen :

...) kimchi, of1gr in nutrient ofmgr ( foodeach ofunit oneper nutrient each ofamount :

...) kimchi, ofgr ( foodeach ofamount :

...) kimchi, ofwon/1gr ( foodeach ofcost unit : where

subject tominimize

ib

ia

x

c

i

ij

j

j

b

A

x

c

0xbAx

xc'

≥≥

Examples

• Hierarchical Compaction– Exploit design hierarchy to reduce computation time– Compact bottom-up– Fixed-cell

• Cell abstraction with protection frame and terminal frame• Interconnections among Sub-cells require routing

– Stretching and pitch matching• Connection by abutment

– Limitations of the previous hierarchical compactor• Protection frame or fixed terminal location

--> Area is wasted.• Stretching and pitch-matching

--> Sub-cells can be distorted.--> New master cells are generated.--> Original layout hierarchy is lost.

– David Marple, "A hierarchy preserving hierarchical compactor," Proc. 27th Design Automation Conference, 1990.

A1

A3

A2A

mastercell

Examples

– Constraints• Flat compaction

– AX ≥ B, A: incidence matrix– Each constraint is related with two object locations. Each row

of A has one '-1' and one '1'.

1 2 3b1 b2

b3⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡≥

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−−

−

b3

b2

b1

x3

x2

x1

101

110

011

x1 + 3 ≤ x2

x1 x23

Examples

• Hierarchical compaction– Each constraint may be related with more than two object

locations.

x1 + x2 + 5 ≤ x3 + x4

x2

x3

x4

5

sub-cell A sub-cell B

x1

x1 + x2 + 4 ≤ x3 - x2or

x1 + 2x2 + 4 ≤ x3

x2

x3

x2

4

sub-cell C sub-cell C (reflect)

x1

Examples

– Compaction algorithm• Linear program

– minimize xt=CTX=[000...1]Xsubject to AX ≥ b

X ≥ 0where xt is the location of the sink vertex

– Example:x1 + 2 ≤ x2x2 - 1 ≤ x3x1 + 5 ≤ x3x1 + 2x2 + 4 ≤ x3

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

≥⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

−−−

−−

4

5

1-

2

x3

x2

x1

121101110011

Basic Feasible Solution

Basic Feasible Solution• Definitions

– Given a standard form

– Basis of A• linearly independent collection of columns of A

• can be represented by an m x m matrix

. isrank theand )(matrix an is Assume

subject tominimize

mnmnm <×≥

=

A0x

bAx

xc'

{ }mjj AA ,...,

1=Β

[ ]mjj AAB ,...,

1=


– Basic solution

– Basic feasible solution (BFS)• basic solution in F (domain of feasible points), i.e. x ≥ 0

[ ]

[ ]⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=+

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

→

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎪⎩

⎪⎨⎧

==

∉=

mj

j

jj

mn

n

j

j

jj

b

b

x

x

b

b

x

x

x

mkkx

x

m

m

k

k

MM

MM

1

11

1

1-

1

1

variablebasic:

,...,1,B ofcomponent th

for 0

0...AA

...AA

b

A B


• Lemma 1– Let x be a BFS of

Ax=bx ≥ 0

corresponding to the basis B. Then there exists a cost vector c such that x is the unique optimal solution of the LP

min c'xAx=bx ≥ 0

• Proof– Let cj=0 if Aj ∈ B

1 otherwiseThen c'x=0 which is optimum (c is non-negative).If there is another feasible solution y such that c'y=0, then yj=0 for Aj ∉ B. Other yis are computed by B-1b.--> y=x --> unique


• Theorem 1– If F is not empty, then at least one BFS exists.

• Proof– WLOG, assume one solution is [x1,...,xn], where x1,...,xt>0

and xt+1,...xn=0. Then A1x1+...+Atxt=b (1)

Let r=rank of [A1,...,At] ≤ m (A is an m x n matrix).WLOG, assume first r columns are linearly independent.Rewrite equations (1) as follows

A1x1+...+Arxr=b-Ar+1xr+1-...-Atxt (2)Solving (2) gives

[x1,...,xr]'=β-αr+1xr+1-...-αtxtAs xt-->0, some of x1,...,xr increase or decrease. If any of x1,...,xr becomes 0, then stop decreasing xt.--> feasible solution with more zero componentContinue to obtain a feasible solution with t ≤ m nonzero components. The corresponding columns are independent. Otherwise, we can reduce t further until t=r.

Geometry of Linear Program

Geometry of Linear Program• Definitions

– Examplea1x1+a2x2+a3x3=b --> dimension: 3 -> 2x ≥ 0

x1

x3

b/a1

b/a2

b/a3

x2


– Linear subspace S of Rd

S = {x ∈ Rd : aj1x1+...+ajdxd= 0, j=1,...m}--> dimension=d-rank([aij])=d-m

– Affine subspace A of Rd

A = {x ∈ Rd : aj1x1+...+ajdxd= bj, j=1,...m}= {u+x : x ∈ S}

– HyperplaneAn affine subspace of Rd of dimension d-1{x ∈ Rd : a1x1+...+adxd= b}Defines two halfspaces{x ∈ Rd : a1x1+...+adxd ≥ b}{x ∈ Rd : a1x1+...+adxd ≤ b}

u


– Convex polytope• Bounded nonempty intersection of a finite number of

halfspaces• Every point in a convex polytope is the convex

combination of its vertices (convex hull)

x2

x3

x1

(2,2,0)

(2,0,0)

(2,0,2)

(0,1,3)(0,0,3) (1,0,3)

(0,0,0)

(0,2,0)

x1 + x2 + x3 ≤ 4x1 ≤ 2

x3 ≤ 33x2 + x3 ≤ 6

x1 ≥ 0x2 ≥ 0

x3 ≥ 0


• Polytope and LP[ ][ ]

[ ][ ][ ][ ] [ ]

P polytope),...,(ˆ

)(,...,1,0

,...,1,0ˆˆ

,...,1,ˆˆ

,...,1,ˆˆ

ˆ|ˆ

ˆˆ

|)(|)(|)(

1

1

1

1

∈=→

⎪⎩

⎪⎨

⎧

−=≥

=≥−→≥

=−=

==+

=→

=→

=−×→=

=−×→=×−×→=

−

−

=

−

=+−

−

=+−

−−−

∑

∑

∑

mn

j

mn

jjiji

mn

jjijiimn

mn

jiimnjij

iij

xxx

mnjx

mixab

mixabx

mibxxa

ba

mnmmnm

mmmnm

0x

xI

bxI|A

bBxIbBAxBbxB

bxbAx

111


[ ] [ ]

[ ][ ][ ] [ ]

∑−

=+−

+

−−−

=−=

=→

=−×→=

⎪⎩

⎪⎨⎧

≥

≥↔

⎩⎨⎧

≥=

mn

jjijiimn

in-m

iij

iij

mixabx

x

ba

mnm

ba

1

1

,...,1,ˆˆ

by obtained becan ,ˆfor solvingAfter

ˆ|ˆ

|)(

ˆ

ˆˆˆ

x

xI

bBxIbBAxB

0x

x0x

bAx

11


– Example 1

– Example 2

x1 + x2 + x3 ≤ 4x1 ≤ 2

x3 ≤ 33x2 + x3 ≤ 6

x1, ..., x3 ≥ 0

x1 + x2 + x3 + x4 = 4x1 + x5 = 2

x3 + x6 = 33x2 + x3 + x7 = 6

x1, ..., x7 ≥ 0

x4 = 4 - (x1 + x2 + x3) ≥ 0

x1 + x2 + x3 = b x1 + x2 ≤ b

x1

x3

x2

b b

b

x1

x2

bb


• Theorem 2

• Proof

P polytopeconvex theof vertex a is ˆ ingcorrespond The

by defined F of bfs a is

*x0x

bAx*x

⇔≥

=

(Lemma1)

satisfying vector unique theis such that r cost vecto a exists There

0xbAx

*xc'xc'*xx

c

≥=≤

=

⇒


*xd'xd'x*c'xc'xd'xc'

xd'

xc'

xc'

ˆˆ*ˆ*

ˆ

ˆ)ˆ(

)ˆˆ(

)(

111

1 11

11

111

≤→≤+=

+=

+−=

−+=

+=

−−=

+==

∑∑∑

∑ ∑∑

∑∑

∑∑∑

=+−

=+−

−

=

=

−

=+−

−

=

=+−+−

−

=

+−=

−

==

kk

bcxacc

xabcxc

xcxc

mnji

xcxcxc

m

iiimn

m

ijijimn

mn

jj

m

ij

mn

jijiimn

mn

jjj

m

iimnimn

mn

jjj

n

mnjjj

mn

jjj

n

jjj


)ˆ defining hyperplane supporting a isˆˆ(vertexpoint unique a ison intersecti

onintersectiPˆ

halfspace)(ˆˆsatisfying Rin point theis )(ˆThen

Pˆ

satisfying vector theis such that r cost vecto a exists There

m-n1

*x*xd'xd'

x*xd'xd'

unique*x

x0x

bAx*xc'xc'

unique*xxc

=→

→⎭⎬⎫

∈≤

=

∈→⎭⎬⎫

≥=

≤=

n-m,...,xx

x1

x3

x2


( )

P.in points ofn combinatioconvex strict a becannot vertex a However,

ˆ 21ˆ

21*ˆvertex

21

21* and F ,then

00

0θ

00

0θ-

and points twoDefine

0θsuch that θ smallly sufficientfor θ

0 somefor Then

t.independenlinearly not are s' Assume

bfs. a is then t,independenlinearly are s' If

0 s.t. ,*

*Pˆ vertexa isˆ

vertexa isˆbfs a is

*

**

*

**

**

**

xxxxxxxx

xx

bA

0A

A

*xA

bA0x

bAx*x*x

*x*x

′′+′=→′′+′=∈′′′

⎪⎩

⎪⎨⎧

=

>+=′′

⎪⎩

⎪⎨⎧

=

>=′

′′′

≥±=±→

≠=

>∀=→⎩⎨⎧

≥=

→∈→

⇐

∑

∑

∑

j

jjjj

j

jjjj

jjj

jjj

jj

jj

j

j

jj

jj

x

xdxx

x

xdxx

dxdx

dd

xjx

x1

x3

x2


• Theorem 3In any instance of LP with bounded F, there is an optimal vertex of P (optimal BFS).

• Proof

solution. optimalan is

Then

cost.lowest th vertex wi thebe Let

0,1 where

P of vertices,

solution. optimalan is Assume

110

1

10

j

N

ijij

N

iii

j

i

N

ii

i

N

iii

αα

αα

α

x

xc'xc'xc'xc'

x

xxx

x0

∴

=≥=

≥=

==

∑∑

∑

∑

==

=

=

x1

x3x2 x4

1-λ1

λ2

λ1

1-λ2

x0

x0 = λ1 x1+(1-λ1)x4= λ1 x1+(1-λ1)(λ2 x2+(1-λ2)x3)= λ1 x1+(1-λ1)λ2 x2+(1-λ1) (1-λ2)x3= α1 x1 + α2 x2 + α3 x3

α1 + α2 + α3= λ1 +(1-λ1)λ2+(1-λ1) (1-λ2)= 1

Moving from BFS to BFS


⎪⎪

⎩

⎪⎪

⎨

⎧

==

≠=⋅−

=′→⎪⎩

⎪⎨⎧ =

=

∞→∀≤=→=→

⋅−

=⋅+⋅−→⋅−

=−→∉=

=→=

==

>

=

==

=

∑

∑∑

∑

otherwise ,0 , θ 0,

and , θ

BFS newotherwise ,0

, BFS

)θ( unbounded is F then ,,0 If minθ

0 becomes ) θ( some until θ Increase

θ) θ()2( θ)1(

)2(,

)1(*

},...,1:{ basis afor BFS a be*Let

*

**

*

0|

*1

*

11

1

*

jiji

lkjiex

xjix

x

kelkex

ex

ex

ee

x

mk

l

kkjj

ikj

i

kjkj

j

ek

kjj

jj

m

kkjj

j

m

kjkjj

m

kjkjj

m

kjj

j

k

k

k

kj

k

kk

kk

kk

k

bAA

0AAAAA

bAbAx

Ax

B

B


• Theorem 4

• Proof{ } basis new a is A,...,A,A,A,...,A

111 mll jjjjjB+−

=′

[ ]

basis a is tindependen linearly are B of columns All0d

,...,1 0, 0A becomes 0(3)

0 0, since0 then 0,(3)set we if

tindependen linearly are ,...,1,A

(3) A)(A

AAAAdB

A,...,A,A,A,...,ABLet

,1

,1

,11,1

111

B

mkdd

deed

mk

deded

deddd

k

m

lkkjk

lljljl

j

m

lkkjkkjljljl

j

m

lkkk

m

kjkjlj

m

lkkkjl

jjjjj

k

k

kl

kkk

mll

′→

′→=→

==→==→

=>→==→

=

++=

+=+=′

=′

∑

∑

∑∑∑

≠=

≠=

≠==≠=

+−

BFS1

BFS2


• Theorem 5

• Proof

{ } { }polytope. the of edge an ]isˆ ,ˆ[ i.e. adjacent, are ˆ and ˆ then

,A)A( i.e. adjacent, are and If

yxyx

BByx kjxy ∪−=

edge.an thereforeis andP with halfspace a ofon intersecti theis ]ˆ,ˆ[only Hence

.ˆˆsatisfy ]ˆ,ˆ[on only P,in Therefore,. and ,0satisfy

and of wnscombinatioconvex only Therefore,s.bfs'such only theare and However,

. of subsets bases with sbfs' ofn combinatioconvex a is Then optimal. is suppose ,uniqueness prove To

optimal.uniquely are and ofnscombinatioconvex are that solutions feasible allThen

otherwise1 if0

r cost vecto aconstruct usLet

yxxd'wd'yxw

c'xc'wb,wAwyx

yx

BBzz

yx

BBAc

yx

yxjj

≤≤≥=

∪

⎩⎨⎧ ∪∈

=

BFS2

BFS1

bAA =⋅+⋅−∑=

jj

m

kkjj kk

ex θ) θ(1

*

Tableau

Tableau• Example

3x1 + 2x2 + x3 = 15x1 + x2 + x3 + x4 = 32x1 + 5x2 + x3 + x5 = 4

Select B={A3, A4, A5} --> make an identity matrix

x1=x2=0, x3=1, x4=2, x5=3 : BFSx3, x4, x5 : basic variables

x1 x2 x3 x4 x5

1 3 2 1 0 0

3 5 1 1 1 0

4 2 5 1 0 1

x1 x2 x3 x4 x5

1 3 2 1 0 0

2 2 -1 0 1 0

3 -1 3 0 0 1

Tableau

x1 x2 x3 x4 x5

1/3 1 2/3 1/3 0 0

4/3 0 -7/3 -2/3 1 0

10/3 0 11/3 1/3 0 1

x1 x2 x3 x4 x5

1 3 2 1 0 0

2 2 -1 0 1 0

3 -1 3 0 0 1

{ }5411

31

*

0|

1

321

3

115431

,,,3

10,34,0,0,

31BFS]'0 0 1[ Make

basic-non becomes 131)

22,

31min(minθ

basis, new a into put To

5,4,323123

11

AAAA

AA

A

AAAAA

=⎥⎦⎤

⎢⎣⎡=→=

=→=→===

===→=−+=⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−=

>

=

=∑

B

lex

jjje

jk

j

ek

m

kjk

k

k

k

Choosing a Profitable Column


( )

[ ]yzyc0y

zc0zccc

Proof

Theorem

′≥′≥→≥→≥−=→≥′=

≥

⋅=−⋅=⎟⎠

⎞⎜⎝

⎛−⋅=

+−→

−→

→

∑

∑∑∑

=

===

,any for 0...

optimuman at are then we, allfor 0 If

θθθΔ

θθ:cost

θ:

θ0:

21

1

11

*

1

*

**

n

j

jjj

m

kkjjjj

j

m

kkjj

m

kjj

m

kjj

kjjjj

j

ccc

jc

czcecc

cecxcxc

exxx

x

k

kkkkk

kkk


[ ]

[ ] [ ][ ][ ] [ ]

optimum global a is **

0) are variablesbasic-non(*

,any For

**

1

1

1

111

111

1

1

1111

1

11

1

11

1

xxcyc

xc

bB

yAByE

y

yyzyc

0y

ABEBEBABAA

∴

′≥′→

′=′

⋅=

⋅=

⋅⋅=⋅⋅=

⋅⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡⋅=

⋅⎥⎦

⎤⎢⎣

⎡=′≥′

≥

=→=⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=→

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

==

−

−

==

−

=

∑∑

∑

mm

m

mm

m

kk

k

jjjj

jj

jjjj

mnm

n

jj

m

kknj

m

kkj

mnm

n

mj

jm

kjkjj

xxcc

cc

cccc

ee

eecc

ecec

ee

ee

e

ee

LL

L

LL

L

MOM

L

L

L

L

MOM

L

M

Simplex Algorithm

Simplex Algorithm

endend

on pivot and

minθ find

else

yes'':unbounded then allfor 0 if

;0such that any choose begin else

yes'':opt then allfor 0 if do no''unbounded and no''opt while

s) terminatealgorithm theyes'' becomeseither (when ;no'':unbounded;no'':opt

beginsimplex procedure

0|0

lj

lj

j

kj

j

ek

kj

j

j

e

ex

ex

ke

cj

jc

lk

kj

=⎥⎥⎦

⎤

⎢⎢⎣

⎡=

=≤

<

=≥==

==

>

bAA =⋅+⋅−∑=

jj

m

kkjj kk

ex θ) θ(1

*

Simplex Algorithm

• Examplez=x1+x2+x3+x4+x5

0 1 1 1 1 1

1 3 2 1 0 0

3 5 1 1 1 0

4 2 5 1 0 1

-6 -3 -3 0 0 0

1 3 2 1 0 0

2 2 -1 0 1 0

3 -1 3 0 0 1

( )( )( )( ) 31311121

31112131

.3

2

1

1

−=⋅+⋅−+⋅−=−=⋅−+⋅+⋅−=

−= ∑=

cc

cecckj

m

kkjjj

1. obtain basis x3 x4 x5

2. make identity matrix

∑=

−m

kjj kk

cx1

.4

21 kkj eexk

Simplex Algorithm

-6 -3 -3 0 0 0

1 3 2 1 0 0

2 2 -1 0 1 0

3 -1 3 0 0 1

jc

ex

c

cj

j

kj

j

ek

j

j

k

kj

allfor 0 until 7-3 stepsrepeat 8.

pivot .7

minθ compute6.

n)computatio (more θ negativemost

or negativemost gives that column select .5

0|

≥

=

⋅

> -9/2 3/2 0 3/2 0 0

1/2 3/2 1 1/2 0 0

5/2 7/2 0 1/2 1 0

3/2 -11/2 0 -3/2 0 1

Beginning the Simplex Algorithm

Beginning the Simplex Algorithm• How to obtain an initial BFS?

– Use slack variables• Ax ≤ b --> Ax + Ixs = b, xs are initial basic variables• What if b < 0? --> -Ax - Ixs = -b, then use artificial variables

– Use artificial variables, then two-phase method• Ax=b --> Ax + Ixa = b, xa are initial basic variables• All the artificial variables are driven out of the basis

problem original

variablesbasicnonotherwise,0

variablesbasic,...,1,0

→

⎪⎩

⎪⎨

⎧

−→=

→=≥=

→

j

j

a

x

mkxk

0x


• Two-phase method– In phase I, minimize the cost function

( ) )θθ (recall

0θ becausepivot a becan 0 case,last In the

variablesoriginal with the basis aget weuntil pivoting continue

:basis in theremain s' somebut 0ξ

problem original theosolution t feasible no:0ξ

ok:basis theofout driven are s' all and 0ξ

ξ

1

1

bAA =⋅+⋅−

=<

⎪⎪⎪

⎩

⎪⎪⎪

⎨

⎧

=

>=

=

∑

∑

=

=

jj

m

kkjj

kj

ai

ai

m

i

ai

kkex

e

x

x

x


endend

cost originalth simplex wi call :II Phase

row; ingcorrespond omit the and yes'':redundant then

)0(out driven becannot and basis in the is variableartificialan if begin else

yes'':infeasible then I Phasein 0ξ if ;ξcost th simplex wi call

; basis, artificialan introduce :I Phase

)yes'' toset thesemay I (Phase ;no'':redundant;no'':infeasible

beginphase- twoprocedure

=

∀=

=>

=

==

∑

je

x

kj

opt

ai

aix


• Examplex1a x2a x3a x1 x2 x3 x4 x5

-z= 0 0 0 0 1 1 1 1 1

-ξ= 0 1 1 1 0 0 0 0 0

1 1 0 0 3 2 1 0 0

3 0 1 0 5 1 1 1 0

4 0 0 1 2 5 1 0 1

x1a x2a x3a x1 x2 x3 x4 x5

-z= 0 0 0 0 1 1 1 1 1

-ξ= -8 0 0 0 -10 -8 -3 -1 -1

x1a= 1 1 0 0 3 2 1 0 0

x2a= 3 0 1 0 5 1 1 1 0

x3a= 4 0 0 1 2 5 1 0 1

kj

m

kkjjj cecc ∑

=

−=1

jc

∑=

=m

i

aix

1ξ



-z= -1/3 -1/3 0 0 0 1/3 2/3 1 1

-ξ= -14/3 10/3 0 0 0 -4/3 1/3 -1 -1

x1= 1/3 1/3 0 0 1 2/3 1/3 0 0

x2a= 3/4 -5/3 1 0 0 -7/3 -2/3 1 0

x3a= 10/3 -2/3 0 1 0 11/3 1/3 0 1

kj

m

kkjjj cecc ∑

=

−=1

∑=

=m

i

aix

1ξ


-z= -1/2 -1/2 0 0 -1/2 0 1/2 1 1

-ξ= -4 4 0 0 2 0 1 -1 -1

x2= 1/2 1/2 0 0 3/2 1 1/2 0 0

x2a= 5/2 -1/2 1 0 7/2 0 1/2 1 0

x3a= 3/2 -15/6 0 1 -11/2 0 -3/2 0 1



-z= -3 0 -1 0 -4 0 0 0 1

-ξ= -3/2 7/2 1 0 11/2 0 3/2 0 -1

x2= 1/2 1/2 0 0 3/2 1 1/2 0 0

x4= 5/2 -1/2 1 0 7/2 0 1/2 1 0

x3a= 3/2 -5/2 0 1 -11/2 0 -3/2 0 1


-z= -9/2 0 -1 -1 3/2 0 3/2 0 0

-ξ= 0 7/2 1 1 0 0 0 0 0

x2= 1/2 1/2 0 0 3/2 1 1/2 0 0

x4= 5/2 -1/2 1 0 7/2 0 1/2 1 0

x5= 3/2 -5/2 0 1 -11/2 0 -3/2 0 1

This example gives an optimum point when Phase I is finished ( )jc j ∀≥ 0

Geometric Aspects of Pivoting

Geometric Aspects of Pivotingx1 x2 x3 x4 x5 x6 x7

-34 -1 -14 -6 0 0 0 0

x4= 4 1 1 1 1 0 0 0

x5= 2 1 0 0 0 1 0 0

x6= 3 0 0 1 0 0 1 0

x7= 6 0 3 1 0 0 0 1

1

x1 x2 x3 x4 x5 x6 x7

-32 -1 -14 -6 0 0 0 0

x4= 2 0 1 1 1 -1 0 0

x1= 2 1 0 0 0 1 0 0

x6= 3 0 0 1 0 0 1 0

x7= 6 0 3 1 0 0 0 1

2

x2

x3

x112


x1 x2 x3 x4 x5 x6 x7

-20 0 -8 0 6 -5 0 0

x3= 2 0 1 1 1 -1 0 0

x1= 2 1 0 0 0 1 0 0

x6= 1 0 -1 0 -1 1 1 0

x7= 4 0 2 0 -1 1 0 1

3

x1 x2 x3 x4 x5 x6 x7

-4 0 0 8 14 -13 0 0

x2= 2 0 1 1 1 -1 0 0

x1= 2 1 0 0 0 1 0 0

x6= 3 0 0 1 0 0 1 0

x7= 0 0 0 -2 -3 3 0 1

4

x2

x3

x11

24

3


x1 x2 x3 x4 x5 x6 x7

-4 0 0 -2/3 1 0 0 13/3

x2= 2 0 1 1/3 0 0 0 1/3

x1= 2 1 0 2/3 1 0 0 -1/3

x6= 3 0 0 1 0 0 1 0

x5= 0 0 0 -2/3 -1 1 0 1/3

5

x1 x2 x3 x4 x5 x6 x7

-2 1 0 0 2 0 0 4

x2= 1 -1/2 1 0 -1/2 0 0 1/2

x3= 3 3/2 0 1 3/2 0 0 -1/2

x6= 0 -3/2 0 0 -3/2 0 1 1/2

x5= 2 1 0 0 0 1 0 0

6

x2

x3

x112

4

3

5

6

=


• Taking an alternative path x1 x2 x3 x4 x5 x6 x7

-34 -1 -14 -6 0 0 0 0

x4= 4 1 1 1 1 0 0 0

x5= 2 1 0 0 0 1 0 0

x6= 3 0 0 1 0 0 1 0

x7= 6 0 3 1 0 0 0 1

1

2

x2

x3

x112

x1 x2 x3 x4 x5 x6 x7

-6 -1 0 -4/3 0 0 0 14/3

x4= 2 1 0 2/3 1 0 0 -1/3

x5= 2 1 0 0 0 1 0 0

x6= 3 0 0 1 0 0 1 0

x2= 2 0 1 1/3 0 0 0 1/3


3

4

x2

x3

x112

43

x1 x2 x3 x4 x5 x6 x7

-2 -1 0 0 0 0 4/3 14/3

x4= 0 1 0 0 1 0 -2/3 -1/3

x5= 2 1 0 0 0 1 0 0

x3= 3 0 0 1 0 0 1 0

x2= 1 0 1 0 0 0 -1/3 1/3

x1 x2 x3 x4 x5 x6 x7

-2 0 0 0 1 0 2/3 13/3

x1= 0 1 0 0 1 0 -2/3 -1/3

x5= 2 0 0 0 -1 1 2/3 1/3

x3= 3 0 0 1 0 0 1 0

x2= 1 0 1 0 0 0 -1/3 1/3

=

Integer Linear Programming

Integer Linear Programming• Problem

integer :

min

x0x

bAxxc

≥=

′

Feasible region of LP

Decreasing cost

Optimum of ILP

Optimum of LP

1x

2x


• Cutting-Plane Algorithm

0xbAxxc

≥=

′ minILP theof Relaxation

x*, LP solution

Decreasing cost

x0, ILP solution

Feasibleset

1x

2x

Rounded x*


– Add constraints to an ILP that do not exclude integer feasible points until the solution to the LP relaxation is integer

Decreasing cost

x*x* x*=x0


⎣ ⎦ ⎣ ⎦⎣ ⎦( ) ⎣ ⎦

⎣ ⎦ ⎣ ⎦ ∑∑

∑

∑

∉∉

∉

∉

−−

−=−+→

+=++

+=+=

∈=+

=→=→=

Bjjijii

Bjjijj

iiBj

jijijj

iiiijijij

jiBj

jijj

jj

k

j

k

k

j

k

xffbxex

fbxfex

fbbfee

BAbxex

AA

A

A

11 bExbBAxBbAx

::

:

:

~

~

~~

,~

~

Then

and Let

cut). (Gomory constraintnew a as Addpoint. feasible integer any excludenot doesBut

LP. relaxation the of 0) 0,( solution the excludes This

(2), and (1) From

(2) and 0 0 Because

(1) integer. an bemust integers, be to and For

is

Bjjij

ij

Bj:Ajiji

Bjjijijiji

Bjjijijj

fxxf

fx

xff

xffxff

xffxx

j

j

j

j

k

−=+−→

>=→

≤−

<−≥<≤<≤

−

∑

∑

∑

∑

∉

∉

∉

∉

A

A

A

0

1011

:

:

:

.

.,,,

linear programmingocw.snu.ac.kr/sites/default/files/note/10130.pdf · 2018-01-30 · • convex...

Documents