1/25

JJIIJI

Back

Close

Informs Annual Conf. Denver 2004

Solving Large Scale Linear NetworkFlow Problems with MOSEK

Bo Jensen

MOSEK ApSC/O Symbion Science ParkFruebjergvej 3, Box 162100 Copenhagen ØDenmarkhttp://www.mosek.com

bo.jensen@mosek.com

2/25

JJIIJI

Back

Close

Talk outline• Introduction to MOSEK

• The pure network flow problem

• Network simplex - exploiting structure

• MOSEK network simplex

• Computational results

• The MOSEK network API - design considerations

3/25

JJIIJI

Back

Close

Introduction to MOSEKAn large scale optimization software package.

Solvers :

• Linear optimization, interior andsimplex

• Mixed integer for linear andquadratic optimization

• Conic optimization

• Generel convex nonlinear optimiza-tion

Interfaces :

• MATLAB

• GAMS

• AMPL

• AIMMS

• Java

• C/C++

• .NET

Current Version 3.1. The next major release Ver. 4.0 will containmany new features and improvements among these network simplex.

4/25

JJIIJI

Back

Close

The pure network flow problemGiven a network G = (N, A) with capacities lxij, ux

ij, and costs cij

on its arcs, find flow satisfying node demands/supply at lowest totalcost.

Standard LP formulation:

min∑

(i,j)∈A

cijxij

s.t. lci ≤∑

{j:(i,j)∈A}

xij −∑

{j:(j,i)∈A}

xji ≤ uci for all i ∈ N

lxij ≤ xij ≤ uxij for all (i, j) ∈ A .

Problem structure:

Every column has two nonzeroes, with value -1.0 and 1.0.

5/25

JJIIJI

Back

Close

Well known facts• A large amount of solutions methods

• Both polynomial simplex and non-simplex algorithms exists

• Practical efficient implementations exists

• Can solve large problems very fast

• Primal network simplex is competitive

6/25

JJIIJI

Back

Close

Network simplex - exploiting structure• Basis is a permuted triangular matrix

B =

2666666666664

x48 x31 x47 x12 x36 x14 x35 x1r

r1 1 −1 −1 −1r2 1r3 −1 −1 −1r4 −1 −1 1r5 1r6 1r7 1r8 1

3777777777775L =

2666666666664

x48 x47 x36 x35 x14 x31 x12 x1r

r8 1r7 1r6 1r5 1r4 −1 −1 1r3 −1 −1 −1r2 1r1 −1 1 −1 −1

3777777777775• Basis can be represented as a spanning tree

7/25

JJIIJI

Back

Close

– Fast FTRAN

∗ We need B−1Aj for entering column j, loop the basis equiv-alent path (BEP).

8/25

JJIIJI

Back

Close

– Fast BTRAN

∗ If pivoting xBiout of basis, we need row i of B−1 to update

free duals. Correspond to subtree hanging from xBi

B−1 =

26666666664

1−1 −1 −1

11

11 1 1

1−1 −1 −1 −1 −1 −1 −1 −1

37777777775

9/25

JJIIJI

Back

Close

MOSEK primal network simplex

What is implemented ?

• No new theoretical ideas

• In generel a brush up and refinement of known methods

• Highly tuned algorithm for large scale problems

10/25

JJIIJI

Back

Close

• Where is the time spend ?

Problem numcon numvar Price FTRAN BTRAN

big8 50000 400000 52% 3% 44%

8 n12 4096 32768 78% 12% 7%

16 n12 4096 65536 86% 8% 4%

i n10 1024 32768 90% 4% 4%

hi7 16384 131431 30% 9% 58%

lo7 16384 131431 34% 11% 52%

Conclusion :Pricing and updating duals are the two key operations. The timespend in dual update is highly affected by the structure of the basistree.

11/25

JJIIJI

Back

Close

• Some problems tend to produce ’cheap’ basis trees

Wide and ’cheap’ basistree Deep and ’expensive’ basistree

Note : Having many slacks in the basis keeps the tree wide.

12/25

JJIIJI

Back

Close

• Exploiting degeneracyFlow problems are often highly primal degenerate. Creates ’ties’in the ratio test, we can choose freely among these. Of course thequestion of cycling arise.

13/25

JJIIJI

Back

Close

• Efficient datastructure for basis treeObjectives :

1. Easy location of smallest subtree size for better dual update

2. Possible to loop subtree above and below a node

3. Cheap identification of BEP

4. Easy to update after basis exchange

Mosek uses a version of the Extended Augmented PredecessorMethod (Lobel,1996 )

14/25

JJIIJI

Back

Close

• Obtaining a feasible solutionMost standard simplex algorithms use a two phase method, wherephase one minimize the sum of infeasibility :

min∑

(i,j)∈I1

(lxij − xij) +∑

(i,j)∈I2

(xij − uxij)

I1 = {(i, j) ∈ A|lxij > xij}

I2 = {(i, j) ∈ A|uxij < xij}

This does not work well for the network simplex.

15/25

JJIIJI

Back

Close

Network simplex feasibility method:

– Network simplex codes use some kind of penalty approach

– Very sensitive if penalty is set too high, disturbs the pricingphase

– MOSEK uses a variant of the Gradual Penalty Method (GPM)(Grigoriadis, 1986 )

16/25

JJIIJI

Back

Close

• Pricing - a historical perspective

Problem numcon numvar Mulvey Gibby Grigoriadis SMP

8 n11 2048 16384 3.61 1.06 1.43 1.00

16 n11 2048 32768 6.66 1.09 1.76 1.00

i n11 2048 92682 16.11 1.19 2.03 1.00

long13 8193 188439 3.15 5.01 2.51 1.00

hi6 11585 93033 1.32 1.60 2.96 1.00

lo7 46341 406225 1.49 2.48 2.57 1.00a

aJensen and Berthelsen, 2003

– Older partial pricing strategies is not competitive

– Keeping a sorted queue seems to be method of choice

17/25

JJIIJI

Back

Close

Computational results• Development in network simplex

• MOSEK network Vs. other specialized implementations

• MOSEK network Vs. standard LP solvers

• Conclusions

18/25

JJIIJI

Back

Close

First an example of development in network simplex

Netflo (Kennington and Helgason, 1980 )Vs.

MOSEK network optimizer 2004

Problem numcon numvar Netflo MOSEK Netflo/MOSEK

8 n11 2048 16384 3.12 2.35 1.33

16 n11 2048 32768 7.12 3.67 1.94

i n11 2048 92682 42.66 10.43 4.09

hi5 8192 65750 11.36 1.5 7.57

lo9 32768 262921 1064.21 59.58 17.86

long13 8193 188439 16.26 1.80 9.03

square15 32762 753526 1452.10 43.41 33.45

wide15 32769 753687 1550.62 40.79 38.01

a b

aAn early version of MOSEK network simplex. Both solvers running oninteger data.

bJensen and Berthelsen, 2003

19/25

JJIIJI

Back

Close

Network Simplex and Cost Scaling implementations

MCF Solvers:

• MCF 1.3 an efficient network simplex implementation by Lobel

• CS2 an efficient Cost-Scaling Push-Relabel algorithm by Goldberg

Test setup :

• In total 77 problems were included in the test, results are shownfor a representive subset

• All test instance were randomly generated

• We used a C++ translation from the MCFClass project at theUniversity of Pisa

• Double precision data is used for all tested codes

• Tested on a AMD Athlon XP 3200++ 1 GB RAM running Linux

20/25

JJIIJI

Back

Close

NETGEN LO instances generated with netgen

Problem numcon numvar MOSEK network MCF CS2

Iter Time Iter Time Time

lo1 2048 17933 21744 0.12 22294 0.17 0.2

lo2 2896 25355 33279 0.25 35229 0.33 0.40

lo3 4096 32877 45577 0.41 49049 0.54 0.63

lo4 5793 50760 67927 0.81 76961 1.18 1.22

lo5 8192 65750 89836 1.32 109025 2.33 2.05

lo6 11585 93033 138303 2.65 165839 7.46 3.92

lo7 16384 131431 193585 5.80 196823 13.27 6.68

lo8 23170 202986 276060 11.51 286881 24.35 13.18

lo9 32768 262921 400955 26.93 423743 63.27 18.86

lo10 46341 406225 608275 73.16 639384 133.32 37.60

Problem MCF / MOSEK CS2 / MOSEK

Iter Time Time

lo1 1.03 1.42 1.67

lo2 1.06 1.32 1.60

lo3 1.08 1.32 1.54

lo4 1.13 1.46 1.51

lo5 1.21 1.77 1.55

lo6 1.20 2.82 1.48

lo7 1.02 2.29 1.15

lo8 1.04 2.12 1.15

lo9 1.06 2.35 0.70

lo10 1.05 1.82 0.51

21/25

JJIIJI

Back

Close

GRIDGEN LONG instances generated with gridgen

Problem numcon numvar MOSEK network MCF CS2

Iter Time Iter Time Time

long11 2049 47127 6532 0.10 9844 0.15 0.4

long11 5 2897 66631 8389 0.14 16073 0.27 0.68

long12 5 5793 133239 15502 0.48 19950 0.82 1.84

long13 8193 188439 22834 1.00 30636 1.97 3.00

long13 5 11585 266455 34172 1.97 48896 5.34 5.29

long14 16385 376855 54242 4.83 87330 17.36 9.52

long14 5 23169 532887 73049 9.67 138079 49.16 16.05

long15 32769 753687 107550 22.02 207611 87.31 30.32

Problem MCF / MOSEK CS2 / MOSEK

Iter Time Time

long11 1.51 1.50 4.00

long11 5 1.92 1.93 4.86

long12 5 1.29 1.71 3.83

long13 1.34 1.97 3.00

long13 5 1.43 2.71 2.69

long14 1.61 3.59 1.97

long14 5 1.89 5.08 1.66

long15 1.93 3.97 1.38

22/25

JJIIJI

Back

Close

GOTO 8 instances generated with GOTO

Problem numcon numvar MOSEK network MCF CS2

Iter Time Iter Time Time

8 n9 512 4096 10375 0.06 11318 0.06 0.04

8 n95 724 5792 15693 0.11 22345 0.15 0.06

8 n10 1024 8192 24814 0.21 31234 0.26 0.10

8 n105 1448 11584 35607 0.38 39052 0.56 0.16

8 n11 2048 16384 58868 0.98 65258 1.30 0.32

8 n115 2896 23168 87171 2.46 103824 3.43 0.59

8 n12 4096 32768 172425 4.67 178520 8.33 1.27

8 n125 5793 46344 187540 9.69 253379 17.87 1.74

8 n13 8192 65536 266201 26.47 365946 51.09 3.13

8 n135 11585 92680 377822 55.57 561301 154.72 5.73

8 n14 16834 131072 522723 130.61 715647 425.83 8.92

Problem MCF / MOSEK CS2 / MOSEK

Iter Time Time

8 n9 1.09 1.00 0.67

8 n95 1.42 1.36 0.55

8 n10 1.26 1.24 0.48

8 n105 1.10 1.47 0.42

8 n11 1.11 1.33 0.33

8 n115 1.19 1.39 0.24

8 n12 1.04 1.78 0.27

8 n125 1.35 1.84 0.18

8 n13 1.37 1.93 0.12

8 n135 1.49 2.78 0.10

8 n14 1.37 3.26 0.07

23/25

JJIIJI

Back

Close

Standard LP Vs. Network Simplex

Problem numcon numvar Primal network simplex Primal simplex IPM

iter Time iter Time Time

hi1 2048 17933 6384 0.06 9886 1.34 7.37

hi2 2896 25355 7870 0.10 14557 2.67 19.30

hi3 4096 32877 10998 0.15 24931 7.99 47.73

hi4 5793 50760 16910 0.32 40704 17.43 136.62

hi5 8192 65750 22897 0.63 68643 52.95 329.58

hi6 11585 93033 34545 1.17 124024 165.36 1027.00

hi7 16384 131431 49894 2.49 219899 492.01 3600.42

hi8 23170 202986 76292 5.89 353020 1205.78 7391.12

hi9 32768 262921 118673 13.73 703132 4194.89 MEM

hi10 46341 406225 194499 35.77 1071190 9016.27 MEM

Problem Primal simplex / network IPM / network

Iter Time Time

hi1 1.55 22.33 122.83

hi2 1.85 26.70 193.00

hi3 2.27 53.27 318.20

hi4 2.41 54.47 426.94

hi5 3.00 84.05 523.14

hi6 3.59 141.33 877.78

hi7 4.41 197.59 1445.95

hi8 4.63 204.72 1254.86

hi9 5.92 305.53 ?

hi10 5.51 252.06 ?a

aTested on a AMD Athlon XP 3200++ 1 GB RAM running Linux.

24/25

JJIIJI

Back

Close

Numerical conclusionsGenerel conclusions

• Network simplex implementations has improved

• Network simplex is surprisingly faster than standard simplex

• Network simplex and cost scaling algorithms are method of choice

– Network simplex performs well optimizing small to large graphsand on ’easy’ problems

– For reoptimization is network simplex superior (see Frangioniand Manca 2004)

MOSEK related conclusions

• MOSEK is the fastest network simplex on 73 of 77 instances

• MOSEK is faster than CS2 on 40 of 77 instances

25/25

JJIIJI

Back

Close

The MOSEK network API - design consid-erations• Own network API or standard LP API with network detection

– Specific network API more efficient

• Easy integration with standard API

– Transparent exchange data between network and standard task

– No terms as edge and nodes

– Interface with two non zeroes in each column

– Extendable API to generalized networks