review of linear programming software

Introduction to LP/QP/MIPLP Software

Analysis and ComparisonConclusion

Review of Linear Programming Software

Peter He, Yang Li, Zhenghue Nie, Nael El Shawwa

Instructor: Prof. Tamas TerlakySchool of Computational Engineering and Science

McMaster University

Jan. 12, 2007

He, Li, Nie, Shawwa Linear Programming



Outline1 Introduction to LP/QP/MIP

Linear Programming(Mixed) Integer ProgrammingQuadratic Programming

2 LP SoftwareIntroductionCPLEXXPRESS-MPMOSEKLPSOLCLP

3 Analysis and ComparisonBenchmark AnalysisComparison





What are LP?

Standard LP form:

min cT xs.t . Ax = b

x ≥ 0

Practical form:min cT xs.t . b ≤ Ax ≤ b̄

l ≤ x ≤ u

Main Methods:Simplex MethodsInterior Point Methods





Simplex Method (Pivot Method)

Source:www.cas.mcmaster.ca/ terlaky/htm/talks/pivot-vs-IPM.pdf

Ax = b defines a polytope,namely, SimplexJump from vertex to vertexPerform well for small andmedium scale, but bad forlarge scale problem, O(2n)

Efficient for MIPDegeneracy





Interior Point Methods


Ax = bAT y + s = c

xT s = µe

follow the central path inthe feasible region





Interior Point Methods


Very efficient forlarge-scale problem, totalcomputational complexityO(n3L)

Bad performance in MIPNo troubles caused bydegeneracy





What is (Mixed) Integer Programming

Integer Programming:

min cT xs.t. Ax = b

xi ∈ Z or subset of Z for all i

Branch and Bound & Cutting Plane.





What is (Mixed) Integer Programming

Mixed Integer Programming:

min cT xs.t. Ax = b

xi ∈ Z or subset of Z for some i

Branch and Bound & Cutting Plane.





What is Quadratic Programming?

Standard QP form:

min cT x + 12xT Qx

s.t. Ax = bx ≥ 0

If Q is semi-definite, the problem is convex. Otherwise,local convergence.




IntroductionCPLEXXPRESS-MPMOSEKLPSOLCLP









Developers’ View

ProgramsAlgorithmsModeling systems

CostFree codesCommercial products - providing the demo version

Othersproducts typeinterfaceplatforms and/or operating systemsdevelopment environmentmodeling systemparallel...





Overview of CPLEX

Created by Robert E. Bixby using C language in 1987Been Acquired by ILOG, inc. in 1997Widespread

95% of papers that mention a solver mention CPLEX at anyconference where papers are presentedStandard solver in supply-chain applications; Industryleaders including SAP, Oracle, i2, JD Edwards, andManugisticsMost of the major airlines including United, American,Delta, Continental, Northwest, and Southwest

2004 INFORMS Impact AwardLatest version: 10.1 (Parallel, Indictor Constraints, etc. )





Problems Covered

Linear ProgrammingMixed Integer Linear ProgrammingQuadratic ProgrammingMixed Integer Quadratic ProgrammingQuadratic Constrained ProgrammingMixed Integer Quadratic Constrained Programming

Millions of constraints and variables





Algorithms

LP:Primal simplexDual simplexNetwork simplexPrimal/dual log barrier

QP:Primal simplexDual simplexPrimal dual log barrier

QCP:Primal dual log barrier





ILOG Optimization Suite

Simplex Optimizers (Primal,Dual, Network) for LP and QPBarrier Optimizer for LP, QPand QCPMixed Integer Optimizer forMIP, MIQP and MIQCPCPLEX Interactive OptimizerCPLEX Component Libraries

CPLEX Callable Library (C and VB6APIs)ILOG Concert Technology (C++, Java,.NET APIs)

Source: http://www.ilog.com/products/optimization/presentations/methods.pdf





Embedding CPLEX

CPLEX Callable Library usesmatrices to represent aproblemILOG Concert Technologyuses objects and methods torepresent a problemILOG OPL Interfaces allowOPL models to be embeddedinside an application

Source: http://www.ilog.com/products/optimization/presentations/methods.pdf





Modeling System, Platforms and Parallel

Modeling System: AMPL, MPL, OPL Studio, and GAMSPlatforms: PC/Windows, PC/Linux, Sun/Solaris, HP, IBMPowerParallel: Yes





Overview of Xpress-MP

Developed by Dash OptimizationUsed in industry:

Finance and investmentItalcementi Group - Cement plant supply chain optimisationHoneywell - Flowsheet Decomposition Heuristic forSchedulingPower Optimization Limited - Generator Scheduling in theElectricity Industry





Problems covered

LP, QPMIP, MIQP





Algorithms

Primal and Dual simplex methodsInterior points method for LP and QPBranch and Bound techniques to solve MIP and MIQPTechniques to reduce problem size





Platforms and OS

WindowsLinux, UnixMac (PowerPC)HP, AIX





Modeling Systems

MoselAMPLGAMSFrontlineMPL





Interfaces

command lineC/ C++JavaFortranVB6, .NETODBC I/O driver for external spreadsheets and databases





Other Features

Size: no fixed limitInput form: Industry standard formats - LP,MPSSupports parallel computingAvailable add-ons to solve other problem types - stochasticproblems, NLPtotal user control of optimization processfast re-solve timeDevelopment tools - GUI development, problemformulation, solving and analysis





Cost

$ 6,000 per license (basic)$ 22,500 per license (bundle)$ 1,000 per additional licenceAcademic: 90% off, free with academic partnershipprogramOther Mosel and Optimizer add-ons available separately

Prices as of January 2007 - Dash Optimization Price List





Overviews of MOSEK

Erling D. Andersen and Knud D. AndersenImplement language: CLatest Version: 4.0





Problems covered

Linear optimizationConvex quadratic optimization.Quadratically constrained convex optimization.Conic quadratic optimization (also known as second orderoptimization).Convex optimization.Geometric optimization (posynomial case).Handle integer valued variables in linearly andquadratically constrained optimization problems





Algorithms

Interior-point optimizer for all continuous problems.Primal or dual simplex optimizer for linear problems.Conic interior-point optimizer for conic quadratic problems.Mixed-integer optimizer based on a branch and cuttechnology.





Platforms and OS

Linux 32 and 64 bitMAC OSXSolaris 32 and 64 bitWindows 32 and 64 bit





Modeling Systems

AMPLGAMSAIMMS





Interfaces

C/C++ APICommand line interfaceJava APIMicrosoft. NET APIMATLAB





Costs:

A: Linux and Windows 32 bit B: Other Platforms, likeSolaris, Mac OS, Linux and Windows 64 bit.Additional maintenance fee chargedDiscount for academic use: 70% to 90%





Other Features

Input format: MPS, LP, MBT(MOSEK binary task format)Parallel Computing





Overviews of LPSOL

Michel Berkelaar and Jeroen DirksImplement language: ANSI CLatest Version: Version 5.5





Problems covered

(Large-scale) Linear ProgrammingMixed Integer Programming





Algorithms

Revised Simplex Method for Linear ProgrammingA Branch and Bound Method for Mixed IntegerProgramming





Platforms and OS

Linux/UnixWindowsMAC OS





Modeling Systems

AMPLGAMSAIMMS





Interfaces

C/C++ APIVisual BasicMicrosoft .NETDelphiJavaMATLABExcel





Other Features

Sizes: Basically, no limit on model sizeInput form: LP format and MPSParallel Computing: NoPrices: Free





Introduction to CLP

CLP means Coin OR Linear Porgramming.It is a highquality simplex code under the term of the Common PublicLicenseAuthor: John Forrest; Published Time: 2002; The firstvesion:version 0.90; Current version:Clp-1.3.3( http://www.coin-or.org/)CLP is written in C++ and it is primarily intended to beused as a callable library.





What can CLP solve?

Linear OptimizationRegularLarge Scale

It has been proved that LP with 1.5 million constraints stillkeeps reliability.

Sparse Coeffient MatrixesBoolean MatrixesNetwork Matrixes

Some Nonlinear Optimization:Such asQuadratic ProgrammingPiecewise Linear Convex function as the objective function





What Are Algorithms Being Used?

Prime simplex method : the column pivot choiceDual simplex method: Dantzig and Steepest edge rowpivot choiceCLP barrier method: solving convex QPs as well as LPs





What Input Form Is Often Used?

CLP often reads or loads MPS file.MPS (MathematicalProgramming System) is a file formatfor LO and MIP.





What Platforms Are Available?

Linux using g++ version 3.1.1 (or later)Windows using Microsoft Visual C++ 6 (or later)Windows using cygwinAIX using xIC (not supported in the current Makefile)





Features of CLP

Apparently as reliable as OSLSlightly slower that OSL on calculation speedBarrier code is not as mature as the simplex codeBut, CPL is easier than OSL, free or low cost, and the highratio of the function to the price .




Benchmark AnalysisComparison









Benchmark

Run on aLinux-PC(3.2GHz P4,4GB RDRAM,Linx-2.6)

Version: CPLEX-10.0;MOSEK-4.0.0.32;CLP-1.03.03;

Times are user timesin seconds includinginput and crossover toa feasible basis for allcodes.B/D/P:barrier/daul/primalsimplex

source:http://plato.asu.edu/ftp/lpcom.html





Parallel CPLEX on MIP problemsCPLEX-10.1 was run in default mode on a single and on a dual processor 2.4GHz Opteron (64-bit, Linux), and a2.2GHz dual-core Opteron (64-bit,Linux)

Times given are elapsed CPU times in seconds. "c" problem convex. AMPL or MPS input.

source: http://plato.asu.edu/ftp/ser_par.html





CPLEX parallel benchmark shows significant improvementwith dual processorsSome problems may have dense coefficient matrix → badfor parallel computingCommercial packages were more robust and efficient thanopen-source softwareBUT there are cases were CPLEX algorithms failed toconverge while MOSEK and CLP did.There are tests where CPLEX has reported in-feasiblesolution while MOSEK convergesIPM is more robust than Primal/Dual for large scaleproblems





Price

Name Commercial Academic EvaluationCPLEX Call $995 Free student

XPRESS-MP $6,000 $600 Free student30 day demo

MOSEK $1,750 $175 Free student15 day demo





Platform Supported

Product CPLEX XPRESS-MP MOSEK solver enginee LPSOL CLPPC/Windows y y y y yPC/Linux y y y y ySun/Solaris y y y y yMac/Mac OS - Power PC - y yOther HP,IBM power HP,AIX - - -Shared Memory Parallel y y y - -

source: http://lionhrtpub.com/orms/surveys/LP/LP-survey.html





Size of Problems solvable by the system

- Internal Max No. of Available AvailableProduct Restrictions Constraints Memory Disk SpaceCPLEX - - y -XPRESS Solver Engine - - y -MOSEK Solver Engine - - y -LPSOL - - y -CLP - - y -






Algorithms

- Primal Dual Interior Branch-and- Branch-and-Product Simplex Simplex Point Cut PriceCPLEX y y y y -XPRESS Solver Engine y y y y yMOSEK Solver Engine y y y y -LPSOL y y - - -CPL y y y y -





Summary

All of them can solve the LP programs, but commercialprograms

provide more flexible interface;support more modeling systems;provide development tools;provide parallel computation;

Parallel computation provide a new way to improve theefficiency and solve large-scale problems, but is limited bythe algorithm used and matrix sparsity.Which program is the fastest one?


Reference

Reference

T. Illes, and T. Terlaky Pivot versus interior point methods: Pros and cons European Journal of OperationalResearch, Volume 140, Number 2, 16 July 2002, pp. 170-190(21)

CPLEX: http://www.ilog.com/products/cplex/

XPRESS: http://www.dashoptimization.com/

MOSEK: http://www.mosek.com/

LPSOL: http://www.cs.sunysb.edu/ algorith/implement/lpsolve/implement.shtml

CLP: http://www.coin-or.org/Clp/index.html

OR/MS Today 2005 Linear Programming Software Survey:http://lionhrtpub.com/orms/surveys/LP/LP-survey.html

Hans Mittelmann’s Benchmarks for Optimization Software: http://plato.la.asu.edu/bench.html

Software for Optimization: A Buyer’s Guide http://www.ampl.com/buyers1.html

Optimization Software Guide: http://www-fp.mcs.anl.gov/otc/Guide/SoftwareGuide/

http://www.informs.org/index.php?


Thanks &Questions Please...

Thanks!

Questions?


review of linear programming software

Documents