nonlinear integer programming€¦ · 2.4.1 linear integer program 29 2.4.2 linearly constrained...

30
Nonlinear Integer Programming

Upload: others

Post on 28-Jun-2020

13 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Nonlinear Integer Programming€¦ · 2.4.1 Linear integer program 29 2.4.2 Linearly constrained separable convex integer program 31 2.4.3 Unconstrained convex integer program 32

Nonlinear Integer Programming

Page 2: Nonlinear Integer Programming€¦ · 2.4.1 Linear integer program 29 2.4.2 Linearly constrained separable convex integer program 31 2.4.3 Unconstrained convex integer program 32

Recent titles in the INTERNATIONAL SERIES IN OPERATIONS RESEARCH & MANAGEMENT SCIENCE

Frederick S. Hillier, Series Editor, Stanford University

Maros/ COMPUTATIONAL TECHNIQUES OF THE SIMPLEX METHOD Harrison, Lee & Neale/ THE PRACTICE OF SUPPLY CHAIN MANAGEMENT: Where Theory and

Application Converge Shanthikumar, Yao & Zijm/ STOCHASTIC MODELING AND OPTIMIZATION OF

MANUFACTURING SYSTEMS AND SUPPLY CHAINS Nabrzyski, Schopf & W?glarz/ GRID RESOURCE MANAGEMENT: State of the Art and Future

Trends Thissen & Herder/ CRITICAL INFRASTRUCTURES: State of the Art in Research and Application Carlsson, Fedrizzi, & Fuller/ FUZZY LOGIC IN MANAGEMENT

Soyer, Mazzuchi & Singpurwalla/ MATHEMATICAL RELIABILITY: An Expository Perspective Chakravarty & Eliashberg/ MANAGING BUSINESS INTERFACES: Marketing, Engineering, and

Manufacturing Perspectives Talluri & van Ryzin/ THE THEORY AND PRACTICE OF REVENUE MANAGEMENT Kavadias & Loch/PROJECT SELECTION UNDER UNCERTAINTY: Dynamically Allocating

Resources to Maximize Value Brandeau, Sainfort & Pierskalla/ OPERATIONS RESEARCH AND HEALTH CARE: A Handbook of

Methods and Applications Cooper, Seiford & Zhu/ HANDBOOK OF DATA ENVELOPMENT ANALYSIS: Models and

Methods Luenberger/ LINEAR AND NONLINEAR PROGRAMMING, 2'"^ Ed. Sherbrooke/ OPTIMAL INVENTORY MODELING OF SYSTEMS: Multi-Echelon Techniques,

Second Edition Chu, Leung, Hui & Cheung/ 4th PARTY CYBER LOGISTICS FOR AIR CARGO Simchi-Levi, Wu & Shen/ HANDBOOK OF QUANTITATTVE SUPPLY CHAIN ANALYSIS:

Modeling in the E-Business Era Gass & Assad/ AN ANNOTATED TIMELINE OF OPERATIONS RESEARCH: An Informal History Greenberg/ TUTORIALS ON EMERGING METHODOLOGIES AND APPLICATIONS IN

OPERATIONS RESEARCH Weber/ UNCERTAINTY IN THE ELECTRIC POWER INDUSTRY: Methods and Models for

Decision Support Figueira, Greco & Ehrgott/ MULTIPLE CRITERIA DECISION ANALYSIS: State of the Art

Surveys Reveliotis/ REAL-TIME MANAGEMENT OF RESOURCE ALLOCATIONS SYSTEMS: A Discrete

Event Systems Approach Kail & Mayer/ STOCHASTIC LINEAR PROGRAMMING: Models, Theory, and Computation Sethi, Yan & Zhang/ INVENTORY AND SUPPLY CHAIN MANAGEMENT WITH FORECAST

UPDATES Cox/ QUANTITATIVE HEALTH RISK ANALYSIS METHODS: Modeling the Human Health Impacts

of Antibiotics Used in Food Animals Ching & Ng/ MARKOV CHAINS: Models, Algorithms and Applications

* A list of the early publications in the series is at the end of the book *

Page 3: Nonlinear Integer Programming€¦ · 2.4.1 Linear integer program 29 2.4.2 Linearly constrained separable convex integer program 31 2.4.3 Unconstrained convex integer program 32

NONLINEAR INTEGER PROGRAMMING

DUAN LI Department of Systems Engineering and Engineering Management The Chinese University of Hong Kong Shatin, N. T Hong Kong

X I A O L I N G SUN Department of Mathematics Shanghai University Baoshan, Shanghai 200444 R R. China

^ S p rineer g«

Page 4: Nonlinear Integer Programming€¦ · 2.4.1 Linear integer program 29 2.4.2 Linearly constrained separable convex integer program 31 2.4.3 Unconstrained convex integer program 32

Duan Li Xiaoling Sun The Chinese University of Hong Kong Shanghai University P.R. China P.R. China

Library of Congress Control Number: 2006922525

ISBN-10: 0-387-29503-8 (HB) ISBN-10: 0-387-32995-1 (e-book)

ISBN-13: 978-0387-29503-9 (HB) ISBN-13: 978-0387-32995-6 (e-book)

Printed on acid-free paper.

© 2006 by Springer Science+Business Media, LLC All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science + Business Media, LLC, 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now know or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks and similar terms, even if the are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights.

Printed in the United States of America.

9 8 7 6 5 4 3 2 1

springer.com

Page 5: Nonlinear Integer Programming€¦ · 2.4.1 Linear integer program 29 2.4.2 Linearly constrained separable convex integer program 31 2.4.3 Unconstrained convex integer program 32

To my mother, Yunxiu Xue, my wife, Xiuying Lu, and my

children, Bowen Li, Kevin Bozhe Li, Andrew Boshi Li

Page 6: Nonlinear Integer Programming€¦ · 2.4.1 Linear integer program 29 2.4.2 Linearly constrained separable convex integer program 31 2.4.3 Unconstrained convex integer program 32

To my parents, Zhiping Sun and Cuifang Lan, my wife, Huijue Yang, and my son,

Lei Sun

Page 7: Nonlinear Integer Programming€¦ · 2.4.1 Linear integer program 29 2.4.2 Linearly constrained separable convex integer program 31 2.4.3 Unconstrained convex integer program 32

Contents

Dedication v List of Figures xiii List of Tables xvi Preface xix Acknowledgments xxii

1. INTRODUCTION 1

1.1 Classification of Nonlinear Integer Programming Formulations 2

L2 Examples of Applications 4 1.2.1 Resource allocation in production planning 4 L2.2 Portfolio selection 4 1.2.3 Redundancy optimization in reliability networks 6 1.2.4 Chemical engineering 7

1.3 Difficulties and Challenges 8

1.4 Organization of the Book 10

1.5 Notes 11

2. OPTIMALITY, RELAXATION AND GENERAL SOLUTION PROCEDURES 13

2.1 Optimality Condition via Bounds 14

2.2 Partial Enumeration 14 2.2.1 Outline of the general branch-and-bound method 15 2.2.2 The back-track scheme 17

2.3 Continuous Relaxation and Lagrangian Relaxation 24 2.3.1 Continuous relaxation 24 2.3.2 Lagrangian relaxation 25

Page 8: Nonlinear Integer Programming€¦ · 2.4.1 Linear integer program 29 2.4.2 Linearly constrained separable convex integer program 31 2.4.3 Unconstrained convex integer program 32

viii NONLINEAR INTEGER PROGRAMMING

2.3.3 Continuous bound versus Lagrangian bound 26

2.4 Proximity between Continuous Solution and Integer Solution 28 2.4.1 Linear integer program 29 2.4.2 Linearly constrained separable convex integer program 31 2.4.3 Unconstrained convex integer program 32

2.5 Penalty Function Approach 37

2.6 Optimality Conditions for Unconstrained Binary Quadratic Problems 38

2.6.1 General case 38 2.6.2 Convex case 42

2.7 Notes 44

3. LAGRANGIAN DUALITY THEORY 45

3.1 Lagrangian Relaxation and Dual Formulation 45

3.2 Dual Search Methods 52 3.2.1 Subgradient method 52 3.2.2 Outer Lagrangian linearization method 57 3.2.3 Bundle method 68

3.3 Perturbation Function 70

3.4 Optimal Generating Multiplier and Optimal Primal-Dual Pair 77

3.5 Solution Properties of the Dual Problem 83

3.6 Lagrangian Decomposition via Copying Constraints 88 3.6.1 General Lagrangian decomposition schemes 88 3.6.2 0-1 quadratic case 91

3.7 Notes 95

4. SURROGATE DUALITY THEORY 97

4.1 Conventional Surrogate Dual Method 97 4.1.1 Surrogate dual and its properties 97 4.1.2 Surrogate dual search 99

4.2 Nonlinear Surrogate Dual Method 104

4.3 Notes 112

5. NONLINEAR LAGRANGIAN AND STRONG DUALITY 113

5.1 Convexification and Nonlinear Support: p-th power Nonlinear Lagrangian Formulation 113

5.2 Nonlinear Lagrangian Theory Using Equivalent Reformulation 118 5.3 Nonlinear Lagrangian Theory Using Logarithmic-Exponential

Dual Formulation 127

Page 9: Nonlinear Integer Programming€¦ · 2.4.1 Linear integer program 29 2.4.2 Linearly constrained separable convex integer program 31 2.4.3 Unconstrained convex integer program 32

Contents ix

5.4 Generalized Nonlinear Lagrangian Theory for Singly-Constrained Nonlinear Integer Programming Problems 137

5.5 Notes 148

6. NONLINEAR KNAPSACK PROBLEMS 149

6.1 Continuous-Relaxation-Based Branch-and-Bound Methods 150 6.1.1 Multiplier search method 151 6.1.2 Pegging method 157

6.2 0-1 Linearization Method 159 6.2.1 0-1 linearization 159 6.2.2 Algorithms for 0-1 linear knapsack problem 161

6.3 Convergent Lagrangian and Domain Cut Algorithm 164 6.3.1 Derivation of the algorithm 165 6.3.2 Domain cut 170 6.3.3 The main algorithm 173 6.3.4 Multi-dimensional nonlinear knapsack problems 176

6.4 Concave Nonlinear Knapsack Problems 181 6.4.1 Linear approximation 181 6.4.2 Domain cut and linear approximation method 184

6.5 Reliability Optimization in Series-Parallel Reliability Networks 188

6.5.1 Maximal decreasing property 190

6.6 Implementation and Computational Results 195 6.6.1 Test problems 196 6.6.2 Heuristics for feasible solutions 198 6.6.3 Numerical results of Algorithm 6.2 for singly constrained

cases 199 6.6.4 Numerical results of Algorithm 6.2 for multiply constrained

cases 201 6.6.5 Numerical results of Algorithm 6.3 202 6.6.6 Comparison results 203

6.7 Notes 206

7. SEPARABLE INTEGER PROGRAMMING 209

7.1 Dynamic Programming Method 209 7.1.1 B ackward dynamic programming 210 7.1.2 Forward dynamic programming 211 7.1.3 Singly constrained case 215

7.2 Hybrid Method 217

Page 10: Nonlinear Integer Programming€¦ · 2.4.1 Linear integer program 29 2.4.2 Linearly constrained separable convex integer program 31 2.4.3 Unconstrained convex integer program 32

X NONLINEAR INTEGER PROGRAMMING

7.2.1 Dynamic programming procedure 218 7.2.2 Incorporation of elimination procedure 219 7.2.3 Relaxation of (RSPk) 221

7.3 Convergent Lagrangian and Objective Level Cut Method 224 7.3.1 Motivation 224 7.3.2 Algorithm description 227 7.3.3 Implementation of dynamic programming 233 7.3.4 Computational experiment 236

7.4 Notes 238

8. NONLINEAR INTEGER PROGRAMMING WITH A QUADRATIC OBJECTIVE FUNCTION 241

8.1 Quadratic Contour Cut 241 8.1.1 Ellipse of quadratic contour 242 8.1.2 Contour cuts of quadratic function 243

8.2 Convergent Lagrangian and Objective Contour Cut Method 245

8.3 Extension to Problems with Multiple Constraints 250 8.4 Extension to Problems with Indefinite q 25A 8.5 Computational Results 257

8.5.1 Test problems 258 8.5.2 Computational results 260 8.5.3 Comparison with other methods 262

8.6 Note 264

9. NONSEPARABLE INTEGER PROGRAMMING 265

9.1 Branch-and-Bound Method based on Continuous Relaxation 265 9.1.1 Branching variables 267 9.1.2 Branching nodes 268

9.2 Lagrangian Decomposition Method 268

9.3 Monotone Integer Programming 272 9.3.1 Discrete polyblock method for {MIP) 273 9.3.2 Convexity and monotonicity 277 9.3.3 Equivalent transformation using convexification 281 9.3.4 Polyblock and convexification method for (MIP) 284 9.3.5 Computational results 286

9.4 Notes 290

Page 11: Nonlinear Integer Programming€¦ · 2.4.1 Linear integer program 29 2.4.2 Linearly constrained separable convex integer program 31 2.4.3 Unconstrained convex integer program 32

Contents xi

10. UNCONSTRAINED POLYNOMIAL 0-1 OPTIMIZATION 293

10.1 Roof Duality 293 10.1.1 Basic concepts 294 10.1.2 Relation to other linearization formulations 297 10.1.3 Quadratic case 299

10.2 Local Search 300

10.3 Basic Algorithm 301

10.4 Continuous Relaxation and its Convexification 304

10.5 Unconstrained Quadratic 0-1 Optimization 306 10.5.1 A polynomially solvable case 307 10.5.2 Equivalence to maximum-cut problem 308 10.5.3 Variable fixation 309

10.6 Notes 313

11. CONSTRAINED POLYNOMIAL 0-1 PROGRAMMING 315

11.1 Reduction to Unconstrained Problem 315

11.2 Linearization Methods 317

11.3 Branch-and-Bound Method 319 11.3.1 Upper bounds and penalties 319 11.3.2 Branch-and-bound method 320

11.4 Cutting Plane Methods 321 11.4.1 Generalized covering relaxation 321 11.4.2 Lower bounding linear function 324 11.4.3 Linearization of polynomial inequality 326

11.5 Quadratic 0-1 Knapsack Problems 328 11.5.1 Lagrangian dual of {QKP) 328 11.5.2 Heuristics for finding feasible solutions 335 11.5.3 Branch-and-bound method 339 11.5.4 Alternative upper bounds 341

11.6 Notes 348

12. TWO LEVEL METHODS FOR CONSTRAINED POLYNOMIAL 0-1 PROGRAMMING 349

12.1 Revised Taha's Method 349 12.1.1 Definitions and notations 350 12.1.2 Fathoming, consistency and augmentation 352 12.1.3 Solution algorithm 357

Page 12: Nonlinear Integer Programming€¦ · 2.4.1 Linear integer program 29 2.4.2 Linearly constrained separable convex integer program 31 2.4.3 Unconstrained convex integer program 32

xii NONLINEAR INTEGER PROGRAMMING

12.2 Two-Level Method for p-Norm Surrogate Constraint Formulation 360

12.3 Convergent Lagrangian Method Using Objective Level Cut 368

12.4 Computational Results 370

12.5 Notes 371

13. MIXED-INTEGER NONLINEAR PROGRAMMING 373

13.1 Introduction 373

13.2 Branch-and-Bound Method 375

13.3 Generalized Benders Decomposition 377

13.4 Outer Approximation Method 382

13.5 Nonconvex Mixed-Integer Programming 389 13.5.1 Convex relaxation 390 13.5.2 Convexification method 393

13.6 Notes 394

14. GLOBAL DESCENT METHODS 397 14.1 Local Search and Global Descent 398

14.1.1 Local minima and local search 398 14.1.2 Identification of global minimum from among local

minima 400

14.2 A Class of Discrete Global Descent Functions 400 14.2.1 Condition (Dl) 403 14.2.2 Condition (D2) 403 14.2.3 Condition (D3) 405

14.3 The Discrete Global Descent Method 410

14.4 Computational Results 412

14.5 Notes 416

References 419

Index 433

Page 13: Nonlinear Integer Programming€¦ · 2.4.1 Linear integer program 29 2.4.2 Linearly constrained separable convex integer program 31 2.4.3 Unconstrained convex integer program 32

List of Figures

1.1 The ARPA complex system (n = 7). 7 1.2 Superstructure of synthesization of a chemical process [53]. 8 2.1 Diagram of the general branch-and-bound method. 16 2.2 Diagram of the back-track scheme. 19 2.3 Diagram of the additive algorithm of Balas. 22 3.1 Illustration of the solution scheme for (SPj). 50 3.2 Error bound (i(A*) — d{X^) in the subgradient method

with Rule 1 for stepsize for Example 3.1. 57 3.3 Error bound (i(A*) - v^ in the subgradient method with

Rule 1 for stepsize for Example 3.1. 58

3.4 Error bound (i(A*) — d{X^) in the subgradient method with Rule 2 for stepsize for Example 3.1. 58

3.5 Error bound d(A*) — v^ in the subgradient method with Rule 2 for stepsize for Example 3.1. 59

3.6 Error bound (i(A*) — d{X^) in the subgradient method with Rule 3 for stepsize for Example 3.1. 59

3.7 Error bound (i(A*) — v^ in the subgradient method with Rule 3 for stepsize for Example 3.1. 60

3.8 Dual function of Example 3.3. 64 3.9 Perturbation function of Example 3.4. 72 3.10 Illustration of perturbation function w and decomposi­

tion of y . 73 3.11 Perturbation function where there exists no subgradient

of t/; at y = p(x*). 80 3.12 Perturbation function where there exists a subgradient

of to at y = p(x*). 81

Page 14: Nonlinear Integer Programming€¦ · 2.4.1 Linear integer program 29 2.4.2 Linearly constrained separable convex integer program 31 2.4.3 Unconstrained convex integer program 32

xiv NONLINEAR INTEGER PROGRAMMING

4.1 Surrogate constraints in the constraint space of Example 4.1. 105 4.2 p-norm surrogate constraints in the constraint space with

/i = (0,4,0.6)^ and b = (3, 2)^. 107 4.3 p-norm surrogate constraints in the constraint space of

Example 4.1 with/x - (0.6316,0.3684)^. I l l 5.1 Set 5 in {xi,X2} space. 114 5.2 Set Sp in {x\, X2} space. 114 5.3 Illustration of Wp{y) and ipp{y) for Example 3.4 with

p - 2 . 117 5.4 Contours of z^ + X^{y'P — bP) = r with different values

of p. 118 5.5 wip and ipip of Example 3.4 with p = 3. 125 5.6 Illustration of w{y) and '^(y) for Example 3.4 with b — 3.5. 125 5.7 Illustration of wip{y) and ipip{y) for Example 3.4 with

p = 2, 127 5.8 Illustration of the Lagrangian dual search for Example

5.3. 129 5.9 Illustration of the contour CM{Z^ X) = a for Example

5.3 ( A - 8 , a - : 1.8). 130 5.10 Behavior of Cp(2:, X) = a {p = 0.7, a — 1) for various A. 132 5.11 Behavior of Cp(^, A) = a (A = 0.7, a = 1) for various p. 132 5.12 Contour Cp{z,A) = a for Example 5.3 (p — 1.2, a =

Cp(P*,4) = 1.2798). 133 6.1 Linear approximation of fj(xj). 160 6.2 Linear Approximation of gj{xj). 160 6.3 Domain X and the feasible region of Example 6.4. 166 6.4 Perturbation function z — w{y) with domain X of Example 6.4.167 6.5 Domain X^ in Example 6.4. 167 6.6 Perturbation function with domain X^ of Example 6.4. 168 6.7 Perturbation function with domain Xl of Example 6.4. 168 6.8 Perturbation function with domain X^ of Example 6.4. 169 6.9 Partition of A \ J5 . 170 6.10 Structural diagram of the convergent Lagrangian and

domain cut method under a branch-and-bound framework. 175 6.11 Domain X and the feasible region of Example 6.5. 179 6.12 A series system with n components. 189 6.13 A series system with redundancies. 189 7.1 The perturbation function of Example 7.4. 225

Page 15: Nonlinear Integer Programming€¦ · 2.4.1 Linear integer program 29 2.4.2 Linearly constrained separable convex integer program 31 2.4.3 Unconstrained convex integer program 32

List of Figures xv

7.2 The perturbation function of problem (7.3.1). 226 7.3 The perturbation function of problem (7.3.2). 227 8.1 Contour cuts for case (a). 244 8.2 Contour cuts for case (b). 245 8.3 Perturbation function of Example 8.1. 246 8.4 Domain X and the objective contour cuts. 247 8.5 The revised domain X ^ 248 8.6 Perturbation function of the revised problem on X^, 248 8.7 S e t Z i - X \ Q i . 258 8.8 Sets X^ = Y^\ {x2} and Q2{xi). 258 9.1 Feasible region of the Example 9.2. 276 9.2 Illustration of Iteration 1 of Algorithm 9,2 for Example 9.2. 276 9.3 The nonconvex function/i(x). 281 9.4 The convexified function ht [y) with t defined in (9.3.11)

andp- :2 . 281 9.5 The convexified function ht (y) with t defined in (9.3.12)

and p = 5, 282 9.6 Feasible region of Example 9.3. 284 9.7 Convexified feasible region of Example 9.3. 284 9.8 A 12-link complex reliabiUty system. 287 12.1 Diagram of revised Taha's method. 358 12.2 Diagram of the p-norm surrogate-constraint algorithm. 365 13.1 Example 13.1 of (MINLP). 375 13.2 Branch-and-bound search tree for Example 13.1. 376 13.3 Branch-and-bound search tree for Example 13.3. 378 14.1 Illustrations of V^{y), V^{y), Aj,{y) = y • V^{y) and

^^fiiy) = y ' ^^{y) i^ Example 14.2 with c = /i == 0.5 andr-=:l . 402

14.2 Illustration of the discrete global descent function. 409

Page 16: Nonlinear Integer Programming€¦ · 2.4.1 Linear integer program 29 2.4.2 Linearly constrained separable convex integer program 31 2.4.3 Unconstrained convex integer program 32

List of Tables

1.1 Multiple discrete local minimizers of a convex function, 9 6.1 Values of Pm (z) in dynamic programming for Example

6.3, 165 6.2 List of noninferior points for a 4-subsystem problem. 194 6.3 Comparison of node selection rules (n = 200, m = 1), 200 6.4 Numerical results for (QPi) with single constraint, 200 6.5 Numerical results for {QP2) with single constraint. 201 6.6 Numerical results for {POLY) with single constraint. 201 6.7 Numerical results for {SAMP) with single constraint. 201 6.8 Numerical results for {LCROP) with single constraint. 202 6.9 Numerical results for {LCOST) with single constraint. 202 6.10 Numerical results for {QPi) with multiple constraints. 203 6.11 Numerical results for {QP2) with multiple constraints. 203 6.12 Numerical results for {POLY) with multiple constraints. 203 6.13 Numerical results for {SAMP) with multiple constraints. 204 6.14 Numerical results for {LCROP) with multiple constraints. 204 6.15 Numerical results for {CCKP), 204 6.16 Comparison results for convex knapsack problems. 205 6.17 Comparison results for nonconvex knapsack problems. 206 6.18 Comparison results for multidimensional knapsack problems. 207 7.1 Solution process for Example 7,1 using backward dynamic

programming. 213 7.2 Solution process for Example 7.1 using forward dynamic

programming. 214 7.3 Dynamic programming table for Example 7,2. 217

Page 17: Nonlinear Integer Programming€¦ · 2.4.1 Linear integer program 29 2.4.2 Linearly constrained separable convex integer program 31 2.4.3 Unconstrained convex integer program 32

xvii NONLINEAR INTEGER PROGRAMMING

7.4 Domination and upper bounds at Iteration 1 of Algorithm 7.1 for Example 7.3. 223

7.5 Domination and upper bounds at Iteration 2 of Algorithm 7.1 for Example 7,3. 224

7.6 Iteration process of Example 7.5. 235 7.7 Numerical results for {PIP) (r = 0.62). 238 7.8 Numerical results for (QIPi) (r - 0.62). 238 7.9 Numerical results for (QIPLi) (r = 0.65). 238 7.10 Numerical results for {QIP2) (r = 0.70). 239 7.11 Numerical results for {QIPL2) (r - 0.70). 239 8.1 Coefficients in the test problems for Algorithm 8.1 and

its extensions. 259 8.2 Numerical results for convex q{x) (r = 0.6). 261 8.3 Numerical results for concave q{x) (r — 0.6). 261 8.4 Numerical results for indefinite q{x) (r — 0.6). 262 8.5 Numerical results for problem {SMV), 262 8.6 Comparison results for convex problems. 263 8.7 Comparison results for nonconvex problems. 264 9.1 Numerical results for test problems with a polynomial

objective function. 288

9.2 Numerical results for test problems with a quadratic objective function. 288

9.3 Numerical results for test problems with a minimax ob­jective function. 288

9.4 Numerical results for the reliability optimization problem of the 12-link complex network. 289

9.5 Comparison results for problems with a minimax ob­jective function {n ~ 15) for different m, 289

9.6 Comparison results of the discrete polyblock method. 290 10.1 Illustrative example of mapping gk. 303 12.1 Numerical results with the revised Taha's algorithm (r =

0.5). 371 12.2 Numerical results with the objective level cut algorithm

(r - 0.5). 372 12.3 Numerical results with the p-th power surrogate algorithm

(r = 0.5). 372 13.1 Summary of the branch-and-bound method for Example 13.3. 377 13.2 Solution process of the GBD method for Example 13.3. 383

Page 18: Nonlinear Integer Programming€¦ · 2.4.1 Linear integer program 29 2.4.2 Linearly constrained separable convex integer program 31 2.4.3 Unconstrained convex integer program 32

List of Tables xviii

13.3 Solution process of the OA method for Example 13.3. 389 14.1 Ui and U2 local minimizers for Example 1.1. 399 14.2 Numerical results for Problems 14.1-14.5. 417

Page 19: Nonlinear Integer Programming€¦ · 2.4.1 Linear integer program 29 2.4.2 Linearly constrained separable convex integer program 31 2.4.3 Unconstrained convex integer program 32

Preface

It is not an exaggeration that much of what people devote in their hfe re­solves around optimization in one way or another. On one hand, many decision making problems in real applications naturally result in optimization problems in a form of integer programming. On the other hand, integer programming has been one of the great challenges for the optimization research community for many years, due to its computational difficulties: Exponential growth in its computational complexity with respect to the problem dimension. Since the pioneering work of R. Gomory [80] in the late 1950s, the theoretical and methodological development of integer programming has grown by leaps and bounds, mainly focusing on linear integer programming. The past few years have also witnessed certain promising theoretical and methodological achieve­ments in nonlinear integer programming.

When the first author of this book was working on duality theory for non-convex continuous optimization in the middle of 1990s, Prof. Douglas J. White suggested that he explore an extension of his research results to integer pro­gramming. The two authors of the book started their collaborative work on integer programming and global optimization in 1997. The more they have investigated in nonlinear integer programming, the more they need to further delve into the subject. Both authors have been greatly enjoying working in this exciting and challenging field.

Applications of nonlinear (mixed) integer programming can be found in various areas of scientific computing, engineering, management science and operations research. Its prominent applications include, for example, portfolio selection, capital budgeting, production planning, resource allocation, computer networks, rehability networks and chemical engineering. Due to nonlinearity, theory and solution methodologies for nonlinear integer programming problems are substantially different from the linear integer programming.

There are numerous books and monographs on linear integer programming [68][109][116][168] [191][188][212][222] [228]. In contrast, there is no book

Page 20: Nonlinear Integer Programming€¦ · 2.4.1 Linear integer program 29 2.4.2 Linearly constrained separable convex integer program 31 2.4.3 Unconstrained convex integer program 32

XX NONLINEAR INTEGER PROGRAMMING

that comprehensively discusses theory and solution methodologies for gen­eral nonlinear integer programming, despite its importance in real world ap­plications and its academic significance in optimization. The book by Ibaraki and Katoh [106] systematically describes solution algorithms for resource al­location problems, a special class of nonlinear integer programming problems. Sherali and Adams, in their book [196], develop a reformulation-linearization technique for constructing a convex hull of a nonconvex domain. Floudas, in his book [60], develops lower-bounding convexification schemes for nonlin­ear mixed integer programming problems with apphcations in chemical engi­neering. Tawarmalani and Sahinidis [213] also investigate convexification and global optimization methods for nonUnear mixed integer programming.

This book addresses the topic of general nonlinear integer programming. The overall goal of this book is to bring the state-of-the-art of the theoretical foundation and solution methods for nonlinear integer programming to readers who are interested in optimization, operations research and computer science. Of note, recent theoretical progress and innovative methodologies achieved by the authors are presented. This book systematically investigates theory and solution methodologies for general nonlinear integer programming and, at the same time, provides a timely and comprehensive summary of the theoretical and algorithmic development in the last 30 years on this topic.

We assume that readers are already famihar with some basic knowledge of linear integer programming. The book thus focuses on the theory and solution methodologies of nonlinear integer programming. The following are some features of the book:

• Duality theory for nonlinear integer programming: Investigation of the re­lationship between the duality gap and the perturbation function has led to the development of the novel nonlinear Lagrangian theory, thus establish­ing a theoretical foundation for solution methodologies of nonlinear integer programming.

• Convergent Lagrangian and cutting methods for separable nonlinear integer programming problems: Performing objective-level cut, objective contour cut or domain cut reshapes the perturbation function, thus exposing eventu­ally an optimal solution to the convex hull of a revised perturbation function and guaranteeing a zero duality gap for a convergent Lagrangian method.

• Convexification scheme: The relationship between the monotonicity and convexity has been explored. Convexification schemes have been developed for monotone and nonconvex integer programming problems, thus extending the reach of branch-and-bound methods whose success depends on an ability to achieve a global solution of the continuous relaxation problem.

Page 21: Nonlinear Integer Programming€¦ · 2.4.1 Linear integer program 29 2.4.2 Linearly constrained separable convex integer program 31 2.4.3 Unconstrained convex integer program 32

PREFACE xxi

• A solution framework using global descent: The exact solution of a nonlin­ear integer programming problem is sought from among the local minima. A theoretical basis has been established to escape from the current minimum to a better minimum in an iterative global descent process.

• Computational implementation for large-scale nonlinear integer program­ming problems with dimensions up to several thousands is demonstrated for several efficient solution algorithms presented in the book.

Readers of this book can be researchers, practitioners, graduate students and senior undergraduate students in operations research and computer science. This book aims at people in academics as well as people in applied areas who already have basic knowledge of optimization and want to broaden their knowl­edge in integer programming. It can be used as a textbook for graduate students in the fields of operations research, management science and computer science. It can be also used as a reference book for researchers, engineers and practition­ers to solve real-world application problems by nonlinear integer programming models and to design and implement sophisticated algorithms for their specific application problems.

Page 22: Nonlinear Integer Programming€¦ · 2.4.1 Linear integer program 29 2.4.2 Linearly constrained separable convex integer program 31 2.4.3 Unconstrained convex integer program 32

Acknowledgments

We appreciate many individuals for their contributions to the materials pre­sented in this book. Especially, we thank Dr. Jun Wang for his contributions in the convergent Lagrangian and objective level cut method and the revised Taha's method, Dr. Kevin Ng and Prof. Liansheng Zhang for their contribu­tions in global descent methods, Prof. Douglas J. White for his contributions in development of the p-ih power Lagrangian, Prof, Yifan Xu and Ms. Chunli Liu for their contributions in nonlinear Lagrangian theory. Prof. Ken McKinnon for his contributions in the convergent Lagrangian and domain cut method, Ms. Fenian Wang for her contribution in the convergent Lagrangian and contour cut method, and Dr. Zhiyou Wu for her help in deriving Proposition 12.1.

We would like to express our gratitude to Dr. Kevin Ng, Mr. Hezhi Luo, Mr. Hongbo Sheng, Ms. Fei Gao, Ms. Chunli Liu, Ms. Fenian Wang, Ms. Jianling Li, Ms. Ning Ruan and Ms. Juan Sun for their careful proofreading of the manuscript of our book.

We are grateful to our institutes. The Chinese University of Hong Kong and Shanghai University, for providing us with excellent working environments. We are grateful for the financial support to our research by the Research Grants Council of Hong Kong and the National Natural Science Foundation of China. The first author is grateful to Prof. Yacov Y, Haimes for the guidance and the all-round support during his career development in the past twenty years.

We are also very thankful for the encouragement from Prof. Frederick S. Hillier and the professional support from Mr. Gary Folven and other staff at Springer.

Finally, we owe a great debt of thanks to our families for their lasting under­standing and support.

Page 23: Nonlinear Integer Programming€¦ · 2.4.1 Linear integer program 29 2.4.2 Linearly constrained separable convex integer program 31 2.4.3 Unconstrained convex integer program 32

Chapter 1

INTRODUCTION

Most of the contents of this book deal with the following general class of nonlinear integer programming problems:

{NLIP) min f{x)

s.t. gi{x) <bi, i = l,..,,m,

X e X,

where / and gi,i = \,., .,m, are real-valued functions on R^, and X is a finite subset in Z^, the set of all integer points in M .

While problem {NLIP) is a nonlinear pure integer programming problem, Chapter 13 of this book deals with the following mixed-integer nonlinear pro­gramming problem

{MINLP) min f{x,y)

s.t. gi{x,y) <bi, i = l,,,.,m,

xeX, yeY,

where / and gi,i = I, ..., m, are real-valued functions on R' ' , X is a finite subset in Z^, and Y" is a continuous subset in R^.

When all functions / and gi, i = I, ..., m, are linear, problems {NLIP) and {MINLP) reduce to linear integer programming and mixed integer linear programming problems, respectively. The focus of this book is on nonlinear integer programming, which implies that at least one function of / and gi.i-1, ..., 771, is nonlinear.

Page 24: Nonlinear Integer Programming€¦ · 2.4.1 Linear integer program 29 2.4.2 Linearly constrained separable convex integer program 31 2.4.3 Unconstrained convex integer program 32

2 NONLINEAR INTEGER PROGRAMMING

1.1 Classification of Nonlinear Integer Programming Formulations

Problem (NLIP) can be classified into different subclasses according to its special structure.

Unconstrained Nonlinear Integer Programming. When the inequality con­straints in {NLIP) are absent, the problem is called unconstrained nonlinear integer programming problem. Two important classes of the unconstrained nonhnear integer programming problems are unconstrained polynomial O-I op­timization problems and unconstrained quadratic 0-1 optimization problems.

Singly Constrained Nonlinear Integer Programming, When m = 1, i.e., there is only one constraint in {NLIP), the problem is called a singly con­strained nonlinear integer programming problem.

Multiply Constrained Nonlinear Integer Programming. When m > 2, problem {NLIP) is called a multiply constrained nonlinear integer program­ming problem. It will be revealed later in this book that there exist essential differences between singly- and multiply-constrained nonlinear integer pro­gramming problems.

Convex Integer Programming. If all functions / and ^ , i = 1, ... , m, are convex on the convex hull of X in problem {NLIP), problem {NLIP) is called a convex integer programming problem. Note that the convexity is a sufficient condition to obtain a global solution to the continuous relaxation of {NLIP).

Separable Integer Programming. When a function is of an additive form with respect to all of its variables, the function is called separable. In many situations, the objective function and the constraint functions of {NLIP) are separable ([44][45]). A separable nonlinear integer programming formulation of {NLIP) takes the following form:

(SIP) mmf{x) = J2M^j)

n

s.t. gi{x) = ^ ^ i j ( x j ) < 6i, z = l , . . . , m ,

X e X = {x e Z^ \ Ij < Xj < Uj^ j = 1 , . . . , n}.

Nonlinear resource allocation problem is a special case of {SIP) where all fj 's and Qij 's are convex functions. In many nonlinear resource allocation problems, only a single constraint is presented in {SIP) with the fovmg{x) = Yl^=i ^j — N (see [ 106]). If all fj's in {SIP) are of a quadratic form of Xj, fj {xj) = QJx'j

Page 25: Nonlinear Integer Programming€¦ · 2.4.1 Linear integer program 29 2.4.2 Linearly constrained separable convex integer program 31 2.4.3 Unconstrained convex integer program 32

Introduction 3

+ CjXj, then (SIP) is classified as a separable nonlinear integer programming problem with a quadratic objective function.

Nonseparable Integer Programming, When at least one of the objective function and the constraint functions of (NLIP) is nonseparable, problem (NLIP) is a nonseparable integer programming problem. There are many real cases of nonlinear integer programming models where some of the functions involved are nonseparable. For example, in reliability optimization, the relia­bility function of an overall system is a multi-linear function of the reliability levels of all individual subsystems.

Nonlinear Knapsack Problem. If in a separable nonlinear integer program­ming problem {SIP), all /j's are nonincreasing while all gij's are nondecreas-ing, then the problem is called a nonlinear knapsack problem. If in a nonlinear knapsack problem, all fj 's are concave and all gij 's are linear, then the problem is called concave knapsack problem.

Monotone Nonlinear Integer Programming. If in a nonseparable integer programming problem (NLIP), f is nonincreasing while all gi's are nonde-creasing, then the problem is called a monotone nonlinear integer programming problem or a nonseparable knapsack problem.

Nonlinear 0-1 Programming. When all the integer variables x / s are re­stricted to be 0 or 1 in {NLIP), problem {NLIP) is called a nonlinear 0-1 programming problem. Theoretically, any integer programming problem can be reduced to a 0-1 integer programming problem ([31][92]). The methodolo­gies for solving nonlinear 0-1 programming problems are inherently different from methods for other problem formulations.

Polynomial 0-1 Programming. If in a nonlinear 0-1 programming formula­tion, all functions / and gi's are of a multi-linear polynomial form:

n

J = l

K

fc=l n

i^S{k)

^it

where S{k) is an index set with 15(^)1 > 2, then the problem is called a poly­nomial 0-1 programming problem ox pseudo-Boolean optimization problem. In particular, if / and ^^'s are of the following form of quadratic functions:

n

/ V ^3 ^3 + 2^ ^^^ XiXj^ j=l '^<i<3<n

then the problem is called a quadratic 0-1 programming problem. The poly­nomial 0-1 programming and quadratic 0-1 programming problems have been extensively studied over the last thirty years (see [31][92][95]).

Page 26: Nonlinear Integer Programming€¦ · 2.4.1 Linear integer program 29 2.4.2 Linearly constrained separable convex integer program 31 2.4.3 Unconstrained convex integer program 32

4 NONLINEAR INTEGER PROGRAMMING

1.2 Examples of Applications Integer programming has its root in various real applications. We present in

this section some nonlinear integer programming models arising from different application areas.

1.2.1 Resource allocation in production planning Optimal lot sizing is often sought in production planning in order to minimize

the total cost via optimal resource allocation of labor and machine-hour among n different items. Let Xj denote the lot size of item j , Dj the total demand of item j , Oj the ordering cost per order of item j , hj the holding cost per period of item j , Cj the storage requirement per item j , and C the total storage capacity. Then (i) the term OjDj/xj represents the total ordering cost of item j since item j is ordered Dj/xj times; and (ii) the term hjXj/2 gives the average holding cost of item j . The optimal lot size problem can be then formulated as

n

(OL) min J2(0jDj/xj + hjXj/2)

n S.t. \ ^ CjXj < C

xezi,

where Z!f: denotes the set of integer points in W^, Notice that problem (OL) is a separable convex integer programming problem.

1.2.2 Portfolio selection Portfolio selection is to seek a best allocation of wealth among a basket of

securities. Quantifying the investment risk by the variance of the random return of the portfolio, the mean-variance formulation proposed by Markowitz [150] in the 1950s provides a fundamental basis for portfolio selection.

The trade practice often only allows trade of integer lots of stocks. Consider a market with n available securities where the purchasing of the securities is confined to integer number of lots. An investor with initial wealth WQ seeks to improve his wealth status by investing his wealth into these n risky securities and into a risk-free asset (e.g., a bank account). Let Xi be the random return per lot of the i-th security (i = 1 , . . . , n) before deducting associated transaction costs. The mean and covariance of the returns are assumed to be known,

fii = E{Xi), and aij = Cov(X^,Xj), ij = l,,..,n.

Let Xi be the integer number of lots the investor invests in the i-ih security. Denote the decision vector in portfoUo selection by a: = ( x i , . . . , Xn)^- Then,

Page 27: Nonlinear Integer Programming€¦ · 2.4.1 Linear integer program 29 2.4.2 Linearly constrained separable convex integer program 31 2.4.3 Unconstrained convex integer program 32

Introduction 5

the random return from holding securities is Ps{x) = ^^1=1 ^i^i- The mean and variance of Ps{x) are

n n

s{x) - E[Ps{x)] = ElY^XiXi] = J^i^iXi i=l i=l

and

n n n

V{x) — Var(Ps(x)) — Var[^XiXi] == ^^^^XiXjcrjj ~ x^Cx,

where C = {o'ij)nxn is the covariance matrix. Let r be the interest rate of the risk-free asset. Assume that the same rate is appHed when borrowing money from the risk-free asset. Let bi be the current price of one lot of the i-th security. The balance XQ = WQ — Yl^=i ^i^i ^^ assumed to be deposited into the risk-free asset and rxo is the corresponding return. Note that a negative XQ implies a debt from the risk-free asset.

The budget constraint of the investor is given by

b^x <Wo + Ub

where b — (61 , . . . , bn)^ and Ut is the upper borrowing limit from the risk-free asset. Let c{x) = Y17=i ^ii^i) ^^ he transaction cost associated with the portfoUo decision x = ( x i , . . . , x^)^. It is always assumed in the Hterature that each Ci(-) is a nondecreasing concave function.

The total expected return of portfolio decision x can be now summarized as:

n n

R{x) = s{x) + rxo - ^ Ci{xi) = ^ [ ( M Z - rbi)xi - Ci{xi)] + TWQ,

Note that R{x) is a convex function since each Ci{xi) is a concave function. In most situations, an investor would like to invest his wealth only to a

limited number of stocks. Thus a cardinality constraint is often necessary to be considered in portfolio selection,

supp(x) < K^

where supp(x) denotes the number of nonzero components in x and K is ?i given positive integer with K <n.

By introducing n zero-one variables, yi, i = I, ..., n, d. discrete-feature constrained mean-variance model can be formulated as follows for an investor who would like to minimize his investment risk while attaining an expected return level higher than a given value, e, under transaction costs and a cardinality

Page 28: Nonlinear Integer Programming€¦ · 2.4.1 Linear integer program 29 2.4.2 Linearly constrained separable convex integer program 31 2.4.3 Unconstrained convex integer program 32

6 NONLINEAR INTEGER PROGRAMMING

constraint:

{MV) min V{x) = x^Cx n

s.t. R{x) = ^ [ ( / i i - rhi)xi - Ci{xi)\ + rW^ > e, i=i

U{x) = b^x <Wo + Ut, n

1=1

X e X ~ {x eZ^ \ kyi <Xi< UiUi, i = 1,2,..., n},

ye{0,ir,

where k and Ui are lower and upper bounds on purchasing the i-th security, respectively, A negative k implies that short selling is allowed. Upper bound Ui is either imposed by the investor or can be set as the largest integer number less than or equal to " " ^ .

Problem (MV) is of a nonhnear nonconvex integer programming formula­tion. Varying the value of s, the efficient frontier in the mean-variance space can be traced out which provides a valuable decision-aid for investors.

1.2.3 Redundancy optimization in reliability networks Systems reliability plays an important role in systems design, operation and

management. Systems reliability can be improved by adding redundant com­ponents to subsystems.

Assume that there are n subsystems in a network. Let n (0 < n < 1) be a fixed value of component reliabiUty in the i-th subsystem and Xi represent the number of redundant (parallel) components in the i-th subsystem. Then, the reliabihty of the i-th subsystem, Ri, is given as follows:

Ri(xi) = 1 - {1 - rif\ z - l , . . . , n .

Let X = ( x i , . . . , Xn)^ be the decision vector for the redundancy assignment. The overall system reliability, Rs{x), is in general a nonlinear increasing func­tion of Ri{xi), . . . , Rn{xn)' For example, if the network is the 7-link ARPA complex system given in Figure 1.1, then we have

Rs = RQRI 4- RiR2R?>{Q^ + RQQI) 4- RIRAR7QQ{Q2 + ^2^3)^

where Qi =1 — Ri,i = I, .. .,n. Determination of the optimal amount of redundancy among various sub­

systems under hmited resource constraints leads to a nonseparable nonlinear

Page 29: Nonlinear Integer Programming€¦ · 2.4.1 Linear integer program 29 2.4.2 Linearly constrained separable convex integer program 31 2.4.3 Unconstrained convex integer program 32

Introduction

Figure 1.1. The ARPA complex system (n — 7).

integer programming problem,

{RELI) max Rs[x) = / ( i ? i (x i ) , . . . , Rn{xn))

, m, s.t. gi{x) = ^ ^ z j ( x j ) < 6 , z == 1 , . . . , :

X e X =^ {x e Z"^ \ 1 < Ij < Xj < Uj, j = 1,..., n},

where gi (x) is the z-th resource consumed; bj is the total available z-th resource, Ij and Uj are lower and upper integer bounds of Xj, respectively. The resource constraints often correspond to the constraints in cost, volume, and weight.

An inherent property in problem (RELI) is that functions Rs and ^ 's are strictly increasing with respect to each variable Xj. Thus, problem (RELI) is a nonconvex nonseparable knapsack problem.

1.2.4 Chemical engineering Chemical engineers often seek at the same time an optimal structure for a

chemical process and the corresponding optimal operating parameters in order to satisfy given design specifications. This often results in nonlinear mixed integer programming formulations at the design stage. Figure 1.2 from [53] presents a superstructure of a chemical process in which all competitive alter­native process configurations have been embedded. A zero-one variable yi is attached to each process unit, while its final value will determine whether or not a process unit is in the final optimal configuration. The continuous variable Xij represents a process parameter such as the flow rate of materials. The objective is to minimize a summation of fixed-charge costs and operation costs, while the constraints correspond to design specifications, topological considerations and physical conservation laws. The nonlinearities in the formulation are often caused by some intrinsic nonlinear input-output relationships of some process units.

Page 30: Nonlinear Integer Programming€¦ · 2.4.1 Linear integer program 29 2.4.2 Linearly constrained separable convex integer program 31 2.4.3 Unconstrained convex integer program 32

NONLINEAR INTEGER PROGRAMMING

3''22 ^ 2 3 '•l'-2A

Figure 1.2. Superstructure of synthesization of a chemical process [53].

1.3 Difficulties and Challenges There is no doubt that the first number system which mankind understood

and utihzed was the integer number system. More specifically, counting fingers (integer number) could be the first step humankind ever took in their long journey in advancing mathematics. In the later development of mathematics, however, focus has been primarily placed on the real number system, mainly due to powerful analytical tools that have been developed for mathematical study under the real number system. Compared to continuous optimization, discrete optimization presents more difficult research tasks, posting great long-standing challenges.

While convexity in continuous optimization guarantees that a local search offers a global solution, this is certainly not the case for discrete optimization or integer programming. To support this argument, let us consider a two-dimensional example:

EXAMPLE 1.1

min f{x) = (x — x)^Q{x — x)

s.t. X G X = {x G Z^ I 0 < XI < 7, 0 < X2 < 6},

where x = (3.1, 2.5)^ and Q = 42.67 -49.41

-49.41 57.38

The global minimizer of this problem is Xgiotai = (6, 5)^ with /{xgiobai) = 1-Since matrix Q is positive definite, there is only one continuous local minimizer X = (3.1, 2.5)^ which is also the global minimizer. An integers G X is defined