Introduction to О ot im um Desian

r _ SP Second Edition

Jasbir S. Arora The University of Iowa

.A i

• • : •

ELSEVIER ACADEMIC

PRESS

Amsterdam Boston Heidelberg London New York Oxford Paris San Diego San Francisco Singapore Sydney Tokyo

Table of Contents

Preface ix

C h a p t e r 1 Introduction to Design 1 1.1 The Design Process 2 1.2 Engineering Design versus Engineering Analysis 4 1.3 Conventional versus Optimum Design Process 4 1.4 Optimum Design versus Optimal Control 6 1.5 Basic Terminology and Notation 7

1.5.1 Sets and Points 7 1.5.2 Notation for Constraints 9 1.5.3 Superscripts/Subscripts and Summation Notation 9 1.5.4 Norm/Length of a Vector 11 1.5.5 Functions 11 1.5.6 U.S.-British versus SI Units 12

C h a p t e r 2 Optimum Design Problem Formulation 15 2.1 The Problem Formulation Process 16

2.1.1 Step 1: Project/Problem Statement 16 2.1.2 Step 2: Data and Information Collection 16 2.1.3 Step 3: Identification/Definition of Design

Variables 16 2.1.4 Step 4: Identification of a Criterion to

Be Optimized 17 2.1.5 Step 5: Identification of Constraints 17

2.2 Design of a Can 18 2.3 Insulated Spherical Tank Design 20 2.4 Saw Mill Operation 22 2.5 Design of a Two-Bar Bracket 24 2.6 Design of a Cabinet 30

2.6.1 Formulation 1 for Cabinet Design 30 2.6.2 Formulation 2 for Cabinet Design 31 2.6.3 Formulation 3 for Cabinet Design 31

2.7 Minimum Weight Tubular Column Design 32 2.7.1 Formulation 1 for Column Design 33 2.7.2 Formulation 2 for Column Design 34

2.8 Minimum Cost Cylindrical Tank Design 35 2.9 Design of Coil Springs 36 2.10 Minimum Weight Design of a Symmetric

Three-Bar Truss 38 2.11 A General Mathematical Model for Optimum Design 41

2.11.1 Standard Design Optimization Model 42 2.11.2 Maximization Problem Treatment 43 2.11.3 Treatment of "Greater Than Type" Constraints 43 2.11.4 Discrete and Integer Design Variables 44 2.11.5 Feasible Set 45 2.11.6 Active/Inactive/Violated Constraints 45

Exercises for Chapter 2 46

C h a p t e r 3 Graphical Optimization 55 3.1 Graphical Solution Process 55

3.1.1 Profit Maximization Problem 55 3.1.2 Step-by-Step Graphical Solution Procedure 56

3.2 Use of Mathematica for Graphical Optimization 60 3.2.1 Plotting Functions 61 3.2.2 Identification and Hatching of Infeasible

Region for an Inequality 62 3.2.3 Identification of Feasible Region 62 3.2.4 Plotting of Objective Function Contours 63 3.2.5 Identification of Optimum Solution 63

3.3 Use of MATLAB for Graphical Optimization 64 3.3.1 Plotting of Function Contours 64 3.3.2 Editing of Graph 64

3.4 Design Problem with Multiple Solutions 66 3.5 Problem with Unbounded Solution 66 3.6 Infeasible Problem 67 3.7 Graphical Solution for Minimum Weight

Tubular Column 69 3.8 Graphical Solution for a Beam Design Problem 69 Exercises for Chapter 3 72

C h a p t e r 4 Optimum Design Concepts 83 4.1 Definitions of Global and Local Minima 84

4.1.1 Minimum 84 4.1.2 Existence of Minimum 89

4.2 Review of Some Basic Calculus Concepts 89 4.2.1 Gradient Vector 90 4.2.2 Hessian Matrix 92 4.2.3 Taylor's Expansion 93 4.2.4 Quadratic Forms and Definite Matrices 96 4.2.5 Concept of Necessary and

Sufficient Conditions 102

xii Contents

4.3 Unconstrained Optimum Design Problems 103 4.3.1 Concepts Related to Optimality Conditions 103 4.3.2 Optimality Conditions for Functions of

Single Variable 104 4.3.3 Optimality Conditions for Functions of

Several Variables 109 4.3.4 Roots of Nonlinear Equations Using Excel 116

4.4 Constrained Optimum Design Problems 119 4.4.1 Role of Constraints 119 4.4.2 Necessary Conditions: Equality Constraints 121 4.4.3 Necessary Conditions: Inequality Constraints—

Karush-Kuhn-Tucker (KKT) Conditions 128 4.4.4 Solution of KKT Conditions Using Excel 140 4.4.5 Solution of KKT Conditions Using MATLAB 141

4.5 Postoptimality Analysis: Physical Meaning of Lagrange Multipliers 143 4.5.1 Effect of Changing Constraint Limits 143 4.5.2 Effect of Cost Function Scaling on

Lagrange Multipliers 146 4.5.3 Effect of Scaling a Constraint on Its

Lagrange Multiplier 147 4.5.4 Generalization of Constraint Variation

Sensitivity Result 148 4.6 Global Optimality 149

4.6.1 Convex Sets 149 4.6.2 Convex Functions 151 4.6.3 Convex Programming Problem 153 4.6.4 Transformation of a Constraint 156 4.6.5 Sufficient Conditions for Convex

Programming Problems 157 4.7 Engineering Design Examples 158

4.7.1 Design of a Wall Bracket 158 4.7.2 Design of a Rectangular Beam 162

Exercises for Chapter 4 166

Chapter 5 More on Optimum Design Concepts 175 5.1 Alternate Form of KKT Necessary Conditions 175 5.2 Irregular Points 178 5.3 Second-Order Conditions for

Constrained Optimization 179 5.4 Sufficiency Check for Rectangular Beam

Design Problem 184 Exercises for Chapter 5 185

C h a p t e r 6 Linear Programming Methods for Optimum Design 191 6.1 Definition of a Standard Linear Programming

Problem 192 6.1.1 Linear Constraints 192 6.1.2 Unrestricted Variables 193 6.1.3 Standard LP Definition 193

Contents xiii

6.2 Basic Concepts Related to Linear Programming Problems 195 6.2.1 Basic Concepts 195 6.2.2 LP Terminology 198 6.2.3 Optimum Solution for LP Problems 201

6.3 Basic Ideas and Steps of the Simplex Method 201 6.3.1 The Simplex 202 6.3.2 Canonical Form/General Solution of Ax = b 202 6.3.3 Tableau 203 6.3.4 The Pivot Step 205 6.3.5 Basic Steps of the Simplex Method 206 6.3.6 Simplex Algorithm 211

6.4 Two-Phase Simplex Method—Artificial Variables 218 6.4.1 Artificial Variables 219 6.4.2 Artificial Cost Function 219 6.4.3 Definition of Phase I Problem 220 6.4.4 Phase I Algorithm 220 6.4.5 Phase II Algorithm 221 6.4.6 Degenerate Basic Feasible Solution 226

6.5 Postoptimality Analysis 228 6.5.1 Changes in Resource Limits 229 6.5.2 Ranging Right Side Parameters 235 6.5.3 Ranging Cost Coefficients 239 6.5.4 Changes in the Coefficient Matrix 241

6.6 Solution of LP Problems Using Excel Solver 243 Exercises for Chapter 6 246

C h a p t e r 7 More on Linear Programming Methods for Optimum Design 259 7.1 Derivation of the Simplex Method 259

7.1.1 Selection of a Basic Variable That Should Become Nonbasic 259

7.1.2 Selection of a Nonbasic Variable That Should Become Basic 260

7.2 Alternate Simplex Method 262 7.3 Duality in Linear Programming 263

7.3.1 Standard Primal LP 263 7.3.2 Dual LP Problem 264 7.3.3 Treatment of Equality Constraints 265 7.3.4 Alternate Treatment of Equality Constraints 266 7.3.5 Determination of Primal Solution from

Dual Solution 267 7.3.6 Use of Dual Tableau to Recover Primal Solution 271 7.3.7 Dual Variables as Lagrange Multipliers 273

Exercises for Chapter 7 275

C h a p t e r 8 Numerical Methods for Unconstrained Optimum Design 277 8.1 General Concepts Related to Numerical Algorithms 278

8.1.1 A General Algorithm 279 8.1.2 Descent Direction and Descent Step 280

xiv Contents

8.1.3 Convergence of Algorithms 282 8.1.4 Rate of Convergence 282

8.2 Basic Ideas and Algorithms for Step Size Determination 282 8.2.1 Definition of One-Dimensional

Minimization Subproblem 282 8.2.2 Analytical Method to Compute Step Size 283 8.2.3 Concepts Related to Numerical Methods to

Compute Step Size 285 8.2.4 Equal Interval Search 286 8.2.5 Alternate Equal Interval Search 288 8.2.6 Golden Section Search 289

8.3 Search Direction Determination: Steepest Descent Method 293

8.4 Search Direction Determination: Conjugate Gradient Method 296

Exercises for Chapter 8 300

C h a p t e r 9 More on Numerical Methods for Unconstrained Optimum Design 305 9.1 More on Step Size Determination 305

9.1.1 Polynomial Interpolation 306 9.1.2 Inaccurate Line Search 309

9.2 More on Steepest Descent Method 310 9.2.1 Properties of the Gradient Vector 310 9.2.2 Orthogonality of Steepest Descent Directions 314

9.3 Scaling of Design Variables 315 9.4 Search Direction Determination: Newton's Method 318

9.4.1 Classical Newton's Method 318 9.4.2 Modified Newton's Method 319 9.4.3 Marquardt Modification 323

9.5 Search Direction Determination: Quasi-Newton Methods 324 9.5.1 Inverse Hessian Updating: DFP Method 324 9.5.2 Direct Hessian Updating: BFGS Method 327

9.6 Engineering Applications of Unconstrained Methods 329 9.6.1 Minimization of Total Potential Energy 329 9.6.2 Solution of Nonlinear Equations 331

9.7 Solution of Constrained Problems Using Unconstrained Optimization Methods 332 9.7.1 Sequential Unconstrained Minimization

Techniques 333 9.7.2 Multiplier (Augmented Lagrangian) Methods 334

Exercises for Chapter 9 335

C h a p t e r 1 0 Numerical Methods for Constrained Optimum Design 339 10.1 Basic Concepts and Ideas 340

10.1.1 Basic Concepts Related to Algorithms for Constrained Problems 340

10.1.2 Constraint Status at a Design Point 342 10.1.3 Constraint Normalization 343

Contents xv

10.1.4 Descent Function 345 10.1.5 Convergence of an Algorithm 345

10.2 Linearization of Constrained Problem 346 10.3 Sequential Linear Programming Algorithm 352

10.3.1 The Basic Idea—Move Limits 352 10.3.2 An SLP Algorithm 353 10.3.3 SLP Algorithm: Some Observations 357

10.4 Quadratic Programming Subproblem 358 10.4.1 Definition of QP Subproblem 358 10.4.2 Solution of QP Subproblem 361

10.5 Constrained Steepest Descent Method 363 10.5.1 Descent Function 364 10.5.2 Step Size Determination 366 10.5.3 CSD Algorithm 368 10.5.4 CSD Algorithm: Some Observations 368

10.6 Engineering Design Optimization Using Excel Solver 369 Exercises for Chapter 10 373

C h a p t e r 11 More on Numerical Methods for Constrained Optimum Design 379 11.1 Potential Constraint Strategy 379 11.2 Quadratic Programming Problem 383

11.2.1 Definition of QP Problem 383 11.2.2 KKT Necessary Conditions for

the QP Problem 384 11.2.3 Transformation of KKT Conditions 384 11.2.4 Simplex Method for Solving QP Problem 385

11.3 Approximate Step Size Determination 388 11.3.1 The Basic Idea 388 11.3.2 Descent Condition 389 11.3.3 CSD Algorithm with Approximate Step Size 393

11.4 Constrained Quasi-Newton Methods 400 11.4.1 Derivation of Quadratic Programming

Subproblem 400 11.4.2 Quasi-Newton Hessian Approximation 403 11.4.3 Modified Constrained Steepest

Descent Algorithm 404 11.4.4 Observations on the Constrained

Quasi-Newton Methods 406 11.4.5 Descent Functions 406

11.5 Other Numerical Optimization Methods 407 11.5.1 Method of Feasible Directions 407 11.5.2 Gradient Projection Method 409 11.5.3 Generalized Reduced Gradient Method 410

Exercises for Chapter 11 411

Chapter 12 Introduction to Optimum Design with MATLAB 413 12.1 Introduction to Optimization Toolbox 413

12.1.1 Variables and Expressions 413

xvi Contents

12.1.2 Scalar, Array, and Matrix Operations 414 12.1.3 Optimization Toolbox 414

12.2 Unconstrained Optimum Design Problems 415 12.3 Constrained Optimum Design Problems 418 12.4 Optimum Design Examples with MATLAB 420

12.4.1 Location of Maximum Shear Stress for Two Spherical Bodies in Contact 420

12.4.2 Column Design for Minimum Mass 421 12.4.3 Flywheel Design for Minimum Mass 425

Exercises for Chapter 12 429

C h a p t e r 1 3 Interactive Design Optimization 433 13.1 Role of Interaction in Design Optimization 434

13.1.1 What Is Interactive Design Optimization? 434 13.1.2 Role of Computers in Interactive

Design Optimization 434 13.1.3 Why Interactive Design Optimization? 435

13.2 Interactive Design Optimization Algorithms 436 13.2.1 Cost Reduction Algorithm 436 13.2.2 Constraint Correction Algorithm 440 13.2.3 Algorithm for Constraint Correction

at Constant Cost 442 13.2.4 Algorithm for Constraint Correction

at Specified Increase in Cost 445 13.2.5 Constraint Correction with Minimum Increase

in Cost 446 13.2.6 Observations on Interactive Algorithms 447

13.3 Desired Interactive Capabilities 448 13.3.1 Interactive Data Preparation 448 13.3.2 Interactive Capabilities 448 13.3.3 Interactive Decision Making 449 13.3.4 Interactive Graphics 450

13.4 Interactive Design Optimization Software 450 13.4.1 User Interface for IDESIGN 451 13.4.2 Capabilities of IDESIGN 453

13.5 Examples of Interactive Design Optimization 454 13.5.1 Formulation of Spring Design Problem 454 13.5.2 Optimum Solution for the Spring

Design Problem 455 13.5.3 Interactive Solution for Spring Design Problem 455 13.5.4 Use of Interactive Graphics 457

Exercises for Chapter 13 462

C h a p t e r 1 4 Design Optimization Applications with Implicit Functions 465 14.1 Formulation of Practical Design

Optimization Problems 466 14.1.1 General Guidelines 466 14.1.2 Example of a Practical Design

Optimization Problem 467

Contents xvii

14.2 Gradient Evaluation for Implicit Functions 473 14.3 Issues in Practical Design Optimization 478

14.3.1 Selection of an Algorithm 478 14.3.2 Attributes of a Good Optimization Algorithm 478

14.4 Use of General-Purpose Software 479 14.4.1 Software Selection 480 14.4.2 Integration of an Application into General-

Purpose Software 480 14.5 Optimum Design of Two-Member Frame with

Out-of-Plane Loads 481 14.6 Optimum Design of a Three-Bar Structure for Multiple

Performance Requirements 483 14.6.1 Symmetric Three-Bar Structure 483 14.6.2 Asymmetric Three-Bar Structure 484 14.6.3 Comparison of Solutions 490

14.7 Discrete Variable Optimum Design 491 14.7.1 Continuous Variable Optimization 492 14.7.2 Discrete Variable Optimization 492

14.8 Optimal Control of Systems by Nonlinear Programming 493 14.8.1 A Prototype Optimal Control Problem 493 14.8.2 Minimization of Error in State Variable 497 14.8.3 Minimum Control Effort Problem 503 14.8.4 Minimum Time Control Problem 505 14.8.5 Comparison of Three Formulations for

Optimal Control of System Motion 508 Exercises for Chapter 14 508

C h a p t e r 1 5 Discrete Variable Optimum Design Concepts and Methods 513 15.1 Basic Concepts and Definitions 514

15.1.1 Definition of Mixed Variable Optimum Design Problem: MV-OPT 514

15.1.2 Classification of Mixed Variable Optimum Design Problems 514

15.1.3 Overview of Solution Concepts 515 15.2 Branch and Bound Methods (BBM) 516

15.2.1 Basic BBM 517 15.2.2 BBM with Local Minimization 519 15.2.3 BBM for General MV-OPT 520

15.3 Integer Programming 521 15.4 Sequential Linearization Methods 522 15.5 Simulated Annealing 522 15.6 Dynamic Rounding-off Method 524 15.7 Neighborhood Search Method 525 15.8 Methods for Linked Discrete Variables 525 15.9 Selection of a Method 526 Exercises for Chapter 15 527

C h a p t e r 1 6 Genetic Algorithms for Optimum Design 531 16.1 Basic Concepts and Definitions 532 16.2 Fundamentals of Genetic Algorithms 534

xviii Contents

16.3 Genetic Algorithm for Sequencing-Type Problems 538 16.4 Applications 539 Exercises for Chapter 16 540

C h a p t e r 1 7 Multiobjective Optimum Design Concepts and Methods 543 17.1 Problem Definition 543 17.2 Terminology and Basic Concepts 546

17.2.1 Criterion Space and Design Space 546 17.2.2 Solution Concepts 548 17.2.3 Preferences and Utility Functions 551 17.2.4 Vector Methods and Scalarization Methods 551 17.2.5 Generation of Pareto Optimal Set 551 17.2.6 Normalization of Objective Functions 552 17.2.7 Optimization Engine 552

17.3 Multiobjective Genetic Algorithms 552 17.4 Weighted Sum Method 555 17.5 Weighted Min-Max Method 556 17.6 Weighted Global Criterion Method 556 17.7 Lexicographic Method 558 17.8 Bounded Objective Function Method 558 17.9 Goal Programming 559 17.10 Selection of Methods 559 Exercises for Chapter 17 560

C h a p t e r 1 8 Global Optimization Concepts and Methods for Optimum Design 565 18.1 Basic Concepts of Solution Methods 565

18.1.1 Basic Concepts 565 18.1.2 Overview of Methods 567

18.2 Overview of Deterministic Methods 567 18.2.1 Covering Methods 568 18.2.2 Zooming Method 568 18.2.3 Methods of Generalized Descent 569 18.2.4 Tunneling Method 571

18.3 Overview of Stochastic Methods 572 18.3.1 Pure Random Search 573 18.3.2 Multistart Method 573 18.3.3 Clustering Methods 573 18.3.4 Controlled Random Search 575 18.3.5 Acceptance-Rejection Methods 578 18.3.6 Stochastic Integration 579

18.4 Two Local-Global Stochastic Methods 579 18.4.1 A Conceptual Local-Global Algorithm 579 18.4.2 Domain Elimination Method 580 18.4.3 Stochastic Zooming Method 582 18.4.4 Operations Analysis of the Methods 583

18.5 Numerical Performance of Methods 585 18.5.1 Summary of Features of Methods 585 18.5.2 Performance of Some Methods Using

Unconstrained Problems 586

Contents xix

18.5.3 Performance of Stochastic Zooming and Domain Elimination Methods

18.5.4 Global Optimization of Structural Design Problems

Exercises for Chapter 18

586

587 588

Appendix A Economic Analysis A. 1 Time Value of Money

A. 1.1 Cash Flow Diagrams A. 1.2 Basic Economic Formulas

A.2 Economic Bases for Comparison A.2.1 Annual Base Comparisons A.2.2 Present Worth Comparisons

Exercises for Appendix A

593 593 594 594 598 599 601 604

Appendix В Vector and Matrix Algebra B.l Definition of Matrices B.2 Type of Matrices and Their Operations

B.3

B.4

B.5

B.6 B.7

B.2.1 B.2.2 B.2.3 B.2.4 B.2.5 B.2.6 B.2.7 B.2.8 B.2.9 B.2.10

Null Matrix Vector Addition of Matrices Multiplication of Matrices Transpose of a Matrix Elementary Row-Column Operations Equivalence of Matrices Scalar Product-Dot Product of Vectors Square Matrices Partitioning of Matrices

Solution of n Linear Equations in n Unknowns B.3.1 Linear Systems B.3.2 Determinants B.3.3 Gaussian Elimination Procedure B.3.4 Inverse of a Matrix: Gauss-Jordan Elimination Solution of m Linear Equations in n Unknowns B.4.1 Rank of a Matrix B.4.2 General Solution of m x n Linear Equations Concepts Related to a Set of Vectors B.5.1 Linear Independence of a Set of Vectors B.5.2 Vector Spaces Eigenvalues and Eigenvectors Norm and Condition Number of a Matrix B.7.1 Norm of Vectors and Matrices B.7.2 Condition Number of a Matrix

Exercises for Appendix В

611 611 613 613 613 613 613 615 616 616 616 616 617 618 618 619 621 625 628 628 629 635 635 639 642 643 643 644 645

Appendix С A Numerical Method for Solution of Nonlinear Equations С1 Single Nonlinear Equation C.2 Multiple Nonlinear Equations Exercises for Appendix С

647 647 650 655

xx Contents

Appendix D Sample Computer Programs D.l Equal Interval Search D.2 Golden Section Search D.3 Steepest Descent Method D.4 Modified Newton's Method

References Bibliography Answers to Selected Problems Index

657 657 660 660 669

675

683

687

695

Contents xxi